Research concerning the measurement of the permeability of fabrics for polymer matrix composites has been ongoing since several decades, but remains in the focus of applied research for liquid composite molding. Today, several systems and technologies for the measurement of in- and out-of-plane permeability are available, but still no approach has gained acceptance as a common standard. A main requirement for reliable permeability measurement technology is the reproducibility of results when comparing different characterization systems. In this context, benchmark studies are an appropriate method to evaluate the reproducibility of a technology. This study presents a benchmark on unsaturated out-of-plane permeability measurement systems based on flow front monitoring via ultrasonic sensors. Two corresponding systems are compared in the study comprising carbon and glass wovens as well as non-crimp fabrics. The out-of-plane permeability was measured with both systems at three different levels of fiber volume content and three repetitive measurements. The results gained with the systems showed good compliance with relative deviation of the permeability values mostly below 50%. Textile-induced inhomogeneities and varying measurement parameters, e.g. injection pressure, were found to be main reasons for the deviations.