Binary measurement systems that classify parts as either pass or fail are widely used. Inspectors or inspection systems are often subject to error. The error rates are unlikely to be identical across inspectors. We propose a random effects Bayesian approach to model the error probabilities and overall conforming rate. We also introduce a feature-subset selection procedure to determine the best inspector in terms of overall classification accuracy. We provide simulation studies that demonstrate the viability of our proposed estimation ranking and subset-selection methods and apply the methods to a real data set.