On Item Selection Algorithms for Noisy-AND Computerized Adaptive Testing

Abstract

Computerized adaptive testing (CAT) with Noisy-AND models integrates adaptive efficiency with fine-grained diagnostic information about skill mastery. While various item selection algorithms have been proposed, their comparative performance across different Noisy-AND parameterizations and test structures remains unclear. This study compares two canonical Noisy-AND models from Bayesian networks-Leak Noisy-AND (equivalent to Reduced Reparameterized Unified Model, RRUM) and Simple Noisy-AND (equivalent to the Deterministic Inputs, Noisy “And” gate model, DINA)—under three item selection algorithms across simple and complex Q-matrix structures, as well as uniform and realistic skill profile distributions. A fair comparison was ensured through careful calibration of item parameters and the specification of appropriate prior distributions. Monte Carlo simulations indicated no statistically significant differences in classification accuracy between the two models and no meaningful performance differences among the three algorithms across all experimental conditions. Mean pattern match ratios differed by less than 2.2% between models and by less than 3.4% among algorithms, with no differences reaching statistical significance. Across all conditions, stable skill profile estimates were achieved within the first three administered items, demonstrating rapid convergence. These findings demonstrate that, when properly calibrated and paired with well-specified priors, model choice and algorithm selection have minimal influence on classification accuracy in CAT using Noisy-AND models. The rapid convergence further indicates that efficient diagnostic assessment may be accomplished with substantially shorter tests than traditional fixed-form assessments. Overall, this work underscores the critical importance of methodologically rigorous parameter calibration and prior specification in comparative studies, and provides empirical evidence that practitioners may select among these models and algorithms based on interpretability and implementation considerations rather than expected differences in classification performance.