Obtaining comparable scores from multiple test forms in case of non-equivalent groups via repeated covariate equating

Abstract

ZÁKLADNÍ ÚDAJE: Frontier Research in Educational Measurement (FREMO): Conference Abstracts. Oslo: Centre for Educational Measurement, University of Oslo, 2023. s. 19-19. [FREMO 2023: Frontier Research in Educational Measurement. 05.09.2023-07.09.2023, Oslo]. ABSTRACT: The traditional way to equate scores from alternate test forms is to include anchor items which are administered to all test takers and are used to adjust for possible differences in ability between the groups taking different forms (Kolen & Brennan, 2004, von Davier, 2011, von Davier, 2013, González & Wiberg, 2017). In the absence of anchor items, one approach is the so-called covariate equating, in which the anchor items are substituted by covariates, such as grades or other test scores (Wiberg & Bränberg, 2015, Longford, 2015, Wallin & Wiberg 2019). This procedure assumes that the covariates can account for differences in ability and that the conditional distribution of the test scores, given the covariates, is the same for both groups. In this work, we consider different violations of the assumptions of the covariate equating method, motivated by real data examples from Czech matura examinations. We consider the case in which the correlation between the covariate and the test score is weak, thus the assumption that the covariates explain the difference in ability is not met. We also consider the case in which the covariate itself is measured using different test forms, thus violating the assumption of the same conditional distribution. We conduct a simulation study showing that, for the latter case, equating the covariate before incorporating it into the primary test scores equating algorithm can improve the accuracy of the resulting equated scores. The real data example demonstrates the proposed repeated covariate method, as well as the effect of low correlation between the score and the covariate.