Dissecting the Impact of DIF/DBF on Ability Estimation and Person Fit

Abstract

Prior research has shown that differential item functioning (DIF) and differential bundle functioning (DBF) can influence ability estimation in unidimensional item repose theory (IRT); however, the relationship between ability estimation and uniform and non-uniform DIF/DBF has not been thoroughly investigated. Therefore, a simulation study was conducted to more thoroughly investigate how DIF/DBF and other related factors influence ability estimation in IRT. The factors examined included bundle size, the sum of uniform DIF in a bundle, magnitude of non-uniform DIF in each item in a bundle, balance of reference and focal group examinees, test length, and impact. Results indicated that an increase in uniform DIF/DBF leads to positive ability estimation bias for reference group examinees ability estimates. The magnitude of non-uniform DIF/DBF was found to influence the root mean squared error (RMSE) of ability estimates and standard error of the estimates. Specifically, lower RMSE and lower standard errors were obtained when items were simulated to be more discriminating for the reference group. Rank order correlations between true and estimated ability were found to be highly consistent regardless of the magnitude of uniform and non-uniform DIF/DBF in the bundle. Finally, Crossing SIBTEST was found to provide acceptable type-I error rates and power when uniform DBF was simulated.