Using Cost-effectiveness Analysis to Select Mathematics Screening Measures in Middle School

Abstract

This study examined the utility of using cost-effectiveness analysis to select universal mathematics screening measures in middle school. Participants (n=1586) were students in Grades 6, 7, and 8 at two suburban middle schools in Wisconsin. Screening data, including previous year criterion-measure (Wisconsin Forward Exam) scores, fall Measures of Academic Progress scores, and curriculum-based measurement scores were collected in the fall of 2016. Multiple imputation was used to account for missingness, and linear combinations of screening scores were created using receiver operator curve analyses. Costs were calculated based on published standards, the CostOut® Toolkit, and the experience of content experts. Results reveal that the single most cost-effective screening method studied was using students’ previous year criterion-measure scores to predict current year risk. The most cost-effective linear combination of screening methods was the Wisconsin Forward Exam and Measures of Academic Progress. An analysis of coefficients of variance revealed that using cost-effectiveness analysis produced more variability among screening methods than when using diagnostic accuracy alone, potentially helping stakeholders select from among multiple screening approaches. Finally, the results of this study were tested for robustness to changes in cost assumptions. Analyses revealed that the results of this study were very robust, even when costs were changed significantly. Implications of this study suggest that cost-effectiveness analysis could prove useful in selecting universal academic screening measures, that schools and districts may be able to utilize criterion-measure data in place of other screening approaches, and that combinations of screening measures, although more expensive than individual measures, may indeed be more cost-effective.