Three Studies of Comparability between Paper-Based and Computer-Based Testing for the ACT. ACT Research & Policy. Research Report

Three mode comparability studies were conducted on the following Saturday national ACT test dates: October 26, 2019, December 14, 2019, and February 8, 2020. The primary goal of these studies was to evaluate whether ACT scores exhibited mode effects between paper and online testing that would necessitate statistical adjustments to the online scores to make them comparable to paper scores. All participants took the full ACT multiple-choice test (English, math, reading, and science), and participants in the February study had the option to take the writing test. The studies each employed an experimental, randomly equivalent groups design. All study participants received college-reportable scores, so their motivation to perform well on the ACT was expected to be like typical examinees during a national testing event. Findings reveal that students who test online tend to perform slightly better than students who test on paper, especially on the English, reading, and writing tests. ACT will equate scores across modes as needed to ensure that ACT scores can be treated as interchangeable regardless of testing mode, and ACT will continue to monitor comparability between paper and online testing and investigate reasons for observed mode effects on the ACT test.