Teleconference 3:
April 29, 2002

Issues and Opportunities for Students with Disabilities in Technology Based Assessments

Computerized Adaptive Testing

In computerized adaptive testing, a student responds to an item, which is followed by more difficult items if the student responded correctly, or easier items if the student responded incorrectly (Hamilton, Klein, & Lorié, 2001). Through this process, a student's performance level is determined. According to Hamilton, Klein and Lorié (2001), "each response leads to a revised estimate of the student's proficiency and a decision either to stop testing or to administer an additional item that is harder or easier than the previous one" (p. 12).

The advantages cited for computerized adaptive testing include short and efficient administration time, with the computer selecting the next item immediately after an item is completed. A proficiency level is determined through the completion of fewer items than a test in which students respond to every item on the test. According to McBride (1985), "A well-constructed adaptive test attains a specified level of measurement precision in about half the length of time a conventional test would require to reach the same level. This is attributable to the adaptive feature; by tailoring the choice of questions to match the examinee's ability, the test bypasses most questions that are inappropriate in difficulty level and contribute little to the accurate estimation of the test-taker's ability" (p. 26).

However Stone and Lunz (1994) found that the inability of students taking computerized adaptive tests to review items and alter their responses may affect the quality of measurement. Students cannot select the order in which they respond to items, or leave some items blank. There is some research that suggests that students who change earlier answers may improve their scores by a small margin (Gerson & Bergstrom, 1995; Stocking, 1996). There is also concern that some students may respond to early items wrong on purpose to get easier questions (Wainer, 1993).

The use of computerized adaptive tests for large-scale assessments has come under scrutiny by federal officials who question whether "levels" testing, defined as testing at a student's ability level rather than at their grade level, meets accountability requirements of Title I (Olson in Education Week, 2002).

These materials are excerpts from a draft NCEO report developed with support from NCS Pearson