Teleconference 3:
April 29, 2002

Issues and Opportunities for Students with Disabilities in Technology Based Assessments

Opportunities

Several advocates have articulated the positive merits of technology-based testing. Some of the advantages over paper/pencil tests that have been cited include: efficient administration, preferred by students, improved writing performance, built-in accommodations, immediate results, efficient item development, increased authenticity, and the potential to shift focus from assessment to instruction. This section describes each of these prospective opportunities.

Efficient Administration

Technology-based assessments can be administered to individuals or small groups of students in classrooms or computer labs, eliminating timing issues caused by the need to administer paper/pencil tests in large groups in single sittings. Different students can take different tests simultaneously in the same room.

Preferred by Students

In an evaluation of testing experience, students overwhelmingly preferred computerized testing to paper/pencil testing (Brown & Augustine, 2001). Most students, regardless of group or ability, believed that the computer was easier, faster, and more fun. Students also responded that using a computer helped concentration by presenting only one question at a time. Brown-Chidsey and Boscardin (1999) interviewed students with learning disabilities and found that the computer helped them deal with limitations that often interfered with the completion of their work.

Improved Writing Performance

As computers become more common in schools, many of today's students are accustomed to using computers in their daily work. Students write and calculate on computers as easily and with more speed and efficiency than previous generations could on paper. Research has shown that writing on computers leads students to write more and revise more than writing with paper/pencil (Daiute, 1985; Morocco & Neuman, 1986). Paper/pencil tests that require writing may underestimate the writing ability of students who have grown accustomed to writing on computers (Russell & Haney, 1997).

Built-in Accommodations

Technology-based testing has been viewed as a vehicle to increase the participation of students with disabilities in assessment programs. According to Greenwood and Rieth (1994), "The primary strength of technology-based assessment is its potential for removing traditional barriers to the inclusion of persons with disabilities in the assessment process through adaptations and accommodations as well as through new forms" (p. 110).

Standardization of accommodated assessment administrations can be facilitated by technology-based assessment. According to Brown-Chidsey and Boscardin (1999), "Using a computer to present a test orally controls for standardization of administration and allows each student to complete the assessment at his/her own pace" (p. 2). Brown and Augustine (2001), cited educator appreciation of a computer's ability to present items over and over, in both written and verbal form, without the need for a non-standard (and sometimes impatient) human reader. Several studies have shown the positive effects of providing a reader for math tests (see Calhoon, Fuchs & Hamlett, 2000; Fuchs, Fuchs, Eaton, Hamlett, & Karns, 2000; Tindal, Heath, Hollenbeck, Almond, & Harniss, 1998). However, Calhoon et al. found that, "teachers are unlikely to provide a reader to meet student needs because teachers prefer test accommodations that require little individualization and do not require curricular or environmental modifications" (p. 272). Thus, technology-based readers may provide access to a necessary accommodation that may not be offered currently, due to issues of convenience.

Just as the use of accommodations on paper/pencil tests has increased awareness and use of accommodations in the classroom, so can opportunities to use built-in accommodation features of technology-based assessments encourage and increase the use of those features in classroom and other environments. For example, Williams (2002) believes, "It is possible that new developments in speech recognition technology could increase opportunities for individual reading practice with feedback, as well as collecting assessment data to inform instructional decision making" (p. 41).

Immediate Results

One of the major drawbacks of state testing on paper has been the long wait for results because of the need to distribute, collect, and then scan test booklets/answer forms. Students tested in the spring often do not receive their results until fall - nor do their teachers or schools. The results of technology-based assessments can be available immediately, providing schools with diagnostic tools to use for improved instruction, and states with information to guide policy. Even open-ended questions can be scored automatically, greatly reducing cost and scoring time (Thompson, 1999). According to a report by the National Governor's Association (2002), cost savings can result from "the elimination of printing and shipping activities when paper testing ceases" (p. 7).

Efficient Item Development

As technology-based assessment becomes more developed, item development will be more efficient, higher quality, and less expensive (National Governor's Association, 2002). Bennett (1998) believes that at some point, items might be generated electronically, with items matched to particular specifications at the moment of administration. "Test design will also be the focal point for responding to diversity. The effects of different test designs on minority group members, females, ...will be routinely simulated in deciding what skills and which task formats to use in large-scale assessments" (Bennett, 1998, p. 9). Baker (2002) cited several research efforts that have significantly advanced the progress of schema or template-based, multiple-choice development and test management systems (see Bejar, 1995; Bennett, 2002; Chung, Baker, & Cheak, 2001; Chung Klein, Herl & Bewley, 2001; Gitomer, Steinbert, & Mislevy, 1995; Mislevy, Steinbert, & Almond, 1999).

Increased Authenticity

Computers allow for increased used of "authentic assessments" - responses can be open-ended rather than just relying on multiple choice. According to Bennett (1998), the next generation of technology-based tests will be "qualitatively different from those of the first generation. This difference will be evident in the test questions (and, in some cases, the characteristics they measure), as well as in development, scoring, and administrative processes" (p. 4). Bennett notes that many Americans are now receiving their news from TV and the World Wide Web, with the expectation that students will increasingly be able to process information from a variety of sources, not just from print. Bennett also suggests that response formats will shift dramatically, perhaps including problems in which a student is not expected to find the best answer, but a reasonable one within certain constraints.

Shifts Focus from Assessment to Instruction

Bennett (1998) believes that eventually large-scale assessment will join with instruction. "Decisions like certification of course mastery, graduation eligibility, and school effectiveness will no longer be based largely on one examination given at a single time but will also incorporate information from a series of measurements" (p. 11). "By virtue of moving assessment into the curriculum, the locus of the debate over performance differences must logically shift from the accuracy of assessment to the adequacy of instruction" (p. 12). Bennett continues this line of thought in a 2001 article, "When well-constructed tests closely reflect the curriculum, group differences should become more an issue of instructional inadequacy than test inaccuracy. As attention shifts to the adequacy of instruction, the ability to derive meaningful information from test performance becomes more critical" (p. 2).

These materials are excerpts from a draft NCEO report developed with support from NCS Pearson