NCEO Logo
Bookmark and Share

Computer-based Testing: Practices and Considerations

Synthesis Report 78

Martha Thurlow, Sheryl S. Lazarus,  Debra Albus, & Jennifer Hodgson

September 2010

All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Thurlow, M., Lazarus, S. S., Albus, D., & Hodgson, J. (2010). Computer-based testing: Practices and considerations (Synthesis Report 78). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.


Table of Contents


Executive Summary

Computer-based testing (CBT) has emerged as one of the recent “innovative” approaches to assessments most pursued by states. CBT is lauded as the answer to having cheaper and speedier test delivery for state and district-wide assessments. It is also seen by some as an avenue toward greater accessibility for students with disabilities. In this report we explore the context of CBT, current state computer-based tests, and considerations for students with disabilities, in part as follow-up to a similar exploration that occurred in the early 2000s when just a few states were beginning to develop and implement CBT for their state assessments. Nine considerations for states and districts are presented:

  • Consider the assumptions and beliefs of various stakeholders about computer-based instruction and assessments.
  • Consider the system as a whole, from the computer infrastructure to classroom and instructional experiences with computers before deciding whether and how to use CBT.
  • Consider the computer or online platform first, with input from individuals who know students with disabilities and their accessibility needs.
  • Consider a process for bringing in the needed expertise to delineate the specific accessibility features of CBT, and to determine what specific accommodations may still be needed by students with disabilities, as well as to determine whether a computer-based test may create new accessibility issues.
  • Determine the policies for which accessibility features will be available to all students and which are designated for specific groups of students, such as students with disabilities.
  • Consider how to track the use of accessibility features incorporated into CBT design.
  • Field test the accessibility features of the computer-based test at the same time that the computer-based test is field tested.
  • Examine results from CBT for students with disabilities to determine whether there are any features or characteristics of the assessment that might need reconsideration.
  • Develop training for teachers and students to ensure that students benefit from accessibility features.

Overview

Computer-based testing (CBT) has become widespread in recent years. Some states now use an online platform as the primary delivery mode for one or more computer-based tests used for accountability purposes. When CBT was emerging in state testing in the early 2000s, Thompson, Thurlow, Quenemoen, and Lehr (2002) examined the implications of CBT for students with disabilities. There was not much literature about the use of CBT for large-scale assessments at that time, and Thompson et al. worked with states to explore what needed to be considered during development for students with disabilities and how states might address the needs of these students for accommodations in a CBT environment.

Since the early 2000s, much has occurred in CBT. CBT seems to have advantages over paper and pencil testing, both for states that run the assessment programs and for the students who participate in them. These advantages are recognized by the U.S. Department of Education, which in one of its major initiatives (Race to the Top Assessment Program), encouraged the development of CBT. There currently is strong interest in CBT and advocates have identified many positive merits of this approach to assessment including: efficient administration, student preference, self-selection options for students, improved writing performance, built-in accommodations, immediate results, efficient item development, increased authenticity, and the potential to shift focus from assessment to instruction (e.g., Becker, 2006; Salend, 2009; Thompson et al., 2002). CBT also allows new ways of assessing students that move beyond the traditional multiple choice and constructed response items. For example, innovative assessments are now being developed that enable students to manipulate data and role play. Yet, as states move forward with CBT they are discovering that it is important to consider not only the positive benefits, but also potential negative unintended consequences. These include, for example, the possibility that additional training will be needed for students with disabilities to interact successfully with computers and the challenges of determining the best way to present some accommodations such as screen readers.

Despite the fairly dramatic increase in attention to CBT, accessibility challenges continue to have the potential to reduce the validity of the assessment results and to exclude some groups of students from assessment participation. In the early years of CBT many fairly basic design issues baffled testing companies and states as they sought to transfer paper and pencil tests onto a computer-based platform (Thompson, Quenemoen, & Thurlow, 2006). Many of those issues have been resolved. Some states also believe that CBT may be more efficient to administer than paper and pencil based tests, and new test designs may have the potential to improve the assessment of students with disabilities. For example, some accommodations can be embedded in computer-based tests and there may be less variability in how some accommodations are delivered (for example, a screen reader may deliver the read aloud accommodation more consistently than a human reader). However, students with disabilities may be at a great disadvantage if their wide range of access needs are not considered from the beginning of the development process (Thompson et al., 2002).

The purpose of this report is to highlight states’ computer-based tests and the issues that should be considered as they are designed and implemented to include students with disabilities. This report also suggests factors to consider when designing computer-based tests. This report has three sections: (a) description of contextual issues; (b) analysis of states’ CBT; and (c) discussion of considerations for ensuring that computer-based tests are accessible for all students, including students with disabilities.


Contextual Issues Related to Computer-based Testing

The implementation of CBT occurs within a context that both supports and limits its use. In this section, we briefly address several of the contextual factors that surround CBT including: (a) the technological capacity of schools to support CBT, (b) universal design applied to CBT, (c) perceived advantages and disadvantages of CBT, and (d) current federal programs that promote CBT, such as the Race to the Top Assessment Program.

Technological Capacity in Schools

Access to computers and Internet capabilities have for some time been a stumbling block for the push to widespread use of computer-based and online assessments. For example, Becker (2006) questioned “digital equity” in computer access, computer use, and state-level technology policies. He used data from the National Assessment of Educational Progress to examine digital equity, and reported that “students in rural schools or schools with high percentages of African American students were likely to have less access to computers” (p. 1). Becker did not examine whether having a disability had any impact on computer availability or use.

A nationally representative school-level survey on information technology conducted by the National Center for Education Statistics (Gray, Thomas, & Lewis, 2010) reported several findings about the availability and use of technology in schools in the fall of 2008. The results generally indicated that computers with Internet access were available for instruction, and that the ratio of instructional computers with Internet access was 3 to 1. The large majority of computers in public schools were used for instruction. Schools also reported using a district network or the Internet to provide assessment results for teachers to individualize instruction or to inform instructional planning; nearly three-fourths provided online student assessments. Full-time technology staff members were available in about one-third of low poverty schools and one-fourth of high poverty concentration schools. The survey did not ask about availability of computers to students with disabilities, or about the use of computers for statewide testing.

Universal Design Applied to CBT

The term “universal design” applied to assessments in general has been defined in several ways (CAST, 2009; Thompson, Thurlow, & Malouf, 2004). Universal design of assessment generally means an approach that involves developing assessments for the widest range of students from the beginning while maintaining the validity of results from the assessment. Universal design also sometimes refers to multiple means of representation, action/expression, and engagement.

The application of universal design to paper-based assessments has received considerable attention (Johnstone, 2003; Johnstone, Altman, & Thurlow, 2006; Johnstone, Altman, Thurlow, & Moore, 2006; Johnstone, Liu, Altman, & Thurlow, 2007; Johnstone, Thompson, Miller, & Thurlow, 2008; Johnstone, Thurlow, Thompson, & Clapper, 2008; Thurlow, Johnstone, & Ketterlin-Geller, 2008; Thurlow, Johnstone, Thompson, & Case, 2008). More recently, attention has been paid to applying the principles of universal design in a technology-based environment. For example, Russell, Hoffman, and Higgins (2009a) demonstrated that the principles of universal design could be applied to computer-based test delivery. Technology also can be used to more seamlessly link instruction and assessment. For example, Salend (2009) specifically identified a variety of technology-based approaches that might be used for classroom assessments—for example, curriculum-based measurement (CBMs) assessment probes, digitalized observations and portfolios, and self-monitoring tools.

Dolan et al. (2009) prepared a set of guidelines specifically for computer-based assessments. The principles address test delivery considerations, item content and delivery considerations, and component content and delivery considerations. A variety of topics relevant to computer-based testing and universal design is addressed in the component content and delivery considerations section of the guidelines (e.g., text, images, audio, video), with each organized according to categories of processing that students apply during testing.

As the application of universal design principles to CBT has been considered, there also has been increased attention to various assistive technology requirements. Assistive technology devices can include such things as speech recognition or text to speech software, as well as sophisticated technology such as refreshable braille displays or sip and puff technology (which allows individuals unable to use a mouse or speech-to-text technology to send signals to a computer via a straw device using air pressure by sipping and puffing). One of the challenges of CBT has been to ensure that the assistive technology that is needed by students with disabilities is available and that the students know how to use it. Russell (2009) has considered this challenge and developed a set of 15 capabilities that should be incorporated into the computer or online platform (e.g., allowing all text appearing within each test item and on all interactive areas of the screen, including menus and navigation buttons, to be read aloud using a human voice and synthesized voices, etc.).

Perceived Advantages and Challenges of CBT

Many of the perceived advantages and challenges of CBT have been addressed in the literature. Yet, most are not focused on students with disabilities. In examining perceptions and research on CBT, we identified several categories or “themes” that can be used to organize the advantages and challenges of CBT:

  • Economic: Factors that have cost implications.
  • Systems implementation: Logistical, test security, and other factors that affect the capacity of local education agencies and state education agencies to implement CBT.
  • Test administration/design: Factors that affect how students (and sometimes teachers) perceive and interact with the test.
  • Accessibility: Factors that affect how accessible a test is to a wide range of students.

Appendix A provides details on the advantages and challenges of CBT for all students and for students with disabilities.

Current Federal Programs that Promote CBT

During the 2009-2010 school year, two competitions for federal funds promoted CBT. The Race to the Top Assessment Program was designed to support the development of innovative assessments based on a set of common core standards that, among many other qualities, made effective and appropriate use of technology. Given that the Race to the Top assessments potentially will be used by most states, the emphasis in this funding on technology is likely to have a significant impact on future assessment development. Responses to this program of funding already have generated thoughts about the interoperability of a technology platform and accessibility considerations (Mattson & Russell, 2010).

In addition to the federal funds for the regular assessment, funding for an alternate assessment based on alternate achievement standards also was made available. This competition again emphasized technology, stating “The Secretary is also interested in projects that propose the development of alternate assessment systems that use approaches to technology, assessment administration, scoring, reporting, and other factors that facilitate the coherent inclusion of these assessments within States’ Comprehensive Assessment Systems” (U.S. Department of Education, 2010, pp. A-9-A-10).


Analysis of States’ Computer-based Tests

As more states develop and implement CBT, it is important to examine the nature of these assessments and the ways in which the developers of them have considered students with disabilities. Without appropriate consideration, it is likely that accessibility issues will not be adequately addressed, with the result being an assessment that fails to produce valid results for this group of students.

To examine states’ CBTs, we collected information in three steps. Initially, a scan of state department of education Web sites was conducted to determine how many states indicated that they administered computer-based assessments. This occurred during late October and early November, 2009. A comprehensive search of state Web sites occurred later in November to collect additional information about states’ CBTs.

As a third step, a second set of researchers replicated the comprehensive search to verify information found in the states initially identified, and to see whether additional states had posted CBT information. This verification occurred in February and March, 2010. The detailed results of the Web searches are summarized in Appendix B; the Web sites used in the analysis of states’ computer-based tests are listed in Appendix C. This includes only information on state Web sites. It is possible that other assessments exist that do not have information on the state Web site.

The results of our search are shown in Table 1. We found 26 states that had at least one state-administered computer-based test, either operational or in field testing stages. States with more than one type of test are noted in the table with the number of tests in parentheses. In the 26 states with CBT, a total of 51 statewide assessments were being administered: 8 formative assessments, 24 regular tests (including retakes, graduation exams, and alternates based on grade level achievement standards), 14 End of Course (EOC) assessments, 3 alternate assessments based on modified achievement standards (AA-MAS), and 2 tests of English language proficiency (ELP) for English language learners (ELLs). One of the regular end-of-course assessments (SATP2) was being phased out in Mississippi. Two other states (Delaware and Hawaii) were field testing their computer-based tests in fall 2010; they are not represented in the table. In addition, Virginia’s Modified Achievement Standard Test (VMAST) will be administered online in 2011-2012 for mathematics and 2012-2013 for reading.

Table 1. Number of States with CBTs Found by Type of Assessment

Formative/
Diagnostic

Regular
(Including Exit)

End of Course (EOC)

Alternate based on Modified Achievement Standards

English
Language
Proficiency Test

Alaska
Indiana
Kentucky
Louisiana
South Dakota (2)
Utah
West Virginia

Florida
Idaho
Indiana
Kansas
Maryland (2)
Minnesota (4)
Nebraska
Oklahoma
Oregon (3)
South Carolina
South Dakota
Utah (2)
Virginia
Washington
West Virginia (2)
Wyoming

Florida
Georgia
Indiana
Oklahoma
Louisiana
Maryland (2)a
Mississippi
Missouri
North Carolina
South Carolina
South Dakota
Texas
Virginia

Connecticut
Kansas
Maryland

Massachusetts
Texas

Number of States

7

16

13

3

2

Number of Tests

8

24

14

3

2

a Maryland has two EOC assessments. One is based on regular achievement standards and the second (AA-MAS) is based on modified achievement standards. The AA-MAS version is counted in this column.

 

Regular Assessment

As seen in Figure 1, most states (N=12) administered some type of regular assessment via CBT in elementary through high school grades. Only one state each administered a computer-based test in the other four categories: elementary and middle school only, middle school only, middle and high school only, and high school only.

Figure 1 Bar Chart 

Table 2 shows the specific states that administered computer-based tests at the various school levels. South Carolina was the only state that administered CBT only at the high school level; South Dakota only administered its test in middle school. Both Oklahoma and Washington administered CBT at two school levels, while the remainder of the states (n = 12) administered CBT across school levels.

Table 2. States Administering Regular CBTs at Different School Levels

Elementary and Middle School Only

Middle School Only

Middle and High school Only

High School Only

Elementary Through High School

Washington

South Dakotaa

Oklahoma

South Carolina

Florida
Idaho
Indiana
Kansas
Maryland
Minnesota
Nebraska
Oregon
Utah
Virginia
West Virginia
Wyoming

Number of States

1

1

1

1

12

a Technology assessment

The availability of paper versions for CBT varied across states (see Table 3), as did the states’ perspectives on what the use of a paper version was considered to be. Paper versions were considered by some states to be an accommodation, whereas other states considered them an approach that anyone could use. One state required that a paper version be used if a student needed any accommodation. Only those states that mentioned paper versions are included in the table.

Table 3. Approaches of States Offering Paper Versions of Regular CBTs

Any Accommodation Requires Paper

Certain
Accommodations Require Paper

Paper
Version is an Accommodation

Other Situations Needing Paper

Anyone May Use Paper

Prohibited

Texas

Connecticut
Florida
Oregon
Virginia

Kansas
Oklahoma

Connecticuta
Marylandb
Minnesotac
Utahd
West Virginiae

Indiana
Marylandf

None

Number of States

1

4

2

5

2

0

a Beginning in March 2010, the CMT/CAPT MAS Reading tests will be administered as a Computer Based Test (CBT) to all eligible students and uses the Measurement Incorporated Secured Testing (MIST) application. This is the same online application used with students who receive the Word Processor/Online computer response accommodation. Since this will be the primary method of taking this test, it is not considered an accommodation for these tests. If students’ disabilities interfere with what they are expected to do on MIST, they will be allowed to take the test in the most appropriate manner. There are, however, various input devices that may be used to interface with the computer. Schools should investigate these devices and determine their appropriateness for particular students. The MIST application includes a practice site that may be used to determine a student’s ability to participate in online testing. If a student’s lack of experience is a true barrier, the student should be administered the test with the appropriate grade-level test booklet(s) and allowable accommodations.

b On Maryland’s Technology Literacy test all students must take the online assessment unless otherwise noted in an IEP.

c Minnesota retests are online. The first time a student takes the test it is paper/pencil. The student is eligible for a paper-administered retest of the GRAD in reading or mathematics if the student has attempted to pass the GRAD in the requested subject via an online retest at least three times and submits a processing fee of $45.

d Utah is currently transitioning to 100% computer-based testing, except for grade 2 and students whose disability warrants paper-based testing. Utah indicated that students with disabilities should be encouraged to test via computer whenever possible.

e In West Virginia, the Westest 2 Online Writing is a Web-based assessment, and students must have on file an Acceptable Use of the Internet Form, signed by a parent or legal guardian, as directed by West Virginia Board of Education Policy. If a student does not have this form on file, he/she must handwrite the composition and a scribe will enter the composition on the Web site.

f In Maryland, the Science test may be taken on paper or online.

As evident in Table 3, two states were in two of the columns (Connecticut, Maryland), with the different columns reflecting different approaches for different tests. A total of 12 states indicated the use of paper versions of assessments otherwise administered on computer. As noted, Texas required any accommodated student to use a paper version. Four other states indicated that the use of specific accommodations may require a paper version instead of the online version (Connecticut, Florida, Oregon, Virginia). Two states considered paper versions to be accommodations (Kansas, Oklahoma). Five states addressed other situations when paper versions were needed or implied (Connecticut, Maryland, Minnesota, Utah, West Virginia); these are reflected in the table footnotes. Two states appeared to allow any student to take a paper and pencil version for a regular assessment (Indiana, Maryland). No state prohibited paper and pencil versions.

 

End of Course Exams

Table 4 shows the states with End of Course (EOC) exams, with an indication of whether the exams are computer based or only available via paper (i.e., not CBT). The existence of each form of EOC was examined to determine whether all EOCs were administered via CBT. As evident in Table 4, only 3 of the 16 states with EOCs provided the EOCs only via paper.

Table 4. States with End of Course (EOC) Assessments That are CBT or Not CBT

EOC Is Computer-based

EOC Is Not Computer-based

Florida
Georgia
Indiana
Louisiana
Maryland
Mississippi
Missouri
North Carolina
Oklahoma
South Carolina
South Dakota
Texas
Virginia

Arkansas
New York
Tennessee

Number of States

13

3

Table 5 presents information on the availability of paper versions for computer-based EOCs. Eleven of the thirteen states with computer-based EOCs had information about paper versions of the EOCs. Most of these states (n=7) offered paper versions to any student. Only a few states addressed paper versions in other ways. One state noted that the paper version was required for certain accommodations (Oklahoma), one state allowed paper only for braille that is considered an accommodation (Louisiana), and two other states addressed other situations where paper would be needed (Mississippi, South Dakota); these are reflected in the table footnotes. As with the regular tests, no state noted a prohibition on paper versions for EOCs. Of those states that considered paper versions available to any student, Texas noted that although tests are primarily administered online, some content areas were still being introduced and that students had the option of taking paper/pencil versions at the time of this report.

Table 5. Approaches of States Offering Paper Versions of Computer-based EOCs

Any Accommodation Requires Paper

Certain
Accommodations Require Paper

Paper
Version is an
Accommodation

Other Situations Needing Paper

Anyone May Use Paper

Prohibited

None

Oklahomaa

Louisianab

Mississippic
South Dakotad

Georgia
Indiana
Maryland
Missouri
South Carolina
Texas
Virginia

None

Number of States

0

1

1

2

7

a In Oklahoma, for each paper/pencil test book ordered for an IEP student, the district must have on file a copy of the section of the student’s IEP that indicates that a paper/pencil test is necessary to provide an appropriate accommodation.

b In Louisiana, all EOC tests are administered online, except braille, which is administered in braille and the student responses entered online by a teacher or test administrator.

c In Mississippi, Paper/Pencil retests in Algebra I and English II are for students who were first-time test takers in 2007-2008 or 2008-2009.

d It is recommended students take the exams using Achievement Series, however it is recognized there may be times when this is not feasible. Specific instructions have been developed for utilizing EOC exams online through Achievement Series in addition to paper/pencil.


Discussion and Considerations

With the widespread and increasing use of computer-based and online testing for state and district assessments, it is time to examine the practices underway or in development, and to examine the considerations for students with disabilities. Since the Thompson et al. (2002) report examined the implications of computer-based testing for students with disabilities and how to address accommodations in CBT, there has been an evolving literature on this and related topics. For the most part, this literature has presented the advantages of CBT and has made recommendations for how to ensure that these assessments are appropriate for all students, including students with disabilities, from the beginning of their development.

Many of the advantages attributed to CBT (see Appendix A) apply to all students, including students with disabilities. The one area in which the challenges seem to outweigh the advantages for students with disabilities is in the area of accessibility. Challenges here ranged from the possibility of less access to computers and less experience with keyboarding than other students, to difficulties specifically related to certain disability characteristics, such as poor fine motor skills. In addition, the possibility that applications related to accessibility are automatically turned off in the CBT environment was identified as a challenge.

The analysis of states’ computer-based tests indicated that over half of the 50 states had at least one computer-based test either currently operational or in the field testing stages. These computer-based tests include regular assessments, end of course assessments, formative or diagnostic assessments, and assessments designated for specific populations of students (e.g., AA-MAS or ELP assessments). And, it was clear from the analysis that additional tests in additional states are under development, but not quite to the field testing stage. An examination of paper versions of computer-based tests indicated that most states either identify specific situations in which a paper version of the test might be used or require that a paper version be used when a student needs certain accommodations.

Overall, these findings suggest that accessibility continues to be an issue for computer-based testing. Although states have moved forward with CBT during the past decade, there remains a need for continued thinking about maximizing the meaningful participation of students with disabilities in computer-based testing. Suggestions made by Thompson et al. (2002) and by Dolan et al. (2009) continue to be appropriate for those considering CBT. In addition, structural considerations, such as those made by Russell and colleagues (2009) and by Mattson and Russell (2010) about the CBT platform and interoperability requirements need to be brought to bear on accessibility for students with disabilities in CBT.

Increasingly, states are exploring new ways to make CBT more sophisticated and interesting for students, including the development of simulations and increased interactivity. As the evolution of CBT continues, several considerations are relevant for those states and districts delving into—or continuing—computer-based testing that will be appropriate for all students, including students with disabilities:

1. Consider the assumptions and beliefs of various stakeholders about computer-based instruction and assessments.

CBT design and development often are based on assumptions that drive decisions about the nature of the assessment, the tools that are provided as part of the assessment, and the ways in which students may respond to the assessment. It is important to delineate assumptions that drive the decisions for CBTs, so that the appropriateness of the assumptions can be examined before they result in an inappropriate CBT. Assumptions and beliefs can be at different levels, from those that underlie the design of the entire system, to those that guide the ways in which individual students interact with CBT. An example of broader assumption is all students interact with computers in the same way, and that they are similarly engaged when a test is on a computer platform. An example of a specific assumption about how students interact with the computer is the assumption that the student must be able to use a keyboard; students with disabilities use a variety of assistive technology (such as sip and puff technology) to provide them access to computers even if they are not able to use a keyboard.

2. Consider the system as a whole, from the computer infrastructure to classroom and instructional experiences with computers before deciding whether and how to use CBT.

Computer-based testing occurs within the broader educational context, which includes the infrastructure of the building and classroom as well as the computer platform itself. It also occurs within a context in which teachers and students have had differential experiences with computers. Each of these, and potentially other factors, should be carefully analyzed as part of making decisions about CBT. Analysis of contextual factors and computer infrastructure and decision making about whether and how to use CBT must be influenced by the characteristics of all students, including those with disabilities. Thus, the capacity of school buildings and classrooms in a state or district must be checked in terms of the availability of computers that could be used for testing. This analysis needs to include schools and classrooms that may be designated to include only students with disabilities who meet eligibility requirements for the assessments to be administered via computers. Similarly, if assessments are to be Web based, schools and classrooms must be checked for connectivity to the Internet. This includes identifying issues that might emerge if every computer in the school is to be used at the same time for testing. Similar issues may result if a large number of computers in the state attempt to connect to an online platform on or around the same time.

Examining the system as a whole also entails assessing the extent to which students, including those with disabilities, have been using computers during instruction. Although recent surveys have determined that most schools have computer networks and plans for replacing computers (Gray et al., 2010), digital equity gaps have been identified for rural schools and schools with high percentages of African American students (Becker, 2006). No recent study has examined the current status of the equity gap, which was documented in the 1990s (Waddell, 1999) for students with disabilities. The concern that students with disabilities still may have less access to computers and fewer experiences in using them during instruction must be examined in every school, district, and state that is planning to use CBT.

The importance of having experiences with accommodations before they are used in the testing situation has been emphasized (e.g., Thurlow, Elliott, & Ysseldyke, 2003), and is required in most states’ accommodation policies (Christensen, Lazarus, Lail, Crone, & Thurlow, 2008). Although computer-based testing is not an accommodation, the same cautions apply. Without previous experiences and facility in using computers, it is possible that the introduction of the computer itself could depress the performance of students who have not have previously used a computer for instruction.

3. Consider the platform first, with input from individuals who know students with disabilities and their accessibility needs.

The platform for CBTs may be strictly designed for an offline computer, or may be designed for an online system. In both cases, it is essential to consider the specifications for the platform and how they potentially interact with accessibility requirements and with various assistive technology that students might use. Russell (2009) has addressed this issue directly by suggesting capabilities that should be incorporated in the platform to be maximally accessible. Similarly, Mattson and Russell (2010) specifically addressed how to meet both interoperability and accessibility requirements for CBTs, noting that codes (i.e., tags) used to specify elements of a test or test item could be also be identified to provide accessibility instructions. Mattson and Russell (2010) recommend that tags and specific behaviors need to be defined that are expected to occur when a given tag is applied in order to “standardize the behaviors that result when [tags] are applied” (p. 2).

4. Consider a process for bringing in the needed expertise to delineate the specific accessibility features of CBT, and to determine what specific accommodations may still be needed by students with disabilities, as well as to determine whether a computer-based test may create new accessibility issues.

A process is needed to identify specific accessibility features for a given computer-based test in a given content area, just as it is needed for item review (Johnstone et al., 2008). This process ideally would involve stakeholders with various perspectives, including those who know the content, those who know students with disabilities, and those who know how to address programming issues that accessibility issues may create. Consideration would be given to the possible needs of all students with disabilities, including those that may not be addressed through accessibility features included in the CBT, in light of the nature of the items and tasks included on the CBT. These considerations must take on challenging situations and issues, such as what happens for a student with significant visual impairments when simulations and other virtual scenarios are part of CBT. In addition, consideration must be given to those accessibility features that cannot be incorporated into the CBT, resulting in the need still for external accommodations.

5. Determine the policies for which accessibility features will be available to all students and which are designated for specific groups of students, such as students with disabilities.

As accessibility features are incorporated into the CBT, they may no longer be considered accommodations, but instead aspects of the assessment that are available to all students. Some of these features, on the other hand, may be perceived as appropriate only for students with disabilities, adding the requirement that they be turned on or off depending on who the student is taking the assessment. Careful consideration must be given to decisions about what accessibility really means and who it is for. These decisions, like other accessibility decisions, need to be made before or early in the design and development process. In addition, decisions about which accommodations a student will use need to be data-based and include students’ perceptions about the usefulness of accommodations. Further, students need to have used an accommodation in instruction before using it on a test. This can be a particularly challenging requirement for some technology-based accommodations that are specific to a test platform.

6. Consider how to track the use of accessibility features incorporated into CBT design.

An important part of considering any accessibility or accommodation feature is knowing the extent to which students are using those features or accommodations that they should be using. CBT should make it possible to track the use of various features through the technology itself. Then, monitoring of the relationship between recommendations on a student’s Individualized Education Program (IEP) and what actually occurs during the assessment (Christensen, Lail, & Thurlow, 2007; Christensen, Thurlow, & Wang, 2009) will be possible. States could track how students used the various options and accommodations as well as capture student input to improve the accommodations decision-making process for individual students.

7. Field test the accessibility features of the computer-based test at the same time that the computer-based test is field tested.

Accessibility features should be considered part of the design of CBT, and thus should be included in all pilot and field tests to which CBT is subjected. At the same time, efforts must be made to ensure that the full range of students is included in the field testing so that accessibility features can be adequately tested.

8. Examine results from CBT for students with disabilities to determine whether there are any features or characteristics of the assessment that might need reconsideration.

Results from field testing and full test administration should be continually examined to determine whether any accessibility features or other characteristics of the assessment are not working as intended. This will allow test developers to determine whether there is a need to obtain additional stakeholder information, conduct cognitive labs with students who are using the accessibility features, or simply revise the features.

9. Develop training for teachers and students to ensure that students benefit from accessibility features.

Just as there is a need for training for teachers and students about accommodations, there is a need for training related to most accessibility features. Students who have not experienced text- to-speech as presented during the assessment (e.g., by phrase instead of by word, for example), need to have training to understand the working of that feature as it exists on the assessment. Similarly, teachers require training on accessibility features to know how they work so that assistance can be provided to students as needed.


Conclusions

For more than a decade CBT has been called the “new frontier of testing” (Thompson et al., 2002). CBT has matured and is no longer a new frontier—more than half of the states now offer at least one CBT and current federal initiatives strongly encourage the continued movement toward technology enhanced assessments. However, there is a risk that the use of technology could lead to tests that are not accessible to some students with disabilities and that do not validly measure student performance.

We now have a better understanding of both the benefits and challenges of CBT. Consideration of these can help guide a thoughtful approach to the development and implementation of CBT and potentially ensure they are accessible for all students, including students with disabilities. Careful planning enables states and districts to move forward in a way that capitalizes on all that we now know without making ambitious changes that may result in tests that are less accessible for some groups of students.


References

Almond, P., Winter, P., Cameto, R., Russell, M., Sato, E., Clarke, J., Torres, C., Haertel, G., Dolan, B., Beddow, P., & Lazarus, S. (2010). Technology enabled and universally designed assessment: Considering access in measuring the achievement of students with disabilities—A foundation for research. Dover, NH: Measured Progress and Menlo Park, CA: SRI International.

Ash, K. (2008). States slow to embrace online testing. Education Week, 28(13), 21.

Becker, J. D. (2006). Digital equity in education: A multilevel examination of differences in and relationships between computer access, computer use and state-level technology policies. Education Policy Analysis Archives, 15(3), 1-38.

Bennett, R. E., Braswell, J., Oranje, A., Sandene, B., Kaplan, B., & Yan, F. (2008). Does it matter if I take my mathematics test on computer? A second empirical study of mode effects in NAEP. Journal of Technology, Learning, and Assessment, 6(9), 1-38.

Bridgeman, B., Lennon, M. L., & Jackenthal, A. (2001). Effects of screen size, screen resolution, and display rate on computer-based test performance. (ETS RR-01-23). Princeton NJ: Educational Testing Service. Retrieved from http://www.ets.org/Media/Research/pdf/RR-01-23-Bridgeman.pdf

CAST. (2009). UDL guidelines. Retrieved from http://www.cast.org/publications/UDLguidelines/

Cater, K., Rose, D., Thille, C., & Shaffer, D. (2010, June). Innovations in the classroom. Presentation at the Council of Chief State School Officers (CCSSO) National Conference on Student Assessment, Detroit MI.

Christensen, L. L., Lail, K. E., & Thurlow, M. L. (2007). Hints and tips for addressing accommodations issues for peer review. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Christensen, L. L., Lazarus, S. S., Lail, K., Crone, M., & Thurlow, M. L. (2008). 2007 state policies on assessment participation and accommodations for students with disabilities (Synthesis Report 69). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Christensen, L. L., Thurlow, M. L., & Wang, T. (2009). Improving accommodations outcomes: Monitoring instructional and assessment accommodations for students with disabilities. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Clapper, A. T., Morse, A. B., Thompson, S. J., Thurlow, M. L. (2005). Access assistants for state assessments: A study of state guidelines for scribes, readers, and sign language interpreters (Synthesis Report 58). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Department of Justice. (2010). ADA best practices tool kit for state and local governments, Chapter 3 - General effective communication requirements under Title II of the ADA. Retrieved from http://www.ada.gov/pcatoolkit/chap3toolkit.htm

Dolan, R. P., Burling, K. S., Harms, M., Beck, R., Hanna, E., Jude, J., Murray, E. A., Rose, D. H., & Way, W. (2009). Universal design for computer-based testing guidelines. Iowa City, IA: Pearson.

Dolan, R. P., Hall, T. E., Banerjee, M., Chun, E., & Strangman, N. (2005). Applying principles of universal design to test delivery: The effect of computer-based read-aloud on test performance of high school students with learning disabilities. Journal of Technology, Learning, and Assessment, 3(7), 1-31.

Erdogan, Y. (2008). Paper-based and computer-based concept mapping: The effects on computer achievement, computer anxiety and computer attitude. British Journal of Educational Technology, 40(5), 821-836.

Fletcher, G. H. (2004). Technology’s role in accountability and assessment. T.H.E. Journal, 31(9).

Gray, L., Thomas, N., & Lewis, L. (2010). Educational technology in U.S. public schools: Fall 2008 – First look (NCES 2010-034). Washington, DC: U.S. Department of Education. Retrieved from http://nces.ed.gov/pubs2010/2010034.pdf

Johnstone, C. J. (2003). Improving validity of large-scale tests: Universal design and student performance (Technical Report 37). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Johnstone, C. J., Altman, J., & Thurlow, M. (2006). A state guide to the development of universally designed assessments. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Johnstone, C., Altman, J., Thurlow, M., & Moore, M. (2006). Universal design online manual. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Johnstone, C., Liu, K., Altman, J., & Thurlow, M. (2007). Student think aloud reflections on comprehensible and readable assessment items: Perspectives on what does and does not make an item readable (Technical Report 48). Minneapolis, MN: National Center on Educational Outcomes.

Johnstone, C. J., Thompson, S. J., Miller, N. A., & Thurlow, M. L. (2008). Universal design and multi-method approaches to item review. Educational Measurement: Issues and Practice, 27(1), 25-36.

Johnstone, C. J., Thurlow, M. L., Thompson, S. J., & Clapper, A. T. (2008, Spring). The potential for multi-modal approaches to reading for students with disabilities as found in state reading standards. Journal of Disability Policy Studies, 18(4), 219-229.

Kamei-Hannan, C. (2008, May). Examining the accessibility of a computerized adapted test using assistive technology. Journal of Visual Impairment and Blindness, 261-271.

Ketterlin-Geller, L. R. (2005). Knowing what all students know: Procedures for developing universal design for assessment. The Journal of Technology, Learning, and Assessment, 4(2), 1-21.

Kettler, R., Scholz, C., Oderman, E., Hixon, N, & Weigert, S. (2010). Innovative uses of technology to support assessment decision-making and curricular alignment for students with disabilities. Presentation at the Council of Chief State School Officers (CCSSO).

Kingston, N. M. (2009). Comparability of computer- and paper-administered multiple-choice tests for K-12 populations: A synthesis. Applied Measurement in Education, 22(1), 22-27.

Luecht, R. M. (2005). Some useful cost-benefit criteria for evaluating computer-based test delivery models and systems. Journal of Applied Testing Technology, 7(2), 1-31.

Mattson, D., & Russell, M. (2010). Meeting interoperability and accessibility requirements by adopting the accessible portable item profile (APIP) standards (White Paper). St. Paul, MN: Minnesota Department of Education.

McGroroy, K., & Sampson, H. (2010, June 7). Glitches delay Florida test score results. Miami Herald. Published online.

McHenry, B., Griffith, L., & McHenry, J. (2004). The potential, pitfalls and promise of computerized testing. T.H.E. Journal, 31(9), 28.

Meyen, E., Poggio, J., Seok, S., & Smith, S. (2006). Equity for students with high-incidence disabilities in statewide assessments: A technology-based solution. Focus on Exceptional Children, 38(7), 1-8.

Olson, J. F., & Dirir, M. (2010). Technical report for studies of the validity of test results for test accommodations. Washington DC: Council of Chief State School Officers.

“Overview information; Race to the Top Fund Assessment Program; Notice inviting applications for new awards for fiscal year (FY) 2010.” 75 Fed. Reg. 18182 (2010, April 9). http://www.gpo.gov/fdsys/pkg/FR-2010-04-09/pdf/2010-8176.pdf

“Race to the Top Fund.” 74 Fed. Reg. 54798 (2009, October 23). http://www.gpo.gov/fdsys/pkg/FR-2009-10-23/pdf/E9-25600.pdf

Russell, M. (1999).Testing on computers: A follow-up study comparing performance on computer and on paper. Education Policy Analysis Archives, 7(20).

Russell, M. (2009). Universal design of computer-based test. Boston: Boston College.

Russell, M., Almond, P., Higgins, J., Clarke-Midura, J., Johnstone, C., Bechard, S., & Fedorchak, G. (2010, June). Technology enabled assessments: Examining the potential for universal access and better measurement in achievement. Presentation at the Council of Chief State School Officers (CCSSO) National Conference on Student Assessment, Detroit MN.

Russell, M., Hoffman, R., & Higgins, J. (2009a). Nimble Tools: A universally designed test delivery system. Teaching Exceptional Children, 42(2), 6-12.

Russell, M., Hoffmann, T., & Higgins, J. (2009b). Meeting the needs of all students: A universal design approach to computer-based testing. Innovate: Journal of Online Education, 5(4).

Salend, S. J. (2009). Technology-based classroom assessments: Alternatives to testing. Teaching Exceptional Children, 41(6), 48-58.

Sawchuk, S. (2010). Open-ended test items pose challenges. Education Week, 29(19), pp. 1, 11.

Stock, S. E., Davies, D. K., & Wehmeyer, M. L. (2004). Internet-based multimedia tests and surveys for individuals with intellectual disabilities. Journal of Special Education Technology. 19(4), 43-47.

Thompson, S. J., Johnstone, C. J., & Thurlow, M. L. (2002). Universal design applied to large-scale assessments (Synthesis Report 44). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thompson, S. J., Quenemoen, R. F., & Thurlow, M. L. (2006). Factors to consider in the design of inclusive online assessments. In M. Hricko (Ed.), Online assessment and measurement: foundations and challenges (pp. 102-117). Hershey, PA: Information Science Publishing.

Thompson, S. J., Thurlow, M. L., & Malouf, D. (2004, May). Creating better tests for everyone through universally designed assessments. Journal of Applied Testing Technology, 10(2). See http:// www.testpublishers.org/atp.journal.htm

Thompson, S. J., Thurlow, M. L., Quenemoen, R. F., & Lehr, C. A. (2002). Access to computer-based testing for students with disabilities (Synthesis Report 45). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M. L., Elliott, J. E., & Ysseldyke, J. E. (2003). Testing students with disabilities: Practical strategies for complying with district and state requirements (2nd ed.). Thousand Oaks, CA: Corwin.

 Thurlow, M., Johnstone, C., & Ketterlin-Geller, L. (2008). Universal design of assessment. In S. Burgstahler & R. Cory (Eds.), Universal design in post-secondary education: From principles to practice (pp. 73-81). Cambridge, MA: Harvard Education Press.

Thurlow, M., Johnstone, C., Thompson, S., & Case, B. (2008). Using universal design research and perspectives to increase the validity of scores on large-scale assessments. In R. C. Johnson & R. E. Mitchell (Eds.), Testing deaf students in an age of accountability (pp. 63-75). Washington, DC: Gallaudet University Press.

U.S. Department of Education. (2010). Application for new grants under the Individuals with Disabilities Education Act (IDEA): Technical assistance on state data collection—IDEA General Supervision Enhancement Grant—Alternate academic achievement standards (CFDA 84.373X). Washington, DC: Office of Special Education Programs.

Waddell, C. D. (1999). The growing digital divide in access for people with disabilities: Overcoming barriers to participation in the digital economy (White Paper for Understanding the digital economy: Data, tools and research). Washington, DC. National Science Foundation. Available at http://www.icdri.org/CynthiaW/the_digital_divide.htm


Appendix A

Advantages and Challenges of Computer-based Tests (CBTs)

Practitioners, state department of education staff, and researchers have identified many advantages and challenges of CBTs. We identified several categories or “themes” that can be used to organize the advantages and challenges:

  • Economic: Factors that have cost implications.

  • Systems implementation: Logistical, test security, and other factors that affect the capacity of LEAs and SEAs to implement CBTs.

  • Test administration/design: Factors that affect how students (and sometimes teachers) perceive and interact with the test.

  • Accessibility: Factors that affect how accessible a test is to a wide range of students.

Many advantages and challenges of CBTs are not specific to students with disabilities.

In Tables A1–A4, we have listed advantages and challenges based on whether they affect all students or students with disabilities. This appendix ends with a brief discussion of advantages and challenges of CBT for ELLs with disabilities. The advantages and challenges identified in these tables should not be considered all-inclusive, and there probably are additional advantages and challenges that we have not identified.

Economic

Table A1 summarizes economic advantages and challenges. In the short run CBTs often cost more than paper and pencil tests because they are costly to develop and implement. Traditional tests require the printing and shipping of test booklets. Less paper is needed with CBT, and it may be greener and reduce the carbon footprint. However, at least in the short run, the carbon footprint may not be reduced. Many schools may not currently have enough computers and related infrastructure—and the manufacture and delivery of these items may increase the carbon footprint. Additional electricity will also be used. Costs may be shifted from State Education Agencies (SEAs) to Local Education Agencies (LEAs). SEAs generally are responsible for printing and shipping test booklets—but in some situations LEAs may be responsible for providing the technology that is needed to administer CBT. CBTs may cost less in the longer run than paper and pencil tests, but if technology continues to rapidly evolve cost savings may not occur in the foreseeable future (Luecht, 2005; Kettler, Scholz, Oderman, Hixon, & Weigert, 2010).

Table A1. Economic Advantages and Challenges of CBTs for All Students and for Students with Disabilities

Advantages

Challenges

All Students

Cost-effective (long-run).

Reduce paper and shipping.

Need to prepare for a more global economy.

Cost (short-run).

Resource-intensive (short-run).

Students with Disabilities

Less costly to provide some accommodations.

Accommodations use tracked more efficiently.

Cater, Rose, Thille, and Shaffer (2010) maintain that students who participate in computer-based instruction and testing may be more prepared to compete in the global economy. They argue that today’s students need to know how to use technology to obtain good jobs, and teaching students to navigate and successfully complete online tests might help prepare them for the future.

Students with Disabilities. It is costly for LEAs to provide some accommodations; and SEAs and LEAs expend resources keeping track of the accommodations that students receive on test day. There can be cost savings when many accommodations—including accommodations that may have been provided by a human (i.e., read aloud, scribe, sign interpret, etc.) for a paper and pencil test— are incorporated into CBTs (Russell, Almond, Higgins, Clarke-Midura, Johnstone, Bechard, & Fedorchak, 2010).

Systems Implementation

Table A2 summarizes the systems implementation factors. With paper and pencil tests, materials need to be distributed to—and collected from—each administration site. Additional test administrators and extra rooms may be needed. However, CBTs may create their own set of logistical issues: Are there enough computers available? Are there technology experts available to assist with any computer problems? How will computer labs—or other test administration sites—be set up to minimize opportunities for cheating? Have students had adequate opportunities to learn how to use computers? (Luecht, 2005; McGrory & Sampson, 2010, McHenry, Griffith & McHenry, 2004). Historically there have also been equity issues related to student access to computers. Becker (2006) found that rural and African American students were less likely to have access to computers than other students.

Table A2. Systems Implementation Advantages and Challenges of CBTs for All Students and Students with Disabilities

Advantages

Challenges

All Students

More efficient administration.

Data are more accurately
collected.

Easier to change the test if mistakes are discovered.

Results and other data can be stored in much less space—and it is easier to retrieve.

Test security may be improved if schools and districts do not have hard copies ahead of time.

On-demand testing.

Potential to shift focus from
assessment to instruction.

Test-day logistics.

Some schools may not have enough computers and related infrastructures.

Finding instructional time to teach students how to navigate the test and how to use online tools.

Practice tests/manuals need to be available far in advance of test so teachers can teach needed computer skills.

Both LEA educators and technology staff need training.

Students with Disabilities

Special test formats do not need to be requested in
advance.

It may take longer for some students with disabilities to learn to use online tools (for example, measurement tools) than other students.

Currently some schools do not have enough computers for all students to take large-scale tests within a reasonable time frame. Expanding the test window, or in some cases on-demand testing, can help resolve this challenge, but it ties up computer labs for a longer period (Ash, 2008). This characteristic may be particularly relevant for retests and for end-of-course tests where there may be a preference for administering the test soon after the completion of the course (Russell et al., 2010).

Also, SEAs and LEAs sometimes lack related infrastructure. For example, according to Ash (2008), there is a need for “beefed-up network capacity to handle the spike in bandwidth for the assessment applications to run smoothly” (p. 21). Students who take a test that is interrupted by technology issues (e.g., loss of Internet connection, hardware or software issues) tend to perform less well on a test when the test administrator attempts to restart the student where they stopped or allow the student to start over (Bennett, Braswell, Oranje, Sandene, Kaplan, & Yan, 2008).

CBT may help ensure that data are more accurately collected, and are easier to store. Responses generally are accurately captured and scored with a CBT. For example, on paper and pencil tests students sometimes make stray marks or messy erasures that can result in inaccurate scoring of an exam. It may also be easier to change a test if mistakes are discovered after it has been finalized (Luecht, 2005; McGroroy & Sampson, 2010; McHenry et al., 2004). Still, test contractors continue to have major problems with scoring and data retrieval. For example, the June 7, 2010 online issue of the Miami Herald contained an article entitled “Glitches Delay Florida Test Score Report” that discussed problems with several CBTs (and other tests) including system-wide technology glitches and students who could not log in. The article quoted Florida K-12 Chancellor Frances Haithcock as saying, “Accuracy must at all times trump expediency, especially given the importance of these results” (McGroroy & Sampson, 2010).

An important potential advantage of CBT at the systems level may include the shifting of focus from assessment to instruction. Currently some educators and others express frustration that assessment is driving instruction. CBTs that are closely aligned with instructional methods may have the potential to move the focus back to instruction (Kettler et al., 2010; Meyen, Poggio, Seok, & Smith, 2006).

Classroom instruction time may be needed to train students how to navigate a computer-based test and how to use test tools. Different tests have different platforms—and students need to learn test-specific computer navigation skills. Also, without prior training students often are unable to correctly use online rulers, protractors, and other online measurement tools. Teachers need to plan lessons and provide instruction so that students know how to use the specific features and tools incorporated into a computer-based test. States and test vendors sometimes fail to make practice tests and manuals available far enough ahead of test day to provide sufficient instructional time. The school day is already packed and decisions need to be about what should be eliminated from the curriculum to find time to teach students these skills (Kettler et al., 2010; Russell et al., 2010; Thompson et al., 2002).

Training is needed not only for test administrators, but also for LEA technology staff. For example, computers may need to have specific software downloaded prior to test day. Also, problems with computers on the day of the test can cause significant disruption and potentially cause additional anxiety in staff and students; technology staff need training so that they will be prepared resolve potential issues that might emerge on test day (Ash, 2008; Thompson et al., 2002).

Students with Disabilities. Occasionally school district staff may not know ahead of time that a particular special format test (e.g., large print, braille, read aloud package, etc.) will be needed—for example, if a student with visual impairments enrolls in a school a few days before test day. Schools and districts may find CBT to be more convenient because test formats for students with disabilities can be customized on test day, and there is no need to order special formats far in advance (Kettler et al., 2010; Russell et al., 2010).

Some students with disabilities are not in fully inclusive settings and may have fewer opportunities to learn and practice keyboarding skills, computer navigation skills, and online tools. And, some students with physical or cognitive disabilities may have particular challenges to become proficient keyboarders (Ketterlin-Geller, 2005).

Test Administration and Design

Table A3 summarizes test administration/design advantages and disadvantages. Some computer-based tests may have the potential to more authentically assess student learning than paper-based tests—though this is a more a function of test design than just the test platform (Fletcher, 2004; Thompson et al., 2002). Innovative CBT formats that may become more common in the future include role playing, simulations, and data manipulations (Russell et al. 2010).

Another advantage of CBT may be student preference. Some students have used computers to play games and they may receive some of their instruction via computers. Students may also prefer CBT because it has the option of customizing the assessment based on personal preferences. For example, all students may be allowed to decide what background color they would like on the screen, or what font size they would prefer. Although some students may prefer CBT, others may prefer paper and pencil tests (Cater et al., 2010; Russell et al., 2010). Interactions with computers may cause anxiety for some students. Computer anxiety does not refer to negative attitudes towards computers, but rather to a student’s “emotional reaction to using computers” (Erdogan, 2008, p. 823). Students with weak computer skills (for example, students who have difficulty with scrolling) are more likely to experience test anxiety (Bridgeman, Lennon & Jackenthal, 2001). Erdogan asserts that providing students with more opportunities to use computers during instruction has the potential to reduce computer anxiety.

Educators and parents often are frustrated by how long it takes to receive test results. Teachers often need timely results for instructional decision-making purposes. Generally with CBT, multiple choice items can be scored with a very short turn-around (Ash, 2008; Kingston, 2009). For both computer-based tests and paper-based tests, there generally is a time lag in scoring constructed response items. Currently many states with CBT hand score open-ended items (often using teachers)—although “test publishers are investing heavily in automated-response systems that use artificial-intelligence software to better ‘read’ student answers” (Sawchuk, 2010, p. 1). According to Sawchuk, these systems would distance teachers from scoring assessments which can be a valuable professional development activity.

Table A3. Test Administration/Design Advantages and Challenges of CBTs for All Students and Students with Disabilities

Advantages

Challenges

All Students

Increased authenticity.

Innovative ways to assess students (role playing, simulations, data manipulation).

May be preferred by students

Self-selection options.

Immediate results (at least for multiple choice items).

If adaptive testing is used, the test might be shorter.

Students may not always make good choices about which embedded resources they use.

Some students may have computer anxiety.

 

Students with Disabilities

Built-in accommodations.

Some accommodations can be delivered more consistently.

Read aloud/text-to-speech voice may be different from what student is used to—different accent, unusual ways of pronouncing words.

Text-to-speech may express terminology or terms in a different way from instruction. (Math and scientific terms are particularly problematic.)

Students may not always make good accommodations choices.

Braille can be a challenge with CBTs.

Innovative test formats may be difficult for some students with disabilities (e.g., those with visual impairments, poor fine motor skills, etc.).

For students with visual impairments, it can be difficult to describe online graphics without giving away answer.

If adaptive testing is used, it can be difficult to ensure that students with gaps in their knowledge have the opportunity to show what they know.

Some computer-based tests are computer adaptive tests (CATs) that tailor the test items that a student receives based on his or her correct or incorrect responses to previous items—and the resulting test is generally shorter than a non-adaptive test that is designed to provide similar information (Meyen, Poggio, Seok, & Smith, 2006). Care must be used to ensure that these types of CBTs appropriately assess all students. An adaptive test must be well designed to ensure that students who miss questions early in the test are not denied the opportunity to answer challenging questions. With some adaptive test designs some students who have gaps in their knowledge—including some students with disabilities—may only have the opportunity to answer low level questions. For example, a student may not be able to correctly answer a computation problem but could respond correctly to a problem that requires the use of higher order skills; the algorithms for many computer-based tests currently are not designed in ways that ensure that such students would have the opportunity to respond to those challenging questions (Almond, Winter, Cameto, Russell, Sato, Clarke, Torres, Haertel, Dolan, Beddow, & Lazarus, 2010; Thompson et al. 2002).

Students with Disabilities. Some accommodations can be delivered more consistently with a computer-based test. For example, for a paper-based test it can be problematic to consistently provide some accommodations that involve a human access assistant to administer (e.g., reader, sign language interpreter, scribe). The quality of human readers varies greatly from one individual to another. Some readers may mispronounce words, use misleading intonation, or influence student responses. With the read aloud accommodation some human readers may use voice tone to suggest the correct response—whereas others would not (Clapper, Morse, Thompson, & Thurlow, 2005). An advantage of CBT is that technology can consistently provide the read aloud accommodation (Dolan, Hall, Banerjee, Chun, & Strangman, 2005) although computer-based tests may use a digitalized or human read aloud/text-to-speech voice that students find difficult to understand. The voice may have unusual ways of pronouncing words or a different accent from what the student is used to (Olson & Dirir, 2010). The text-to-speech function may also express terminology or terms in a different way from instruction. Math and science terminology are particularly problematic. For example, if a science test item refers to H2O should it be read as “H 2 O” or as “water”? Christopher Johnstone conducted a think aloud study and found a clear student preference for “water” rather than “H 2 O” but that may affect the difficulty of the test item (Russell et al., 2010).

In a small study focused on individuals with intellectual disabilities, Stock, Davies, and Wehmeyer (2004) found that many study participants preferred computer-based tests. The participants particularly liked being able to take the test with little assistance. Still, when students have the opportunity to self-select accommodations on a computer-based test, they sometimes make poor decisions. Requiring the pre-selection of accommodations prior to test day can help mitigate this issue. Also, some students with disabilities may find innovative test formats challenging. Innovative formats may be especially challenging for students with visual impairments or poor fine motor skills. For example, it is difficult to braille innovative test items. It is also sometimes difficult to describe some online graphics without giving the answer away (Kamei-Hannan, 2008; Kettler et al., 2010; Russell et al., 2010; Thompson et al., 2002).

As noted previously, if a computer-based test is used, it can be difficult to ensure that students with gaps in their knowledge have the opportunity to show what they know. This can be particularly problematic for some students with disabilities. As a result of their disability some may lack lower level skills and knowledge, yet know more complex information and have higher order knowledge. Many CATs can fail to provide these students with opportunities to show what they know (Thompson et al., 2002).

Accessibility

Table A4 summarizes accessibility factors that we identified. Well-designed, more universally designed assessments benefit all students—including students with disabilities. According to Ketterlin-Geller (2005), “computer-based technology presents an efficient tool for customizing assessments to meet individual needs within a universally designed environment” (p. 5). For example, all students would benefit if a computer-based test has allowable features that make the test easier to understand and navigate. Computer-based tests often allow many options for interacting with the assessment, which fits well with the concept of universal design. (Dolan et al., 2005; Russell, Hoffman, & Higgins, 2009b). However, younger students, as well as some other students, may have never learned how to use a keyboard or a mouse. Some early elementary grade students also may lack the hand size and fine motor skills to successfully use a keyboard. Students with weaker keyboarding skills tend to do less well on CBTs than other students—though this effect is diminished as keyboarding skills improve (Russell, 1999).

Table A4. Accessibility Advantages and Challenges of CBTs for All Students and Students with Disabilities

Consideration

Advantages

Challenges

All Students

Potential for more universally designed assessments.

Some students may not have good keyboarding skills.

Students with Disabilities

For some students with disabilities the test may be more accessible than a paper/pencil test.

Online tests can be difficult for students with severe visual impairment/blind or who have poor fine motor skills.

CBTs may require more working memory.

Accessibility an issue if applications (for example, accessible mouse) that come with computer are automatically turned off.

Can create challenges for students who need a more concrete representation of things.

Students with Disabilities. Some test design changes (font size, read aloud, screen background color, etc.) may be incorporated into the design of some CBTs. These embedded resources can be used to create customized, more universally designed tests for students who need an accommodation to meaningfully access the test. It is much easier to provide some accommodations based on individual student needs with a CBT than with a paper and pencil test (Ketterlin-Geller, 2005; Russell et al. 2009b; Thompson et al., 2002). On a CBT some of these features may no longer be considered an accommodation, but instead an allowable resource that all students may use (Almond, et al. 2010).

Students with some visual impairments may find CBT to be more accessible than paper and pencil testing because the font size can be adjusted easily. However, for students with more severe visual impairments or blindness, a CBT may be less accessible than a paper and pencil test. And, as previously discussed, braille is a challenge for current CBTs—especially with innovative items (Russell et al., 2010). Similar examples could be given for students with disabilities who have other characteristics. CBTs can create challenges for students who need more concrete representations because they often use new ways of representing information, and they may require the use of online tools that are visual representations (Russell et al., 2010; Thompson, et al., 2002).

Some CBTs may require more working memory than similar paper and pencil tests. There are cognitive differences in how students interact with CBTs and paper and pencil tests. For example, for a reading passage, less text typically appears on the screen at any one time than on a page in a test booklet. Students who need to scroll through a passage may miss or forget key information that is needed to answer an item. CBTs that minimize the need for scrolling or allow students to highlight text can help mitigate this issue (Kingston, 2009).

Delivery systems can be an issue if a student does not have access to an assistive technology delivery device that he or she normally uses. For test security, or other reasons, computer applications are sometimes automatically turned off on CBTs. For example, a student may normally use an accessible mouse or a spellchecker that is not allowed on a CBT. Bennett et al. (2008) found that student performance is negatively affected when students are not allowed to use a familiar computer.

ELLs with Disabilities

Many of the same economic, systems implementation, test administration/design, and accessibility factors noted for students with disabilities in general are applicable—and sometimes even more of an issue—for ELLs with disabilities. For example, a digitalized or human read aloud/text-to-speech voice using an unfamiliar dialect can present particular challenges for ELLs with disabilities because dialects may not be as easily perceived by ELLs if the computer voice speaks with a different dialect from the one that the student hears every day. CBTs may also have benefits that are specific to ELLs with disabilities. For example, a CBT could offer an ELL with disabilities (as well as other ELLs) more levels of glosses than a paper and pencil test. A CBT could give a picture of an item, a glossed word tailored to a student’s language background, or other options.


Appendix B

Computer-based Tests: Specifications and Details

Online Test

Test Type

Content Area

Grade

Status

Notes

Alaska

Alaska Computerized Formative Assessments (ACFA)

Formative

Math, Reading

3-10

Active

Free for teachers, linked to grade level expectations, instant reporting; includes item pool for teachers to create own tests; online tools available for accommodations; no indication of paper/pencil version.

Connecticut

CAPT/CMT MAS

Alternate based on modified achievement standards

Reading

3-8, 10

Active

"The Bureau of Student Assessment is committed to providing every student with the most appropriate access to the state tests. If students’ disabilities interfere with what they are expected to do on MIST, they will be allowed to take the test in the most appropriate manner. There are, however, various input devices that may be used to interface with the computer. Schools should investigate these devices and determine their appropriateness for particular students."

Florida

Florida Comprehensive Assessment Test (FCAT)

Regular retakes

Math, Reading, Science, Writing

3-11

Active

For schools participating in the computer-based Retake, the number of paper-based tests should only reflect the number of students whose IEP or Section 504 plan indicates that they must test on paper, as explained in the Accommodations section below.

End of Course (EOC) Exams

End of Course

Algebra I, Biology, American History

After course is taken

Active/Devel-oping

Algebra I field test to occur in 2009-2010; all tests computer based only except for limited paper for accommodation for SWD.

Georgia

End of Course Tests (EOCT)

End of Course

Math I, II, US History, Econ, Biol, Phys Sci, 9th Lit and Comp, Amer Lit and Comp

After course is taken

Active

Optional paper version on main administrations.

Idaho

Idaho Standards Achievement Tests (ISAT)

Regular

Math, Reading, Language Usage, Science

3-11

Active

Indiana

Indiana Statewide Testing for Educational Progress (ISTEP)+

Regular

Math, ELA, Science Social Studies

3-8

Active

Can also be taken via paper and pencil but was unclear for whom.

End of Course Assessments (ECAs)

End of Course

Algebra I, English 10, Biology I

After course is taken

Active

Both online and paper and pencil versions are available but unclear for whom.

Grade 3-8 Diagnostic Tools (AcuityTM)

Formative

Math, English Language Arts, Science, Social Studies

3-8

Active

May be active only in some schools; voluntary; schools have to pay a per student fee.

Kansas

Kansas Computerized Assessments (KCA)

Regular

Math, Reading, Science

3-8,

High School

Active

A paper/pencil copy of the assessment is available only as an accommodation.

Kansas Assessment of Modified Measures (KAMM)

Alternate based on modified achievement standards

Math, Reading, Science

3-8,

High School

Active

The KAMM is considered part of the KCA. A paper/pencil copy of the assessment is available only as an accommodation.

Kentucky

Test of Primary Reading Outcomes (T-PRO)

Diagnostic/ Formative

Reading

K - 3

Active

Louisiana

End of Course (EOC)

End of Course

Algebra I, Geometry, English II

After course is taken

Active

Administered online only with the exception of the braille test.

EAGLE

Enhanced Assessment of Grade Level Expectations

Formative

ELA, Math, Algebra

4 ELA/ Math,

9 Algebra

Active

They are adding grades 8,10-12 to math and 8-12 to ELA.

Maryland

Maryland School Assessment – Science (MSA– Science)

Regular

Science

5, 8

Active

Paper/pencil and online versions of this assessment are available but not clear for whom.

Maryland Modified High School Assessments (Mod-HSA)

End of Course

alternate based on modified achievement standards

English, Algebra/

Data Analysis, Biology, Government

High School

Active

Paper/pencil and computer versions of this assessment are available but not clear for whom.

Modified Maryland School Assessments (Mod-MSA)

Alternate based on modified achievement standards

Reading, Math

3-8

Active

Paper/pencil and online versions of this assessment are available but not clear for whom.

High School Assessment

End-of-course

English, Algebra/Data Analysis, Government, Biology

High School

Active

Paper retakes available.

Maryland Measure of Students Technology Literacy for 7th Grade

Regular

Technology skills

7

Active

Online only, except as required by an IEP.

Massachusetts

MEPA RW

English language proficiency test

Reading, Writing,

K-12

Active

In 2010 a small number of schools are being given the test on voluntary basis.

Minnesota

Science Minnesota Comprehensive Assessment – II (Science MCA-II)

Regular

Science

5, 8,

High School

Active

Mathematics Minnesota Comprehensive Assessment III

Regular

Math

Active/Field Test

Field tested in selected districts in Fall 2009.

Graduation Assessment Required for Diploma (GRAD)

Graduation Test

Reading, Math, Written Composition

High school

Active

Retests are online. (The first time a student takes the test it is paper/pencil.) The student is eligible for a paper-administered retest of the GRAD in reading or mathematics if the student has attempted to pass the GRAD in the requested subject via an online retest at least three times and submits a processing fee of $45.

Mathematics Test for English Language Learners (MTELL)

Regular

Math

3-8, 11

Active

Mississippi

Subject Area Testing Program, Second Edition (SATP2)

End of Course

English I, English II, Algebra I, Geometry, Integrated Math III, Biology, Government

High school

Active/Phas-ing Out

Students who enrolled in the course prior to 2007-08 must take retakes online. New test framework for students who took course in 2007-08 or later is a paper/pencil test.

Missouri

Missouri End of Course Assessments (EOC)

End of Course

English I, English II, Algebra I, Algebra II, Geometry, Integrated Math II, Integrated Math III, Biology, American History, Government

High School

Active

Paper/pencil and online versions of this assessment are available.

Nebraska

Nebraska State Accountability (NeSA)

Regular

Reading

3-8, 11

Active

The first administration will be Spring 2010. The Math and Science tests are in development. The first operational Math test will be 2011; Science will be 2012.

North Carolina

End-of-Course (EOC) Tests

End of Course

Algebra I, Algebra II, Biology, Chemistry, Civics and Economics, English I, Geometry, Physical Science, Physics, U.S. History

After course is taken

Active

Students may take an EOC test during the final week or final two weeks of the instructional period.

Oklahoma

Oklahoma Core Curriculum Tests (OCCT)

Regular

Geography (Gr.7), Math (Gr. 8), Reading (Gr. 8).

See Content Area

Active

Students may receive paper/pencil accommodation if documented within their IEP.

Students may receive a read-aloud accommodation during online testing (Grade 7 Geography and Grade 8 Math). An audio accompaniment may be administered via headphones, or a test monitor may read to the student.

End-of-Instruction (EOI) Assessments

End of Course

Algebra I, Algebra II, Biology I, English II, English III, Geometry, U.S. History.

After course is taken

Active

Online format not available for Writing portions of English II and English III assessments. Paper/pencil accommodation available for all other EOI assessments (must be documented within student’s IEP).

Oregon

Oregon Assessment of Knowledge and Skills (OAKS) Computer Adaptive

Regular

Reading/

Literature, Math,

Science, Social Sciences

3-8, HS

3-8, HS

5,8,HS

5,8,HS

Active

Paper/pencil accommodation available for students if documented within their IEP.

Paper version available if a district notes it is inappropriate for student (decision made by subject/specific content).

"(10) School districts may only assess students in the content areas listed in Section 9 (a)-(c) of this rule using a paper-based administration of the OAKS assessment instead of OAKS Online if the following conditions are met: (a) For students with an IEP or 504 Plan, the student’s Plan indicates separately for each content area to be assessed that the student requires a paper-based administration; or (b) For students without either an IEP or 504 Plan, the school district determines separately for each content area to be assessed that the web-based testing application is not appropriate for the particular student to demonstrate his or her level of proficiency. The school district must base its determination on an individual evaluation of the student and on documentation of the student’s needs maintained by the school district. Such documentation is subject to audits by the ODE."

Writing Test

Regular

Writing

High School

Active

Part 1 is multiple choice and is online only. Part 2 of test is either online or paper.

Spanish Reading/

Literature

Regular

Grade 3

Active

South Carolina

End-of-Course Examination Program (EOCEP)

End of Course

Algebra I, Biology, Math for Technologies II, English I, Physical Science, U.S. History and the Constitution

After course is taken

Active

Optional paper versions.

South Dakota

End-of-Course (EOC) Exams

End of Course

Algebra I and II, Biology, Chemistry, Geography, Geometry, Government, Physical Science, Physics, U.S. History, World History

After course is taken

Active

It is recommended that students take the EOC examination online. If online administration is not possible, the district may print off a paper copy of the EOC test to administer to students. Thus, paper version is optional.

Dakota Assessment of Content Standards (DACS)

Formative

DACS assesses performance in each of the South Dakota standards.

2-12

Active

Standards Based Adaptive Measurement.

8th grade Technology Literacy Assessment

Regular

Technology Literacy

8 on 6-8 standards

Active

Achievement Series

Formative

Classroom Tests

Reading, Math, Science

1-12

Active

Texas

Texas English Language Proficiency Assessment System (TELPAS)

English Language Proficiency Test

Listening, Speaking, Reading, Writing

K-12

Active

All tests (listening, speaking, reading, writing) administered online. Paper/pencil test is available in rare circumstances. Texas Education Agency must grant approval for paper/pencil administration.

Started Spring 2009.

End-of-Course (EOC)

End of Course

Operational: Algebra I, Geometry, Biology, Chemistry, U.S. History

Field Testing: Physics, World Geography, Algebra II, World History, English I, II, III

After course is taken

Active/

Devel-oping

Currently, EOC assessments are administered in the spring semester only and are primarily administered through the online testing system. The 2009 World Geography EOC field test will be offered in both online and paper formats in 2009. No retests are offered for EOC assessments because they are not used for high-stakes decisions about individual students and are not used for state or federal accountability reporting.

For the 2008–2009 school year, the Algebra I, Geometry, Biology, Chemistry, and U.S. History EOC Assessments will be offered on a voluntary basis as operational online test administrations. The Physics and World Geography EOC field tests are mandatory for selected campuses. Students taking TAKS accommodated form must use paper version.

Utah

Criterion Referenced Test (CRT)

Regular

Math

Element-ary and Second-ary

Active

Utah is currently transitioning to 100% computer-based testing, except for grade 2 and students whose disability warrants paper-based testing. Students with disabilities should be encouraged to test via computer whenever possible.

Direct Writing Assessment (DWA)

Regular

Writing

5, 8

Active

Administered beginning in Spring 2010.

Utah Test Item Pool Service (UTIPS)

Formative

ALL

ALL

Active

Optional local use of test.

Virginia

Standards of Learning (SOL) Assessments

Regular

Reading, Mathematics, Science,

U.S. History to 1877, 1877 to the present, and Civ/Econ

6-8

Active

Optional paper versions noted for all subjects but unclear for whom. Plain English Math (gr.6) and writing were not online as of spring 2007. ELLs or students with IEP or 504 plans may qualify to take the plain English math test.

End of Course (EOC) Assessments

End of Course

English: Reading, Algebra I, Geometry, Algebra II, Earth Science, Biology, Chemistry, Virginia and U.S. History, World Geography, World History I, II

After course is taken

Active

Optional paper version for all subjects.

Washington

Measurements of Student Progress (MSP)

Regular

Reading, Math, Writing, Science

3-8

Active/

Devel-oping

Online testing begins in Spring 2010 for Grades 3-8 (Reading, Math), Grades 5 and 8 (Science). In 2010-2011, online practice writing tests will be available. Majority of online assessments will be operational by 2012.

West Virginia

West Virginia Education Standards (WESTEST) 2 Writing

Regular

Writing

3-11

Active

Paper must be used for students without an acceptable use of the Internet form (varies by county), then, composition will be entered by a scribe.

Online Technology Assessment (TechSteps)

Regular

Computer skills

K-8

Active

Online District Benchmark Interim Assessment (ODBIA)

Formative

Mathematics, Reading/

Language Arts, Science, Social Studies

3-11

Active

Administered quarterly via Acuity (web-based platform).

Unclear if optional or not.

Note state also has other supported assessment programs: Riverdeep, Writing Roadmap, Techsteps.

Wyoming

Proficiency Assessments for Wyoming Students (PAWS)

Regular

Reading, Writing, Math, Science

3-8 and 11 for Reading, Writing, and Math. Grades 4, 8, and 11 for Science

Active/

Devel-oping

Beginning in 2008, multiple choice items for Reading, Math, and Science were administered online. Constructed response items for Reading, Math, and Science were administered paper/pencil. For grade 11, both items types were online for Reading. For writing all were paper for grades 3-8, with grade 11 online.

 


Appendix C

Web Sites used in Analysis of States’ Computer-based Tests

State/Test

Source

Alaska

Alaska Computerized Formative Assessments (ACFA)

Alaska Department of Education. Statewide system of support. www.eed.state.ak.us/nclb/pdf
/Statewide_System_of_Support.pdf - 2009-03-26 (see page 26)

Alaska Department of Education. Formative assessment resources.
http://www.eed.state.ak.us/tls/assessment/FormativeAssessmentResources.html

CAL TestBuilder. http://alaska.caltesting.org/about.html; http://alaska.caltesting.org/aca_testbuilder.html

Connecticut

CAPT/CMT MAS

Assessment Guidelines

http://www.csde.state.ct.us/public/cedar/assessment/agl/resources/AssessmentGuideline2009-10.pdf

MAS memo http://www.csde.state.ct.us/public/cedar/assessment/common/MAS2010memo.pdf

Delaware

Delaware Comprehensive Assessment System (DCAS)

Delaware Department of Education. RFP for DCAS. http://www.doe.k12.de.us/AAB/files
/DCAS%20RFP%20Review%20Process%20and%20Next%20Steps%20-%204.2.09.pdf

Online testing main page: http://de.portal.airast.org/

Users guide: http://de.portal.airast.org/resources/DE_TA_UserGuide.pdf

Frequently asked questions: http://de.portal.airast.org/FAQ.html

Administration manual: http://de.portal.airast.org/resources/DE_TAM_Final.pdf

Florida

Florida Comprehensive Assessment Test (FCAT)

Florida Department of Education. Frequently asked questions. h
ttp://www.fldoe.org/faq/default.asp?Dept=179&ID=972#Q972

Florida Department of Education. Next generation assessments and end of course exams.
http://www.fldoe.org/board/meetings/2009_03_17/exam.pdf

Memos on changes: http://www.fldoe.org/asp/k12memo/

Memo on FCAT Updates for 2009-2010 http://www.fldoe.org/asp/k12memo/pdf/FCATUpdatesFor2009-10.pdf

Georgia

End of Course Tests (EOCT)

Georgia Department of Education. End of course tests. http://www.doe.k12.ga.us/ci_testing.aspx?PageReq=CI_TESTING_EOCT

Hawaii

Hawaii State Assessment (HSA)

Hawaii Department of Education. The Hawaii States Assessment is moving online. http://sas.sao.k12.hi.us/STATE/SAO/SASWebsite.nsf/5c93c85c1627a0e78a256c2f007f47bc
/061958571641bc050a2576350005c53f/$FILE/tech_coordinator_brochure.pdf

About the online assessment: http://sas.sao.k12.hi.us/STATE/SAO/SASWebsite.nsf/By+Category
/6F54B0336C8F1DA50A257635000026EB?OpenDocument

Grades and subject field test dates: http://sas.sao.k12.hi.us/STATE/SAO/SASWebsite.nsf/10d1a575953d0e908a256c340001adab
/061958571641bc050a2576350005c53f/$FILE/Scheduling%20Options%20Brochure.pdf

Online assessment FAQs: http://www.alohahsa.org/Events/index.php/2009/10/01/online-testing-faqs/

Idaho

Idaho Standards Achievement Tests

Idaho Department of Education. ISAT online testing. http://isat.caltesting.org/about.html

ISAT winter TAMS from here: http://www.sde.idaho.gov/site/assessment/ISAT/testCoordinators.htm

Paper and pencil tam: http://www.sde.idaho.gov/site/assessment/ISAT/docs/testCoordinators/ISAT-Winter-2009-p-p_TAM.pdf

Federal and state assessments: http://www.sde.idaho.gov/site/assessment/docs/TC%20Guide%202009-10_02_09_10_version.pdf

ISAT spring 2009 Q and A: http://www.sde.idaho.gov/site/assessment/ISAT/docs/training/ISAT_Accommodated-Materials-Training-QA.pdf

Test dates pdf: http://www.sde.idaho.gov/site/assessment/docs/State%20Test%20Dates.pdf

"New" LEP policy from here: http://www.sde.idaho.gov/site/assessment/ISAT/testAdmin.htm

Indiana

ECA Kick-off letter for 09-10: http://www.doe.in.gov/eca/pdf/2009-10_ECA%20_Kick-off_Memo.pdf

Assessment window: http://www.doe.in.gov/istep/

Testing Schedule for different formats: http://www.doe.in.gov/eca/pdf/Opening_of_Year_Newsletter.pdf

October 14 memo about online testing: http://www.doe.in.gov/super/2009/10-October/101609/documents/memo_istep_reg.pdf

Indiana Department of Education. ISTEP. http://www.doe.in.gov/edmatters/ed_matters_winter08/pdf/ISTEP+%20results%20with%20parent%20network.pdf

Indiana Department of Education. Core 40 End of Course Assessments. http://www.doe.in.gov/eca/ and https://ineca.questarai.com/Admin/

Indiana Department of Education. Computer-based assessment tools/Grades 3-8 Diagnostic Tools. http://www.doe.in.gov/news/2009/09-September/istep+_2009.html and http://www.doe.in.gov/istep/pdf/INStatewideAssessmentSystQA-021808.pdfISTEP+ Program Manual 2009-2010http://www.doe.in.gov/istep/ProgramManual.html

Kansas

Kansas Computerized Assessment (KCA)

Kansas Department of Education. 2009-10 Assessment Examiner’s Manual http://www.cete.us/docs/2010/KS_Examiners_Manual_2010.pdf

And revised manual http://www.ksde.org/LinkClick.aspx?fileticket=W0ahzUs6CUA%3d&tabid=2374

Kansas Assessment of Modified Measures (KCA)

Kansas Department of Education. 2009-10 Assessment Examiner’s Manual http://www.cete.us/docs/2010/KS_Examiners_Manual_2010.pdf

And revised manual http://www.ksde.org/LinkClick.aspx?fileticket=W0ahzUs6CUA%3d&tabid=2374

Kentucky

T-PRO

Kentucky Department of Education. T-PRO Kentucky http://www.cia.indiana.edu/TPRO/kentucky.shtml

TPro FAQ page; http://www.education.ky.gov/KDE/Instructional+Resources/Read+To+Achieve/FAQ/default.htm

Testing system information: http://www.education.ky.gov/KDE/Administrative+Resources/Testing+and+Reporting+/Kentucky+School+Testing+System/

Louisiana

End of Course

Louisiana Department of Education. Algebra I End of Course (EOC) Test Assessment Guide http://www.doe.state.la.us/LDE/uploads/10252.pdf

Louisiana Department of Education. Geometry End of Course (EOC) Test Assessment Guide http://www.doe.state.la.us/LDE/uploads/13211.pdf

Louisiana Department of Education. English II End of Course (EOC) Assessment Guide http://www.doe.state.la.us/LDE/uploads/11619.pdf

EOC coordinator manual: http://www.louisianaeoc.org/Documents/EOC_Test_Coordinators_Manual.pdf

Test schedules: http://www.doe.state.la.us/lde/saa/781.html

EAGLE

Assessment news for Eagle link: http://www.doe.state.la.us/lde/saa/2610.html

Maine

Maine Educational Assessments

Notice of discontinuation of online test:

Maine Department of Education. Info Letter 31— Changes to the Maine Educational Assessment at Grades 3 – 8. http://mailman.informe.org/pipermail/doe_letters/2008-October/000290.html

Maryland

Maryland School Assessment – Science (MSA – Science)

Maryland Department of Education. Testing Overview http://www.marylandpublicschools.org/MSDE/testing/msa/?WBCMODE=pr

Maryland’s Accountability Assessment Program, 2008: http://www.marylandpublicschools.org/nr/rdonlyres/9659f357-134c-4040-aeb9-5972246e764d/19193/accountabilityassessmentprogramdec08.pdf

Accommodation Manual: http://www.marylandpublicschools.org/nr/rdonlyres/840efbb6-cd7d-404e-8a77-e978f6d508aa/11347/mdaccommodationsmanual.pdf and

http://www.marylandpublicschools.org/NR/rdonlyres/840EFBB6-CD7D-404E-8A77-E978F6D508AA/16337/MDAccommodationsManual_21108.pdf

Maryland State Department of Education Student Testing Calendar School Years 2009-2010 through 2013-2014

http://www.marylandpublicschools.org/nr/rdonlyres/840efbb6-cd7d-404e-8a77-e978f6d508aa/24340/msde_testing_calendar_2009_2010_201314_rev_03_17.pdf

Maryland Modified High School Assessments (Mod-HSA)

Maryland Department of Education. HSA: High School Assessment Program. http://mdk12.org/assessments/high_school/index_d2.html

Maryland’s Accountability Assessment Program, 2008: http://www.marylandpublicschools.org/nr/rdonlyres/9659f357-134c-4040-aeb9-5972246e764d/19193/accountabilityassessmentprogramdec08.pdf

Accommodation Manual: http://www.marylandpublicschools.org/nr/rdonlyres/840efbb6-cd7d-404e-8a77-e978f6d508aa/11347/mdaccommodationsmanual.pdf and

http://www.marylandpublicschools.org/NR/rdonlyres/840EFBB6-CD7D-404E-8A77-E978F6D508AA/16337/MDAccommodationsManual_21108.pdf

Anne Arundal County Public Schools. Testing Calendar for Federal and State Mandated Assessments and College Board Exams 2009 – 2010. http://www.aacps.org/testing/testing.pdf

Maryland Modified Middle School Assessments (Mod-MSA)

Maryland Department of Education. Parent Guide to the Modified School Assessment – Mod-MSA http://marylandpublicschools.org/NR/rdonlyres/5F4F5041-02EE-4F3A-B495-5E4B3C850D3E/22818/ModMSA_WhatItMeans_010709.pdf

Maryland’s Accountability Assessment Program, 2008:

http://www.marylandpublicschools.org/nr/rdonlyres/9659f357-134c-4040-aeb9-5972246e764d/19193/accountabilityassessmentprogramdec08.pdf

Accommodation Manual: http://www.marylandpublicschools.org/nr/rdonlyres/840efbb6-cd7d-404e-8a77-e978f6d508aa/11347/mdaccommodationsmanual.pdf and

http://www.marylandpublicschools.org/NR/rdonlyres/840EFBB6-CD7D-404E-8A77-E978F6D508AA/16337/MDAccommodationsManual_21108.pdf

Anne Arundal County Public Schools. Testing Calendar for Federal and State Mandated Assessments and College Board Exams 2009 – 2010. http://www.aacps.org/testing/testing.pdf

Massachusetts

MEPA

New features for MEPA (ELP test): http://www.doe.mass.edu/news/news.aspx?id=4542

Minnesota

Science Minnesota Comprehensive Assessment – II (Science MCA-II)

Minnesota Department of Education. Online Testing http://education.state.mn.us/MDE/Accountability_Programs/Assessment_and_Testing/DAC_Corner/Online_Testing/index.html

Minnesota Department of Education. Procedures Manual for Minnesota Assessments: 2009-2010 http://education.state.mn.us/mdeprod/groups/Assessment/documents/Manual/035664.pdf

Minnesota Department of Education. Minnesota Comprehensive Assessments – Science http://education.state.mn.us/mdeprod/groups/Assessment/documents/FAQ/007863.pdf

Graduation Assessment Required for Diploma (GRAD)

Minnesota Department of Education. Online Testing http://education.state.mn.us/MDE/Accountability_Programs/Assessment_and_Testing/DAC_Corner/Online_Testing/index.html

Minnesota Department of Education. Procedures Manual for Minnesota Assessments: 2009-2010 http://education.state.mn.us/mdeprod/groups/Assessment/documents/Manual/035664.pdf

Minnesota Department of Education. Graduation Assessment Required for Diploma (GRAD) http://education.state.mn.us/MDE/Accountability_Programs/Assessment_and_Testing/Assessments/GRAD/index.html

Mathematics Test for English Language Learners (MTELL)

Minnesota Department of Education. http://education.state.mn.us/MDE/Accountability_Programs/Assessment_and_Testing/Assessments/ELL_Tests/index.html

Minnesota Department of Education. Procedures Manual for Minnesota Assessments: 2009-2010 http://education.state.mn.us/mdeprod/groups/Assessment/documents/Manual/035664.pdf

Minnesota Department of Education. Frequently Asked Questions: Mathematics Test for English Language Learners (MTELL) http://education.state.mn.us/mdeprod/groups/Assessment/documents/FAQ/030683.pdf

Minnesota Department of Education. Online Testing http://education.state.mn.us/MDE/Accountability_Programs/Assessment_and_Testing/DAC_Corner/Online_Testing/index.html

Mississippi

Subject Area Testing Program, Second Edition (SATP2)

Mississippi Department of Education. Subject Area Testing Program, Second Edition (SATP2) http://www.mde.k12.ms.us/acad/osa/2009/2SATP_DTC_FALL_09v8.pdf

SATP2 School Test Coordinator Manual: http://www.mde.k12.ms.us/acad/osa/2010_Guidelines/SATP2%20School%20Test%20Coordinator%20Manual.pdf

09-10 testing calendar: http://www.mde.k12.ms.us/acad/osa/cal.html

District Coordinator Training: Fall 2009: http://www.mde.k12.ms.us/ACAD/osa/dtc.html

Missouri

Missouri End of Course Assessments (EOC)

Missouri Department of Education. Missouri End of Course Assessments (EOC) Test Examiner’s Manual Phase I Assessments 2009-2010. http://dese.mo.gov/divimprove/assess/documents/2009-2010_EOC_Phase_I_TEM_000.pdf

Missouri Department of Education. Missouri End of Course Assessments (EOC) Test Examiner’s Manual Phase I Assessments 2009-2010. http://dese.mo.gov/divimprove/assess/documents/2009-2010_EOC_Phase_II_TEM_000.pdf

Discontinued Integrated Math on EOC site: http://dese.mo.gov/divimprove/assess/eoc.html

2009-10 Test Examiner Manual: http://dese.mo.gov/divimprove/assess/documents/2009-2010_EOC_Online_TEM_000.pdf

EoC TCM 09-10: http://dese.mo.gov/divimprove/assess/documents/2009-2010_EOC_TCM_000.pdf

EoC assessments 2009-10: www.dese.mo.gov/divimprove/assess/documents/EOC-Assessments2009-2010.ppt

Nebraska

Nebraska State Accountability (NeSA)

Nebraska Department of Education. About Nebraska State Accountability: NeSA http://nesa.caltesting.org/about.html

NESA online manual : http://www.nde.state.ne.us/Assessment/documents/Final-Spring2009NeSA-RTAM.pdf

NESA accommodations document: http://www.nde.state.ne.us/Assessment/documents/NeSA_Accommodations.pdf

North Carolina

Computer Skills Test

North Carolina Department of Public Instruction. Memorandum: Transition Plan for Competency and Computer Skills

http://www.ncpublicschools.org/docs/accountability/policyoperations/transition/transitionplanletter.pdf

Eliminated tests as of 09-10: http://www.dpi.state.nc.us/accountability/testing/suspendadmin

http://www.dpi.state.nc.us/accountability/testing/

End-of-Course (EOC) Tests

North Carolina Department of Public Instruction. North Carolina End-of-Course Tests

http://www.ncpublicschools.org/accountability/testing/eoc/

NCTest link: http://cuacs8.mck.ncsu.edu/nctest/

EoC tests: http://www.dpi.state.nc.us/accountability/testing/eoc/

Oklahoma

Oklahoma Core Curriculum Tests (OCCT)

Oklahoma State Department of Education. Oklahoma School Testing Program, http://www.sde.state.ok.us/AcctAssess/pdf/OSTPBrochure.pdf

Core test prep manual: http://www.sde.state.ok.us/AcctAssess/pdf/Core/TestPrepManual.pdf

2009 Test Interpretation Manual Grades 3–8 Oklahoma Core Curriculum Tests

http://www.sde.state.ok.us/AcctAssess/pdf/forms/TIMGr3_8.pdf

End-of-Instruction (EOI) Assessments

Oklahoma State Department of Education. Frequently Asked Questions Oklahoma School Testing Program (2009-2010), http://www.sde.state.ok.us/AcctAssess/pdf/FAQ_OSTP.pdf

EOI Online Guide: http://sde.state.ok.us/AcctAssess/pdf/forms/EOIOnlineGuide.pdf

EOI Test prep manual: http://www.sde.state.ok.us/AcctAssess/pdf/forms/EOITestPrep.pdf

Test prep manual modified: http://www.sde.state.ok.us/AcctAssess/pdf/OMAAP/TestPrepManual.pdf

User Guide: http://sde.state.ok.us/AcctAssess/pdf/forms/UserGuide.pdf

Oregon

Oregon Assessment of Knowledge and Skills (OAKS) Computer Adaptive

Test Administration manual: http://www.ode.state.or.us/search/page/?=486

Oregon Department of Education. Test Administration Manual, Appendix A: 2009-2010 Test Schedule and Required Ship Dates, http://www.ode.state.or.us/wma/teachlearn/testing/admin/appendix_a.pdf

Oregon Department of Education. Test Administration Manual, Appendix N: OAKS Paper and Pencil, http://www.ode.state.or.us/wma/teachlearn/testing/admin/appendix_n.pdf

Accommodations Manual http://www.ode.state.or.us/teachlearn/testing/admin/alt/ea/oregon-accommodations-manual-2009-2010.pdf

Writing Test

Test Administration manual: http://www.ode.state.or.us/search/page/?=486

Oregon Department of Education. Oaks Online User Guides. http://www.ode.state.or.us/search/page/?=391

Accommodations Manual http://www.ode.state.or.us/teachlearn/testing/admin/alt/ea/oregon-accommodations-manual-2009-2010.pdf

Spanish Reading/

Literature

Oregon Department of Education. 2009-2010 Test Administration Manual. http://www.ode.state.or.us/search/page/?=486

Accommodations Manual http://www.ode.state.or.us/teachlearn/testing/admin/alt/ea/oregon-accommodations-manual-2009-2010.pdf

South Carolina

End-of-Course Examination Program (EOCEP)

South Carolina Department of Education. End-of-Course Examination Program (EOCEP).

http://ed.sc.gov/agency/Accountability/Assessment/old/assessment/programs/endofcourse/End-of-CourseExaminationProgramEOCEP.html.

Updated 09-10 materials with DTC supplement: http://ed.sc.gov/agency/Accountability/Assessment/old/assessment/publications/manuals2.html

South Dakota

End-of-Course (EOC) Exams

South Dakota Department of Education. End of Course Exams: Procedure Manual

http://doe.sd.gov/octa/assessment/EOC/index.asp

EOC procedure manual http://doe.sd.gov/octa/assessment/EOC/docs/manual/1-6.pdf

Assessment overview: http://doe.sd.gov/octa/assessment/docs/SD_AssessmentSystemOverview_808.pdf

Dakota Assessment of Content Standards (DACS)

South Dakota Department of Education. DACS Frequently Asked Questions. http://doe.sd.gov/octa/assessment/dacs/faqs.asp

DACS administration link: http://doe.sd.gov/octa/assessment/dacs/manual.asp

8th Grade Technology Literacy Test

8th grade technology literacy test page: http://doe.sd.gov/contentstandards/NCLB/assessment.asp

Achievement Series

South Dakota Department of Education. Achievement Series. http://doe.sd.gov/octa/assessment/documents/AS%20Website%20info.pdf

Texas

Texas Assessment of Knowledge and Skills (TAKS)

Texas Education Agency. Chapter 2: Texas Assessment of Knowledge and Skills (TAKS) Overview. http://www.tea.state.tx.us/index3.aspx?id=948&menu_id=793

2010 DCCM overview: http://ritter.tea.state.tx.us/student.assessment/resources/guides/coormanual/dccm_overview.pdf

2010 supplement here from same main page: http://ritter.tea.state.tx.us/student.assessment/resources/guides/coormanual/dccm_supplement.pdf

Accommodations Manual http://ritter.tea.state.tx.us/student.assessment/resources/accommodations/AccommManual_2009_10.pdf

Texas English Language Proficiency Assessment System (TELPAS)

Texas Education Agency. TELPAS: Texas English Language Proficiency Assessment System: Manual for raters and test administrators http://ritter.tea.state.tx.us/student.assessment/ELL/TELPASManual.pdf

TELPAS 2010: http://ritter.tea.state.tx.us/student.assessment/resources/guides/coormanual/dccm_telpas.pdf

2010 TELPAS coordinators’ supplement (main page link): http://www.tea.state.tx.us/index3.aspx?id=3680&menu_id=793#telpas

Accommodations Manual http://ritter.tea.state.tx.us/student.assessment/resources/accommodations/AccommManual_2009_10.pdf

End-of-Course (EOC)

Texas Education Agency. End of Course (EOC) Assessments. http://www.tea.state.tx.us/index3.aspx?id=3302&menu_id=793

Accommodations Manual http://ritter.tea.state.tx.us/student.assessment/resources/accommodations/AccommManual_2009_10.pdf

Utah

Criterion Referenced Test (CRT)

Utah State Office of Education. 2009 Utah Elementary Mathematics CRT Online (CBT) Test Overview. http://www.schools.utah.gov/assessment/info_math.aspx

Special needs accommodations policy: http://www.schools.utah.gov/assessment/documents/Special_Needs_Accommodations_Policy.pdf

Test administration calendar 09-10: http://schools.utah.gov/assessment/documents/DA_Test_Administration_Calendar_09-10.pdf

Site Coordinator Manual: http://www.schools.utah.gov/assessment/documents/DA_Site_Coordinator_Manual_20090311.pdf

Direct Writing Assessment (DWA)

Utah State Office of Education. Direct Writing Assessment. http://www.schools.utah.gov/assessment/info_math.aspx

Special needs accommodations policy: http://www.schools.utah.gov/assessment/documents/Special_Needs_Accommodations_Policy.pdf

Utah Test Item Pool Service (UTIPS)

Utah State Office of Education. Utah Test Item Pool Service: UTIPS for CRT and All Subjects. http://www.schools.utah.gov/assessment/utips/default.aspx

Special needs accommodations policy: http://www.schools.utah.gov/assessment/documents/Special_Needs_Accommodations_Policy.pdf

Virginia

Standards of Learning (SOL) Assessments

Virginia Department of Education. 2009-2010 Non-Writing Examiners Manual (Grades 6,7,& 8). http://www.doe.virginia.gov/testing/test_administration/index.shtml

End of Course (EOC)

Virginia Department of Education. 2009-2010 Non-Writing Examiners Manual (End of Course: Reading).

http://www.doe.virginia.gov/testing/test_administration/index.shtml

Virginia Department of Education. 2009-2010 Non-Writing Examiners Manual (End of Course: History).

http://www.doe.virginia.gov/testing/test_administration/index.shtml

Content specific history manual: http://www.doe.virginia.gov/testing/test_administration/manuals/examiners/content_specific_history.pdf

Fall 2009 writing tests: http://www.doe.virginia.gov/testing/test_administration/manuals/test_implementation/fall_writing.pdf

Virginia Department of Education. 2009-2010 Non-Writing Examiners Manual (End of Course: Mathematics).

http://www.doe.virginia.gov/testing/test_administration/index.shtml

Virginia Department of Education. 2009-2010 Non-Writing Examiners Manual (End of Course: Science).

http://www.doe.virginia.gov/testing/test_administration/index.shtml

Washington

Measurement of Student Progress (MSP)

Washington Office of Superintendent of Public Instruction.

State Testing: Rollout Schedule for Online Testing.

http://www.k12.wa.us/assessment/StateTesting/OnlineTestingRolloutSchedule.aspx

Online MSP testing page: http://wa-online.caltesting.org/about.html

High School Proficiency Exam (HSPE)

Washington Office of Superintendent of Public Instruction.

State Testing: Rollout Schedule for Online Testing.

http://www.k12.wa.us/assessment/StateTesting/OnlineTestingRolloutSchedule.aspx

West Virginia

West Virginia Education Standards (WESTEST) 2 Writing

West Virginia Department of Education. WESTEST 2 Online Writing Overview. http://wvde.state.wv.us/oaa/writing/wa_index.html

Westtest 2 Online Writing FAQs: http://wvde.state.wv.us/oaa/writing/wa_w2faq.html

Online Technology Assessment

West Virginia Department of Education. Assessments and Accountability. http://wvde.state.wv.us/oaa/

Online District Benchmark Interim Assessment (ODBIA)

West Virginia Department of Education. Assessments and Accountability. http://wvde.state.wv.us/oaa/

ODBIA information: http://wvde.state.wv.us/oaa/acuity.php

Techsteps

Information about Techsteps:

http://wvde.state.wv.us/teach21/techsteps.html

http://www.techsteps.com/WV/techStepsK-8.htm

Wyoming

Proficiency Assessments for Wyoming Students (PAWS)

Wyoming Department of Education. Proficiency Assessments for Wyoming Students (PAWS) Pre-Administration Training December 1-7, 2009. http://www.k12.wy.us/SA/Paws/Paws_Info/2009_10/PAWS%202010%20Pre-Administration%20Training%20Powerpoint.pdf

Technical Manual, Spring 2009: here:http://www.k12.wy.us/SA/Paws/Resources/Docs/2009%20Technical%20Manual.PDF

Accommodations Manual

http://www.k12.wy.us/SA/Paws/Paws_Info/2009_10/AccommodationsManual.pdf

Accommodations Information:

http://www.k12.wy.us/SA/Paws/Paws_Info/2009_10/PAWS_accomodations.PDF

2014 by the Regents of the University of Minnesota.
The University of Minnesota is an equal opportunity educator and employer.

Online Privacy Statement
This page was last updated on January 03, 2013

NCEO is supported primarily through a Cooperative Agreement (#H326G050007, #H326G110002) with the Research to Practice Division, Office of Special Education Programs, U.S. Department of Education. Additional support for targeted projects, including those on LEP students, is provided by other federal and state agencies. Opinions expressed in this Web site do not necessarily reflect those of the U.S. Department of Education or Offices within it.