A New Series of Briefs for the Race to the Top (RTTT) Assessment Consortia Don’t Forget Accommodations!Five Questions To Ask When Moving to Technology-based AssessmentsAccommodations are an important part of paper and pencil testing. They give students with disabilities and English language learners (ELLs) access to the assessment and provide results that have greater validity for those students. (The term “students with disabilities” in this document includes both students with Individualized Education Programs (IEPs) and students with 504 accommodation plans.) As the Race to the Top (RTTT) Consortia and states move to technology-based assessments, some states might have hoped that accommodations would go away. They might have thought that all needed changes in materials or procedures could now be incorporated into the design of the technology-based test itself, and the need for “external” accommodations could be eliminated. Although this sounds ideal, there are several questions that the Consortia and states should be asking themselves. Five questions are addressed here to help Consortia and states think through accommodations issues as they move into technology-based assessments:
This Brief focuses on general technology-based assessments, but the information also is applicable to technology-based alternate assessments. 1. Do technology-based assessments mean that students will no longer need accommodations?Technology-based assessment platforms offer new opportunities and ways for accommodations to be provided to students who need them, but they will not eliminate the need for accommodations. Technology-based assessments can be developed with all students in mind from the beginning (universal design) so that the assessments are accessible to the greatest number of students right from the start. Yet even with the best designed test some students still will require accommodations. The ways in which accommodations are provided will likely change via technology so that some accommodations (for example, a pop-up glossary) are built into the testing platform itself, while other accommodations (for example, providing frequent breaks or a scribe) may require additional planning and tracking. Technology-based assessments present an opportunity to improve the processes and procedures for providing accommodations to students who require them. Consortia and states should address both policy and implementation issues surrounding accommodations as they move to technology-based assessments. For example, the Consortia and states should develop procedures that ensure that decision makers are aware of all available accommodations provided as part of the testing platform. Decisions makers also should be made aware of which accommodations may need to be provided in addition to those embedded in the assessment. Consortia and states also should have policies in place to address situations in which the technology may not meet all students’ needs. For example, the testing platform may have embedded video files in American Sign Language (ASL), but a student who recently arrived in this country may use French Sign Language (LSF) instead. In such a situation, it may be necessary to maintain the option of having a human provide sign interpretation on the spot. In addition, Consortia and states also should have policies for what to do when technology-based accommodations break down (for example, when the built-in microphone stops working for a student who needs a speech-to-text accommodation), just as they have policies for other technology-related breakdowns. Technology-based tests may create a need for new accommodations. For example, students with some physical disabilities that affect coordination may be able to take a paper and pencil test without accommodations, but may need accommodations to navigate a technology-based assessment. Also, some technology-based assessments may require the use of more working memory than paper-based tests. For example, less information may be visible on a screen than on a page in a test booklet. Consortia and states may need to make data-based decisions about possible new accommodations in such situations.
2. What is the difference between accommodations, embedded features, and good testing practices?Consortia and states should develop mechanisms to ensure that all embedded features within technology-based assessments are developed with universal design principles. The primary goal of applying universal design in the development process is to ensure that embedded features do not hinder students from accurately showing what they know during the testing experience. Still, the use of universal design will not eliminate the need for specific accommodations that also can be embedded within a technology-based assessment platform. Consortia and states should determine which embedded features will be made available to all students and which are accommodations that are used to meet specific documented student access needs above and beyond the embedded features available to all. (See Figure 1 for a display of the decisions that Consortia and states need to make as they develop technology-based assessments.) Some people have the misconception that accommodations will be fluid when technology-based assessments are used. Depending on the test platform there may be situations in which accommodations are needed for a student to meaningfully access the test, but the decision about whether to provide the accommodation is based on the student’s characteristics and needs. For example, while a pop-up glossary could be made available to all students, a visual/pictorial dictionary, which is otherwise unnecessary for all students, could be embedded and selected for students with an identified need for this specific accommodation. Consortia and states should continue to make explicit the distinction between embedded features available to all and embedded features that are accommodations. If students need to use specific embedded features to access a test—as when they are on the student’s IEP, 504 plan, or ELL plan—the features would be considered accommodations for those students. For example, the Consortia may decide that the ability to enlarge font size is an embedded feature available to all students and provide one setting for larger font that all students can turn on during the test; yet, some students with visual impairments will still require a specific font size (such as 18 point or 24 point). Although these font sizes could also be embedded, Consortia and states may decide to make them available only to those students who need them as an accommodation. Other practices that might have been considered accommodations in the past (for example, minimize distractions) now might be viewed simply as good testing practices. When determining their accommodations policies, Consortia members also may decide that certain features such as item masking and color overlays are embedded features that could be made available to every student. The process of identifying which embedded features will be available to all and which will be available only to those students needing accommodations will be essential to the development of accessible technology-based assessments. Figure 1. Decision Tree for Use of Embedded Features, Accommodations, and Good Testing Practices for Technology-based Assessments* 
* Two points should be considered within the decision tree: (1) A “turned on” feature is visible and accessible to the student during the test, but the student may or may not choose to interact with that feature; (2) While an embedded feature may fall into the “before test day” category, it will be important to clarify the time frame needed to ensure the decision is appropriately made and communicated to the responsible person prior to test day.
3. Who makes decisions about what is used during technology-based assessments, and when are these decisions made?Technology-based assessments have embedded features that can be used to customize each student’s testing experience. Consortia should consider which changes students are allowed to select themselves and which the teacher selects for them. Care should be taken to minimize the likelihood of students (or teachers) making poor decisions. For example, if a student has not previously used and demonstrated success with certain embedded features, he or she may just end up experimenting with them. The degree of distraction that embedded features create, and thus the degree of teacher involvement required, may vary with student grade level or other factors. Consortia and states may want to consider whether students at higher grade levels should be allowed to make more selection decisions than those in lower grades. It is important to consider how and when the selection is made. A more rigorous approach may be needed for students with disabilities and students who are ELLs. Although a wide range of embedded features may be available to students on test day, careful consideration must be given to ensure that students know and understand the accessibility features, and are provided further accommodations as needed (such as testing in a separate room, timing accommodations, use of a scribe or other response assistance). Further, Consortia and states may want to consider requiring that accommodations be pre-programmed into the technology-based assessment so that only those students who need them are allowed to select needed personal preference features. It is likely that as Consortia and states move to technology-based assessments, unanticipated needs for new types of accommodations will emerge. Many states already have procedures for teachers or teams to request accommodations that are not on the approved list. States that do not currently have sound procedures in place for requesting new accommodations will need to develop them.
4. What training is needed for teachers and for students?Educators have always needed training to support good decision making about accommodations. Training needs will be even greater with technology-based assessments. In addition to accommodation decisions, educators will need to know how to use technology during instruction and how to assist students in making good decisions about student-selected embedded features. All students should have interactive learning opportunities that are technology-based if the assessments that measure and evaluate that learning are technology based. This is especially important for some students with disabilities and students who are ELLs. Educators should know how to use technology-based platforms during instruction to build skills, comfort, and familiarity with tools (for example, navigation tools, electronically represented measurement tools such as rulers) that students will use when taking a technology-based assessment. For example, it can be problematic if a student learns how to use one screen reader during instruction and a different screen reader is embedded in the assessment. Educators might also be encouraged to use technology-based classroom tests so that students can practice using embedded features and accommodations. Training can build teacher capacity to help students learn how to use accommodations on technology-based platforms during state tests. Students should be provided opportunities to communicate feedback about how well the embedded features and accommodations worked for them. For example, some Spanish-speaking ELLs who have been instructed in their native language may find a written Spanish translation of a test helpful, while other Spanish-speaking ELLs who lack literacy in their native language may not find a translation beneficial. Some students may find calming music played during the test helpful, and others may find it distracting.
5. What should be tracked and documented within the technology-based platform?It is important for Consortia and states to consider how the powerful possibilities of collecting data associated with technology-based assessments can be used to support continuous improvement for both students and the assessment system. Data should be collected to evaluate how well the system is working so that there can be iterative improvement. A technology-based assessment platform has the potential to create many data points about the student’s interaction with the test delivery system. With the ability to track and document, there are a host of new issues. Consortia and states should consider the value of recording, compiling, and analyzing data. Systems can be designed to capture everything from whether embedded features are turned on, to keystrokes and moves of a mouse, to student or teacher selection and de-selection of tools to improve test access. An important question to ask is which data are of value. Also to be considered is whether there are personnel available to view and make sense of some of the detailed data that could potentially be collected. Analyzing data can be costly, and its value may have diminishing returns as it becomes more detailed and specific. There may be new roles for those who monitor tests during their administration to ensure that students have appropriate access to needed technology-based accommodations. It may be possible to gather some of the data used for monitoring in new ways with technology-based assessments. Finally, a person should be available to address needed accommodations outside those features embedded within the technology-based assessment and to ensure that they actually are available to those students who have been designated as needing them. This person also should document whether the student used non-embedded accommodations. If only one person is to be present during the technology-based assessment, that person should be familiar with accommodations and the need to document them.
ConclusionTechnology-based assessments will not eliminate the need for accommodations. They will create a need to think more carefully about what features can be embedded within assessments, which of these are considered accommodations, and which are simply good testing practices. Technology-based assessments will require states to think again about when decisions are made about accommodations and who makes them. They will need to carefully consider training needs, and they should consider what to track and document as part of the technology platform. Thoughtfully addressing the questions raised here will help to ensure that the Consortia and states provide meaningful access to students with disabilities and students who are ELLs.
ResourcesThe following resources can be found on the NCEO Web site (nceo.info) under the Reports section: A Principled Approach to Accountability Assessments for Students with Disabilities (Synthesis Report 70) (2008). Thurlow, M. L., Quenemoen, R. F., Lazarus, S. S., Moen, R. E., Johnstone, C. J., Liu, K. K., Christensen, L. L., Albus, D. A., & Altman, J. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Computer-based Testing: Practices and Considerations (Synthesis Report 78) (2010). Thurlow, M., Lazarus, S. S., Albus, D., & Hodgson, J. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Considerations for the Development and Review of Universally Designed Assessments (Technical Report 42) (2005). Thompson, S. J., Johnstone, C. J., Anderson, M. E., & Miller, N. A. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Meeting the Needs of Special Education Students: Recommendations for the Race-to-the-Top Consortia and States (2011). Thurlow, M. L., Quenemoen, R. F., & Lazarus, S. S. Washington DC: Arabella Advisors. Also visit the NCEO Data Viewer to create an accommodations map that details which states have policies that allow for the use of specific accommodations.
|
NCEO
Brief #1
National
Center
on
Educational
Outcomes