Universally Designed Assessments

Frequently Asked Questions

1. What are the elements of "universally designed assessments"?
2. What specific steps are involved in the universal design of assessments?
3. Are universally designed assessments specifically meant to benefit students with disabilities?
4. Will universal design result in reducing or eliminating the use of accommodations during assessments?
5. Are universally designed assessments easier, resulting in artificially higher scores?

 

1. What are the elements of "universally designed assessments"?
NCEO developed seven elements of universally designed assessments based on a review of the literature on universal design, assessment, and instructional design. Test developers have used many of these elements to increase assessment validity and accessibility. The seven elements are:

• Inclusive assessment population
• Precisely defined constructs
• Accessible, non-biased items
• Amenable to accommodations
• Simple, clear, and intuitive instructions and procedures
• Maximum readability and comprehensibility
• Maximum legibility

 

2. What specific steps are involved in the universal design of assessments?
To develop assessments that conform to the elements of universal design, test developers can integrate practices such as the following at each stage of test development: 

A. Test conceptualization:

• Define the constructs to be measured precisely and explicitly so the test can be designed to measure the construct while minimizing the effects of irrelevant factors.
• Include the full range of students in the definition of the target population.

B. Test construction:

• Develop items that minimize the effects of extraneous factors and that can be used with accommodations as appropriate (e.g., avoid unnecessary use of graphics that cannot be presented in braille, use font size and white space appropriate for clarity and focus, avoid unnecessary linguistic complexity when it is not being assessed). It is the construct that must be held constant, not the design features; there are times, for example, when linguistic complexity is appropriate and necessary.
• Provide for a full range of test performance to avoid ceiling or floor effects.
• Undergo a review of items using tools such as NCEO's Considerations for Universally Designed Assessments. By promoting a structured review of items, test designers can determine the design strengths and weaknesses of items before field testing. Determining well-designed items−and items that need minor adjustments−may save time and money over unstructured item reviews that simply eliminate potentially problematic items.

C. Test tryout (field testing):

• Include a full range of students in the tryout sample (e.g., students with disabilities, other students with special needs). Because there may be constraints in sampling due to the low numbers of students with specific characteristics, there may be a need to use over-sampling strategies (e.g., select groups of items for which additional sampling will occur).
• Include the use of accommodations during the test tryout.

D. Item analysis:

• Analyze item characteristics to determine which items can be used with the full range of students and with accommodations.
• Use a wide range of statistical tests to determine whether items are functioning differently for particular populations. Populations of students with particular disabilities are often small in number, so using multiple analysis techniques will help test designers to see patterns of items to "flag" for further investigation. Examples of statistical techniques can be found in Analyzing Results of Large-scale Assessments to Ensure Universal Design.

E. Analysis of "flagged" items:

• Conduct cognitive labs, also called think aloud studies, with a small number of students who take the test. Student data can be used to determine whether the design of items set forth by test designers is comprehensible to students. Involving a wide range of students is helpful in gathering particular perspectives. See Using the Think Aloud Method (Cognitive Labs) to Evaluate Test Design for Students with Disabilities and English Language Learners.
• Conduct an expert review. Another method of analyzing flagged items is to enlist a group of expert reviewers. Use Considerations for the Development and Review of Universally Designed Assessments to examine possible design problems of flagged items.

F. Flexibility in assessment practices:

• Provide opportunities for students to understand and respond to assessments in varied ways.

G. Test revision:

• Eliminate items with evidence of disability bias.
• Include the full range of students and the use of accommodations in the test administration.

For more, see A State Guide to the Development of Universally Designed Assessments.

 

3. Are universally designed assessments specifically meant to benefit students with disabilities?
Universal design does not apply only to people with disabilities. It applies to all individuals, with wide ranging characteristics. Many of the principles of universal design are similar to general principles for increasing validity in assessments for all students. For example, by designing assessments to provide for a full range of test performance, and thus avoiding ceiling or floor effects, the assessments provide better information on the performance of all students. Similarly, reducing extraneous features, such as unnecessary linguistic complexity and confusing or low contrast graphics, allows all students to better show their skills on the constructs being tested. By thinking about all students during test conceptualization, construction, field testing, item analysis, and test revision, universal design results in more usable and valid assessments for all students, reducing the need for different forms, booklets, or assessments. It is important to check assessment changes to make sure that they do not add new issues in testing by changing constructs or introducing new bias against particular populations.

 

4. Will universal design result in reducing or eliminating the use of accommodations during assessments?
Universally designed assessments will not eliminate the need for all accommodations. However, they may reduce the need for them. They can also increase the utility of accommodations that can be used without threat to the validity and comparability of the scores. This will result in inclusive accountability measurement, and provide instructionally supportive information across the full range of students. Some students will still need accommodations, however. For example, students who are easily distracted by the presence of other students may still need to be tested individually. Also, some students may need assistive technology to access and respond to the test questions. Finally, students who cannot read print in a size less than 18 point, or who must use braille, will still need a large print or braille test.

 

5. Are universally designed assessments easier, resulting in artificially higher scores?
Developing assessments using universal design principles may result in more valid scores. Valid test results reflect actual student knowledge and skills, and not extraneous factors. Universal design principles include careful thought about the construct, level of difficulty, and nature of the measurement problem. Design decisions do not change features necessary to the intended measurement problem nor the range of content tested. 

Most standards-based assessments used for school accountability are NOT intended to measure student characteristics and skills such as visual acuity, hand/eye coordination, the ability to find isolated facts within a diagram with distracting information, and speededness. A more accurate assessment of the intended construct can be developed when these kinds of extraneous and confounding factors are removed. 

Top of page