NCEO Policy Directions

Published by the National Center on Educational Outcomes
Number 14 / June 2002


Universally Designed Assessments: Better Tests for Everyone!

Prepared by Sandra Thompson and Martha Thurlow


Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:

Thompson, S., & Thurlow, M. (2002). Universally designed assessments: Better tests for everyone! (Policy Directions No. 14). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/Policy14.htm


Background

The standard administration of assessments is not appropriate for all students who must participate in state and district assessments today. The use of accommodations – changes in administration procedures or materials – is evidence that there are students who cannot participate in assessments or receive valid scores unless something is changed. Only a very small percentage of students need a completely different assessment, identified in federal special education law as an alternate assessment. A much larger group of students need changes in the regular assessment.

Because of the emphasis on testing and including all students, the provision of accommodations and decisions about who should participate in alternate assessments has become very complex. There is a great deal of controversy about the “fairness” of many test accommodations and about which students should have access to accommodations and how decisions are made. Research to validate accommodation use is growing, but the research is difficult to conduct and rarely provides conclusive evidence about the effects of accommodations on validity. States grapple with decisions about which accommodations should be included in school accountability and which invalidate assessment scores. Repeated revisions in state accommodation policies is just one indicator of the controversy surrounding the need to provide accommodations for students to be able to participate and show their knowledge and skills in assessments. It is time to take a more global approach to addressing these testing issues, an approach in which increased access for all students is considered from the beginning.


Applying Universal Design to Assessments

The concept of universal design is not new. Its use began in the field of architecture, but its application has spread rapidly into environmental initiatives, recreation, the arts, health care, and education. Principles of universal design that traverse all of these areas have been developed (see Table 1). It is reasonable to expect that they can apply equally as well to large-scale assessments.

 

Table 1. Principles of Universal Design in Architecture and Other Areas

Principle

Explanation

Equitable Use

The design is useful and marketable to people with diverse abilities.

Flexibility in Use

The design accommodates a wide range of individual preferences and abilities.

Simple and Intuitive Use

Use of the design is easy to understand, regardless of the user's experience, knowledge, language skills, or current concentration level.

Perceptible Information

The design communicates necessary information effectively to the user, regardless of ambient conditions or the user’s sensory abilities.

Tolerance for Error

The design minimizes hazards and the adverse consequences of accidental or unintended actions.

Low Physical Effort

The design can be used efficiently and comfortably and with a minimum of fatigue.

Size and Space for Approach and Use

Appropriate size and space is provided for approach, reach, manipulation, and use regardless of user’s body size, posture, or mobility.

Source: The Center for Universal Design, North Carolina State University (1997).

 

The goal of applying universal design principles to assessments is to be able to design and develop assessments that allow participation of the widest range of students, and result in valid inferences about their performance. The need that many students have for accommodations could be reduced if assessments could be universally designed. Universally designed assessments are not intended to eliminate individualization, or to take away from the IEP process. Instead, they could make the IEP process richer by focusing on instructional needs rather than on all the changes that will have to be made for the student to participate in the assessment. Universal design is the best way to increase participation in general state and district assessments.

Universal design is based on the same ethics of equity and inclusiveness that are expected for people with disabilities and others in schools, communities, and on the job – an ethic that values differences in age, ability, culture, and lifestyle. Testing conditions should not be affected by disability, gender, race, English language ability, or levels of anxiety about tests. On the other hand, it is important to remember that universal design does not address deficiencies in instruction. Students who have not had an opportunity to learn the material tested will be disadvantaged during testing no matter how universal the design of the assessment.


Elements of Universally Designed Assessments

NCEO has conducted an extensive review of all research relevant to the assessment development process and the principles of universal design (see Synthesis Report 44 in Resources). This review produced a set of seven elements of universal design that apply to assessments (see Table 2).

 

Table 2. Elements of Universally Designed Assessments

Element

Explanation

Inclusive Assessment Population

Tests designed for state, district, or school accountability must include every student except those in the alternate assessment, and this is reflected in assessment design and field testing procedures.

Precisely Defined Constructs

The specific constructs tested must be clearly defined so that all construct irrelevant cognitive, sensory, emotional, and physical barriers can be removed.

Accessible, Non-Biased Items

Accessibility is built into items from the beginning, and bias review procedures ensure that quality is retained in all items.

Amenable to Accommodations

The test design facilitates the use of needed accommodations (e.g., all items can be Brailled).

Simple, Clear, and Intuitive Instructions and Procedures

All instructions and procedures are simple, clear, and presented in understandable language.

Maximum Readability and Comprehensibility

A variety of readability and plain language guidelines are followed (e.g., sentence length and number of difficult words are kept to a minimum) to produce readable and comprehensible text.

Maximum Legibility

Characteristics that ensure easy decipherability are applied to text, to tables, figures, and illustrations, and to response formats.

Based on Thompson, Johnstone, and Thurlow (2002).

 

Inclusive Assessment Population
When tests are first conceptualized, they need to be thought of in the context of who will be tested. If the test is designed for state, district, or school accountability purposes, the target population must include every student except those who will participate in accountability through the alternate assessment. Assessments need to be responsive to growing demands – increased diversity, increased inclusion of all types of students in the general curriculum, and increased emphasis and commitment to accountability for all students.

Precisely Defined Constructs
An important function of well-designed assessments is that they actually measure what they are intended to measure. Test developers need to carefully examine what is to be tested and design items that offer the greatest opportunity for success within those constructs. Just as universally designed architecture removes physical, sensory, and cognitive barriers to all types of people in public and private structures, universally designed assessments must remove all non-construct-oriented cognitive, sensory, emotional, and physical barriers.

Accessible, Non-Biased Items
Items are reviewed through bias review or sensitivity review procedures to ensure that they do not create barriers because of lack of sensitivity to disability, cultural, or other subgroups. But, perhaps more important, items are developed by individuals who understand the varied characteristics of students, and the characteristics of items that might create difficulties for any group of students. Accessibility is incorporated as a primary dimension of test specifications, so that accessibility is woven into the fabric of the test rather than being added after the fact.

Amenable to Accommodations
Even though items on universally designed assessments will be accessible for most students, there will still be some students who continue to need accommodations. Thus, another essential element of any universally designed assessment is that it is compatible with accommodations and a variety of widely-used adaptive equipment and assistive technology.

Simple, Clear, and Intuitive Instructions and Procedures
Assessment instructions should be easy to understand, regardless of a student’s experience, knowledge, language skills, or current concentration level. Directions and questions need to be in simple, clear, and understandable language. Knowledge questions that are posed within complex language certainly invalidate the test if students cannot understand how they are expected to respond to a question.

Maximum Readability and Comprehensibility
A variety of guidelines exist to ensure that text is maximally readable and comprehensible. These features go beyond what is measured by readability formulas. Readability and comprehensibility are affected by many characteristics, including student background, sentence difficulty, organization of text, and others. All of these features need to be considered in developing the text of assessments.

Plain language is a concept now being highlighted in research on assessments. Plain language has been defined as language that is straightforward and concise. Strategies for editing text to produce plain language have been identified (see Table 3).

 

Table 3.  Plain Language Editing Strategies

Strategy

Description

Reduce excessive length

Reduce wordiness and remove irrelevant material.

Use common words

Eliminate unusual or low frequency words and replace with common words (e.g., replace “utilize” with “use”).

Avoid ambiguous words

For example, “crane” should be avoided because it could be a bird or a piece of heavy machinery.

Avoid irregularly spelled words

Examples of irregularly spelled words are “trough” and “feign.”

Avoid proper names

Replace proper names with simple common names such as first names.

Avoid inconsistent naming and graphic conventions

Avoid multiple names for the same concept. Be consistent in the use of typeface.

Avoid unclear signals about how to direct attention

Well-designed heading and graphic arrangement can convey information about the relative importance of information and order in which it should be considered. 

Mark all questions

Give an obvious graphic signal (e.g., bullet, letter, number) to indicate separate questions.

 Source: Brown (1999).

 

Maximum Legibility
Legibility is the physical appearance of text, the way that the shapes of letters and numbers enable people to read text easily. As delineated by Schriver, a leading document designer, text that is legible can be read “quickly, effortlessly, and with understanding” (see Resources). Despite a great deal of research on what the characteristics of maximum legibility are, the personal opinions of editors about how they want text to look often prevail.

Bias results when tests contain physical features that interfere with a student’s focus on or understanding of the constructs that test items are intended to assess. Dimensions can include contrast, type size, spacing, typeface, leading, justification, line length/width, blank space, graphs and tables, illustrations, and response formats (see Table 4).

 

Table 4. Dimensions of Legibility and Characteristics of Maximum Legibility

Dimension

Maximum Legibility Characteristics

Contrast

Black type on matte pastel or off-white paper is most favorable for both legibility and eye strain.

Type Size

Large type sizes are most effective for young students who are learning to read, students with visual difficulties, and individuals with eye fatigue issues. The legal size for large print text is 14 point.

Spacing

The amount of space between each character can affect legibility. Spacing needs to be wide between both letters and words. Fixed-space fonts seem to be more legible for some readers than proportional-spaced fonts.

Leading

Leading, the amount of vertical space between lines of type, must be enough to avoid type that looks blurry and has a muddy look. The amount needed varies with type size (for example, 14-point type needs 3-6 points of leading).

Typeface

Standard typeface, using upper and lower case, is more readable than italic, slanted, small caps, or all caps.

Justification

Unjustified text (with staggered right margin) is easier to see and scan than justified text especially for poor readers.

Line Length

Optimal length is about 4 inches or 8 to 10 words per line. This length avoids reader fatigue and difficulty locating the beginning of the next line, which causes readers to lose their place.

Blank Space

A general rule is to allow text to occupy only about half of a page. Blank space anchors text on the paper and increases legibility.

Graphs and Tables

Symbols used on graphs need to be highly discriminable. Labels should be placed directly next to plot lines so that information can be found quickly and not require short-term memory.

Illustrations

When used, an illustration should be directly next to the question for which it is needed. Because illustrations create numerous visual and distraction challenges, and may interfere with the use of some accommodations (such as magnifiers), they should be used only when they contain information being assessed.

Response Formats

Response options should include larger circles (for bubble response tests), as well as multiple other forms of response.

 Based on Thompson, Johnstone, and Thurlow (2002).


Summary

The concept of universally-designed assessments is relatively new, and therefore what it actually means is still undergoing clarification. It is likely that the elements of universally designed assessments will be expanded and become more concrete as they are applied to assessment design and development. With the increased emphasis on testing in the nation’s schools in response to federal and state mandates, it is essential that this progress occurs as rapidly as possible. This will require the consolidation and application of current best practices in assessment, along with research and innovation to expand our knowledge in this area. Universal design opens the door to ways to rethink assessments to ensure that it is not the assessment itself that produces barriers to improved learning. The concept of universal design helps us to rethink our basic assumptions about how to create national, state, and district assessments that give a more accurate picture of what all students know and can do so that educators can focus on the critical target of providing universally designed standards-based instruction.


Resources

2001 State Policies on Assessment Participation and Accommodations (Synthesis Report 46). Thurlow, M.L., Lazarus, S., Thompson, S.J., & Robey, J. (2002). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Accommodations Online Bibliography. National Center on Educational Outcomes. (2002). See http://education.umn.edu/NCEO/AccomStudies.htm.

Assessment Accommodations Research: Considerations for Design and Analysis (Technical Report 26). Thurlow, M.L., McGrew, K.S., Tindal, G., Thompson, S.J., Ysseldyke, J., & Elliott, J.L. (2000). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Dynamics in Document Design. Schriver, K.A. (1997). John Wiley & Sons.

Findings of the 1999 Plain Language Field Test. Brown, P.J. (1999). University of Delaware, Newark, DE: Delaware Research and Development Center.

Test Science, Not Reading. Rakow, S.J., & Gee T.C. (1987). Science Teacher, 54 (2), 28-31.

The Truth about Testing: An Educator’s Call to Action. Popham, W.J. (2001). Alexandria, VA: Association for Supervision and Curriculum Development.

Universal Design Applied to Large-Scale Assessment (Synthesis Report 44). Thompson, S.J., Johnstone, C.J. & Thurlow, M.L. (2002). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Universal Design in Education: Teaching Nontraditional Students. Bowe, F.G. (2000). Westport, CT: Bergin & Garvey.

What is Universal Design? Center for Universal Design. (1997). North Carolina State University. See http://www.design.ncsu.edu.

© 2007 by the Regents of the University of Minnesota.
The University of Minnesota is an equal opportunity educator and employer.

Online Privacy Statement
This page was last updated on May 20, 2013

NCEO is supported primarily through a Cooperative Agreement (#H326G050007) with the Research to Practice Division, Office of Special Education Programs, U.S. Department of Education. Additional support for targeted projects, including those on LEP students, is provided by other federal and state agencies. Opinions expressed in this Web site do not necessarily reflect those of the U.S. Department of Education or Offices within it.