NCEO Policy Directions
Published by the National Center on Educational Outcomes
Number 14 / June 2002
Universally Designed Assessments: Better Tests for Everyone!
Prepared by Sandra Thompson and Martha Thurlow
Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:
Thompson, S., & Thurlow, M. (2002). Universally designed assessments: Better tests for everyone! (Policy Directions No. 14). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/Policy14.htm
Background
The standard administration
of assessments is not appropriate for all students who must participate in state
and district assessments today. The use of accommodations – changes in
administration procedures or materials – is evidence that there are students who
cannot participate in assessments or receive valid scores unless something is
changed. Only a very small percentage of students need a completely different
assessment, identified in federal special education law as an alternate
assessment. A much larger group of students need changes in the regular
assessment.
Because of the emphasis on
testing and including all students, the provision of accommodations and
decisions about who should participate in alternate assessments has become very
complex. There is a great deal of controversy about the “fairness” of many test
accommodations and about which students should have access to accommodations and
how decisions are made. Research to validate accommodation use is growing, but
the research is difficult to conduct and rarely provides conclusive evidence
about the effects of accommodations on validity. States grapple with decisions
about which accommodations should be included in school accountability and which
invalidate assessment scores. Repeated revisions in state accommodation policies
is just one indicator of the controversy surrounding the need to provide
accommodations for students to be able to participate and show their knowledge
and skills in assessments. It is time to take a more global approach to
addressing these testing issues, an approach in which increased access for all
students is considered from the beginning.
Applying Universal Design to Assessments
The concept of universal
design is not new. Its use began in the field of architecture, but its
application has spread rapidly into environmental initiatives, recreation, the
arts, health care, and education. Principles of universal design that traverse
all of these areas have been developed (see Table 1). It is reasonable to expect
that they can apply equally as well to large-scale assessments.
Table 1. Principles of Universal Design in Architecture and Other Areas
Principle |
Explanation |
Equitable
Use |
The
design is useful and marketable to people with diverse abilities. |
Flexibility in Use |
The
design accommodates a wide range of individual preferences and abilities. |
Simple
and Intuitive Use |
Use of
the design is easy to understand, regardless of the user's experience,
knowledge, language skills, or current concentration level. |
Perceptible Information |
The
design communicates necessary information effectively to the user,
regardless of ambient conditions or the user’s sensory abilities. |
Tolerance
for Error |
The
design minimizes hazards and the adverse consequences of accidental or
unintended actions. |
Low
Physical Effort |
The
design can be used efficiently and comfortably and with a minimum of
fatigue. |
Size and
Space for Approach and Use |
Appropriate size and space is provided for approach, reach, manipulation,
and use regardless of user’s body size, posture, or mobility. |
Source: The Center for Universal Design, North Carolina State University (1997).
The goal of applying
universal design principles to assessments is to be able to design and develop
assessments that allow participation of the widest range of students, and result
in valid inferences about their performance. The need that many students have
for accommodations could be reduced if assessments could be universally
designed. Universally designed assessments are not intended to eliminate
individualization, or to take away from the IEP process. Instead, they could
make the IEP process richer by focusing on instructional needs rather than on
all the changes that will have to be made for the student to participate in the
assessment. Universal design is the best way to increase participation in
general state and district assessments.
Universal design is based
on the same ethics of equity and inclusiveness that are expected for people with
disabilities and others in schools, communities, and on the job – an ethic that
values differences in age, ability, culture, and lifestyle. Testing conditions
should not be affected by disability, gender, race, English language ability, or
levels of anxiety about tests. On the other hand, it is important to remember
that universal design does not address deficiencies in instruction. Students who
have not had an opportunity to learn the material tested will be disadvantaged
during testing no matter how universal the design of the assessment.
Elements of Universally Designed Assessments
NCEO has conducted an
extensive review of all research relevant to the assessment development process
and the principles of universal design (see Synthesis Report 44 in Resources).
This review produced a set of seven elements of universal design that apply to
assessments (see Table 2).
Table 2. Elements of Universally Designed Assessments
Element |
Explanation |
Inclusive
Assessment Population |
Tests
designed for state, district, or school accountability must include every
student except those in the alternate assessment, and this is reflected in
assessment design and field testing procedures. |
Precisely Defined Constructs |
The
specific constructs tested must be clearly defined so that all construct
irrelevant cognitive, sensory, emotional, and physical barriers can be
removed. |
Accessible, Non-Biased Items |
Accessibility is built into items from the beginning, and bias review
procedures ensure that quality is retained in all items. |
Amenable
to Accommodations |
The test
design facilitates the use of needed accommodations (e.g., all items can be
Brailled). |
Simple,
Clear, and Intuitive Instructions and Procedures |
All
instructions and procedures are simple, clear, and presented in
understandable language. |
Maximum
Readability and Comprehensibility |
A variety
of readability and plain language guidelines are followed (e.g., sentence
length and number of difficult words are kept to a minimum) to produce
readable and comprehensible text. |
Maximum
Legibility |
Characteristics that ensure easy decipherability are applied to text, to
tables, figures, and illustrations, and to response formats. |
Based on Thompson, Johnstone, and Thurlow (2002).
Inclusive
Assessment Population
When tests are first conceptualized, they need to be thought of in the context
of who will be tested. If the test is designed for state, district, or school
accountability purposes, the target population must include every student except
those who will participate in accountability through the alternate assessment.
Assessments need to be responsive to growing demands – increased diversity,
increased inclusion of all types of students in the general curriculum, and
increased emphasis and commitment to accountability for all students.
Precisely Defined
Constructs
An important function of well-designed assessments is that they actually measure
what they are intended to measure. Test developers need to carefully examine
what is to be tested and design items that offer the greatest opportunity for
success within those constructs. Just as universally designed architecture
removes physical, sensory, and cognitive barriers to all types of people in
public and private structures, universally designed assessments must remove all
non-construct-oriented cognitive, sensory, emotional, and physical barriers.
Accessible,
Non-Biased Items
Items are reviewed through bias review or sensitivity review procedures to
ensure that they do not create barriers because of lack of sensitivity to
disability, cultural, or other subgroups. But, perhaps more important, items are
developed by individuals who understand the varied characteristics of students,
and the characteristics of items that might create difficulties for any group of
students. Accessibility is incorporated as a primary dimension of test
specifications, so that accessibility is woven into the fabric of the test
rather than being added after the fact.
Amenable to
Accommodations
Even though items on universally designed assessments will be accessible for
most students, there will still be some students who continue to need
accommodations. Thus, another essential element of any universally designed
assessment is that it is compatible with accommodations and a variety of
widely-used adaptive equipment and assistive technology.
Simple, Clear, and
Intuitive Instructions and Procedures
Assessment instructions should be easy to understand, regardless of a student’s
experience, knowledge, language skills, or current concentration level.
Directions and questions need to be in simple, clear, and understandable
language. Knowledge questions that are posed within complex language certainly
invalidate the test if students cannot understand how they are expected to
respond to a question.
Maximum Readability
and Comprehensibility
A variety of guidelines exist to ensure that text is maximally readable and
comprehensible. These features go beyond what is measured by readability
formulas. Readability and comprehensibility are affected by many
characteristics, including student background, sentence difficulty, organization
of text, and others. All of these features need to be considered in developing
the text of assessments.
Plain language is a concept now being highlighted in research on assessments. Plain language has been defined as language that is straightforward and concise. Strategies for editing text to produce plain language have been identified (see Table 3).
Table 3. Plain Language Editing Strategies
Strategy |
Description |
Reduce
excessive length |
Reduce
wordiness and remove irrelevant material. |
Use
common words |
Eliminate
unusual or low frequency words and replace with common words (e.g., replace
“utilize” with “use”). |
Avoid
ambiguous words |
For
example, “crane” should be avoided because it could be a bird or a piece of
heavy machinery. |
Avoid
irregularly spelled words |
Examples
of irregularly spelled words are “trough” and “feign.” |
Avoid
proper names |
Replace
proper names with simple common names such as first names. |
Avoid
inconsistent naming and graphic conventions |
Avoid
multiple names for the same concept. Be consistent in the use of typeface. |
Avoid
unclear signals about how to direct attention |
Well-designed heading and graphic arrangement can convey information about
the relative importance of information and order in which it should be
considered.
|
Mark all
questions |
Give an
obvious graphic signal (e.g., bullet, letter, number) to indicate separate
questions. |
Maximum Legibility
Legibility is the physical appearance of text, the way that the shapes of
letters and numbers enable people to read text easily. As delineated by Schriver,
a leading document designer, text that is legible can be read “quickly,
effortlessly, and with understanding” (see Resources). Despite a great deal of
research on what the characteristics of maximum legibility are, the personal
opinions of editors about how they want text to look often prevail.
Bias results when tests contain physical features that interfere with a student’s focus on or understanding of the constructs that test items are intended to assess. Dimensions can include contrast, type size, spacing, typeface, leading, justification, line length/width, blank space, graphs and tables, illustrations, and response formats (see Table 4).
Table 4. Dimensions
of Legibility and Characteristics of Maximum Legibility
Dimension |
Maximum Legibility Characteristics |
Contrast |
Black
type on matte pastel or off-white paper is most favorable for both
legibility and eye strain. |
Type Size |
Large
type sizes are most effective for young students who are learning to read,
students with visual difficulties, and individuals with eye fatigue issues.
The legal size for large print text is 14 point. |
Spacing |
The
amount of space between each character can affect legibility. Spacing needs
to be wide between both letters and words. Fixed-space fonts seem to be more
legible for some readers than proportional-spaced fonts. |
Leading |
Leading,
the amount of vertical space between lines of type, must be enough to avoid
type that looks blurry and has a muddy look. The amount needed varies with
type size (for example, 14-point type needs 3-6 points of leading). |
Typeface |
Standard
typeface, using upper and lower case, is more readable than italic, slanted,
small caps, or all caps. |
Justification |
Unjustified text (with staggered right margin) is easier to see and scan
than justified text especially for poor readers. |
Line
Length |
Optimal
length is about 4 inches or 8 to 10 words per line. This length avoids
reader fatigue and difficulty locating the beginning of the next line, which
causes readers to lose their place. |
Blank
Space |
A general
rule is to allow text to occupy only about half of a page. Blank space
anchors text on the paper and increases legibility. |
Graphs
and Tables |
Symbols
used on graphs need to be highly discriminable. Labels should be placed
directly next to plot lines so that information can be found quickly and not
require short-term memory. |
Illustrations |
When
used, an illustration should be directly next to the question for which it
is needed. Because illustrations create numerous visual and distraction
challenges, and may interfere with the use of some accommodations (such as
magnifiers), they should be used only when they contain information being
assessed. |
Response
Formats |
Response
options should include larger circles (for bubble response tests), as well
as multiple other forms of response. |
Summary
The concept of
universally-designed assessments is relatively new, and therefore what it
actually means is still undergoing clarification. It is likely that the elements
of universally designed assessments will be expanded and become more concrete as
they are applied to assessment design and development. With the increased
emphasis on testing in the nation’s schools in response to federal and state
mandates, it is essential that this progress occurs as rapidly as possible. This
will require the consolidation and application of current best practices in
assessment, along with research and innovation to expand our knowledge in this
area. Universal design opens the door to ways to rethink assessments to ensure
that it is not the assessment itself that produces barriers to improved
learning. The concept of universal design helps us to rethink our basic
assumptions about how to create national, state, and district assessments that
give a more accurate picture of what all students know and can do so that
educators can focus on the critical target of providing universally designed
standards-based instruction.
Resources
2001 State
Policies on Assessment Participation and Accommodations (Synthesis
Report 46). Thurlow, M.L., Lazarus, S., Thompson, S.J., & Robey, J. (2002).
Minneapolis, MN: University of Minnesota, National Center on Educational
Outcomes.
Accommodations
Online Bibliography. National Center on Educational Outcomes.
(2002). See
http://education.umn.edu/NCEO/AccomStudies.htm.
Assessment
Accommodations Research: Considerations for Design and Analysis
(Technical Report 26). Thurlow, M.L., McGrew, K.S., Tindal, G., Thompson, S.J.,
Ysseldyke, J., & Elliott, J.L. (2000). Minneapolis, MN: University of Minnesota,
National Center on Educational Outcomes.
Dynamics in
Document Design. Schriver, K.A. (1997). John Wiley & Sons.
Findings of the
1999 Plain Language Field Test. Brown, P.J. (1999). University of
Delaware, Newark, DE: Delaware Research and Development Center.
Test Science,
Not Reading. Rakow, S.J., & Gee T.C. (1987). Science Teacher, 54
(2), 28-31.
The Truth about
Testing: An Educator’s Call to Action. Popham, W.J. (2001).
Alexandria, VA: Association for Supervision and Curriculum Development.
Universal
Design Applied to Large-Scale Assessment (Synthesis Report 44).
Thompson, S.J., Johnstone, C.J. & Thurlow, M.L. (2002). Minneapolis, MN:
University of Minnesota, National Center on Educational Outcomes.
Universal
Design in Education: Teaching Nontraditional Students. Bowe, F.G.
(2000). Westport, CT: Bergin & Garvey.
What is
Universal Design?
Center for Universal Design. (1997). North Carolina State University. See
http://www.design.ncsu.edu.