NCEO’s Universal Design project has published Analyzing Results of Large-scale Assessments to Ensure Universal Design (Technical Report 41). This report describes a series of statistical analyses states can use to determine whether particular items function differently for students with disabilities and non-disabled students. The authors of the report (Johnstone, Thompson, Moen, Bolt and Kato) noted that none of the analyses suggested are meant to be used for low incidence populations, such as students with less common disabilities like autism or hearing impairments, but that useful information can be found when patterns exist across disability groups or analysis techniques.

Statistical analyses of statewide assessment data typically take place after field testing and live administrations of tests. Therefore, assessment personnel and data analysts in states, both in special and general education offices, may find this report of interest. This report can be found on the NCEO Web site at http://education.umn.edu/nceo/OnlinePubs/Technical41.htm.

A second report, Considerations for the Development and Review of Universally Designed Assessments (Technical Report 42) is nearing publication. This report highlights results from a Delphi study conducted with assessment, disability, English language learner experts, and others. Delphi studies are an approach to gathering opinions from experts in particular fields. These experts contributed to a list of "considerations" that states and assessment companies can use to ensure that tests are universally designed. For example, experts grappled with timely issues of language use, translation, readability, and computerization of tests.

The Delphi study produced a list of ideas that states and test companies can consider when designing assessments. The ideas set forth by experts were organized into larger themes such as "Respect the Diversity of the Assessment Population," "Measures What it Intends to Measure," "Have Concise and Readable Text," and "Allow Changes to its Format Without Changing its Meaning or Difficulty." Each of these themes should help state educational leaders decide how a test must look in order for it to be considered "universally designed." This report will soon be available on the NCEO Web site at www.nceo.info.

A third report from the Universal Design project focuses on research that employed "think aloud" methods, or cognitive lab techniques, to solicit information from students about test items. In this study, students with disabilities "thought aloud" while they solved large scale assessment items from Grade 4 and 8 math tests. Overall, researchers found that think aloud methods were a good way to find design-related issues in tests. For example, when a particular item was not clearly worded or had distracting pictures or irrelevant background information, students would clearly demonstrate that they were mislead by such aspects. On the other hand, some items that researchers thought would cause students problems proved easy for students to solve. The lesson learned from this study is that perspectives of students with disabilities are very important to consider when making assessment decisions.

The three above-mentioned studies will be synthesized into a "How To" manual for States this fall. The manual will contain information about how to use the techniques NCEO used and important tips to remember in the test development process. For more information about the Universal Design studies, contact Sandy Thompson at thomp178@umn.edu or Chris Johnstone at john4810@umn.edu.

NCEO is putting the finishing touches on the most recent survey of state directors of special education, 2005 State Special Education Outcomes. This tenth survey provides a snapshot of the new initiatives, trends, accomplishments, and emerging issues involving the participation and achievement of students with disabilities during standards-based reform.

The report contains good news: the number of students with disabilities achieving proficiency on state accountability tests is increasing. Most states now have at least three years of trend data and enough evaluation data to be able to attribute increased proficiency to several positive efforts by schools and districts. There have been many changes during the past two years as increasing numbers of students receive the instruction they need to meet grade level proficiency.

When states were asked which factors contributed to the positive trends found in the survey, at least half of the states credited the following six factors:

There is wide variation between states and districts in the types of accommodations that are approved for use on state assessments. Similarly, there is a range of policies and practices related to the use of scribes, readers, and interpreters for students with disabilities. Less than half of the states have guidelines for these assistants, and there are many variations across state guidelines extending from breadth and depth to format and ease of access. Some states have only a few sentences about access assistants, while other states devote entire documents to the roles and qualifications of these individuals.

Several challenges seem to be inherent to state development and use of guidelines for access assistants. First, each state needs clear policies that define and provide parameters for the accommodations provided by readers, scribes, and sign language interpreters. Accommodation policies are very complex, and access assistants must be aware of all the complexities in the policies.

The next challenge is to carefully and completely describe the work of each type of access assistant. This should be done by a groups of people from the state department of education and from classrooms, as well as by experts familiar with the accommodations and populations.

The third challenge is to make sure that all access assistants in all schools have the guidelines, whether in written or online format. An access assistant should not be a volunteer who just appears on test day without preparation and prior knowledge. Because of the need for consistency and standardization, to the extent possible, access assistants need comprehensive guidelines and training on how to use the guidelines.

Because the use of scribes, readers, and sign language interpreters introduces human variability into the testing situation, the potential for challenges to the validity and comparability of resulting scores is greatly increased. Therefore, the availability of guidelines that direct the work of these individuals, and the specificity of language in these guidelines, is critically important to states, students, and the individuals who work as access assistants themselves.

The full report entitled Access Assistants for State Assessments: A Study of State Guidelines for Scribes, Readers, and Sign Language Interpreters (Clapper, Morse, Thurlow, and Thompson) will be available on the NCEO Web site in the near future.