The Mission of the National Center on Educational Outcomes
Acknowledgments
State Directors of Special Education
Executive Summary
Overview of 2005 Survey
Participation and Performance
Accommodations
Alternate Assessments
Emerging Practices
Current and Emerging Issues
Preferred Forms of Technical Assistance
References
Appendix A: Working Definitions of Students with the "Most Significant Cognitive Disabilities"
Appendix B: Examples of Current and Emerging Issues

NCEO Staff
Deb Albus
Christopher Johnstone
Jane Krentz
Kristi Liu
Ross Moen
Michael Moore
Rachel Quenemoen
Dorene Scott
Sandra Thompson
Martha Thurlow, Director

With the collective efforts of State Directors of Special Education, we are able to report on the activities of all 50 states and six of eleven federally funded entities (unique states). Because of the thoughtful and knowledgeable responses of the directors and their designees who completed this survey, we are able to share new initiatives, trends, accomplishments, and emerging issues during this important period of education reform. The purpose of this report is to make public the trends and issues facing states, as well as the innovations states are using to meet the demands of Federal legislation. We appreciate the time taken by respondents to gather information from people outside of special education, and we hope that this collaborative effort provided an opportunity to increase awareness within and across state programs and departments.

For their support, special thanks go to:

• Dave Egnor and Lou Danielson, of the Office of Special Education Programs in the U.S. Department of Education (OSEP);
• David Malouf, formerly with OSEP, now with the Institute of Education Science in the U. S. Department of Education;
• Eileen Ahearn, of the National Association of State Directors of Special Education;
• Michael Moore, online survey designer and communications director for the National Center on Educational Outcomes; and Khoa Vang, student assistant.

2005 State Special Education Outcomes was prepared by Sandra Thompson, Christopher Johnstone, Martha Thurlow, and Jason Altman.

States continue to work on the participation of students with disabilities in state assessments. The calculation of participation rates is based on different denominators in different states. Most states use either the number of students with disabilities enrolled within a month of the test or the number of students with disabilities counted on test day, yet some still use the number of students tested or the enrollment on December 1. Use of accommodations is also documented in some way by every state.

The report contains good news: the number of students with disabilities achieving proficiency on state accountability tests is increasing. Most states now have at least three years of trend data and enough evaluation data to be able to attribute increased proficiency to several positive efforts by schools and districts. There have been many changes during the past two years as increasing numbers of students receive the instruction they need to meet grade level proficiency.

When states were asked which factors contributed to positive trends found in the survey, at least half of the states credited the following six factors:

·         Clearly communicated participation policy

·         Better alignment of IEPs with standards

·         Improved professional development

·         Development and provision of accommodation guidelines and training

·         Increased access to standards-based instruction

·         Improved data collection

Updates on alternate assessments show continued evolution in various aspects, from the approach itself, to the content, setting of standards, and the scoring criteria that are used. Areas of emerging practice include for a few states item analyses and disaggregating assessment results by English language learners. For about half the states, emerging practice includes field testing in alternate formats and disaggregating assessment results by disability category. Nearly all of the states are conducting further work in the area of universally designed assessments.

Overview of 2005 Survey

This report marks the tenth time over the past 14 years that the National Center on Educational Outcomes (NCEO) has collected information from state directors of special education about the participation and achievement of students with disabilities during standards-based reform. As this survey reached final preparation, the government reauthorized the Individuals with Disabilities Education Act (IDEA). As in 1997, IDEA 2004 requires states to report the number of students with disabilities who participate in state assessments. Likewise, the No Child Left Behind Act of 2001 (NCLB) requires the reporting of participation rates by subgroup, including students with disabilities.

We are able to report, for the first time, that the number of students with disabilities achieving proficiency on state tests for accountability is improving. Most states now have at least three years of performance data and enough evaluation data to be able to attribute increased proficiency to several positive efforts by schools and districts. This report begins with descriptions of trends in participation and performance and then, as in past years, takes a careful look at accommodations use, alternate assessment, emerging practices and issues, and technical assistance needs. Readers will see many changes during the past two years as increasing numbers of students receive the instruction they need to meet grade level proficiency and as states and districts continue to strive to meet goals for adequate yearly progress (AYP) as required by the No Child Left Behind Act of 2001. One of the greatest improvements is the increased trend toward proficiency on state assessments by students with disabilities. At least half of the states attribute these positive trends to six factors:

·         Clearly communicated participation policy

·         Better alignment of IEPs with standards

·         Improved professional development

·         Development and provision of accommodation guidelines and training

·         Increased access to standards-based instruction

·         Improved data collection

There is no doubt that difficult issues remain, but progress over the past two years has been strong and positive, providing hope for the same in the future.

Participation and Performance

The 2004 Individuals with Disabilities Education Improvement Act requires states to report the number of students with disabilities who participate in state assessments. Likewise, the No Child Left Behind Act of 2001 (NCLB) requires the reporting of participation rates by subgroup, including students with disabilities. In this survey, NCEO asked states how students were included in 2003-2004 state assessment accountability reports for NCLB.

States responded that they include students with disabilities in their participation reports in different ways, depending on whether they are: students who do not participate in state assessments in any way (e.g., absent on test day, parent refusal, hospitalized); students who attended (sat for) assessments, but did not complete enough items to score; students who used invalid accommodations (e.g., non-standard, modifications); or students who tested at a lower grade than their level of enrollment. Table 1 demonstrates the ways in which states count students in these categories.

The majority of states (n=35, 70%) do not count students who did not participate in the state assessment at all, and give them no score. Other states count non-participating students but give them no score (n=4, 8%), or count students as participants and score them at the lowest proficiency level or a score of zero (n=7, 14%).

Reporting practices differ among states for students who attended (sat for) an assessment but did not complete enough items to score. Twenty-one states count students as participants and score them at the lowest proficiency level or give them a score of zero (42%), and 17 states count these students as participants and give them no score (34%).

Students who use accommodations defined as “invalid” (e.g., read aloud accommodation on a reading test, calculator accommodation on a portion of a mathematics test that does not allow calculators) are also counted in varied ways. States most often count these students as participants but give them no score (n=7, 14%), or a score of zero or the lowest proficiency level (n=17, 34%). Twelve state directors (24%) report that their state does not allow “invalid” accommodations; therefore, they do not consider reporting issues. Other states noted that students who use invalid accommodations are not counted as participants (n=8, 16%).

A practice in some states is to test students on content outside of their current grade level. Twenty-five states (50%) report that they currently allow out-of-level testing. Among states that do allow students to be tested at a grade level lower than their grade of enrollment, approximately half count students as participants and give a score of zero or the lowest proficiency level (n=11, 22%). Of the rest, most states count the earned score as valid (n=10, 20%). Unique states count students in proportions similar to those of regular states.

Nine states do not use any of the formulae for calculating participation rates that were provided as prompts in the NCEO survey. In these states, directors chose “other” as a survey response, and reported a number of ways participation rates are determined. Within this category, states vary in how to count the number of students tested (the numerator), and the total number of students (the denominator). For example, one state reports the number of students with disabilities with valid test results divided by the number of students reported by the December 1 special education child counts. Another state takes the number of test takers with disabilities tested divided by the number of students with disabilities enrolled as of the 4^th Friday in September of that school year. Finally, one state director confidently reports that the participation rate is always 100 percent because all students are tested.

As noted in previous NCEO Surveys of State Directors of Special Education (Thompson & Thurlow, 2001, 2003), inclusive assessment and accountability are critical areas of information for states. As shown in Figure 3, at least half of the states attributed positive trends in performance to each of these six factors:

·         Clearly communicated participation policy (n=36, 72%)

·         Better alignment of IEPs with standards (n=32, 64%)

·         Improved professional development (n=28, 56%)

·         Development and provision of accommodations guidelines and training (n=27, 54%)

·         Increased access to standards-based instruction (n=26, 52%)

·         Improved data collection (n=26, 52%)

In addition to the attributes listed in Figure 3, state directors made these comments:

“We are in the early stage of development. With implementation of a new [statewide] test, we will need to re-examine this question as we observe emerging patterns under new test program. Most of [the] current change is probably due to increasing public awareness of expectation and consequences.”

 “Concurrent to general trends in improved student achievement, most of our students with disabilities are served in general education environments. Achievement has generally improved for all students.”

Fifteen state directors indicated that avoidance of NCLB consequences was partially responsible for current trends (30%). Far fewer state directors selected negative attributes than positive. Four state directors expressed concern that students with disabilities may not be able to reach proficiency on a state’s general assessments (8%).

Accommodations

All states now document accommodations use on test day, although with different levels of specificity. This is a major change in the past few years.  In the report, 2001 State Special Education Outcomes, 21 states reported that they did not document the use of accommodations on state assessments at the state level. This number decreased to three states in 2003.

Table 2 shows the ways that states document accommodation use in 2005. Approximately half of the regular states (n=26, 52%) and fewer of the unique states (n=2, 18%) mark specific accommodations on the test or response form, whereas only 16% of the regular states and no unique states mark only “standard” or “nonstandard.” This is a contrast to 2003 results, where 15 states (30%) indicated they documented accommodations in the dichotomous (standard/nonstandard) form.

NCEO began tracking alternate assessments in 1997, and states were required to start implementing them in 2000. IDEA 2004 describes alternate assessments based on alternate achievement standards as assessments for students unable to participate in regular state or district-wide assessments because they are students with the most significant cognitive disabilities.

Most states (n=45, 90%) offer an alternate assessment based on alternate achievement standards. Ten states (20%) indicate they also offer an alternate assessment based on grade-level achievement standards.    

Almost half of the states (22, 44%) in the U.S. have changed their assessment participation policies/guidelines since the December 9, 2003 regulation on alternate assessments based on alternate achievement standards. One state reported that the alternate assessment is a “work in progress, as it appears our extended indicators may still not meet the federal requirements.”

Alternate assessments based on alternate achievement standards are intended for students with the most significant cognitive disabilities. We asked states how they would define this population of students and found that each state had its own distinct definition.  For example, New Hampshire looks for the following three criteria:

1.  The student has an IEP.

2.  Evidence that the student’s demonstrated cognitive ability and adaptive behavioral skills: prevent him/her from demonstrating achievement of the proficiency standards described in the NH Curriculum Frameworks, even with appropriate accommodations, and require individualized instruction in multiple settings (school, work, home and community environments) to acquire, generalize, and transfer skills necessary for functional application.

3.  There is historical data (current and longitudinal across multiple settings) that confirms the individual student criteria listed above.

Kentucky uses policy from state regulation in determining what qualifies a student as one with the most significant cognitive disabilities. Wyoming defines a student with the most significant cognitive disabilities as one with “significantly sub-average general intellectual functioning, existing concurrently with deficits in adaptive behavior and manifested during the developmental period that adversely affects a child’s educational performance.” The Department of Defense defines these students as “students with an identified moderate to profound intellectual disability as measured on a standardized IQ assessment.”

These are some of the varied definitions of students with the “most significant cognitive disabilities” that states use to guide participation in alternate assessments. Appendix A provides the definitions used by all the regular and unique states able to provide this information. Several states were not able to provide definitions.

*One state has not developed any statewide alternate assessment approaches.
**Of these 25 states, 13 use a standardized set of performance/events/tasks/skills.
***Of these 7 states, 3 require the submission of student work.
 

It may be that the traditional way of describing alternate assessment approaches is no longer the best because there is considerable overlap across approaches that states take. For example, of the 25 states using portfolio (body of evidence) assessments in 2005, 13 states use a standardized set of performance events, tasks, or skills. Three of the seven states using a rating scale of performance on a standardized set of events, tasks, or skills require the submission of a body of evidence.

Of the six unique states that responded to this year’s survey, five responded to this question.  Two unique states are currently using grade-level content standards.

As demonstrated in Figure 5, there is great variation in the specific scoring criteria in states that use rubrics. Skill/competence and level of assistance are the most frequently used scoring criteria. This variation is also evident among unique states. Despite the differences in approaches, rubrics are still the most common approach for scoring a student’s skill/competence on a task or level of assistance needed to complete a task.

States also indicated the types of standard-setting processes they use. Table 6 describes the most commonly used standard-setting processes for alternate assessments and lists the number of states that use them. Most states use a body of work approach (n=11, 22%), reasoned judgment (n=10, 20%), or bookmarking/item mapping (n=9, 18%).  A few states selected more than one technique for standard setting. One unique state responded that it performs standard-setting, but did not select a technique.

Emerging Practices

The 2005 survey asked states about several emerging practices, specifically to indicate whether they were addressing each issue. Issues that were reported in the 2003 survey were item analysis, universally designed assessments, and computer-based state assessments. In the 2005 survey, respondents also answered questions about field testing, disaggregating assessment results, and RFPs and item review process (for universally designed assessments).

States are also beginning to disaggregate data for English language learners. Twelve regular states (24%) and four unique states currently disaggregate assessment data by language groups. Among these states, eight states and one unique state disaggregate data by both language group and disability.

* Did not complete survey.
 

The areas in which states are addressing universal design are shown in Figure 7. More than half of the states are addressing universal design at the item development level (n=31, 62%), item review level (n=30, 60%), and by including requirements for universal design in a request for proposals (RFP) for test development (n=27, 54%).