May 2004

When investigators document the consequences of high stakes assessments for students with disabilities, many negative consequences are cited. We examined both empirical and anecdotal evidence for positive consequences of large-scale high-stakes assessments for students with disabilities. Multiple methodologies were used to gather data on positive consequences: a qualitative media survey, an environmental scan of State Special Education Directors, a focus group, and a national survey on state assessment practices. Four primary positive consequences for students with disabilities were found consistently across all methodologies: increased participation of students with disabilities in testing programs, higher expectations and standards, improved instruction, and improved performance. Secondary findings were found in fewer sources (usually two or three sources) and are as follows: improved assessments, improved diploma options, decreased dropout rates, and increased collaboration and communication between parents and special education and general education teachers. Overall, the findings suggest that large-scale high stakes assessments can have intended and unintended positive consequences for students with disabilities. We discuss the limitations of the study, and contend that the results provide an interesting initial starting point for further research.

Overview

High stakes assessments are drawing considerable attention in both the professional literature and popular press. It does not take long to find stories in the general media about low scores on state tests, declining scores, teachers providing students with practice on the actual test, test company scoring mistakes, and students who get discouraged and drop out. When professionals attend to or describe consequences for individual students with disabilities, the focus typically is on “narrowing of the curriculum,” the anxiety and frustration the students experience in taking tests, the validity of assessment accommodations (like reading the math test), test preparation or teaching to the test, increased referral to special education, and “unrealistic” expectations for students.

Although so much has been reported about the consequences of high stakes testing, the actual empirical evidence that is available is scarce. In Consequences of Assessment: What is the Evidence? Mehrens (1998) cites Daniel Koretz in summing up existing research: “Despite the long history of assessment-based accountability, hard evidence about its effects is surprisingly sparse and the little evidence that is available is not encouraging.…The large positive effects assumed by advocates…are often not substantiated by hard evidence…” ( p. 5). Cizek (2001) also agrees about the lack of research on the consequences of high-stakes testing programs, but finds further that the existing literature overwhelmingly focuses on the negative aspects of testing, and any positive effects have been “…assumed, unrecognized, or unarticulated” (p. 23). In fact, Cizek cites the results of a literature search (undertaken by a colleague) on the effects of high-stakes testing and found that out of 59 entries, only two articles could be considered “positive in nature.”

Although the empirical evidence is not sound enough to provide causal statements about the impact of large-scale assessment, the existing data do suggest intended and unintended consequences (Mehrens, 1998; Lane & Stone, 2002). In this report we examine the extent to which implementation of high stakes assessments (whether high stakes for individual students or local education agencies) has resulted in positive outcomes and experiences for students with disabilities. Those who wrote the assessment provisions of the 1997 Amendments to the Individuals with Disabilities Education Act had intended that the legislation would result in positive consequences. They reasoned that requiring the participation of students with disabilities in state assessments would result in increased participation in those assessment systems, and that educators would increasingly assume responsibility for educational outcomes for students with disabilities. It was thought that increased participation in assessments would result in increased inclusion in the general education curriculum; this, in turn, would lead to improved educational achievement for the students.

We examined some of the obvious intended consequences of high-stakes tests for students with disabilities such as increased rates of participation, improved performance on state assessment measures, higher expectations for students with disabilities, and improved instruction. We also discuss some unintended positive consequences. Although focusing on intended positive effects of high stakes tests cannot provide a complete picture of the efficacy of such programs (negative results also have to be taken into consideration), “…an evaluation of the intended effects is first necessary” (Lane & Stone, 2002).

Focusing on the positive effects of large-scale testing is not an argument for or against such assessment programs but simply an acknowledgement of the current educational landscape students with disabilities must now face (annual testing in grades three through eight must be undertaken by all states in the coming years due to the passage of the No Child Left Behind Act of 2001) and a need for the assessment programs themselves to be evidenced-based. Both proponents for and critics of high stakes testing must acknowledge the lack of research on the issue and begin to conduct serious empirical inquiry to determine not only what the consequences are or will be of large-scale assessments, but also what can be done to minimize negative outcomes and maximize intended consequences. This report should be considered an initial attempt to examine any evidence that large-scale assessments are producing intended positive effects, and as a starting point for future research. Much of the information reported is based on anecdotal information and on secondary sources and we are not able to establish causal relationships. However, this report still serves as an important starting point for reexamining what is actually known or reported about the impact of large-scale assessments on students (large-scale assessments also impact teachers, parents, schools, administrators, etc. but we chose to focus on students for this report).

Methods

Due to the paucity of research on consequences of large-scale assessments, we used multiple methodologies to conduct a survey of positive consequences of high-stakes testing and accountability systems for students with disabilities. We looked for both anecdotal and evidence-based reports. The primary methodology we used was media analysis through “…tracking discourse” or “following certain issues words, themes and frames over a period of time” (Altheide, 1996, p. 70). We chose media reports as a source because general news sources are the public’s basis of information on educational issues (Altheide, 1996). As one focus group finding highlighted, many parents said they were well aware of the “crisis in education” from news reports that left the parents with the impression that “…our children don’t know anything at all” (Barksdale-Ladd, & Thomas, 2000). Besides the media reporting on school test scores and performance, education officials also use the media to communicate important information, and to highlight achievements.

For this study we used the electronic database of Lexis-Nexis (news archival system for major newspapers across the U.S.) and monitored the daily news headlines certain educational organizations chose to highlight (i.e., ASCD SmartBrief). We also looked at professional journals, magazines, and publications (e.g., association newsletters) geared toward educational professionals. We were able to qualitatively establish themes and categories of intended and unintended consequences of large-scale assessments both for students with disabilities and general education students. This methodology allowed us to analyze a large number of articles in a short amount time and opened up other avenues for further research on unexpected findings. To validate the themes, concepts, and ideas found in the newspaper reports, several other sources were used. One was an electronic environmental scan (a one-question survey asking for evidence—anecdotal or otherwise—of positive consequences of large-scale assessment) of State Directors of Special Education. Directors either answered personally or circulated the request among other state or local education officials. In addition, we used the results of a comprehensive survey of state assessment practices compiled by the National Center for Educational Outcomes. We also conducted a focus group on high-stakes testing in Minnesota (Nelson, 2002).

In examining the consequences of high-stakes large-scale assessments, it is important that the reader take into account a couple of context realities: there is considerable variability among states in their assessment and accountability practices; these practices sometimes have high stakes for individuals, systems, or both, and even when it is thought that there are no “high stakes” consequences present. Schulte, Villwock, Whichard, and Stallings (2001) write:

State accountability systems vary on a number of important dimensions that are likely to play a role in the overall impact of the policy, as well as who is affected by them …States vary not only in the form of assessment or the content sampled, but also in how achievement is measured: whether or not certain pre-established levels have been met, or whether there has been growth/gains made by students during the school year. (p. 489)

States that are considered high stakes generally are those that attach consequences to individual students such as grade retention/promotion or withholding diplomas. Huebert and Hauser (1999) indicate that no matter what stakes are attached, students will inevitably bear the consequences of them directly or indirectly. For these reasons, the terms “high stakes” or large-scale assessments will be used interchangeably throughout this paper.

Results

The results of our findings are listed in Tables 1 and 2. Four positive consequences were apparent across methodologies and represent the primary positive consequences. There has been increased participation of students with disabilities in assessment and accountability systems, students with disabilities have shown improved performance, professionals and the general public hold higher expectations/standards for students with disabilities, and people repeatedly talk about improvements in the instruction the students receive. Secondary positive findings (Table 2) were results found in two or three sources. Three of the four sources reported improved assessment and diploma options, while two sources reported decreased dropout rates and increased collaboration between general education and special education teachers. A number of unique findings were cited by only one source; however, they were significant enough to warrant mentioning. Again, it must be emphasized that what has been found and what will be discussed in this paper should be viewed as an initial indication of positive consequences and starting points for future research. In the sections that follow we highlight our findings.

Table 1. Primary Positive Findings

Table 2. Secondary Positive Findings

Results

Increased Participation in Assessments

It is clear that there have been significant increases in the percentages of students with disabilities participating in assessment and accountability systems. This is an anticipated consequence of the IDEA mandate that students with disabilities participate. It seems clear that participation has gone up, yet at the same time this is difficult to assess. State Education Agencies report annually in the fall on the numbers of students with disabilities (referred to as child count information), most accountability testing is completed in the spring, and numbers of students classified as disabled changes during that time (Erickson, Thurlow, & Ysseldyke 1996). Despite being unable to determine precise percentages, in focus groups and in our environmental scan, people repeatedly indicated that the increased participation of all students is providing a more accurate picture of student performance and progress. It is also argued that this increased participation has led to higher expectations for the performance of students. People described the effect of increased participation as “raising the bar” for students for whom we traditionally have held low expectations (e.g., students with disabilities, minority students, and low-income students). Cizek (2001) cites a Chicago School Research Consortium finding that underscores this fact: “…students, especially those who had a history of past failures, said that high-stakes testing forced teachers to pay more attention to them and continued failures were no longer acceptable” (p. 23).

One unfortunate assumption that fuels the desire to exclude certain students from assessment programs is that many believe the scores of Title 1 students (low-income students), students at risk, students with disabilities, and English Language Learners (ELL students) would severely compromise overall state or district scores. Filbin (2002) discounted this argument by writing:

In the not so long ago past, students with learning challenges would not have taken the Colorado Student Assessment Program (CPSAP)…Most would have been sent out of the building on field trips to the zoo so that that they would not interfere with their peers attention to the task at hand. The performance of Colorado students [with disabilities on the CPSAP] confirms that an inclusive accountability system can result in higher expectations and improved learning outcomes. (p. 3)

In Minnesota, the Star Tribune (Draper, May 30 2000) reported that fewer special education students in the state are being excluded or having their scores adjusted to allow them to pass than in previous years. Nationally, about two percent of students with disabilities are excluded from state tests, while Minnesota’s rate of exemption is 1% (Draper, May 30, 2000).

Personnel at the National Center on Educational Outcomes (NCEO) have conducted eight surveys on state assessment practices over the past 10 years. In their 2001 survey they report increased participation rates comparable to those reported in the local and national media (Thompson & Thurlow, 2001). When NCEO began recording national participation rates in 1991, they found that 54% of the states were not aware of or had no mechanism to report participation rates for students with disabilities. The rates of the remaining states (40% with two states not responding) ranged from two percent (Michigan) to 98 % (North Carolina). Ten years later, in 2001, not only do most states record participation rates, but well over half report increases in rates of participation, with 12 states reporting that all students are included in their assessments (Alaska, Arkansas, Connecticut, Delaware, Florida, Illinois, Montana, Nebraska, New Jersey, New Mexico, Rhode Island, and Vermont) (Thompson & Thurlow 2001).

The significance of including students with disabilities in accountability measures can be seen in the following media reports. Vava Guthrie is a parent advocate for PACER (Parent Advocacy Coalition for Educational Results) who has a child who was eligible to take an exemption from the state’s exit exam. Guthrie declined the exemption and encouraged her son to take the state test and admits the first two times he failed: “He was humiliated by it [failing the test],” Guthrie said. “But we used it as a learning moment for him. It’s not something you want to get out of. These are the skills you need to be a productive citizen. . . . He was able to hang in there and pass it on the third try” (Draper, May 30, 2000). In August 2000, The Times-Picuyane reported that Louisiana passed a one-year exemption for students with disabilities from the state’s high-stakes testing program. Some parents in the state did not welcome the exemption because they viewed it as an admission by the state that “we don’t know really what to do about these kids (students with disabilities).” However, St. Bernard Parish did not take the exemption, and 64% of their students passed the state exams during a summer retest. St. Bernard Parish not only had the highest passing rate for students with disabilities in the state at the time, but they also had the highest graduation rate for this population in Louisiana (Thevnot, August 23, 2000). In New York, the move to test all special education students “quadrupled” the passing rates for students with disabilities, and led some educators to realize that many of the students had been misclassified (Gloeckler 2001).

All of these examples show that including students with disabilities and other traditionally low-performing students in accountability measures can produce good results and become the first step in improving learner outcomes. Including students with disabilities in state testing programs alone does not produce desired results; however, increasing their participation certainly provides a greater opportunity and probability for improved student outcomes.
 

Higher Expectations and Standards

Educators report that required participation of students with disabilities in assessments is leading to higher academic expectations. For example, Thompson and Thurlow (2001) indicated that 12% of states (N=6) reported raised academic expectations for students with disabilities and an increased focus on achievement in IEP planning. They indicate further that 20% of states reported that students with disabilities were getting a more rigorous education. West Virginia educators who responded to our environmental scan indicated that special education teachers now receive the teacher’s guide for textbooks because administrators now believe that their students may need to be tested and thus taught (personal communication, April 12, 2002). A district director of assessment from our focus group stated:

I think with special education students, generally speaking, expectations were raised for those kids. I think a lot of caring people who work with them [didn’t] want to damage their [students with disabilities] psyches, they don’t want to see the bar set too high, and I think in the course of that, most often times, the bar was set too low and those kids can achieve more sometimes. And they [students with disabilities] get into the mode of doing less and getting by with less and less...as they go through school because of that expectation…it’s the self-fulfilling prophecy thing. If people are going to expect less, then I will do less…now [in context of raised expectations and large-scale tests] we have lots of special education students who passed [the Minnesota Basic Skills] test. (Nelson, 2002, p. 65)

Not only are teachers and other professionals who work with them raising expectations for students with disabilities, but students and their families are also beginning to have higher expectations. In New York it is reported that academic expectations by families of students with disabilities have increased, and more students with disabilities are expecting the opportunity to pursue postsecondary education. The Boston Globe reported a story of a parent named “Nancy (last name withheld)” of Waltham, Massachusetts pushing for her son to take the regular state exam (MCAS), even against the advice of her son’s teachers. The end result for the student was that he not only passed the state exam, but was so encouraged by the success he is also contemplating attending college (Vashinav, May 12, 2002).

Improved Instruction

Improved instruction was another positive consequence that consistently was indicated. According to a brief literature review on the subject: “Curricular and instructional reform typically means changing the contents of the curriculum or the process of instruction…. That kind of ‘reform’…is at least arguably, a valuable consequence” (Mehrens, 1998, p. 6). In an article on the “standards movement,” which intuitively contains some form of standardized assessment for all students, a pair of authors write:

How does the standards movement change the instructional scene for educators? Content standards for student learning articulate the entire domain of learning in a particular area. Well-written standards emphasize understanding of a discipline’s foundation, not discrete bits of knowledge and mastery of particular techniques. And if student standards define achievement as the demonstration of understanding, the implications for instruction is profound. (Holloway & Pearlman, 2001 p. 40)

Cimbricz (2002) reviewed several studies that examined the relationship between state mandated testing and teacher classroom instruction. She found that although there were other factors that could influence teacher’s instructional practices, state-mandated testing “… does matter and does influence what teachers say and do.” Shepard (1991) conducted a survey of teachers in two high-stakes school districts to discover if state tests influenced teacher instruction. One major positive finding was that teachers were able to give several positive impacts of testing on instruction such as clear instructional goals, help to pinpoint student weaknesses and strengths, and “to identify gaps in instruction.”

In the following sections we discuss the different aspects of improved instruction that were found such as: increased access for students with disabilities, alignment of teaching with state standards, professional development/awareness of teaching practices, utilizing data to inform instruction, and academic interventions for all students, especially those who historically have not done well in large scale assessments.

Access to General Education Curriculum

Along with increased participation in assessments, it is reported that students with disabilities are gaining increased access to the general education curriculum. Twenty eight percent (N=14) of states report more students with disabilities are accessing the general curriculum (Thompson & Thurlow, 2001). In fact, Pamela McCabe (personal communication, April 9, 2002) of the California Department of Education stated that increased exposure to the general curriculum was the single most important “positive effect” of large-scale testing: “Parents and educators are realizing that students can’t pass the high-stakes test without being given standards-based instruction. It forces educators to examine how to adapt to unique learning styles.”

One unexpected finding concerning access to the general curriculum was the link between participation and accommodation decisions. DeStefano, Shriner, and Lloyd (2001) conducted a study in one district on including students with disabilities in large-scale testing programs. Among other steps taken to help school officials was the modifications to the IEP process in which participation decisions were made first, while curriculum and instructional needs were second. “The change in organization allowed the [accommodation and participation decision–making] process to flow from curriculum to assessment in a manner that addressed student needs in relation to the general curriculum first, before deciding on provisions for assessment and accountability” (p. 11). This is an important finding because the focus moves from complying with the letter of the law in allowing students with disabilities to participate, to looking at the level and extents of general education exposure for students with disabilities. If students with disabilities have little exposure to the general curriculum, this is not a disability or accommodation issue, but an issue of changing curriculum and teaching strategies to help the student learn. Students may still need some form of accommodation to take large-scale tests; yet, their chances for success will be increased because they have had an opportunity to learn the skills that are being assessed. The study cites McDonnell, McLaughlin, and Morison (1997) as saying: “Special educators must ensure that the [general] curriculum is accessible to students [with disabilities] as testing becomes more regulated” (DeStefano, et al., 2001, p. 8).

Alignment with State Standards

One reason for large test failure rates in some states like Arizona (Kossan, November 22, 2000), and one very valid criticism against high stakes testing is that the curriculum that was taught in schools was not aligned with what is tested. When this happens, teachers inadvertently have to focus on test preparation and “teaching to the tests” in order to help their students to pass tests that have some consequences attached to them (whether for the individual students or teachers themselves). Huebert and Hauser (1999) cites Bonds et al. in stating that in the 1990s, many states have had to go back and “revamp” assessment systems to reflect state standards, and the rest of the states were in the process of realignment. Tom Conner of Anne Arundel Schools in Annapolis, Maryland commented in our environmental scan (personal communication, April 12, 2002) that in his district the alignment of curriculum with testing is the best it has ever been. Gloeckler (2001) reported that school districts are realigning their curricula and increasing efforts to ensure that students with disabilities have access to the general education curriculum. Without alignment between the classroom curriculum, state standards, and the assessments, students—especially those with disabilities—would face an unfair and almost impossible challenge to prove what they have learned.

Improved Teaching Strategies

Mehrens (1998, p. 8) cites a Chudsoky and Behuniak report of teacher focus groups that even in the midst of negative perceptions that preparing students for the Connecticut Academic Performance Test “resulted in narrowing of the curriculum,” schools also reported trying to “move beyond direct test preparation into instructional approaches (Mehrens 1998, p. 8). Firestone and his colleagues at Rutgers University completed the first two phases of a three-year study to determine the effects of New Jersey’s testing program on mathematics and science classroom instruction. The researchers also observed 63 classrooms. They found that the state assessment had encouraged teachers to try out more inquiry-oriented instruction in their classrooms, including placing a greater emphasis on problem-solving, having students explain their thought processes, assigning students more writing, and making greater use of hands-on material (Olson, April 18, 2001). Abby B. Bergman, Principal at Ralph S. Maugham School in Tenafly, New Jersey indicated that:

New Jersey’s core curriculum content standards provided the impetuous for us to examine the content and instructional practices that we had been using for years…we connected each of our local objectives to the cumulative progress indicators defined in the state’s standards. As we revised our curriculum, we devised new units and eliminated needless repetition. Our students have performed extremely well on state assessments, but more important, the process stimulated fruitful dialogue among professionals. (Bergman, 2002, p. 80)

In looking at instructional changes, one study reported on an interesting teaching strategy that can help students with disabilities (especially secondary students) access the general curriculum in a large-scale testing environment. The SCREAM method mandates Structure, Clarity, Redundancy, Enthusiasm, Appropriate pace, and Maximized engagement for a curriculum to be made accessible. This technique was reported as being particularly effective in helping students with disabilities develop “…higher-level content knowledge, independent study skills, and [cope with] the pace of general education classroom instruction” (Mastropieri & Scruggs, 2001).

Professional Development

Although at first glance professional development for teachers does not seem to directly benefit students with disabilities, we found evidence to the contrary. Wenglinsky (2002) conducted a study on the link between teacher classroom practices and student performance on the 1996 NAEP. He found that teacher practices did influence student test scores: students of teachers that received professional training on teaching diverse groups of students (including special student populations) “substantially” outperformed students of teachers without such training, and students whose teachers focused on higher-order thinking skills performed at higher levels than students with teachers who focused teaching on lower–order skills (Wenglinsky 2002). Our focus group findings revealed that Minnesota school districts have been sending teachers to workshops that not only teach test preparation, but also how to teach reading, math, and writing within their content areas: “…teacher[s] have had to change their teaching strategies. And in some classes, they’ve had to go for more staff development and more training…on how to teach students, and that’s good because we’re constantly trying to find ways to teach” (urban high school principal, Nelson, 2002, p. 157).

Use of Test Data

The 2001 ESEA (No Child Left Behind Act) includes language indicating that tests should be appropriate both for provision of policy information and provision of data that will enable teachers to plan instructional interventions. To date, professionals have been arguing that this is not possible; that different kinds of tests and assessment activities are appropriate for these different purposes. Yet, we now are starting to see standards reflected in IEPs, and increased efforts to use data from large-scale assessments to plan instruction. Mayo and White (2001) reported ways for schools to make better use of test data. They suggested that item analysis (looking at the percentage of items on a group of questions that teach a particular skill) is the most effective way to use test scores to identify by grade and class areas of weakness and collaborate on strategies to improve those areas. Mayo and White further suggest 11 steps to take to use test scores by involving parents, teachers, and students. In fact, Joan Herman, co-director of the National Center for Research on Evaluation, Standards and Student Testing at the University of California at Los Angeles is quoted in the St. Louis Dispatch as saying: Analyzing test scores to make changes in the classroom “is a trend across the states.” “Used properly, the extra information can help teachers focus on skills students need to improve” (Bower & Hacker, September 02, 2001).

In our focus groups, a Minnesota middle school special education staff person shared how they are using test data (engaging in unit analysis) to improve/inform instruction for students with disabilities: She indicated that “…we are …doing better in math versus in reading so I think that our principal still uses the data to make us aware of when we need to have changes in instructional practice, and certainly, I think, congratulates us when we see things like marked improvement in test scores” (Nelson, 2002, p. 159). Roger A. Stock, principal at Chesterfield Elementary School in St. Louis, says his school is using test data to refine what is taught in the classroom. A year ago, after getting test results for his school, Stock and his teachers came up with areas to focus on in each grade. For instance, teachers used flashcards and phonics to build vocabulary in early grades. The school organized a campaign for students to read a million pages worth of books. The school offered a before-school tutoring program that students could volunteer to attend. Scores in reading and most other subjects jumped across the grades on the spring exams this year (Bower & Hacker, September 2, 2001). In Normandy School District in Missouri, school leaders noticed that scores in the area of science inquiry were lower than hoped for. So they required all students to enter projects in a science fair. Since then, scores in science have risen. One suburban junior high school teacher in a high performing district in Ohio also reported using state test data:

When we get the test back, we [the teachers] looked at categories as far as each of the elements of …writing, and we noticed, for instance, capitalization and punctuation were weak, So by looking at those, we were able to change …not change our curriculum but maybe add some things we know we were not getting at. So we do use them in that way, not maybe individual student results, but the results of the individual areas of the test we look at to see, …how we can change and improve. (Kubow & Debard, 2000, p. 21)

Use of Assessment Accommodations

The legal requirement that students with disabilities participate in large-scale assessment and accountability systems has lead to increased provision of accommodations so that students can demonstrate their knowledge and skills. In surveys (Thompson & Thurlow, 2001) state department of education personnel report increased use of accommodations. Our environmental scan suggested that the better use of accommodations for students with disabilities has allowed more students to take states’ regular assessments (appropriate because most students with disabilities are not severely disabled, therefore, only a small percentage should be eligible for alternate assessments) and to pursue diploma options that were otherwise unavailable to them.

Filbin (2002) credits the use of accommodations for enabling more students to take CSAP and pass, while New York reported on our environmental scan that accommodations allowed more students with disabilities to obtain a standard diploma. An Oregon State Department of Education official reported:

Evidence suggests that improved teaching and learning are occurring in our state as school personnel become more familiar with the many options available to include students with disabilities in statewide assessments. The key to this success is providing assessment options which address instructional levels of students, collaboration between general and special education teachers, and tying service plans to state standards. (personal communication, April 12, 2002)

Mandated Remedial Programs/Academic Interventions

School personnel have been taking many kinds of actions in efforts to raise student test scores. They sometimes change curricula, mandate summer school, and implement remedial programs. In response to failure of students to pass the Ohio state test, the Ohio department of education changed its practice from negative consequences for students to getting them the help they need (Krantz, June 14, 2001). A new law eliminates the grade promotion requirement for students in fourth grade, and mandates that schools provide intensive remedial programs for students who do not pass the fourth grade exams. In New York State a program of academic intervention services (during the school year and during summer) was established for students who do not perform well on the 4th and 8th grade assessments. Students with disabilities are also getting those interventions.

Cenziper (December 17, 2001) reported in the Charlotte Observer that schools in Kannapolis, North Carolina are developing in-school support programs for students who struggle to pass tests. Such programs include Saturday academies, longer summer school sessions, and launching statewide campaigns to attract mentors for needy students. In response to our environmental scan (personal communication, April 12, 2002), education officials in Jordan, Utah reported that administrators responded to student difficulty in passing state tests by providing appropriate assessment and instructional accommodations, layering the curriculum, diversifying instructional delivery systems, implementing multi-level testing, using resource teachers better, and giving general education teachers the tools they need to help them tackle the precise difficulties students experience.

Margaret Byrd (2002), principal of Stovall Middle School in Houston, Texas relates the story of her school’s “alternate route to success”:

When I assumed the principalship at Stovall in 1996, half the student body was not performing at grade level. Less than 70% of the students passed each subject area on the TAAS [Texas Assessment of Academic Skills]…. We initiated a program…in which students who fail the TAAS in reading, math, or both, who have limited English proficiency, or who have a teachers recommendation take special elective classes in reading and math in addition to their regular… classes. The electives are designed for students who need to fill small gaps in their learning as well as those who are several years behind grade level…Students participated in the program until they catch up to grade level and achieve a score of 85% in the courseware (computer program that provides instant and ongoing feedback on students progress). When students graduate from the program, almost all of them have gained two or more years in coursework levels and go on to pass the TAAS. Many students, particularly LEP (English language learners) gain three or four years in the coursework levels. Thanks to the program, the improvement of our lowest performing students has had tremendous effect on Stovall’s overall TAAS results. In 2001, we earned our highest test scores ever…[and] the highest rating a school in the state can receive …and [student] success is long lasting. In my eleven years of experience, no student has ever had to retake the program. (pp. 38-40)

Stovall’s success on Texas’ high stakes exam, as well as its lasting student improvement, can provide convincing evidence that increased test scores are indeed indications of improved learning.

Improved Performance

Since enactment of the reauthorization of the IDEA in 1997 states and districts are required to include students with disabilities in their large scale assessment and accountability systems, and report the scores that all students earn in their accountability reports. There is beginning evidence of improved educational performance, and states are starting to include in their reports information on the performance of students with disabilities.

Gloeckler (2001) documented gains in the performance of students with disabilities on the New York statewide tests. He reported that in 1977 the New York State Board of Regents established Basic Competency Tests, which later evolved into Regents Competency Tests (RCT). All students graduating from a local district had to pass either the Regents examinations in order to receive a Regents diploma or the RCT in order to receive a local diploma. In 1996 the Board of Regents decided to phase out local diplomas and require all students to pass the regents exam due to the perception that students who received local diplomas were unprepared for postsecondary opportunities. In years prior to this decision, 60% of students with disabilities were graduating with a local diploma and less than five percent of students with disabilities were receiving a Regents diploma. Some districts in New York did not even offer the option of taking the regents exam to special education students. By 1999 more students with disabilities were passing the Regents exam than who took it in 1997.

Louisiana Department of Education personnel reported in our environmental scan that there have been increased passing rates for students with disabilities on the state’s high stakes exam from 2000 to 2001: fourth grade math increased by nine percent, fourth grade English by five percent, and eighth grade Math five percent. As in New York, students in Louisiana must take a high school exit exam to receive a state standard diploma and there has been a 3.1% increase of students with disabilities receiving such a diploma (Wartelle, 2002). In Virginia, where passing the state exams will become a requirement for students to receive a diploma beginning in 2004, schools nearly doubled their rate of success on the Standards of Learning exams in 2001, with 40% meeting the state’s benchmarks, up from 23% in 2000 (Seymour, October 17, 2001). Students with disabilities in Virginia also improved their performance modestly from 2000 to 2001 (average gains of three-five percent), with the greatest gains being made on fifth grade English, reading, and writing tests; the fifth grade math test; and high school Algebra tests (Virginia Department of Education Web site, accessed June 13, 2002).

Thompson and Thurlow (2001), in their survey of state directors of special education, indicated that three states (Colorado, Maine, and New York) report improved performance of students with disabilities. In Colorado (which has consequences for local education agencies, but not individual students) it was reported that since 1998, students with disabilities “have demonstrated a continuous increase in performance over time, with fourth grade students improving 107% from 1997 to 2001 (Filbin, 2002). The Denver Post further highlighted the improved performance of students not traditionally expected to do well by reporting that a third of students with disabilities met the state standard, up from just 19 percent in 1998 and for the first time, more than 50% of third-graders in each major ethnic group, including black, Hispanic, and American Indian, were proficient or advanced in reading. Low-income students also improved with more than 50% of students in federally funded Title I programs scoring proficient or advanced in reading, up from 33% in 1998 (Bingham, May 4, 2001). Reading scores for special education students in California have increased steadily since 1998, despite little gains or decreases in performance in the general student population. According to the special education director, Florida also has reported improved performance for students with disabilities on test scores from 1999–2001, with the greatest gains being made on the eighth grade reading test.

School District Gain

In addition to gains in performance noted by state education agencies, many large urban school districts report performance gains for their students. According to a report published in the Urban Educator (“Milwaukee Test Scores Up,” 2001), the performance of Milwaukee students on the 4th, 8th, and 10th grade tests improved for three years in a row. Students in the Baltimore Schools posted the highest gain in eight years in 2001 (“Baltimore Schools,” 2001). In May 2001, the Council of Great City Schools published a report entitled Beating the Odds in which it was indicated that 92% of large, urban school districts had improved math scores in a majority of grades tested, and 80% had improvements in reading. It was reported that half of the schools made faster improvements in math gains than the state average and 34% improved faster in reading than the state average (Casserly, Lewis, Jepson, & Baker, 2001). The math gains were further supported by the fact that there were also increases on ACT and the national NAEP. Four districts in particular were successful in narrowing the racial performance gap: Charlotte-Mecklenburg, North Carolina; Fort Worth, Texas; Houston, Texas; and Miami-Dade, Florida.

Upon further research, we found that the Charlotte-Mecklenburg (North Carolina) school district also showed improvements on test performance by students with disabilities from 1992 to 2001 (with a three percent decrease since 1999 for students with emotional behavioral disorders) (North Carolina Department of Education Web site, accessed June 13, 2002). A five-year study on a suburban North Carolina school district also found increased achievement for students with disabilities (Byrd, 2002). Special education students in suburban Minnesota school districts are also making great gains in passing the Minnesota graduation exam (Draper, May 30, 2000). For example, in the Rosemount-Apple Valley-Eagan School District, only three of the 182 seniors have yet to pass the exit exam. Hartford, Connecticut school officials reported continued year to year gains on the their state’s assessment. The gains have increased year to year despite the fact that the district tested more special education and bilingual students than other state school districts (“Scores Rise, Goal Higher,” 2002).

Although many may take issue with the reported gains (i.e., rises in test scores are products of students becoming more familiar with the test or teachers “teaching to the test”), it is clear that when increases in test scores correspond to increases in scores on national assessments (such as the NAEP and SAT) learning is taking place. Another reason that test gains can be seen as true reflections of learning is by examining what some high-performing districts are doing to improve student learning. If examinations of teacher practices reveal that the curriculum is mostly made up of memorization of basic facts, test prep and practice tests, then any gains on test scores would be questionable. However, as stated in the preceding section, when students receive improved instruction, gains in test scores are inevitable.

Secondary Positive Findings

In the section above we described findings (we called them primary findings) that were reported across our multiple methodologies. We also had a number of findings that came up repeatedly, but not in all methodologies. Consequences included changes in graduation rates, dropout rates, and diploma options; development of better tests and better use of assessments; increased collaboration between general and special educators; and increased parental understanding and awareness of the standards their children needed to meet.

Graduation, Diploma Options and Dropout Rates

One of the intended consequences of requiring participation of students with disabilities in state tests was increased graduation and decreased dropout rate. There is an ongoing debate about calculating dropout rates that is much too complex for this paper to delve into (see Haney, 2000). We will take a conservative standpoint in reporting decreased dropout rates, stating that it can happen, especially when there are alternate routes that students can go to earn a high school diploma. For example, the Chicago Public Schools instituted summer school for failing students and an end to social promotion resulted in a one percent drop in dropout rate over three years (Whitmore, April 14, 2000). Ohio developed multiple ways to exit high school: students can graduate by passing a state exam in 2007, or pass end-of-course exams in a variety of subjects at the end of 10th grade (Krantz, May 14, 2001). In New York, it was reported that graduation rates for students with disabilities also increased (Gloeckler 2001).

Many states have moved to multiple diplomas and allow students with disabilities to meet graduation requirements by meeting their IEP objectives. Some states have changed their requirements in response to high failure rates. The Chicago Schools made an initial proposal that would give high school students the option of an additional year to complete graduation requirements (Quintanilla, January 9, 2002). Some states are moving away from exit exams and instead requiring students to pass end-of-course exams. It is thought that in this way testing will reflect more nearly what is taught. (Olson, June 6, 2001). States have been experimenting with various diploma options. Some have one diploma, others multiple diplomas; some have one diploma with a note about the student having met individualized requirements. One parent in Georgia was successful in getting the Gwinnet County Local School Board to look at a change in the wording and endorsements on diplomas that special education students can receive. The parent, Gene Rowan, wanted the change because special education students have a range of abilities, and their diplomas should reflect that. Instead of having “special education” on the top of the diploma, two new terms, such as “individualized education diploma” or “specialized instructional diploma” were proposed. Special achievements by students with disabilities could also be noted with a seal on the diploma, if the student passes a prerequisite proficiency test (Jones, February 22, 2002).

Better Tests and Use of Assessments

There has been a significant increase in concern about use of performance on a single measure to make promotion or school exit decisions. As concerns are expressed about the kinds of tests used (discussions, for example, of “mere multiple choice measures”), there are moves to develop better tests. Increasingly states are requiring use of multiple measures in decision-making. For example, Minnesota Senator Paul Wellstone introduced legislation to limit the consequences of testing and require use of multiple measures in decision-making. The American Psychological Association Committee on Psychological Tests and Assessments has issued opinions on the meaning of “multiple measures.” Efforts to improve tests have included attempts to align assessments with state standards, increase appropriate use of tests, and provide assessment accommodations.

It is becoming common practice for test companies to standardize (norm) their tests on inclusive populations of students. A company may, for example, include blind students in their standardization and permit them to use accommodations (Braille or large print). By doing so, states and districts can later use the test with students who are blind and derive scores (using the norms) for blind students in the same way they do for sighted students.

Aligning Assessments with State Standards

State Education Agency personnel now are contracting with test publishers to develop state and district assessments. The push for relevance to instruction, or ability to derive instructional applications is leading test publishers to change their test development practices. Also, the legal requirement for participation of students with disabilities in assessments is leading to changes in test development. Following scoring errors by NCS Pearson in 1999 that resulted in 49 students inappropriately denied a Minnesota high school diploma, there was a heavy focus on double and triple checking of student responses. Likewise, following a scoring error by CTB/McGraw Hill in New York City that resulted in 9,000 students sent to summer school who had actually passed, there was a significant push to double check scoring of tests. Both situations led to increased awareness of the need for better tests and better scrutiny of the tests, test makers, test users, and test scorers.

Other test publishers have also scrambled to develop classroom tests that are linked in some way to states’ standards. For example, Renaissance Learning Inc. developed “Standards Master,” a computer-generated paper and pencil test that is referenced to state standards in multiple states, and that gives teachers immediate suggestions on instructional changes to make. The Northwest Educational Association developed the Northwest Achievement Levels Test (NALT) a computer-based measure that uses branching technology along with immediate feedback in assessment of students. The extent to which such tests actually link to standards and assist teachers in aligning instruction with standards is still a topic needing study.

Use of Test Scores

Using test data to inform instruction and pinpoint curriculum strengths and weakness have already been discussed in the previous section under improved instruction. Another positive use of test scores seems to be in placement of students in appropriate class sections. In Fall 2001 school personnel in Charlotte-Mecklenburg schools began placing students who had been placed in lower track classes into higher track ones based on their test scores (Cenziper, 2001). In response to our environmental scan, a Louisiana education official reported as positive consequences for their state’s high-stakes exam: 1997-2000 a 14,000 increase of students with disabilities placed in regular education and initial evaluation decrease by 500, 3,800 decrease in the number of students participating in alternative assessment, 250 decrease of out-of-level testing, and an increase of approximately 2,000 students participating in on-level testing (Wartelle, 2002).

Increased Collaboration Between General and Special Education Teachers and Related Services Personnel

One way collaboration benefits students with disabilities is that special education teachers can be “effective consultants” to general education teachers about the most effective teaching strategies for individual students (Mastropieiri & Scruggs, 2001). DeStefano et al. (2001) conducted a study on improving participation and performance of students with disabilities: They report that collaboration between special and general education teachers is important because “…there is a strong need for special education professionals to be aware of the content of the general curriculum. Also, general education professionals need to be aware of the instructional accommodations that will help students access the general curriculum as much as is appropriate and to make sure that the expectations for students achievement are high and relevant to individual students needs” (p. 14).

Although some Minnesota special educators noted that because at times they are receiving different messages at workshop trainings, they have realized the need to request that they attend the same workshops (not separate ones for regular and special educators) and collaborate on specific IEP goals. They also mentioned the important positive change of everyone within a building being a reading, math, and writing instructor. Everyone takes responsibility for ensuring these basic skills are addressed within their classroom as well.

... in other classes I even find, in social studies, that aren’t math or reading really, but the teachers do things in there and they’ll say, “This is the kind of skill you need to know for the test,” and they’ll have an opportunity to practice it…
(Middle school special educator, Nelson, 2002, p. 42)

Increased communication and collaboration among teachers and related personnel provides consistency to the student curriculum.

Increased Parental Understanding and Awareness of Standards and Student Requirements

Many of the newspaper reports that we scanned included information indicating that parents are developing a better awareness and understanding of state standards and why it is important for students with disabilities to work toward attainment of those standards. Oregon Department of Education personnel reported in response to our environmental scan that they have conducted “massive training” for school personnel and parents on the options for participation for students with disabilities, based on the instructional level of the student. They indicated that “Parents are beginning to understand the options and more importantly the accommodations involved.”

Minnesota parents and special educators also noted increased communication with parents about their student’s progress. These parents are more aware of their child’s math, reading, and writing skill levels than they were before. Special educators shared how parents were much more likely to ask in IEP meetings about their child’s skill levels and how they could improve those skills. For example, one parent of a student with a disability explained:

They showed me where they broke down their learning process into different groups. They pointed out those skills that he was doing the worst and they work on them one at a time and that’s how they got him to pass the test during the summer months. (Nelson, 2002, p. 82)

Parents are asking better questions regarding skill levels, standards’ requirements, and how to help improve their child’s learning and success. This increased parent involvement can only lead to more positive consequences for students.

Changes in Newspaper Headlines

Over the past 10 years there have been significant changes in newspaper headlines for articles about the performance of students with disabilities on state tests. Gloeckler (2001) cites these:

“A Ray of Hope in Disabled Kids’ Tests.” Rochester Democrat and Chronicle, April 5, 2000

“More special education students taking and passing the regents exam.” New York Times, April 5, 2000.

“Regents: Special Education Students Rising to Higher Standards.” Associated Press, April 23, 2001

“Special Needs Students Scoring Higher on Tests.” Times Union, Albany, April 24, 2001

The changes in headlines are important because public discussion concerning students with disabilities and high stakes testing is moving away from focus on the process and debating the rights and wrongs of standardized testing to what can be done and is being done to help children achieve. As Gloeckler summarizes:

The value of this effort should be judged by the extent to which students with disabilities, by virtue of their educational experience, grow to be independent adults, participating and competing for all that life has to offer. That end will not be obtainable for all these students, but it is way past time to realize that most have a real chance, if we open the door of opportunity to everyone. (Gloeckler, 2001, p. 25)

Conclusions

We used multiple methodologies in an effort to identify the positive consequences of large-scale assessment and accountability systems for students with disabilities. The ultimate positive consequence, of course, will be improved performance on tests and increased numbers of students with disabilities meeting state standards. There is evidence that this is happening. As we noted in our introduction, however, the preponderance of newspaper reports of student performance on tests, especially performance of students with disabilities, is negative and the empirical evidence on large scale is assessment is scarce. Yet, as we have noted throughout this paper, the evidence that does exist points to some correlational information about the intended and unintended consequences for high-stakes testing. Many of the intended positive consequences envisioned by lawmakers are happening, and there are also a number of unintended positive consequences.

The methodologies we used rely heavily on anecdotal and secondary information. Yet, when multiple sources of data are used, and when they provide consistent evidence of positive consequences, we can be reasonably sure we have a good starting point for further research. For example, Lane and Stone (2002) have suggested the use of multiple methodologies that focus on what stakeholders (students, parents, teachers, administrators) have to say about or perceive consequences of large-scale assessments. Using this type of research framework for any consequence reported, such as improved instruction, has the potential of not only determining effectiveness of the assessment program, but help to develop a body of much needed empirical evidence. We have identified many positive consequences of large-scale assessments for students with disabilities, and we encourage others to join us in our continued efforts to track both positive and negative consequences.

References

*References marked with an asterisk can be found using the Lexis-Nexis Search Engine at: http://www.lexisnexis.com. All newspaper articles were retrieved electronically unless otherwise stated.

Altheide, D. L. (1996). Qualitative Media Analysis (one ed. Vol. 38). London: Sage Publications.

Baltimore schools post largest test-score gains in eight years. (2001). The Urban Educator, 10(1). Retrieved July 9, 2002 from http://www.cgcs.org/urbaneducator/january-vol_10_no1_article12.html.

Barksdale-Ladd, M. A., & Thomas, K. (2000). What’s at stake in high-stakes testing: Teachers and parents speak out. Journal of Teacher Education, 51(5), 384-397.
Bergman, A. B. (2002, April 2002). Having our say on...standards. Principal Leadership, 2, 80.

*Bingham, J. (2001, May 4). Grade 3 reading results. Students’ scores best yet. Three quarters meet or exceed state standard. The Denver Post, pp. A-01.

*Bower, C., & Hacker, H. K. (2001, September 2). Schools are using specific scoring data to gauge areas for improvements. The St. Louis Post-Dispatch.
Byrd, M. R. (2002). An alternate route to success. Principal Leadership (Middle School Ed.), 2(8), 38-40.

Casserly, M., Lewis, S., Jepson, J., & Baker, N. (2001, May). Beating the odds: A city-by-city analysis of student performance and achievement gaps on state assessments. Washington: Council of the Great City Schools.

Cenziper, D. (2001, December 17). Schools cry out for state’s help. The Charlotte Observer. Available from: www.charlotte.com/mld/observer/.

Charlotte-Mecklenburg test scores for students with disabilities. North Carolina State Education Web site: http://www.dpi.state.nc.us/ accessed June 13, 2002.

Cimbricz, S. (2002). State-mandated testing and teacher’s beliefs and practice. Education Policy Analysis, 10(2), 1-21. Retrieved June 23, 2002 from http://epaa.asu.edu/epaa/v2010n2002.html.

Cizek, G. (2001). More Intended Consequences of high-stakes testing. Educational Measurement: Issues and Practice, 20(4), 19-27.

DeStefano, L., Shriner, J. G., & Lloyd, C. A. (2001). Teacher decision making in participation of students with disabilities in large-scale assessment. Exceptional Children, 68 (1), 7-22.

*Draper, N. (2000, May 30). Suburban special-ed kids make gains on skills tests; Fears are proving unfounded that learning-disabled students would have trouble passing. The Star Tribune.

Erickson, R., Thurlow, M., & Ysseldyke, J. (1996). Neglected numerators, drifting denominators, and fractured fractions: Determining participation rates for students with disabilities in statewide assessment programs (Synthesis Report No. 23). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/synthesis23.html.

Filbin, J. (2002). No more field trips on test days: How students with disabilities benefit from the CSAP. Inside Special Education, 3(2), 2. Retrieved July, 15,2002 from http://www.cde.state.co.us/cdesped/download/pdf/INSIDE_Fall2001.pdf.

Gloeckler, L. C. (2001, Autumn). The door of opportunity. The State Education Standard, 2, 21-25.

Haney, W. (2000). The myth of the Texas miracle in education. Education Policy Analysis Archives, 8(41).

Heubert, J. P., & Hauser, R. M. (Eds.). (1999). High stakes: Testing for tracking, promotion, and graduation. Washington, DC: National Academy Press.
Holloway, J., & Pearlman, M. (2001, January). Reform through standards: the ultimate challenge. Principal Leadership (High School Ed.), 1, 40-44.

Jones, A. (2002, February 22). Special ed diplomas under review. The Atlantic Journal/Constitution, p.1JJ.

Kossan, P. (2000, November 22). Keegan Backs off AIMS requirement. Arizona Republic. http://www.azcentral.com/.

Krantz, C. (2001, June 14). Ohio changes testing focus. The Cincinnati Enquirer. Available from: http://enquirer.com/today/.

Kubow, P. K., & DeBard, R. (2000). Teacher perceptions of proficiency testing: A winning Ohio suburban school district expresses itself. American Secondary Education, 29(2), 16-25.

Lane, S., & Stone, C. A. (2002). Strategies for examining the consequences of assessment and accountability programs. Educational Measurement: Issues and Practice.

Mastropieri, M. A., & Scruggs, T. E. (2001). Promoting inclusion in secondary classrooms. Learning Disability Quarterly, 24(Fall 2001), 265-274.

Mayo, R., & White, G. P. (2001, January). Putting test results to work. Principal Leadership (High School Ed.), 1, 36-39.

McDonnell, L. M., McLaughlin, M. J., & Morison, P. (Eds.). (1997). Educating one and all: Students with disabilities and standards based reform. Washington, DC: National Academy Press.

Mehrens, W. A. (1998). Consequences of assessment: What is the evidence? Education Policy Analysis Archives. 6(13).1-30. Retrieved June 3, 2002, from http://epaa.asu.edu/epaa/v6n13.html

Milwaukee test scores up. (2001). The Urban Educator, Vol. 10,(2). Retrieved July 9, 2002 from http://www.cgcs.org/urbaneducator/.

Nelson, R. (2002). Closing or widening the gap of inequality?: The intended and unintended consequences of Minnesota’s Basic Standards Tests for students with disabilities. Unpublished Doctoral Dissertation, University of Minnesota, Minneapolis.

Olson, L. (2001, April 18). Researchers identify the impact of N.J. testing on teaching. Education Week. Available from http://www.edweek.org/.

Olson, L. (2001, June 6). States turn to end-of-course tests to bolster high school curriculum. Education Week. Available from http://www.edweek.org/.

*Quintanilla, R. (2002, January 9). Five-year high school idea is back: Daley endorses option to trim rate of dropouts. The Chicago Tribune.

Schulte, A., Villwock, D. V., Whichard, S. M., & Stallings, C. (2001). High-stakes and expected progress standards for students with learning disabilities: A five-year study of one district. School Psychology Review, 30(4), 487-506.

Scores rise, goal higher. (2002, February 7). The Hartford Currant, p. A12.
*Seymour, L. (2001, October 17). Schools making headway on SOL: Va. passing rate nearly doubles. The Washington Post.

Shepard, L. (1991, April). Effects of high stakes testing on instruction (Paper). Washington, DC: Spencer Foundation U.S. Dept. of Education, Office of Educational Resources Information Center.

*Thevnot, B. (2000, August 23). Special education waiver for LEAP backed, BESES may extend exemption one year. The Times-Picayune, p. 1.

Thompson, S., & Thurlow, M. (2001). 2001 State special education outcomes: A report on state activities at the beginning of a new decade. Minneapolis, MN University of Minneapolis: National Center on Educational Outcomes. Available at http://education.umn.edu/nceo/OnlinePubs/2001StateReport.html.

*Vaishnav, A. (2002, May 12). Some see crisis for special ed pupils on MCAS. The Boston Globe, p. B1.

Virginia 2000-2001 Sol scores. (2000-2001). Virginia Department of Education. June 13, 2002. Retrieved from http://www.pen.k12.va.us/VDOE/Assessment/home.shtml.

Wartelle, N. (2002). Louisiana-Students with disabilities in high stakes assessment. Unpublished manuscript, New Orleans.

Wenglinsky, H. (2002). How schools matter: The link between teacher classroom practices and student academic performance. Education Policy Analysis, 10 (12). Retrieved June 3, 2002 from epaa.asu.edu/epaa/v10n12/.

Whitmore, R. (2000, April 14). Exit exams blamed for rise in dropouts: Minorities most affected by push for stricter testing in Texas. The Detroit News. Retrieved July 9,2002 from http://detnews.com/2000/nation/0004/14/a12-36798.htm.