Synthesis Report 62

J. Ruth Nelson
Bethel University

This present study examined the intended and unintended consequences of Minnesota’s high stakes graduation exam on students with disabilities. Historically, little empirical data have been collected and the scant data available suggest some significant unintended consequences for educational accountability systems (e.g., the retention of students and increased referrals to special education). Fifty-two parents of students with disabilities and 39 special educators from six schools in two large urban and suburban school districts participated in focus groups. I also conducted a focus group with the Minnesota Association of Educational Assessment and Evaluation (MAEAE) as well as 15 interviews with building administrators, school district representatives, and a state representative. The results show that the Minnesota Basic Standards Tests have had some very positive and negative consequences for students with disabilities, including increased exposure to the curriculum, increased participation in testing, raised expectations, high levels of anxiety and frustration, and drop-out concerns. There did not seem to be any retention as perceived by the consumers nor an increase in special education referrals, although many times the referral data were not collected in a systematic manner. Further studies are needed to document empirically whether these consequences are occurring for students with disabilities.

Overview

Personnel in two midwestern school districts were dismayed to learn that nearly 40% of their students had failed the recently implemented basic skills exit exam in their state. Performance on this state-created test is linked to graduation from high school. In efforts to improve students’ low performance, one district required failing students to attend summer school (like many other school districts are doing) and retake the exam at the end of the summer. Schools in this district took the results of their high stakes exams and moved forward to provide remediation with the hope that all students, including students with disabilities, would be able to pass and graduate. But is this enough? Who is thinking about and documenting the consequences? Clearly, there are intended consequences. Are they achieved? Are there other unintended consequences for students, especially students with disabilities? Do bad things happen when good things are done? What happens to students with disabilities when they do not pass high stakes graduation exams?

In this report, I briefly describe the historical context of graduation exams, discuss why the participation of students with disabilities in such exams and larger accountability systems is critical, and review the literature on the current status and outcomes of high stakes assessment for students with disabilities. Finally, I present new data on the intended and unintended effects of an exit exam for students with disabilities.

Graduation Exams

Graduation exams have been a part of American education since the late 1960s and early 1970s, when several states implemented minimum competency testing as a partial requirement for high school graduation. Florida was one such state that was pulled into court and made to defend its testing program. In Debra P. v. Turlington (1981), the courts mandated that students have a recognized property interest in receiving a high school diploma, and so these tests should measure what students have been taught, and there must be at least four years of advance notice of the high‑stakes test requirement. For students with disabilities and the lack of coordination between special education resource rooms and the classroom, the implementation of a graduation exam may well serve to widen the gap of successful life outcomes between these students and regular education students.

According to a survey by Guy, Shin, Lee, and Thurlow (1999), 20 states had high school exit exams. In nearly all of the 20 states with an exit exam, students with disabilities were allowed multiple opportunities to take the exam. Four of the 20 states allowed modifications to the exam requirements for students with disabilities (Minnesota, New Jersey, Ohio, Texas), and Minnesota was the only state that allowed students to be exempted from the test and still able to receive a standard diploma (1999). Texas was the only state in which students with disabilities who were exempted from the graduation exam were required to participate in another assessment. Those states that required a graduation exam generally had more exit documents available to students, but, on the other hand, were more stringent in how students with disabilities could earn a standard diploma (1999). Since then, 22 states added a required graduation exam to their curriculum, and five are in the process of piloting such an exam (Olson, Jones, & Bond, 2001). The growing trend is to require such exit exams that are contingent upon receiving a diploma.

However, there is also quite a bit of flexibility for students with disabilities. Of the 27 states with only course credit requirements for graduation, 20 allowed their students with disabilities to meet the requirements by taking modified coursework or completing IEPs or by having IEP teams or local educational agencies (LEAs) decide the requirements. Of the 19 states that required both credit and exams for graduation, 12 allowed changes in requirements for students with disabilities to earn a standard diploma. Those students who passed their exit exam were all eligible for a standard diploma. Nearly one third of the states had either changed the number of options available to students with disabilities, with more options available, or had changed the standard diploma requirement since Thurlow, Ysseldyke, and Anderson’s 1995 study of graduation requirements for students with disabilities.

Graduation tests are the most popular type of individual accountability mechanism aimed at students and are also “high‑stakes” (Olson, Jones, & Bond, 2001). Individual student accountability is applied as the student must take a test that measures whether he or she has mastered the essential basic skills necessary of a high school graduate. Although nine states use tests that are considered to measure minimum competency (based on 9th grade or lower standards) and most allow an unlimited number of chances to take the exam (Bond & King, 1995), there is very little research that addresses the consequences of graduation testing, especially for students with disabilities. One study by Kreitzer, Madaus, and Haney (1989) compared the 10 states with the highest dropout rates and the 10 states with the lowest dropout rates. They found that 9 of the 10 states with the highest dropout rates had high‑stakes graduation tests, and none of the states with low dropout rates used their tests for high‑stakes purposes.

The limitation in the above research on high‑stakes testing and dropout rates is the lack of evidence that supports a causal relationship or more direct link than mere associations. Further, the researchers did not specifically study the effects for students with disabilities. Reardon (1996) showed with National Educational Longitudinal Study (NELS) data that schools most likely to have high‑stakes testing policies were those with high concentrations of students with low social-economic status (SES). Other groups, such as African‑Americans, Hispanics and English language learners, and low SES students are overrepresented in schools in which high‑stakes tests are given and tend to fail these tests at a higher rate than high‑SES and white students (Ecklund, 1980). This leads one to consider what the impact is for students with disabilities who also are members of those above groups. Are students with disabilities among those students who may be encouraged to drop out as a result of their performance on high school graduation exams?

This is distressing as Hauser (1997) provided evidence that failing to complete high school, whether due to graduation tests or other reasons, is increasingly associated with problems in employment, earnings, family formation and stability, civic participation, and health. To compound matters, the earning power of high school dropouts has significantly fallen relative to that of high school graduates. For example, Bishop and Mane (2001) looked at two nationally representative data sets—the High School and Beyond (HSB) seniors of 1980 and the NELS students graduating in 1992. They analyzed the effects of Minimum Competency Exam (MCE) high schools on a person’s earnings (controlling for quality of the high school, individual’s academic achievement through test scores, grade point average, participation in extracurricular activities, and an indicator for taking remedial courses in either math or English). They found that students who graduated from MCE high schools obtained significantly higher paying jobs and kept their pay advantage for the next five years. Students from low socioeconomic backgrounds who graduated from an MCE high school earned $694 extra, which amounted to more than a 10 percent increase in comparison to those from non-MCE high schools (2001).

National Outcomes for Students with Disabilities

There is some data from the few studies of students with disabilities’ outcomes that have not been satisfactory (Rossi, Herting, & Wolman, 1997; Wagner, D’Amico, Marder, Newman, & Blackorby, 1992; Wagner, Newman, D’Amico, Jay, Butler‑Nalin, Marder, & Cox; 1991). Limited data are available on the results of outcomes for students with disabilities. The few reports have presented a bleak picture. Most of these reports are from special government studies rather than on‑going data collection programs.

In the mid‑1980s Congress mandated a longitudinal study of students with disabilities. Wagner et al. (1991) found that only 15% of students with disabilities attended a post‑secondary school one year after high school, 30% had not held a paid job, 40% of those employed only worked part‑time, one in five overall had been arrested, and nearly 40% of youth left school by dropping out. After three to five years, about 25% of these same youth were enrolled in post‑secondary vocational schools or 2‑year or 4‑year colleges (Wagner et al., 1992). More recent analyses of the National Education Longitudinal Study (NELS) of 1988 have shown that students identified by teachers and parents as having a disability earned lower high school grades in core courses, scored lower on math and reading proficiency tests, and were more likely to drop out of school than their counterparts (Rossi, Herting, & Wolman, 1997). These students also had lower educational expectations for themselves and by their parents. These outcomes should motivate researchers to study the outcomes of students with disabilities more carefully, especially in the context of graduation exams and the larger frame of accountability systems.

How Do States Now Account for Educational Results of Students with Disabilities?

In the past, students with disabilities were excluded from the general curriculum, state and district assessments, and accountability systems (Elliott & Thurlow, 1997; Erickson & Thurlow, 1997; Roach & Raber, 1997). A recent study confirms that some teachers are still being encouraged to not include “certain children” in the test because “it would lower our school average” (Barksdale-Ladd & Thomas, 2000, p. 392), and it is allowable in this state’s test guidelines to exempt special education students from having to take the state test.

In a telling study Thurlow, Ysseldyke, Gutman, and Geenen (1998) report that nearly all states related that their standards were for all children in the state, yet few states defined all to include students with disabilities. Furthermore, when Rhim and McLaughlin (1997) interviewed state officials about whether or not any of the content standards would apply to students with disabilities, 35 states reported that their standards would apply while nine states’ standards would not.

Yet it seems the exclusion of students with disabilities from assessments or accountability systems has in general decreased in recent years. In 1993, only 28 states had formal policies on the participation of students with disabilities in statewide assessments and only 21 states had written policies on accommodations (Thurlow, Ysseldyke, & Silverstein, 1993). In the most recent update of this study, nearly every state (excluding Nebraska who at the time did not have a statewide assessment and Iowa in which districts gather their own data and set their own policies by district) had policies on the participation and accommodation of students with disabilities in statewide assessments (Thurlow, House, Boys, Scott, & Ysseldyke, 2000). Although these numbers look promising, the states vary widely in what they permit as accommodations and some are more flexible than others. Researchers also found that in nearly every state the decision to participate is first determined by the IEP team, but the second most frequent criterion is whether the student had access to the course content.

Recent changes have been made to the participation criteria of the National Assessment of Educational Progress (NAEP) to include more students with disabilities and to offer various accommodations to students with special needs (Ingels & Quinn, 1996; Olson & Goldstein, 1996). Prior to 1995, students with disabilities often were excluded from NAEP for several reasons, including erroneous beliefs that these students could not participate meaningfully or no test accommodations or adaptations were available.

Yet having active policies does not mean that students with disabilities’ scores are included in the scores that are released to the public (Zlatos, 1994). Two Florida elementary schools appealed their rankings because several students with disabilities were “mistakenly” included in their test scores (Gainesville Sun, October 2, 2000, p. B1 as cited in Allington, in press). Even if states may be including more students with disabilities in their assessments, they still may not be publicly reporting student results. Researchers at the National Center on Educational Outcomes (NCEO) reported that only 16 states reported disaggregated test‑based outcome data for students with disabilities in their annual state educational accountability documents (Thurlow, Nelson, Teelucksingh, & Ysseldyke, 2000).

Few Data Available

We know some things about the consequences of educational accountability systems for state education agencies, school districts, individual schools, administrators, teachers, parents, and students. However, even fewer data exist on consequences for students with disabilities. This is partially due to the dynamic and highly politicized nature of statewide assessments and educational accountability systems and the lack of attention to students with disabilities.

Now with federal mandates (e.g., IDEA ’97; IASA ‘94) in place that require the reporting of results for students with disabilities, there is a need to track the consequences of these educational accountability reforms. Lane, Park, and Stone (1998) suggest that one should evaluate the intended and unintended consequences of statewide educational accountability and assessment systems. These systems are designed to have an effect on the following: the implemented curriculum; the instructional content and strategies; the content and format of classroom assessments; student, teacher and administrator motivation and effort; the improvement of learning for all students; the nature of professional development support; student, teacher, administrator, and public awareness and beliefs about assessment and student performance; the use of assessment results; and the use and nature of test preparation materials (Frederiksen & Collins, 1989; Koretz, Barron, Mitchell, & Stecher, 1996; Linn, 1993; Messick, 1992).

However, there may also be unintended consequences such as the following: the narrowing of the curriculum and instruction to only focus on the specific learning outcomes assessed; ignoring the broader construct reflected in the specified learning outcomes; the use of test preparation materials that are closely linked to the assessment without making changes to the curriculum and instruction; using questionable test preparation materials (e.g., secure assessment items); finding differential test performance for subgroups of students; inappropriate or unfair uses of test scores; failure to develop higher order thinking skills; and tracking which can lead to lowered achievement (Darling‑Hammond & Wise, 1985; Haladyna, Nolan, & Hass, 1991; Lane et al., 1998; Mehrens, 1998; McNeil & Valenzuela, 2001; National Research Council, 1999; O’Day & Smith, 1993; Rothman, 1996; Shepard & Dougherty, 1991; Texas Education Agency, 1995).

Other researchers (Koretz & Barron, 1998; Neill & Gayler 2001) agree that high-stakes testing does not necessarily equate with improved learning, at least as measured by NAEP and the American College Testing Program (ACT) college admission tests. States without a mandatory high school graduation exam were more likely than those with a test to make gains, both in the percentage of students reaching the basic level of proficiency and the proficient and advanced levels at the eighth grade on the 1992 and 1996 NAEP (2001). Although it is voluntary to participate in the NAEP, this type of evidence seems to indicate that high-stakes testing is not a significant factor to educational gains by students as measured by the NAEP.

On the other hand, in an in-depth case study of a single school district responding to a newly implemented high stakes graduation exam, Schleisman (1999) found that testing policy helped to promote greater curricular coherence to the system. Instead of narrowing the curriculum as suggested by some in the case of Texas (McNeil & Valenzuela, 2001), this exam appeared to increase students’ exposure to similar curriculum across the district and highlighted those students who needed additional help. For example, this district adopted a new math curriculum for the middle and high schools that “reinforce[d] the eight strands of the MBST [Minnesota Basic Standards Test—high school graduation exam] math portion” (p. 10). They also adopted a new K-6 reading series that all elementary schools began using in the fall of 1999. One administrator also mentioned that they needed to continue to align the curriculum and daily teaching across the district in both math and reading, implying that all students would receive a common curriculum (1999).

Texas has been hailed as a leader in its educational reforms, especially with its claim that they have narrowed the gap between students of color and whites. However, Haney argued in the court case GI Forum Image De Tejas et al. v. Texas Education Agency, 87 F. Supp. 667 (W.D. Tex. 2000) and did further empirical study (2000) that he felt showed that this gap only narrowed because large numbers of students were retained in grade 9 and many left school before taking the 10th grade exit exam. RAND researchers examined NAEP performance in 1994 and 1998 and Texas gains on the TAAS (Klein, Hamilton, McCaffrey, & Stecher, 2000). They questioned the validity of their gains because the average test score gains on the NAEP in Texas, except for fourth grade math, were comparable to those experienced nationwide during the same time period. Further, Linton (2000) studied TAAS data from four years to discover why the 1999 TAAS passing rate did not decrease as expected as the Texas Education Agency began to include students with disabilities’ TAAS results. The passing rates remained constant in reading and increased in math and writing, but the percent of special education students receiving exemptions from the TAAS increased significantly. In fact, across the state, almost two-thirds of the African-American students and one-half of the Hispanic students in special education were exempted from the TAAS while less than 40% of white special education students received exemptions (2000). It appears that increasing passing scores on a statewide test do not guarantee an increase in students’ knowledge base. Further, with the pressure to include students with disabilities, these students may be pushed out of the testing picture through exemption, retention, or even drop-out.

Limitations of Previous Research

Previous research has failed to take into account students with disabilities. Some research has been published on teacher and parent perceptions of high stakes testing (Barksdale-Ladd & Thomas, 2000), but very few studies are available on the outcomes for students with disabilities, and even fewer studies relate the unintended and intended consequences of students with disabilities to educational accountability systems in place today. Some researchers have begun to explore the effects or consequences of these educational accountability systems for general education, but fewer (e.g., Allington & McGill‑Franzen, 1992) actually document the unintended consequences of these reforms for students with disabilities including retention and overidentification of students with disabilities. This is partially due to the dynamic and highly politicized nature of statewide assessments and educational accountability systems (e.g., the Minnesota state legislature went back and forth about requiring a Profile of Learning for their students). Even Allington and McGill‑Franzen’s work is limited to the state of New York which had state‑specific policies that may have contributed to the observed consequences. Studies on tracking, retention, social promotion, and graduation have begun to show negative consequences for low SES and minority youth. What are the consequences for students with disabilities?

Studies done on over-identification for special education services in connection with high stakes assessment are few. Minnesota is one such state in which researchers found that many school districts did not collect the data in a similar manner from school to school (Minnema, Thompson, Thurlow, & Barrow, 2000). For those few districts with some data on high school referrals, there did not appear to be a trend toward increased referral for services (Minnema et al., 2000), yet the first class which had to pass the Minnesota Basic Standards Tests had not yet graduated at the time of their study, and Minnesota is the only state that allows a modified passing score for students with disabilities.

In 1996, the Minnesota State Legislature mandated that all students must take a BST in 8th grade in order to graduate from high school. A voluntary pilot run was completed during the 1995–1996 school year, and five years with numerous testing sessions have since occurred. Nearly 70% of regular education students passed the Reading Test in 1996, while only 24% of students with disabilities passed the same exam (Thurlow, Albus, Spicuzza, & Thompson, 1998). At the same time, 38% of students with disabilities passed the Math Test while 83% of students without disabilities passed (1998). However, these results only reflect the results of 70% of students with disabilities. Of the most recent data available on the MBST results of students with disabilities, Thompson, Thurlow, and Spicuzza (2000) found that over 90% of eighth grade students with disabilities had been included in the Spring 1999 testing session. As the required percentage correct has risen (e.g., from 70% to 75%) and participation has increased, students with disabilities’ scores have actually improved in reading (33% on the Reading Test), but have fallen in math (27% on the Math Test) (Thompson et al., 2000). Over one-third of all students who participated in summer school in 2001 passed the MBSTs in reading, math, and writing (Gray, 2001). More recent scores compiled by the Minnesota Department of Children, Families and Learning (CFL) (2001) include students with disabilities’ scores in the aggregate state and district scores, but do not specifically disaggregate their results.

Though the math scores from 1996 through 1999 may seem discouraging, it is important to acknowledge the work of Ysseldyke and Bielinski (2002) that shows that these scores may be affected by changes in classification and may not be the most accurate reflection of the academic progress of students with disabilities. Overall, however, their performance is considerably lower than students without disabilities, and yet they have the potential to be just as successful on these exams. With these data in mind, it is an opportune time to document the perceived intended and unintended consequences of the MBST for students with disabilities in the state of Minnesota.

Purpose and Research Questions

The purpose of this study was to document perceived intended and unintended consequences of the MBSTs for students with disabilities through focus group and interview methodology in two large school districts, an urban site and a suburban site. In completing this study, I addressed the following research questions:

• What do principals, parents, teachers, and district personnel observe as intended and unintended consequences of the Minnesota Basic Standards Tests for students with disabilities?
• What are the observed consequences for special education services? To what extent are there increased referrals for special education services? Is there more retention of students (with disabilities) from grade to grade?

Study Design

Conceptual Framework

The conceptual framework for this study of intended and unintended outcomes is based on Stake’s countenance model of evaluation (Stake, 1967), which calls for documentation of antecedent conditions, transactions, and outcomes (See Figure 1 for Stake’s model adapted to the current proposed study). Stake suggests that evaluators document both intended and observed antecedent conditions, transactions, and outcomes. For this particular study, I focused on observed event(s) or outcomes in the form of intended and unintended consequences as perceived by parents of students with disabilities, special educators, and school district and state department personnel. As Minnesota’s Basic Standards Tests have been in place for the past five years, I will not be looking at the “Anticipated Event(s)” construct of Stake’s model.

Figure 1: Adaptation of Stake’s Model

 
For purposes of this study, I documented perceived intended and unintended outcomes for students with disabilities through focus groups and interviews in two large Minnesota school districts.

Data Gathering Tools

I completed an exploratory study of key stakeholders’ perceptions of the anticipated and observed (intended and unintended) consequences of including students with disabilities in a developing educational accountability system (i.e., Minnesota). Measures and analysis procedures for each research question are shown in Table 1. Both measures are described briefly here. I completed 20 focus groups with special education teachers (N=6) and parents of students with disabilities (N=14) as well as 15 interviews with school administrators, district representatives, and state representatives (see Table 2). Tables 2 and 3 outline the number of sites and respondents in this study. Data were collected in one school district from 1999-2000, and in a second school district from 2000-2001..

Table 1. Research Questions and Methodology
 

Table 2. Number of Focus Groups & Interviews by Site

Table 3. Number of Respondents in Minnesota

Focus groups. To address the three research questions, one hour focus groups were held with 10–12 participants (e.g., teachers and parents of students with and without disabilities). I obtained approval from the district Human Subjects’ Committees (as well as the University of Minnesota’s Human Subjects Committee), and then obtained cooperation from three principals in each district. I completed at least two parent focus groups for every school included in the study (six schools) for a total of 12 parent focus groups. Two more focus groups were conducted for sites in the suburban district for a total of 14 parent focus groups because of low attendance across the first two parent groups. One special education teacher focus group was completed for each school building, usually at an early morning or after school staffing (total of six focus groups). I also completed a focus group with seven members of the Minnesota Association of Educational Assessment and Evaluation. These focus groups occurred in two middle schools and four high schools in the school districts. Parents and special educators were reimbursed $20 for their participation in a one-hour focus group (AEAE members were not reimbursed for their time). Part of these funds were provided by a University of Minnesota doctoral dissertation grant.

Interviews. I conducted and tape recorded one‑hour interviews with every school building principal and one assistant principal who was in charge of the testing in the high school building (seven administrator interviews with two school districts), the district directors of research and assessment (2), the district directors of special education (2), the district directors of curriculum and instruction (2), and state department personnel (2). The state department personnel included an individual who no longer worked for the state, but was a state trainer in administering the MBST. The other individual was an Educational Specialist whose responsibilities included coordination of the Comprehensive System of Personnel Development or the system that mandates staff development for special education teachers. This person had been in the position for 12 years, and served as the point of contact for all of graduation standards requirements, including Minnesota’s Basic Standards Tests. This CFL official was recommended by the Director of the National Center on Educational Outcomes, Dr. Martha Thurlow, who in the past had worked closely with this individual on a subgrant research project pertaining to the MBST.

I mailed or faxed a two‑page abstract of the study’s purpose and design to invite principals of middle and high schools to participate, and followed up with a phone call to each principal.

Data Management and Analysis

While conducting interviews and focus groups, I listened for inconsistent comments and offered a summary of key questions and statements in order to seek confirmation. I tape recorded each focus group session (except for one due to investigator error) as well as each interview. Immediately after each focus group, we drew a diagram of seating arrangements, spot checked the tape recording, and labeled any field notes taken. Only one tape was difficult to hear; all other tapes were transcribable. A typist was hired to transcribe the majority of the large number of data tapes. I reviewed each transcript checking for major errors. When all focus groups and interviews had been completed, I coded the data into categories through the help of qualitative analysis software—QSR N5 (formerly known as Nu Dist). As instructed by Miles & Huberman (1994) and Krueger & Casey (2000), I first coded the data, organized it into an accessible, compact form to facilitate analysis, and drew conclusions by looking for emerging themes by research questions. The categorized list of consequences was examined for big ideas, internal consistency, frequency or extensiveness, intensity, and specificity of comments. Some of the ideas were only mentioned by one or two people, but if it was said with intensity and great emotion, I made a note of it as instructed by Krueger and Casey (2000). Analysis procedures for each research question are listed in Table 1. Written reports with illustrative quotations for each research question were prepared.

Reliability Study

Three research associates and a research assistant from the National Center on Educational Outcomes of the University of Minnesota completed a reliability check in August, 2001, after the principal investigator had completed two months of data analysis. These evaluators were given quotations categorized into various subgroup topics (e.g., participation and accommodation, exposure to curriculum, referral for special education services, tracking, high expectations, IEP, anxiety and stress, frustration and drop-out.). Evaluators were asked to compose their own themes after reading the quotations by parents of students with disabilities, special educators, administrators, and directors of assessment. Percent agreement was calculated to determine the extent to which generated themes were defined by the outside researchers. The percent agreement was 92%, suggesting a high level of inter rater agreement.

Results

The perceived consequences of the MBSTs for students with disabilities included: curriculum changes; participation and accommodation use; performance results; a growing seriousness among students; logistics of testing; frustration by students with disabilities; and long-term opportunities for students with disabilities (see Table 4 for frequency counts of these perceived consequences). Among the chief perceived consequences, all groups discussed the perceived impact on formal and informal curriculum opportunities.

Test-Driven Curriculum

There was consensus among parents of students with disabilities, special education staff, principals, and members of the AEAE that there has been a concerted effort to revise, redesign or add to curriculum for students with disabilities, whether it be formal opportunities (remediation curriculum) or informal opportunities (test preparation activities, tutoring). As several special education staff and administrators reported, these basic skills of reading, mathematics, and writing are being worked on in all curriculum areas if they weren’t before. Student results have spurred on new learning opportunities and helped to raise expectations for students with disabilities.

Test-driven curriculum demanded consistency across staff. One staff put together a “SamePage booklet” that listed concepts and ideas that every teacher in the middle school building had to follow when they taught certain things (such as the writing process, editing symbols, how to complete group work, and doing research).

Table 4. Frequency Counts of Perceived Consequences of the MBST for Students with Disabilities (SWD)

Reading is now being explicitly taught at the upper grade levels. However, one member of the AEAE discovered that for English language learners and students with disabilities, there may be some confusion as to who is actually teaching them reading. The language arts department may be teaching literature, and another department may assume that reading is being taught by the language arts department. Some members of the AEAE thought that the MBST helps to highlight missing instruction and to correct it. However, this director of assessment was very concerned because this lack of coordination of instruction was happening in her district for both of those special populations even with all the extra attention given to reading preparation.

Developing Skills Through More Opportunities

Some teachers have perceived that they have become more creative in providing additional opportunities for students to develop and hone their reading, writing, and math skills. One district instituted an independent reading program in which students received a grade at the end of the quarter for the number of books they read. Due to the influence of the writing test, special education staff and parents have noted that their students are being required to write more papers across their various classes. One special education teacher noted that “many did not know how to write a paragraph.” One special education staff member also observed a couple English teachers offering more short stories with 10 comprehension questions, similar to the format of the MBST, on a weekly basis. Special education teachers also discussed the increased focus on comprehension development, especially in the 7th and 8th grades.

Another creative teacher began a daily math problem for students to complete. One parent noted the change in curriculum towards a more problem-solving approach in math. Although this cannot be verified, some of the teachers, administrators, and school district personnel perceived that curriculum has been changed, but not just from the impact of the MBSTs, but more so in relation to higher standards as set forth in the Profile of Learning. As both basic standards tests and high standard graduation requirements were being implemented at the time of this study, these above statements cannot be verified.

Increased Exposure to the Regular Education Curriculum and Raised Expectations

As noted above, reading is being taught explicitly at the middle and high school levels now in both districts. High schools have hired reading specialists because students have needed direct reading instruction. Due to these changes in the curriculum, staff and parents have seen that students are being exposed to a basic skills curriculum and are meeting raised expectations. Both special education staff and parents have noted “in the past... [students with disabilities] weren’t being exposed to some of the curriculum that they maybe would have needed.” State department personnel also mentioned the increased inclusion of students with disabilities in choosing classes and special education teachers paying close attention to what’s happening in the general education curriculum. Students with disabilities have surprised their parents and teachers in being able to meet those high expectations placed on them.

Evidence of skills-related goals in the IEP. One special education staff was concerned about how the MBST has influenced the writing of the IEP. This staff believed that their IEPs have become more and more based on skills needed for the MBST, and not on the basis of the student’s personal needs and skill development. Some educators argued that this focus on basic academic skills is very appropriate for students with disabilities. However, a few parents voiced their concerns that their child’s individual needs, especially in other areas (e.g., social skills, other content areas) were not addressed. One parent felt that her child was conveniently placed into different standards preparatory classes and that the individuality of her child’s IEP was not honored at the high school level. In other words, she did not feel that her daughter’s particular academic skill weaknesses were being addressed in the prep class which, in her mind, could lead to due process issues.

No room for electives. A perceived unintended consequence of the focus on the basic skills of math and reading is the lack of opportunity for students with disabilities to take elective courses that might help to develop their strengths and vocational interests. Special education staff, administrators and members of the AEAE also discussed the tension between offering electives and taking courses in order to get the diploma. Both parents and educators appear to want students to be competent in math and reading, and yet still be able to develop other interests and skill areas. These groups, however, also see the necessity of students earning a diploma.

Focus on the Basics and Test Preparation

Administration, parents, and special education staff described the use of practice tests across both metropolitan districts. These tests are typically produced by the district office. Much time and effort goes into preparing for the basic skills tests in the classrooms. As one special education teacher said, “That’s done from the beginning of September and they start that from when they walk into school….” In one middle school, the special education staff described how the school staff has set aside 46 minutes of uninterrupted time (“primetime”) one day a week for math and one day a week for reading in 8th grade. Staff in the 5th and 7th grades have also begun to follow this pattern. The staff in that building has made a concerted effort to teach content in addition to some test-taking skills right before the exams.

Practice tests via the Internet. For those parents and students who have access to the Internet, the two school districts have given students Web site addresses for practicing their math and reading skills. Students can take the pretest and get their results. However, many urban parents discussed that they did not have Internet access at home. For those families that used this Internet tool, parents stated that they really appreciated it for checking on their student’s progress. A lack of access to computer technology varies considerably within both school districts.

Other test preparation resources. As a result of the incredible pressure to do well on these tests, a number of publishers are producing self-help books such as Passing the Minnesota Basic Standards Test in Math (Pintozzi & Pintozzi, 2000a) and Passing the Minnesota Basic Standards Test in Reading (Pintozzi & Pintozzi, 2000b). One school district created a guide that gives practice exams, testing strategies, and lists tutoring resources. One principal discussed that they had purchased an online testing service for math and reading for $1,000 a year. Another principal bought software that students could walk through in a computer lab on their own and work on their skills. He was surprised how many students chose to make use of this after-school option versus one-on-one instruction with a teacher. Two suburban parents described paying $250 for their student to take a course at a university for three hours a day for eight weeks. Both school districts are also providing meals and snacks and sending notes home to parents to remind them to get their child to bed, to give them a good breakfast, and to bring several pencils the day of the test.

Test-taking and relaxation techniques. In trying to prepare anxious students and students who struggle with test-taking in general, quite a few special education teachers reported teaching more test-taking strategies to students with disabilities. In order to help alleviate students’ anxiety, a couple special education staff members discussed visiting the testing site (especially if it’s not in the school building) and going through practice runs with all the students. Entire buildings often will run a simulation of the testing day. This helps the administration logistically figure out what needs to happen so that the testing day runs smoothly.

Teaching to the test or meeting needs? With all of this test preparation, staff may question whether they are teaching to the test or meeting the needs of students. One administrator shared that the building in his district that had the highest gain scores also devoted the most time to test preparation and building math and reading skills. There seems to be a fine line between preparing students adequately for the MBST and teaching to the test. Several parents were not happy with the amount of time some schools devoted to test preparation. One special educator at a middle school setting described how she began naturally adapting her curriculum to match the test, even in creating her class tests.

Individual tutoring. One district provided individual tutoring to those students who still had not passed the MBST by the end of 10th grade. In the suburban district of this study, one of the high school’s 11th and 12th graders received individual tutoring in both math and reading, as one administrator stated, “…even our special ed. kids by regular ed. teachers...” The high schools in both districts offered extra after school help, one for the months of October through January. Both of the middle schools in both districts held Tuesday/Thursday sessions after school for four weeks in January. One district representative discussed what he called “bird dogging” where teachers are given a small group of students to target throughout the school year and to encourage their progress in preparation for the MBSTs. This is another informal opportunity for students with disabilities to have exposure to the curriculum and to increase their skills.

Remediation Options

The suburban school district personnel were not pleased with the small number of students who scored in the upper levels on the third grade reading Minnesota Comprehension Assessments (MCAs). In response to that concern, the district has been working on an “improvement in reading” project as well as connecting with early childhood special education programs in order to encourage early literacy development. The urban district also had a literacy project that was recently formed to build student reading skills. At the high schools in the suburban district, a reading specialist was hired and a reading improvement course was offered as well as a reading and English resource class. One principal at a high school discussed his surprise when he observed a student in the reading class reading at a 2nd grade level. He said, “Oooh, this kid’s a junior! How did that happen?”

Out of the need to offer students appropriate classes for their skill levels, the special education staff also created a course for those who couldn’t make it in the mainstream math course because there was nothing else that was appropriate for them. At one of the urban high schools, a two hour reading block for a semester was offered for those students who scored at the 55th percentile or below on the Basic Standards Reading test. They also offered a standards preparation math class at this same high school. The other urban high school offered morning and evening classes at local Alternative Learning Centers and then some intervention classes for reading and math during the day.

Age-appropriate curriculum. Although new opportunities are being offered to students at the upper grade levels, people reported concerns about the age-appropriateness of the material and whether the material will engage the students. An official at the Minnesota State Department described the ongoing hunt and search for remediation curriculum among school districts. Special education staff in both school districts raised this same concern: that there was not any good math curriculum to support the MBSTs. The principal who described his amazement at the student who was reading at the 2nd grade level was concerned about the lack of engaging materials for students at this skill level.

Summer school. Another opportunity to build student skills was through summer school. A couple parents shared that their student enjoyed summer school, and one student made a special connection with the counselor and worked hard to learn the skills. Most students in summer school were 8th graders (one director mentioned over 90%), and these students were serious about passing the test. Some were there for social reasons as a few parents mentioned, but these extra opportunities to learn and practice basic math and reading skills were mandatory for students in both districts. A bigger group of parents voiced their frustration with summer school, particularly when it did not seem to help their daughter or son at all even when they had gone two or three times. One of the perceived unintended consequences is that summer enrichment programs have basically disappeared for the most part, according to members of the MAEAE.

Tracking

Tracking has been a part of education for many years, yet may be perceived as increasing as a result of this test. The suburban district has been giving a reading achievement test, but not in math. The urban district also gives an in-house achievement test that regularly monitors student achievement levels across the grades (K-12). The urban middle school principal said that they give a 6th grade pre-test to measure students’ abilities in math and reading. One of the unintended consequences is that students of the suburban district are remediated in 7th grade in math before they have actually taken the MBST. As a result, this district representative explained that they are trying not to track as much in math. The administrator at the urban middle school described a 6th grade pre-test that is given to see how they are doing in math and reading. In math, a couple school district personnel were concerned that highly skilled students were being held back. However, they also both acknowledged that remediation provides some students with disabilities access to the content, especially in reading.

Improvement in Reading Skills, but Not Math

With this increased focus on the basics, administrators and district personnel saw the MBST as providing an “early indicator of help” for students. Staff and students alike were more aware of their skill levels and needs, as noted by parents and teachers. All students receive an Individual Learning Plan that identifies skill areas to work on. Parents support the notion of their children knowing the basics. For example, one parent stated “There’s no reason why our kids shouldn’t know basics.” One special educator noted that it’s had a “very positive influence on all kids and special ed. kids in terms of focus and movement.” Members of the MAEAE believe that “our kids with disabilities are now really improving.” One director of assessment looked at longitudinal pass rates of students with disabilities. For those kids with disabilities who were scoring below 50%, “ it took them four years, some of them are going to have to retest, but the graph is very steadily going up.” Another director noted that there are two routes: a flat line for one group of students with disabilities (indicating no improvement) and other groups of students with disabilities exhibiting steady growth, but they need more time to pass the MBST. Special educators and quite a few members of the MAEAE have noted that performance on math for students with disabilities has not been as encouraging—basically a flat line with no growth.
This may reflect a change in the cut scores for math. One special education teacher and a member of MAEAE mentioned that the reason that students with disabilities do not make progress is because there is not emphasis on maintaining skills that they have developed. Members of MAEAE, parents of students with disabilities, and special education staff were concerned about students’ curriculum schedules. Some students may not have math or reading for a semester or even an entire year. Especially when a school runs on a four period day, special educators and parents are concerned about cramming a full year of math into one semester. Some teachers and parents would advocate that students need to be in consecutive math courses in order for them to retain and refine their mathematical skills.

Increased Test Participation of Students with Disabilities

Across both districts, special education staff, parents, and administrators described the importance of having high expectations and goals for students with disabilities. Many times these students will meet those goals. Special education staff encouraged students to take the test the first few years of high school and did not tell them about the pass individual option right away. If appropriate, the members of the IEP team, including the student, will determine whether the student should be allowed to pass the MBSTs at a different passing rate than that required for the other students; thus, an individual passing rate is set for these students. Teachers and parents want their students to attempt to do their best, and then the IEP team will look at the individual pass option or exemption in the 11th or 12th grades.

Unfortunately, some staff and parents reported that their students realized that they could receive the pass individual option and did not give their best effort on the MBST. One group of special education staff said that they specifically waited until students’ junior or senior year before contemplating modifications, but they regularly gave small group setting and short segment test booklet accommodations. A small group setting accommodation is an accommodation in which a student with a disability can take the test with a small number of students (not a large classroom or auditorium full of students). A short segment test booklet accommodation is an accommodation that allows the student with a disability to take the MBSTs in short segments (reading, particularly) with more breaks allowed. A few special education staff felt that the test was inflexible and did not allow enough accommodations that students with disabilities are receiving in the classroom (e.g., spell check). One teacher mentioned that because she did not know the students well, she wrote in all the accommodations in their IEPs to give them the flexibility to use whatever they needed. She further explained that students learn better through various modalities such as proposed by Gardener’s theory of multiple intelligences, and she gave different accommodations based on the students’ weaknesses.

Based on recommendations from the federal and state governments, it appears that more students with disabilities are participating in the MBST. A state department official mentioned that since every student has a MARS or student identification number, every student will test. And participation has been high, especially for students with academic disabilities. One state trainer mentioned that over 90% of students with learning disabilities were taking the MBST. If a parent chose exemption for their student, one of the district representatives emphasizes to special education staff the importance of providing informed consent when a parent wants to have his or her child exempted from the MBST (e.g., what are the consequences of exemption?).

Based on parent and staff comments, a general theme was that if a student used an accommodation, they felt that you had lowered the expectations for those students. Some staff and parents waited as long as possible to provide accommodations and especially modifications. Special education staff and a few parents were reluctant to provide modifications because of the change noted on the diploma.

Two high schools and a middle school staff in the urban district all stated that absenteeism on the day of the test was a real problem, especially for students with disabilities. This may also be related to the high mobility rate of students within these schools. For instance, one high school administrator said that approximately 700 out of 859 total students showed up for the test. The principal was not aware of how many of the 160 students who did not test were students with disabilities. However, special education staff knew some high school students with disabilities who have never taken the test. Many special education staff felt that the test appeared to be overwhelming for some students with disabilities, and they would do anything to avoid the test at all costs. A former administrator at an urban Level V setting (more restrictive educational setting for students who have severe educational and/or emotional needs) described that because students came and went so quickly and were emotionally unable to take the test, many of these students did not test.

Growing Seriousness

The first time the writing test was given, special education staff noticed that special education and regular education students blew it off, but that has given way to a growing seriousness among students with and without disabilities. District personnel from both the urban and suburban districts, state department officials, and principals have observed this growing seriousness among students as they realize the consequences attached to not passing the MBST such as being held accountable by having to attend summer school or not receiving a high school diploma. This growing seriousness by students may be a positive sign that students realize the importance of the exams and are working towards graduation.

Logistics

Increased seriousness may be a result of the emphasis placed on the test and preparations made for it by staff and administration. There is much behind the scenes that occurs so that students are tested appropriately and smoothly. Briefly, there is typically not enough space to complete the testing and have on-going classes for those who are not testing. One school uses a church a couple blocks away. Overcrowded high schools of 3,000 students do not have much extra space, especially to give small setting accommodations. One director of assessment described how students have been bussed to district buildings to take their test. One district representative mentioned that they may ask the state for a waiver from giving the test all in one day due to space limitations. Teachers and administrators are concerned about disrupting a student’s testing environment and heightening his or her anxiety by traveling to a different location to take the test.

The testing system lost credibility when the mis-scoring of math tests occurred in the Spring of 2000 by the testing service contracted by the state department. A couple parents discussed how it really impacted their children, especially a case in which a test was lost and never found.

Appropriate Measure for All Students?

Even with the increased seriousness and attention given to this graduation exam, some consumers are worried about whether this exam appropriately measures the skill growth of all students, particularly students with disabilities. For instance, one member of the MAEAE had concerns about how well the MBST measures students who have lower skills in reading and math. This individual felt that the MBST does not have a low enough floor for some students. One director of assessment saw an individual pass rate for a student with a disability set at the 10th percentile. Among teachers and school district personnel, there are questions about when it is appropriate to set an individual passing rate, when modifications should be put in place, and when is it not an appropriate test for a particular student.

Reading Level of the MBST

Another question that addresses the appropriateness of this exam for the purposes for which it is used is what exactly is the reading level of the MBST. The reading level of the MBST has been questioned by administrators and parents. One administrator did outside testing and was surprised by the results that the reading passages were 10 through 12th grade levels instead of the expected 8th grade reading levels. Two high school principals from both school districts questioned whether the reading and math levels of the MBSTs were truly at an eighth grade level, especially when they considered that the majority of their students were English language learners.

Fair Assessment for English Language Learners and English Language Learners with a Disability? A question of fairness arises when we consider the diversity in both school districts of students who speak various languages taking this exam. As one of the high schools and the middle school in the urban district had a large number of English language learners (40%–50%), there was some concern by staff, parents, and administrators about the fairness of the MBSTs. Staff and administration recognized that it takes seven years to gain cognitive academic language proficiency (CALP), yet students must take the MBST after 4 years. One administrator was not concerned about students with disabilities passing the test as they could write their own passing score; he was concerned about students with little formal education or knowledge of the English language. His building had over 25 different language groups represented in his population, and many of these students came with no formal schooling.

Special education staff in the urban district have noted that students with disabilities who are also English language learners did far worse than other students with similar disabilities on the comprehension questions of the MBST. In the suburban district that had a graduation exam for many years prior to the implementation of the MBST, English language learners could go through an appeals process to become exempt from taking the exam if they were not conversant in English. One principal also mentioned that they were working towards providing more appropriate accommodations to ELL students such as reading the math test in their own language. Respondents believe that if these tests are unfair for certain populations of students or if they do not accurately measure all student progress and growth, then students can become quickly discouraged and anxious.

Increased Exposure Leads to Unintended Consequences of Failure, Frustration, and High Anxiety. Nine of the 12 parent groups were concerned about the anxiety that the MBST had caused for their children. Parents at all six schools discussed it as well as all six special education staff groups, four principals, all district personnel in both school districts, and an official from the state department mentioned that for some students, the stress of not passing is overwhelming. The key words of anxiety (N=81), stress (N=53), failure (N=25), and frustration (N=25) showed up frequently in the transcripts. Though the special education staffs were concerned about those students who become anxious about testing and feel overwhelmed, a couple were quick to point out that there are only a few that are so anxious that it hinders their performance. However, from the parents’ perspective, many felt that there were several students in addition to their own children who are stressed about taking the MBST.

One parent described that her son has said that no matter how hard he studies, he will flunk the test. Her son had not passed it five separate times between summer school programs, actual administrations, and pre-tests given at school. A couple parents described how their children got As and Bs in English, yet did not pass the reading MBST. Another parent described the frustration of her daughter who has repeatedly failed the writing MBST. Her daughter has a written expression disability, uses the computer regularly (e.g., spell check) to write her papers in high school, and gets As on them. However, that accommodation was not available to her on the MBST. In addition, half of the parent groups mentioned how failure on these tests led to perceived lowered self-esteem and embarrassment. Several mentioned that their students are so overwhelmed and frustrated that they gave up when they took the test. Four parent groups talked about how their children did not want to be seen differently or labeled, which makes their children feel dumb. Some parents and special education staff noticed that students will rush through and not finish the test because they do not want to be the last one left.

Failure and Frustration: Can They Lead to Dropping Out?

Respondents reported that high levels of frustration can discourage students and parents. When students feel frustrated, have worked hard, and yet still do not pass, parents and educators have heard, “what is the point of staying in school when I won’t get a diploma anyway?” Three of the four high schools and one middle school (six different parent focus groups plus two special education teacher focus groups at high schools) brought up their child’s and student’s feelings of giving up and dropping out. Staff were quick to point out that usually these students were those who were close to dropping out for other reasons (e.g., few credits, high absenteeism, etc.), but the MBST served as the last straw for some of those students.

Referral for Special Education Services: Unclear Data

One of the common themes from consumers in both school districts was the uncertainty of whether the MBST resulted in more students being referred for special education services and 504 services (services given to students with a disabling condition as defined by Section 504 of the Rehabilitation Act of 1973, but do not meet the requirements to receive special education services as defined by state guidelines). Whether the school districts had accurate data to determine whether or not referral rates for special education services had actually risen was also a concern. Not all schools collected the data or in the same manner. Some thought that more students were being referred for special education services, while others thought more students were receiving 504 Plan services. In the suburban district, the high school principal did not see an increase in special education referrals, but special education staff commented that there were more referrals to student study teams from mainstream parents because “they don’t think their kids can pass these tests so they want them labeled ‘special ed.’” They described even a little flurry at the end of the previous school year with seniors and parents panicking and wondering what options existed for their student. Special education staff at the other suburban high school also mentioned that they often hear at student study team meetings that “this student has never been able to pass the test.” The director of special education services of the suburban district also noticed an increase in special education parent referrals based on the graduation standards. This district representative went on to say that the growth in special education has far exceeded the rate of growth of the district for the last two years, but stabilized as of 2000. On the other hand, there has been a dramatic increase in 504 plans in their district. These concerns were also discussed by members of the MAEAE.

Urban district personnel report an increase in referrals and services in the high schools districtwide, although the overall district child count has not increased dramatically. In trying to understand this trend, it might be partially a result of increased awareness by parents of their child’s levels of ability due to the MBST. Special education staff in the urban district mentioned that they have seen parents who may have previously denied special education services for their child take another look at that option more seriously, especially if their child had not passed the MBST by the 8th grade.

In the urban district, one of the special educators visited various schools and presented on how to include students with disabilities in the MBSTs. She did see some pressure being put on teachers to get kids special education services or 504 Plans. Suburban special education staff mentioned that sometimes unnecessary referrals were made at the middle school level. A comment made by a suburban high school staff was that when they had a bilingual liaison, they did not have increased referrals of English language learners for special education services. The bilingual liaison was able to help the assessment team tease out whether this student had a language problem or a true learning disability.

Overall, it appears that the number of referrals for special education typically has not increased in the two districts recently. There have been some increased parent referrals that point to the fact that their child is still not passing the MBST; however, overall special education referral rates are not collected in the same manner from school building to school building. It appears unwise to make a generalization based on a couple school districts’ data in which data collection is not comparable within buildings. However, there was some legitimate concern about the number of 504 Plans increasing based on feedback from members of MAEAE and special education staff.

Long-term Consequences for Students with Disabilities

For Those Who Pass

For those students with disabilities who pass, teachers and parents described that it was a huge accomplishment. One parent shared that her son scored at the 93rd percentile and “he was so excited he still has it in his bedroom.” Special education staff have observed that for those students who have struggled and taken the MBSTs three or four times, and then finally passed it, these students believe that “I can do it—I did this myself.” One district actually delivered results with sparkling juice with a ribbon and a big sign to a student who finally passed the MBST.

For Those Who Do Not Pass

For those few students who do not pass, a few administrators and members of the MAEAE also discussed the need to think beyond a four year high school. For those who do not graduate, one of the urban district personnel discussed the need for employees with technical degrees and has supported the concept of a middle college or partnership with businesses to give students a chance to improve their skills and prepare for their future job.

Several parents and special educators were concerned about the consequences of their child not receiving a diploma, especially one parent who works in human resources. She has shared with her daughter how important it is to have a diploma as her company will not hire people with GEDs. One special educator noted her concerns with those parents who push for an easy answer in an exemption, noting that there are long-term consequences associated with a notation of “exempt.”

One of the perceived unintended consequences of the MBST is that special education staff have worked with a number of parents who do not want dismissal from special education services because even if the student is doing well in the mainstream classes, the parent believes that their child will need accommodations on the MBST or, even later down the road, on the ACT or SAT. Parents also expressed concern about the identification of their student passing at a different percentage (e.g., individual pass rate marked on transcript). Some of the parents felt strongly about marking their child’s diploma.

Gray areas of assessment. Some educators and members of MAEAE talked about “gap kids” or more appropriately about “gray areas of assessment” where assessments do not accurately measure all student growth. They were referring to students who do not receive special education services, but are not passing in the classroom and are not gaining skills in the basic areas as measured by the MBSTs. A few respondents supported the idea that there should be some other options for students who do not have disabilities and are doing satisfactorily in their courses, but still do not pass the MBSTs. Others would argue that this is another validity issue as the MBST apparently has “gray areas of assessment,” and thus does not accurately measure all students’ gains in skills. Thus, there are no “gap kids,” but gray areas of assessment with the MBST.

Relevance down the road. A director of special education services talked about the relevance of student programming for all students with disabilities. This individual expressed concern about students having relevant functional skills for future jobs. There is a balance to education and preparing all students for some type of work. With increased focus on the basics and elective classes that address reading, writing, and math, will students with disabilities be given the opportunity to learn functional skills that will also enhance their opportunities down the road? This administrator feels that relevance should not be easily forgotten for this group of students.

Summary of the Results

There are a number of perceived unintended consequences of the MBSTs as reported by parents of students with disabilities, special education staff, school district administrators and personnel, and state department personnel. One of the most salient perceived consequences brought up by all respondents was the increased exposure to curriculum so that students with disabilities can gain skills in the basic areas of reading, math, and writing. Some of this increased exposure has come from increased test preparation, extra tutoring and after school sessions, and remediation curriculum. On the other hand, this increased exposure appears to have raised levels of stress, anxiety, and frustration among students as well as limited their ability to take electives which may interest them. It was unclear whether respondents perceived that special education referral rates had increased (e.g., no clear or consistent data), but a few respondents felt that the number of 504 plans had increased as a result of the MBSTs. The validity of the MBSTs was questioned by a few respondents (e.g., reflective of 8th grade level material, low enough floor, gray areas of assessment, etc.). Overall, members of the MAEAE had seen improvement in reading skills, but not as great gains in math skills. Participation rates for students with disabilities, especially those with learning disabilities, appear to be high as reported by respondents. Many special education staff do not present the individual pass option until the student’s senior year in order to maintain high expectations and provide exposure to the curriculum. Absenteeism is high in some schools which many correlate with the high mobility rates of those schools. There was concern that the MBSTs did not offer all the appropriate accommodations students may regularly receive in the classroom, and there was still much confusion about offering appropriate accommodations.

Discussion

High stakes assessment accountability systems can have some unintended negative consequences for students with disabilities. There are also positive, both intended and unintended, consequences associated with high stakes graduation exams. The previous research defined the perceived intended and unintended consequences of a high school graduation exam (the MBSTs) for students with disabilities as described by parents of these students, special educators, administrators, school district personnel including directors of assessment, and a representative and state trainer from the Department of Children, Families and Learning.

Perceived Positive Consequences of the MBSTs

A number of positive themes quickly became apparent, but one of the most exciting consequences was the increased exposure to the curriculum in various forms (e.g., access to content through mainstream courses, greater participation in the BSTs, remediation courses, summer school, reading specialists now in the high schools, tutoring, explicit IEP goals that focused on math, reading, and writing, and greater curricular coherence) and along with that came higher expectations for students with disabilities. Both special education staff and parents had noted that “in the past... they [students with disabilities] weren’t being exposed to some of the curriculum that they maybe would have needed.”

Others (Firestone & Mayrowetz, 2000; Frase-Blunt, 2000; Schleisman, 1999) have noted that high-stakes testing or external pressure can be useful for changing content taught, it can identify students with problems early on, and if you set high expectations for students with disabilities, they will meet them. Teachers and parents in this study discussed that the basic subjects are being worked on in all content areas and there is more consistency across the curriculum (e.g., writing a persuasive essay with specified steps that all teachers follow in one building). Teachers also reported that parents asked more questions at IEP meetings, and parents said that they were much more aware of their student’s actual skill levels.

More Participating

When Thompson, Thurlow, and Spicuzza (2000) analyzed participation rates across the state, they found that over 90% of students with disabilities participated in the 1999 spring testing session of the MBST. Parents and educators from this current study echoed that these statistics were valid, although a few had misgivings about lower functioning students participating in the MBST. This high level of participation in testing and in reporting of test scores does not appear to be uniform across the United States (e.g., Guthrie, 1999; Haney, 2000; Thurlow & Johnson, 2000). Although many students are participating in this state, there were still many questions from all consumers about appropriate accommodation use for students.

Growing Seriousness

The first time the writing test was given, special education staff noticed that special education and regular education students blew it off, but that has given way to a growing seriousness among students with and without disabilities. District personnel from both the urban and suburban districts, state department officials, and principals have observed this growing seriousness among students as they realize the consequences attached to not passing the MBST such as being held accountable by having to attend summer school or not receiving a high school diploma.

Perceived Unintended Negative Consequences of the MBSTs

Increased Anxiety with Exposure to the Curriculum

Increased exposure to the curriculum was not always seen as a blessing, but rather a double-edged sword, as parents and educators perceived that it increased student anxiety, frustration, and fear of failure. This is similar to what Jones et al. (1999) found when they asked teachers to assess the impact of the North Carolina ABCs accountability program on their students, where 61% felt that their students felt more anxious, and 24% felt that their students were less confident. In fact, 48.5% of teachers indicated that the ABCs program had a negative impact on students’ “love of learning” (1999, p. 201). Another study in two large states noted parents discussed high levels of anxiety and nervousness in their children that was related to taking the tests and being disappointed with the test results even when higher scores were achieved (Barksdale-Ladd & Thomas, 2000). Again, Adams and Karabenick (2000) administered a survey in Michigan to teachers and the majority thought that students had shown signs of stress over having to participate in mandated state testing programs. This current study also found that parents from all six schools, all six special educator groups, all administrators and the representative from the CFL mentioned that anxiety can be overwhelming for some students with disabilities (N=81 or 67% of all transcripts). Anxiety and failure affects one’s self-esteem, as reported in this study.

However, Public Agenda and Education Week (2001) surveyed communities about educational progress and the effects of standards and statewide assessment. One of the findings was that nearly 91% of 600 students interviewed across the nation voiced very little resentment or anxiety over testing and promotion in their schools, and most (80%) said that the tests they take seem fair. Only 5% said that they get so nervous before standardized tests that they cannot take them while two-thirds (67%) said that they get nervous but they can handle it (2001). However, this study did not specifically include students with disabilities in their sampling procedure.

Absenteeism

There was also the unintended consequence of absenteeism for students. I was not able to tell from the data what percentage of the 159 students who were absent on the day of testing for one urban high school was actually students with disabilities, although special educators in that building mentioned that some students with disabilities had never yet tested. However, the literature does relate that some parents or teachers have even encouraged their students to stay home (Barksdale-Ladd & Thomas, 2000; Kantrowitz & Springen, 1997). Barksdale-Ladd and Thomas (2000) reported that several parents in their focus group knew of large numbers of parents of high school students (11th grade) who kept their children home in case a low score would become part of their child’s record and prevent college acceptance. As this practice became more common in this state, the governor offered $2,500 in scholarships to students who performed well on the state tests (2000).

Test Preparation and Resources

Much time and effort goes into preparing for basic skills tests in classrooms. Other teachers have reported that they spend too much time preparing children for tests and administering tests, including giving them tests that use similar response formats (Adams & Karabenick, 2000; Barksdale-Ladd & Thomas, 2000; Jones et al., 1999). One administrator in this study shared that the building in his district that had the highest gain scores also devoted the most time to test preparation and building math and reading skills. Several parents and district representatives were not happy with the amount of time some schools devoted to test preparation and felt that the narrowing of the curriculum was not appropriate.

Narrowing of the curriculum is one of the criticisms leveled at high-stakes testing (Haladyna, Nolan, & Hass, 1991; Mehrens, 1998; Phelps, 1999). McNeil and Valenzuela (2001) make a powerful argument in the case of the TAAS that although certain students who were previously not taught much math are now getting more math (e.g., bilingual students, recently immigrated), they may not be learning rich, complex material that encourages higher order thinking. For example, one Texas teacher cautioned, “but of course, it’s not real math—it’s not what you would want for your children. It’s just TAAS math” (p. 141).

Another criticism is that funding, especially in poor schools, is being diverted from curriculum materials into test preparation materials. In this current study, one principal discussed that they had purchased an online testing service for math and reading for $1,000 a year; another principal bought software by Jostens that students could walk through in a computer lab on their own and work on their skills. Directors of assessment also mentioned that summer programming has declined greatly. Other researchers who have investigated high-stakes testing are concerned about the amount of resources that are diverted into test preparation materials instead of books for the library, etc. (McNeil & Valenzuela, 2001). Chicago sends students to “academic prep centers” if they have not passed the eighth grade test by age 15 (Hurwitz & Hurwitz, 2000). These centers provide small transitional classes and intensive test preparation, but the cost per pupil is more than one and a half times the cost of sending a student to a Chicago high school. Remediation takes much funding and resources are going to be diverted.

Tracking

Tracking has been a part of education for many years, and may be a concern associated with high stakes testing. For example, the suburban district gave a placement test in reading, but not in math, while the urban district also gave an in-house achievement test that regularly monitored student achievement levels across the grades (K-12). The urban middle school principal said that they also give a 6th grade pre-test to measure students’ abilities in math and reading. One of the unintended consequences is that students of the suburban district are remediated in 7th grade in math before they have actually taken the MBST.

An unintended negative consequence is that students are inappropriately placed in classes in the 7th grade before their skill levels are known. One of the issues with tracking is whether or not this discourages students such as those in Texas who do not receive any course credit for TAAS preparation courses and have less opportunity to take courses that might interest them (McNeil & Valenzuela, 2001). That was an unintended consequence for the students with disabilities in this study (e.g., not being able to take electives that they would enjoy). The question remains whether this form of tracking is valid when one director of assessment mentioned TIMSS results and pointed to the fact that many countries that outperform us do not track. The negative consequences of tracking that have been documented include that students generally learn less than they are capable of, and those in low-track classes tend to receive poor instruction, low-quality curriculums, and typically are not greeted with high expectations of academic performance (Glaser and Silver, 1994; National Research Council, 1999; Oakes, 1995).

Dropping Out

More distressing was that many parents and teachers noted that student frustration and failure on the high school graduation exam may encourage students to drop out which is a correlational finding noted by previous research (Clark, Haney, & Madaus, 2000; Griffin & Heidorn, 1996; Reardon, 1996). When students feel frustrated, have worked hard, and yet still do not pass, parents and educators have heard, “what is the point of staying in school when I won’t get a diploma anyway?” How widespread dropping out might be occurring in relation to the high stakes exam was difficult to ascertain (and not methodologically appropriate as well) through the use of focus groups and interviews, but the word “frustration” came up in 39% of the transcripts (N=13), “failure” came up in 30% of the transcripts (N=10), and the phrase “give up” came up in 12% of the transcripts (N=4). The phrase “drop out” was mentioned nine times or found in 18% of the transcripts (N=6).

Even if students do not drop out of school, failure and frustration can undermine a student’s best performance. Further study on examinee motivation in the testing process shows that the disposition of test takers is central to performance (Sundre, 1999). Sundre (1999) supported an earlier study by Wolf and Smith (1995) that having consequences was associated with increases in self-reports of motivation and enhanced test performance. Further, Wolf, Smith and Birnbaum (1994) explored the difficulty of a task and demonstrated that “arduousness” requires greater motivation to complete a more difficult task. They also concluded that lack of motivation negatively influences test performance above and beyond known ability levels. The findings in these studies on examinee motivation in previous research are noteworthy and demand further empirical examination in relation to high stakes testing, especially since these feelings of failure and “giving up” can impact student performance significantly and even lead to drop out, which is a severely unintended negative consequence in today’s world as illustrated by Bishop and Mane (2001). Further, as cited before, motivation or lack of motivation may be an artifact of a student’s own beliefs about their abilities, achievement levels, or of a student’s own sociological situation (Cook & Ludwig, 1998; Madaus & Clarke, 2001; Steele, 1997).

Appropriate, Valid Measure for All Students with Disabilities?

Some consumers are worried about whether the MBST appropriately measures the skill growth of all students, particularly students with disabilities. For instance, one member of the MAEAE had concerns about how well the MBST measures students who have lower skills in reading and math, and wonders if the MBST has a low enough floor for some students. Along with that idea, parents and a director of special education wondered if people realized that students with disabilities do plateau, and these tests do not seem to allow for that without penalizing them of a diploma. One assessment director had actually seen an individual pass rate for the MBST set at 10%.

These comments seem valid, but Minnesota is the only state to provide an individual passing score on a high school graduation exam that will still lead to a standard diploma. It seems that the stakeholders and policymakers must have realized that students’ abilities are on a continuum, and not every student fits neatly into a box. This individual pass option gives flexibility for such situations. Further, these comments might also demonstrate some misguided and lowered expectations for students with disabilities (Yell & Shriner, 1996). If we hold high expectations, the majority of students with disabilities will meet them (Gloeckler, 2001). Teachers and administrators in this present study discovered that many times students met their expectations when they set them high.

A few administrators raised the concern that the level of the MBSTs was truly beyond an 8th grade level. One administrator even had an independent company come and give an evaluation that found that the test was actually at the 12th grade level, particularly the reading test. If students with disabilities’ achievement is typically behind by a couple grade levels, then it will require more time for them to attain the skills as mentioned by the group of directors of assessment (MAEAE). They have found that there is a group of students with disabilities who are making progress and real growth, but at a slower rate than their counterparts in regular education services.

Heubert and Hauser (1999; Heubert, 2001) offer three principal criteria for appropriate test use, and the first criterion that must be met is measurement validity. Measurement validity refers to whether a test is valid for a particular purpose and whether it accurately measures the test taker’s knowledge in the content area being tested. If the MBST is a basic skills exam that is supposed to be reflective of 8th grade material, then it might be wise to reanalyze the exams given over the past five years to document that they do.

Long-term Consequences

For those few students who do not pass, a few administrators and members of the MAEAE (directors of assessment) discussed the need to think beyond a four year high school. Several parents and special educators were concerned about the consequences of their child not receiving a diploma, especially one parent who works in human resources, and knows that her own company would not hire someone without a diploma. Bishop and Mane (2001) found that MCEs had significant positive effects on the probability of being in college for a majority of student subgroups during the four years following high school graduation. In fact, effects were largest for students in the middle and bottom of the test score distribution. Historically, students with disabilities continue to be underrepresented in colleges (e.g., 12% versus 54% rate for students without disabilities) (Fairweather & Shaver, 1990). Again, as stated before, those from low socioeconomic backgrounds who graduated from an MCE high school earned $694 extra than those who graduated from a non-MCE high school (2001). These results point to the importance of the high school diploma and validate parent, teacher, and student concerns that a diploma does make a difference, and all efforts to attain one should be made, even if the timeline needs to be extended.

Perceived Unintended Positive and Negative Consequences of the MBSTs

Referral for Special Education Services

There has been some increased parent referrals that point to the fact that their child is still not passing the BST; however, overall special education referral rates are not collected in the same manner from school building to school building just as Thompson, Thurlow, Parson, & Barrow (2000) previously found. It appears unwise to make a generalization based on a couple school districts’ data in which data collection is not comparable within buildings and an individual pass option with a standard diploma is allowable. However, there was some legitimate concern about the number of 504 Plans increasing based on feedback from members of MAEAE and special education staff in the suburban district and further investigation is warranted.

Conclusion

Identifying and limiting the unintended negative consequences for students with disabilities as they participate in high stakes assessment is increasingly important. Much of what we see and hear in the public media points to negative results and images associated with high stakes graduation exams.

The results of this study were positive in that nearly all parents supported the idea of all students knowing the basic skills which means all students have access to the curriculum. It forced the issue of “what content should this student with a disability be working on?” Parents and teachers spoke to the fact that their students have had increased exposure to the curriculum. And when parents and teachers raised their expectations, the majority of students with disabilities met them and some were surprised.

Unfortunately, the process still needs some work as it appears that many students face frustration, failure, and anxiety with these exams. If accommodated for appropriately, and if students have had the opportunity to learn, students should not continue to fail without any progress. There is a concern that students have not had the opportunity to learn, or are not being provided appropriate accommodations, or there is poor IEP team decision-making occurring. If a student does not receive an accommodation, the test is merely measuring his or her disability. A few parents stated that their child was not receiving accommodations on the MBSTs that he or she was receiving in the classroom. Generalized use of classroom accommodations and testing accommodations is needed.

Another conclusion is the concern that real learning is occurring in all classes from standards preparatory classes to daily instruction in the basic areas. It is important, as Cohen and Hill (1998 as cited in Firestone & Mayrowetz ) suggest, that we spend less time thinking about how to best the test and more time thinking about what mathematics (or whatever content area) is and how to teach it. It was disturbing to see six weeks being devoted to purely MBST preparation, especially when “grill and drill” strategies do not engage the student and do not promote higher-level thinking and problem-solving. Is that another factor impacting the static percentage of students passing the math MBST? If we focus on improving and using “best practices” in teaching reading, math, and writing, then hopefully, a few years down the road, we won’t be hearing, “but of course, it’s not real math—it’s not what you would want for your children. It’s just MBST math.”

It has been suggested that effective change requires a blend of pressure and support (Fullan, 1991). Some pressure was good and induced positive changes (e.g., increased access to the curriculum and raised expectations), but these programs need a lot of fiscal and training support. One consistent theme from the study was the amount of cost (hidden costs, too) and resources that need to be devoted to an effective teaching and test program. Recently, the state of Minnesota has felt budget crunches from the State Legislature and from their failed referendums. On top of that, high standards exams are now being put into place. How are we going to sustain the MBST and improve the effectiveness of teaching as well as fund new exams and “high standards packages” without providing more resources? From the perspective of these results, it looks like a recipe for disaster, especially when there are perceptions of low morale among special and regular education staff already.

Finally, this study further emphasized the need to document empirically these intended and unintended consequences for students with disabilities. Few others are doing so, and the impact it can have on students can be life-changing (better and worse outcomes). The MBST has the potential to be a highly effective tool of access for students with disabilities, but, if used inappropriately, can easily undermine student motivation and a student’s long-term potential for success.

References

Adams, L., & Karabenick, S. A. (2000). Impact of state testing on students and teaching practices: Much pain, no gain? Paper presented at the annual conference of the American Educational Research Association, New Orleans.

Allington, R., & McGill-Franzen, A. (1992a). Does high-stakes testing improve school effectiveness? Spectrum, 10 (2), 3–12.

Allington, R., & McGill-Franzen, A. (1992b). Unintended effects of educational reform in New York. Educational Policy, 6, 396–413.

Barksdale-Ladd, M. A., & Thomas, K. F. (2000). What’s at stake in high-stakes testing: Teachers and parents speak out. Journal of Teacher Education, 51 (5), 384–397.

Bishop, J. H., & Mane, F. (2001). The impacts of minimum competency exam graduation requirements on college attendance and early labor market success of disadvantaged students. In G. Orfield & M. L. Kornhaber (Eds.), Raising standards or raising barriers? Inequality and high-stakes testing in public education (pp. 51–84).

Bond, L., & King, J. (1995). State high school graduation testing—Status and recommendations. Oak Brook, IL: North Central Regional Educational Laboratory.

Clarke, M., Haney, W., & Madaus, G. (2000). High stakes testing and high school completion. The National Board on Educational Testing and Public Policy Statements, 1 (3), 1–11.

Cook, P. J. , & Ludwig, J. (1998). "The burden of ‘acting white’": Do black adolescents disparage academic achievement? In C. Jencks & M. Phillips (Eds.), The black-white test score gap (pp. 375–400). Washington, DC: Brookings Institute Press.

Darling-Hammond, L., & Wise, A. (1985). Beyond standardization: State standards and school improvement. Elementary School Journal, 85 (3), 315–336.

Ecklund, B. (1980). Sociodemographic implications of minimum competency testing. In R. M. Jaeger & C. Tittle (Eds.), Minimum competency achievement testing: Motives, models, measures, and consequences. Berkeley, CA: McCutchan Publishing.

Elliott, J., & Thurlow, M. (1997). Opening the door to educational reform: Understanding educational assessment and accountability. Boston: The Federation for Children with Special Needs.

Erickson, R., & Thurlow, M. (1997). 1997 state special education outcomes. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Fairweather, J. S., & Shaver, D. M. (1990). Making the transition to postsecondary education and training. Exceptional Children, 57, 264–270.

Firestone, W. A., & Mayrowetz, D. (2000). Rethinking "high stakes": Lessons from the United States and England and Wales. Teachers College Record, 102 (4), 724–749.

Frase-Blunt, M. (2000). High stakes testing a mixed blessing for special students. Today, 7 (2), p. 1, 5, 7, 15.

Frederiksen, J. R., & Collins, A. (1989). A district’s approach to educational testing. Educational Researcher, 18 (9), 2742.

Fullan, M. (1991). The new meaning of educational change. New York, NY: Teachers College Press.

Glaser, R., & Silver, E. (1994). Assessment, testing and instruction: Retrospect and prospect. Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing.

Gloeckler, L. C. (2001). The door to opportunity: Let’s open it for everyone. State Education Standard, 2 (3), 21–25.

Gray, D. (2001, September 14). Summer work helps more students pass state tests. Retrieved October 31, 2001, from: http://www.educ.state.mn.us/newsrel/01BSTsumrel.html

Griffin, B., & Heidorn, M. (1996). An examination of the relationship between MCT performance and dropping out of high school. Educational Evaluation and Policy Analysis, 18, 243–251.

Guthrie, J. (April 12, 1999). Improvement may be smoke and mirrors. San Francisco Examiner, p. 1.

Guy, B., Shin, H., Lee, S. Y., & Thurlow, M. L. (1999). State graduation requirements for students with and without disabilities (Technical Report No. 24). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Haladyna, T. H., Nolan, S. B., & Haas, N. S. (1991). Raising standardized achievement test scores and the origins of test score pollution, Educational Researcher, 20 (5), 2–7.

Haney, W. (2000). The myth of the Texas miracle in education. Education Policy Analysis Archives, 8 (41). Retrieved January 20, 2002 from http://epaa.asu.edu/epaa/v8n41

Hauser, R. M. (1997). Indicators of high school completion and dropout. In R. M. Hauser, B. V. Brown, & W. R. Prosser (Eds.), Indicators of children’s well-being. New York: Russell Sage Foundation.

Heubert, J. P. (2001). High-stakes testing and civil rights: Standards of appropriate test use and a strategy for enforcing them. In G. Orfield & M. L. Kornhaber (Eds.), Raising standards or raising barriers? Inequality and high-stakes testing in public education (pp. 179–194). New York, NY: Century Foundation, Inc.

Heubert, J. P., & Hauser, R. M. (Eds.). (1999). High stakes: Testing for tracking, promotion, and graduation. Washington, DC: National Research Council, National Academy Press.

Hurwitz, N., & Hurwitz, S. (2000). Tests that count: Do high-stakes assessments really improve learning? American School Board Journal, 187 (1), 20–25.

Ingels, S. J., & Quinn, P. (1996). Sample exclusion in NELS: 88. Characteristics of base year ineligible students, changes in eligibility status after four years (NCES 96–723). Washington, DC: National Center on Education Statistics.

Kantrowitz, B., & Springen, K. (October 6, 1997). Why Johnny stayed home. Newsweek, p. 60.

Klein, S. P., Hamilton, L. S., McCaffrey, D. F., & Stecher, B. M. (2000). What do test scores in Texas tell us? Issue paper. Santa Monica, CA: RAND Corporation.

Koretz, D. M., Barron, S., Mitchell, K. J., & Stecher, B. M. (1996). Perceived effects of the Kentucky instructional results information district (MR-792-PCT/FF). Santa Monica, CA: RAND Corporation.

Koretz, D. M., & Barron, S. I. (1998). The validity of gains in scores on the Kentucky Instructional Results Information System (KIRIS) (RAND report MR-1014-EDU). Washington, DC: RAND Corporation.

Kreitzer, A. E., Madaus, F., & Haney, W. (1989). Competency testing and dropouts. In L. Weiss, E. Farrar, and H. G. Petrie (Eds.), Dropouts from school: Issues, dilemmas, and solutions (pp. 129–152). Albany: State University of New York Press.

Krueger, R. A., & Casey, M.A. (2000). Focus groups: A practical guide for applied research (3rd Ed.). Thousand Oaks, CA: Sage Publications.

Linn, R. L. (1993). Educational assessment: Expanded expectations and challenges. Educational Measurement: Issues and Practice, 13 (3), 5–16.

Linton, T. H. (2000, February). High stakes testing in Texas: An analysis of the impact of including special education students in the Texas Academic Excellence Indicator System. Paper presented at the Texas Assessment Conference, Austin, TX.

McGill-Franzen, A., & Allington, R. (1993). Flunk ’em or get them classified: The contamination of primary grade accountability data. Educational Researcher, 22 (1), 19–22.

McNeil, L., & Valenzuela, A. (2001). The harmful impact of the TAAS system of testing in Texas: Beneath the accountability rhetoric. In G. Orfield & M. L. Kornhaber (Eds.), Raising standards or raising barriers? Inequality and high-stakes testing in public education (pp. 127–150). New York, NY: Century Foundation, Inc.

Madaus, G., & Clarke, M. (2001). The adverse impact of high-stakes testing on minority students: Evidence from one hundred years of test data. In G. Orfield & M. L. Kornhaber (Eds.), Raising standards or raising barriers? Inequality and high-stakes testing in public education (pp. 85-106). New York, NY: Century Foundation, Inc.

Mehrens, W. A. (1998). Consequences of assessment: What is the evidence? Vice-Presidential Address for Division D, annual meeting of the American Educational Research Association, San Diego.

Messick, S. (1992). The interplay of evidence and consequences in the validation of performance assessments (ETS RR-92-39). Princeton, NJ: Educational Testing Service.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. (2nd Ed.). Thousand Oaks, CA: Sage Publications, Inc.

Minnema, J., Thompson, S. J., Thurlow, M. L., & Barrow, S. (2000). Unintended consequences of the Minnesota Basic Standards Tests: Do the data answer the questions yet? (Minnesota Report 23). Minneapolis, Minnesota: University of Minnesota, National Center on Educational Outcomes.

National Research Council. (1999). Testing, teaching, and learning: A guide for states and school districts. Washington, DC: National Academy Press.

Neill, M., & Gayler, K. (2001). Do high-stakes graduation tests improve learning outcomes? Using state-level NAEP data to evaluate the effects of mandatory graduation tests. In G. Orfield & M. L. Kornhaber (Eds.), Raising standards or raising barriers? Inequality and high-stakes testing in public education (pp. 107–126). New York, NY: Century Foundation, Inc.

Oakes, J. (1995). Two cities’ tracking and within-school segregation. Teachers College Record, 96 (4), 681–693.

O’Day, J., & Smith, M. (1993). Systemic reform and educational opportunity. In S. Fuhrman (Ed.), Designing Coherent Educational Policy (pp. 250–312). San Francisco: Jossey-Bass.

Olson, J. F., & Goldstein, A. (1996). Increasing the inclusion of students with disabilities and limited English proficient students in NAEP. Focus on NAEP, 2 (1), 1–5.

Olson, J. F., Jones, I., & Bond, L. (2001). Annual survey of state student assessment programs, summary report and vol. 1 & 2 (1998–1999) data. Washington, DC: Council of Chief State School Officers (CCSSO).

Phelps, R. P. (1999). Why testing experts hate testing. Fordham Report, 3 (1).

Pintozzi, F., & Pintozzi, D. (2000a). Passing the Minnesota Basic Standards Test in math. Woodstock, GA: American Book Company.

Pintozzi, F., & Pintozzi, D. (2000b). Passing the Minnesota Basic Standards Test in reading. Woodstock, GA: American Book Company.

Public Agenda & Education Week. (February 21, 2001). Reality check 2001. Retrieved January 22, 2002, from http://www.publicagenda.org/specials/rc2001/reality.htm

Reardon, S. (1996). Eighth-grade minimum competency testing and early high school drop-out patterns. Paper presented at the annual conference of the American Educational Research Association, New York.

Rhim, L., & McLaughlin, M. (1997). State level policies and practices: Where are students with disabilities? College Park, MD: Institute for the Study of Exceptional Children and Youth, Center for Policy Research on the Impact of General and Special Education Reform, and Consortium for Policy Research in Education.

Roach, V., & Raber, S. (1997). State educational reform: District response and the implications for special education. A cross-site analysis based on four case studies. Alexandria, VA: Center for Policy Research on the Impact of General and Special Education Reform, National Association of State Boards of Education.

Rossi, R., Herting, J., & Wolman, J. (1997). Profiles of students with disabilities as identified in NELS: 88 (NCES 97–254). Washington, DC: National Center on Educational Statistics.

Rothman, R. (1996). Taking aim at testing. The American School Board Journal, 183 (2), 27–30.

Schleisman, J. (1999, October). An in-depth examination of one school district’s response to an externally-mandated, high-stakes testing program in Minnesota. Paper presented at the annual conference of the University Council of Educational Administration, Minneapolis.

Shepard, L., & Dougherty, K. (1991). Effects of high-stakes testing on instruction. Paper presented at the annual conferences of the American Educational Research Association and the National Council on Measurement in Education, Chicago.

Stake, R. (1967). The countenance of educational evaluation. Teachers College Record, 68, 523–540.

Steele, C. (1997). A threat in the air: How stereotypes shape the intellectual identities and performance of women and African Americans. American Psychologist, 52 (6), 613–629.

Sundre, D. (1999). Does examinee motivation moderate the relationship between test consequences and test performance? Paper presented at the annual conference of the American Educational Research Association, Montreal, Canada.

Texas Education Agency. (1995). Including students with disabilities in statewide assessment and accountability systems: A study of the issues. Houston, TX: Region IV Education Service Center.

Thompson, S., Thurlow, M., Parson, L., & Barrow, S. (2000). Initial perceptions of educators as they work toward including students with disabilities in Minnesota’s High Standards (Minnesota Report 25). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thompson, S., Thurlow, M., & Spicuzza, R. (2000). 1999 Report on the participation and performance of students with disabilities on Minnesota’s Basic Standards Tests (Minnesota Report 29). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow M., Albus, D., Spicuzza, R., & Thompson, S. (1998). Participation and performance of students with disabilities: Minnesota’s 1996 Basic Standards Tests in reading and math (Minnesota Report 16). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M., House, A., Boys, C., Scott, D., & Ysseldyke, J. (2000). State participation and accommodation policies for students with disabilities: 1999 update (Synthesis Report 33). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M. L., & Johnson, D. R. (2000). High-stakes testing of students with disabilities. Journal of Teacher Education, 51 (4), 305–314.

Thurlow, M. L., Nelson, J. R., Teelucksingh, E., & Ysseldyke, J. E. (2000). Where’s Waldo? A third search for students with disabilities in state accountability reports (Technical Report 25). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M. L., & Ysseldyke, J. E & Anderson, C. L. (1995). High school graduation requirements: What’s happening for students with disabilities? (Synthesis Report 20). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M., Ysseldyke, J., Gutman, S., & Geenen, K. (1998). An analysis of inclusion of students with disabilities in state standards documents (Technical Report 19). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Thurlow, M., Ysseldyke, J., & Silverstein, B. (1993). Testing accommodations for students with disabilities: A review of the literature (Synthesis Report 4). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Wagner, M., Newman, L., D’Amico, R., Jay, E. D., Butler-Nalin, P., Marder, C., & Cox, R. (1991). Youth with disabilities: How are they doing? Menlo Park, CA: SRI International.

Wagner, M., D’Amico, R., Marder, C., Newman, L., & Blackorby, J. (1992). What happens next? Trends in postschool outcomes of youth with disabilities. Menlo Park, CA: SRI International.

Wolf, L. F., & Smith, J. K. (1995). The consequences of consequence: Motivation, anxiety, and test performance. Applied Measurement in Education, 8 (3), 227–242.

Wolf, L. F., Smith, J. K., & Birnbaum, M.E. (1994). Consequences of performance, test motivation, and mentally taxing items. Applied Measurement in Education, 8 (4), 341–351.

Yell, M., & Shriner, J. (1996). Inclusive education: Legal and policy implications. Preventing School Failure, 40 (3), 101–107.

Ysseldyke, J., & Bielinski, J. (2002). Effect of different methods of reporting and reclassification on trends in test scores for students with disabilities. Exceptional Children, 68 (2), 189–200.

Zlatos, B. (1994). Don’t test, don’t tell: Is "academic red-shirting" skewing the way we rank our schools? The American School Board Journal, 191 (11), 24–28.