Alternate assessment originated in Kentucky in 1992 (National Alternate Assessment Center, 2005), and was mandated nationally by the Individuals with Disabilities Education Act of 1997. Alternate assessment acts as a mechanism for inclusion in large-scale educational assessments for those students with disabilities who cannot participate in regular state and district assessments, even with accommodations and modifications. The No Child Left Behind Act of 2001 (NCLB), the Individuals with Disabilities Educational Improvement Act of 2004 (IDEA), and subsequent regulations reinforce the requirement that states develop alternate assessments for students who need them in order to be appropriately included in assessment and accountability systems. The law and regulations also clarify that alternate assessments on grade-level achievement standards can be designed to assess the achievement of students with disabilities other than the most significant cognitive disabilities, who, often, cannot show what they know on the general assessment, even with accommodations. NCLB requires that alternate assessments for these students must be comparable to the general assessment both in content standard coverage and achievement standards. Regardless of whether alternate or grade-level achievement standards are set, all assessment options are to be aligned to grade-level content standards.

In addition, several states have implemented graduation tests as a condition of receiving a high school diploma. There is much that states must still do to expand opportunities for students with disabilities to demonstrate what they know and can do through ways other than those typically used in large-scale assessments. One option that states have considered is to provide students with alternative methods for showing proficiency (Krentz, Thurlow, Shyyan, & Scott, 2005).

To ensure full participation in assessment and accountability systems, whether for NCLB accountability purposes or for graduation or promotion, students with disabilities unable to take statewide paper-and-pencil tests may require alternate assessments. Some of these students have significant cognitive disabilities and are working substantially below the performance expectations of other students in their grade. Sometimes, though, the nature of the student’s primary disability is not cognitive, though still complex and significant enough to require an alternate assessment, for example a student with cerebral palsy or an intensive emotional disability. This latter group of students may be working at or close to a performance level comparable with typical peers, but cannot take the on-demand paper-and-pencil test, even with accommodations, due to their disability. The alternate assessments taken by these students must be able to measure their performance against grade-level achievement standards, rather than the alternate achievement standards normally associated with students with the most significant cognitive disabilities.

The focus of this discussion will be on those students who require alternate assessments based on grade-level achievement standards, and how such alternate assessments can reliably measure their achievement in a manner that is equivalent to students who are taking the regular state assessment for that grade. The Massachusetts Department of Education (the "Department") has developed an alternate assessment based on grade-level achievement standards consisting of a structured portfolio of student work collected over a period of time, and reviewed by experts in the content area. This option can be used for two purposes: to ensure appropriate participation for all students in system accountability for NCLB purposes and to ensure appropriate assessments for all students for the purpose of competency determination for a diploma. In this paper, the "competency portfolio" will be discussed, including how the model was developed and validated, and how a small but growing number of students are using this format as an alternative pathway to participate in the Massachusetts Comprehensive Assessment System (MCAS) and, ultimately, satisfy the state’s graduation requirement.

First, it is important to clarify why students take alternate assessments of any type. There are many reasons why a student with a disability might need an alternate assessment. IEP teams must make careful decisions about how each student will participate in statewide assessments, based on their knowledge of and familiarity with the student and his or her prior instructional and assessment experiences. State assessment policies must also require teams to revisit previous assessment decisions on an annual basis and make adjustments to their earlier determinations, as needed, rather than maintaining a student on an alternate assessment participation track for the duration of his or her education. Students should also be permitted to take standard tests with accommodations when this is appropriate and possible in certain subjects, while taking alternate assessments in another.

Students with the most significant cognitive disabilities comprise the majority of students taking alternate assessments. These students are required to work on grade-level content standards, but will likely be focusing on these standards at levels of complexity substantially below that of their non-disabled peers. As a result, the standard test at their grade-level would not allow the student access to the assessment in order to participate, nor would it yield meaningful results or information needed to improve subsequent instruction for the student.

Students with other significant (non-cognitive) disabilities may also require alternate assessments, but for different reasons, especially when the effects of their disabilities cannot be overcome in order to participate in standard tests. The number of such students is relatively low in Massachusetts, about 5-6% of the number of students taking alternate assessments based on alternate achievement standards. Some of these students might be able to take the general assessment if appropriate accommodations were identified, available, and provided, though not always.

At whatever rate and frequency the disabilities of these students hinder their participation in general assessments, they are entitled to take assessments that allow them to demonstrate knowledge and skills at the highest and most challenging levels of complexity. This means the alternate assessment must measure the same grade-level achievement as their peers, though using a different assessment format and context.

These students present a dilemma to states charged by law with assessing all students using valid and reliable methods. They exist in small, but significant numbers, and without their meaningful participation in appropriate assessments, the state’s assessment system cannot truly be said to be fully inclusive. Still, few states have developed alternate assessments based on grade-level achievement standards. This places these students in a difficult and vulnerable position. Their IEP teams are forced to choose from among several unsuitable assessment options, and it raises the possibility of legal challenges in states where graduation testing requirements may restrict their access to a diploma.

It is important to review characteristics of students who require alternate assessments based on grade-level achievement standards in order to understand why it is necessary to develop alternate assessments for them. The following profiles are meant to expand an awareness of the characteristics of these students, though it is not meant to be an exhaustive list. The intent is to illustrate the attributes that should impel IEP teams to consider alternate assessments based on grade-level achievement standards for these students, due to the complexity, severity, and exceptional nature of their disabilities. Examples of these students might include:

• Students unable to "maintain sufficient concentration" to participate in standard testing, even with test accommodations, as a consequence of a severe emotional disability, traumatic brain injury, autism or Aspberger’s Syndrome, or other disability or combination of disabilities.

• Students for whom the "demands of a prolonged test administration" would present a significant challenge, as a consequence of a health-related, multiple physical, or other disability.

• Students who require "more time than is reasonable or available" for testing, even with the allowance of extended time, as a consequence of cerebral palsy, deaf-blindness, or a significant motor, communication, or other disability.

• Students for whom the "format of the standard test is inappropriate," and the necessary accommodations are unavailable or would "give away" or hint at the answers.

• Students who do not have significant cognitive disabilities, but whose disabilities result in other "unique and significant challenges" to taking the standard test.

From the qualities listed above, it should be clear to policy-makers and members of IEP teams that such students cannot participate in standard paper and pencil on-demand tests, even with accommodations, without seriously compromising the validity of those results. Alternate assessments based on alternate achievement standards are likewise inappropriate.

When its statewide assessment advisory committee recommended to the Massachusetts Department of Education in 1999 that students with "unique and significant challenges" also be permitted to take alternate assessments, and that the alternate assessment ultimately allow them to meet the state’s graduation requirement, the idea of a "competency portfolio" was born.

IEP teams in Massachusetts have been trained to recognize the characteristics of students who should be considered for alternate assessments, including both students with the most significant cognitive disabilities and those with "unique and significant" (primarily non-cognitive) disabilities. Guidelines for IEP team decision-making on alternate assessment participation have been circulated since 1999.

In broadening the participation criteria to include both groups of students, the alternate assessment itself had to become sufficiently flexible to accommodate the creation and scoring of portfolios for students working at levels of complexity ranging from substantially below to comparable with, and even above, the level of their non-disabled grade-level peers.

As a result, the reporting system required a seamless progression of achievement levels reflecting student performance based on alternate as well as grade-level achievement standards. Figure 1 shows how the Massachusetts Comprehensive Assessment System (MCAS) integrated its performance levels for statewide reporting on alternate assessments to allow comparisons of all students taking alternate assessments, whether measured against alternate or grade-level achievement standards.
 

Portfolios do not assess student learning precisely in the same way as standard assessments, due in part to the large number of learning standards and skills required for documentation. However, portfolios have several distinct advantages over on-demand tests, including demonstration of growth and learning over a period of time, development of an individualized learning sequence and pace of instruction (Seidel et al., 1997), and relieving students of the intense physical and emotional demands of test-taking. New methods and criteria for examining student work will need to be established for this alternate assessment, and information will need to be included with student work telling how the instructional tasks were conducted, and describing the frequency and types of assistance, if any, given to the student.

Further, the design of these methods and criteria must be based on a foundation of rigorous and challenging content. Thus, the direct participation and involvement of content experts is perhaps the most critical element in the design of an alternate assessment based on grade-level achievement standards, since individuals most familiar with the standards in the content area are those who are most qualified to identify the key skills and content to be assessed, and to assist the state in determining the benchmarks needed to reach each level of achievement.

The role of content experts in developing the Massachusetts "competency portfolio" was identified in early discussions. It became obvious that not every learning standard assessed on the general assessment could be documented in a portfolio, due to the unrealistic amount of time and effort such a range of learning standards would entail. Particularly, in a state like Massachusetts with a state requirement to demonstrate competency in English language arts (ELA) and mathematics, practitioners had to agree on the primacy of certain learning standards, so students could demonstrate proficiency in essential areas deemed most significant by secondary-level content experts. These experts assisted the Department in identifying the most critical standards for documentation, and continue to advise the Department on fine-tuning this process.

States must be thorough, thoughtful, and collaborative in their approach to developing alternate assessments based on grade-level achievement standards because of the inherent differences in assessment formats, particularly when results are used to determine high stakes for students, such as promotion or graduation. Massachusetts students, for example, must eventually pass the grade 10 MCAS assessments in English language arts and mathematics as part of the state’s requirement to earn a diploma. The cut score for passing these standard assessments is a score of 220 on the tests, the lowest scaled score within the needs improvement performance level. Performance levels on MCAS tests are scaled as follows: warning/failing (200-218); needs improvement (220-238); proficient (240-258); advanced (260-280). Although performance at the needs improvement level in grade 10 is sufficient for students to meet the state’s graduation requirement, a score of proficient is required to meet the federal definition of proficiency for No Child Left Behind.

While some students with disabilities in Massachusetts score at the needs improvement and proficient levels on their grade 10 assessments, those who do not are encouraged to work until the end of grade 12, or longer, to achieve a score of at least needs improvement and earn their diplomas. It is important for states to provide incentives to continue the intensive instruction of these students beyond grade 10, even though their scores do not count in the AYP calculations of their schools and districts, and to provide them with academic support programs and opportunities to take successive retests. It is equally important that IEP teams be permitted to designate these students for alternate assessments, when necessary, and that these students be permitted to meet all state requirements to graduate when they take these alternate assessments. Taking an alternate assessment should not automatically remove any student from the possibility of earning a diploma, since in many cases it is not known precisely how much these students are capable of learning.

In each of the classes of 2003 and 2004 in Massachusetts, 80-82% of students with disabilities met the threshold for graduation by the end of grade 12. As of this writing, about seventy-five Massachusetts students have successfully earned their diplomas by taking alternate assessments based on grade-level achievement standards. The process of reviewing the portfolios is described more fully in the next section.
 

The evidence in the MCAS Alternate Assessment (MCAS-Alt) portfolio determines whether a student’s performance will be measured based on grade-level achievement standards or alternate achievement standards. During the portfolio review process, scorers (primarily special educators) determine whether a portfolio demonstrates student performance either at, or close to, grade-level complexity and expectations. Through this winnowing process, portfolios identified as "at or close to grade-level expectations" are set aside and subsequently reviewed by content experts who verify that the portfolio should indeed be reviewed based on grade-level achievement standards. The expert panels then review for depth, breadth, and overall comparability with a student who has passed the corresponding MCAS test, based on their familiarity with the learning standards and with the performance expectations for students who have passed the grade 10 tests. Specific requirements to meet the Competency Determination standard are described in the Appendix.

For students in grades 3–8, where the stakes are arguably lower (i.e., no retention or promotion, although sanctions may result for schools and districts based on AYP determinations), the alternate assessment based on grade-level expectations is nevertheless held to high standards for breadth, depth, and quantity. Documentation is required in at least three learning standards in each of three "strands" (or subdomains) in the content area being assessed. When work samples are determined to be at a level of complexity comparable with a student in that grade, as determined by a panel of content experts in grades 3–8, and the work is both accurate and independently produced by the student, then the student earns a score of needs improvement, proficient, or advanced, based on an equivalent performance of a student scoring at these performance levels on the test.

At grade 10, the requirements to meet the graduation standard were set considerably higher. Content experts advised the Department that evidence of only three learning standards per strand would be inadequate to meet the Competency Determination standard. Portfolio submission requirements for grade 10 students were developed with specific products in mind, based on learning standards selected for documentation by curriculum and assessment experts in English and math.

With support from technical advisors and state contractors, the Department developed separate requirements for the English language arts and mathematics competency portfolios. Since the ELA general assessment is primarily an assessment of language, reading comprehension, and writing skills, test developers and assessment experts conceived a portfolio based on writing samples that responded to a piece of grade 10 literature in which the student addressed themes in the work, rather than simplistic plot summaries. Students were required to show successive drafts and revisions, with self-generated edits guided by teachers’ comments and feedback. Specific ELA competency portfolio requirements are shown in the Appendix.

Math test developers and curriculum framework writers, on the other hand, took a more prescriptive approach to the math competency portfolio. They identified between two and four learning standards in each of five Math "strands" for documentation in the portfolio through a collection of work samples focused on each required standard. The selected learning standards typically were either the most difficult, most comprehensive, or those that subsumed other standards within them. In some strands (geometry, for example), teachers and students could choose any three standards on which to base their work samples. In other strands, specific standards were pre-selected for documentation. Students are required to solve problems accurately and independently, and to show all steps in solving the problems, plus show evidence of applying the standard in novel situations, rather than simply recalling or repeating information.

A Level of Complexity ("5") was added to the MCAS-Alt scoring rubric to reflect a student who had submitted work of this depth and breadth. A score of 5 is necessary for a student to "pass" the assessment in order to earn a Competency Determination. Figure 2 shows the Level of Complexity scoring area in the MCAS-Alt scoring rubric, reflecting a range of complexity and alignment of portfolio evidence with grade-level academic learning standards.

Students are given multiple opportunities beyond grade 10 to pass these assessments and earn a diploma by resubmitting their competency portfolios with additional work samples in grades 11 and 12, and even beyond grade 12. A detailed summary of the portfolio review is returned to each school that allows educators to focus instruction in precisely the areas in which it is needed, and resubmit the enhanced portfolio, until such time as the student demonstrates competency on the grade-level content. The decision to resubmit a competency portfolio beyond grade 10 is at the discretion of the student’s IEP/504 team, based on whether the student, in their judgment, is approaching a grade 10 level of achievement.

It is clear that portfolio submissions will need to occur over the course of several school years for some students beginning in grade 10. A collaborative approach among general and special educators will be needed to do this effectively. There is now ample evidence to indicate that this portfolio process has fostered both that collaboration and thoughtful, long-range planning among general and special educators for these students.

States will need to document the validity of assessments used to assess and report the academic performance of all students, based on the requirements in the No Child Left Behind law. A small but significant number of these students will not be able to take the standard test because of the nature and complexity of their disabilities, but are also ill-suited to take alternate assessments based on alternate achievement standards, since their knowledge and skills are at comparable levels with their non-disabled peers. The assessment results of these students based on either of these existing assessment formats will be invalid for the purpose of reporting and accountability. A more flexible and responsive approach is needed to assess these students. As assessments are used to make decisions that have a direct impact on students, states will run the risk of being held accountable by the public to show that results were obtained using fair and precise assessments.

States in increasing numbers are also looking to verify that students graduating from high school can demonstrate the core competencies acknowledged as important in becoming productive citizens and lifelong learners. These states must balance resources and capacity, external pressures, and political concerns in considering whether and how to implement graduation requirements for students. Approaches that rely on the use of only one assessment method by which students can meet these high-stakes requirements will inevitably be challenged by the families of students who claim to have these skills and competencies, but cannot demonstrate them effectively on standard tests. States must therefore decide whether to develop alternative pathways to a diploma for the comparatively small number of students who need them. In so doing, they will ensure that their assessment system remains responsive to the needs of a full range of students, so all will have the opportunity to demonstrate their skills effectively. This report presented one such model of an alternative pathway used by a state to meet diploma requirements.

National Alternate Assessment Center. (2005). The assessment triangle and students with significant cognitive disabilities: Models of student cognition. Lexington, KY: Interdisciplinary Human Development Institute, University of Kentucky.

Krentz, J., Thurlow, M., Shyyan, V., & Scott, D. (2005). Alternative routes to the standard diploma (Synthesis Report 54). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.

Massachusetts Department of Education. (1999). Participation guidelines for MCAS alternate assessment. Malden, MA: Author.

Massachusetts Department of Education. (2004). 2005 educator’s manual for MCAS alternate assessment. Malden, MA: Author

Massachusetts Department of Education. (2000). Massachusetts mathematics curriculum framework. Malden, MA: Author.

Massachusetts Department of Education. (2001) Massachusetts English language arts curriculum framework. Malden, MA: Author.

Seidel, S., Walters, J., Kirby, E., Olff, N., Powell, K., Scripp, L., & Veneema, S. (1997). Portfolio practices: Thinking through the assessment of children’s work. Cambridge, MA: President and Fellows of Harvard College.