Using Growth for Accountability: Considerations for Students with DisabilitiesPolicy Directions 21 Martha Thurlow • Sheryl Lazarus • Rachel Quenemoen • Ross Moen January 2010 All rights reserved. Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as: Thurlow, M., Lazarus, S., Quenemoen, R., & Moen, R. (2010). Using growth for accountability: Considerations for students with disabilities (Policy Directions 21). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/Policy21 Background The academic progress of students is an important indicator of educational success. For school accountability systems, using a growth model to report on a student’s academic growth can be viewed as an addition to or a replacement for a status model. Status models compare the performance of a cohort of students in a grade one year with the performance of another cohort of students in the same grade the preceding year. Growth models typically track the achievement of individual students from year to year. General Considerations There are both strengths and limitations in each accountability approach (see Table 1). Part of the appeal of status models is that they offer a simple, clear way of seeing current achievement levels. This is useful for reporting on an educational system’s progress in helping students in a given grade achieve higher than students in that grade did previously. Status models also are especially well suited for showing students’ standing relative to established knowledge and skills targets. Thus, status models fit well with contemporary educational practices that emphasize content and achievement standards. A criticism leveled against status models that focus on a small number of achievement targets is that they give an incomplete picture of the effects of an educational system on student achievement. For example, improvements in student achievement might go unnoted if the achievement does not cross target thresholds. As a consequence, educators might concentrate their efforts only on student learning that is most visible. Part of the appeal of growth models is that they can be designed to show changes in achievement that status models might miss. Growth reporting that is not tied to particular academic targets can provide an indication of how much achievement changes for students regardless of how high or low their performance is or how close it is to achievement thresholds. Growth models also are appealing because they fit well with the popular view of education as helping students progress from one level of knowledge and skills to a higher level. These models take into account students’ different starting points. Some people believe that this is a more accurate and fairer way to compare education systems that differ in student composition. Growth models face several technical and conceptual challenges. One challenge is that there are a variety of approaches and measures that may be used to track growth. Each involves specific assumptions and each has implications for the ways in which student performance is included (see Table 2). A difficult technical challenge for some, but not all, growth models is that they require vertical scaling of tests. Test developers have found it difficult to create such tests. Another technical challenge for most growth models is that they require longitudinal data systems that track individual students across years. Associated with this technical challenge is the conceptual challenge of what to do with missing data for students who enter, leave, or are absent. A key conceptual challenge for growth models is choosing a comparison base. For example, if a "year’s worth of growth" is defined as the average growth for all students each year, by definition half of all students will be below the year’s growth criterion. More complex approaches that establish different definitions of a year’s growth for different groups raise other concerns.
Table 1. Strengths and Limitations of Status and Growth Models for Accountability
Table 2. Description and Implications of Approaches to Growth Models a
a The approaches shown here from CCSSO (2008), pages 7-12. Considerations in Assessing Students with Disabilities The many ways of talking about growth become more complex when considering students with disabilities. In special education, growth often refers to changes in performance on curriculum-based or norm-referenced measures designed to assess basic skills, regardless of the student’s grade level. Although important for diagnostic purposes, these measures may not be the best for growth models. A basic skills focus can inaccurately measure the performance of students who have mastered grade level higher-order skills (such as problem solving) without mastering lower level skills (such as computation). Similarly, measuring growth toward IEP goals is very different from growth models used for accountability. Standards-based growth models do not measure growth toward individualized student goals. One of the technical challenges of growth models for students with disabilities is ensuring that their assessment results are incorporated into the same accountability reports as for other students. Most growth models do not include results from tests with different scales. This is important because some students with disabilities may participate in alternate assessments based on alternate, modified, or grade-level achievement standards. Including scores from these assessments along with scores from general assessments may create challenges for some growth models that states must address. For students with disabilities who participate in the same assessments as other students, including them is not a challenge. Although some of these students may take the assessment with accommodations, it is generally recognized today that the purpose of accommodations is to obtain a valid measure of what the student knows and is able to do, and therefore scores from accommodated assessments can be treated the same as other scores. The challenge of missing data has potentially more negative implications for students with disabilities than other students. For example, most growth models either exclude students with missing scores or create estimated scores. Estimates of scores may be less reliable if a group of students (such as students with disabilities) has more missing scores than other groups. The greater mobility and poorer attendance of students with disabilities has been documented, and thus are important factors in considering growth models that are appropriate for all students. Other considerations for students with disabilities involve concerns about the impact that different accountability approaches might have on educational practices. Evidence of low expectations for students with disabilities continues to emerge today. One result is that students with disabilities often are denied access to grade-level content. When these students do have access to standards-based grade-level content and are included in accountability, their performance on grade-level assessments often exceeds expectations. Growth models can reinforce low expectations for students with disabilities if the models explicitly set different comparison bases for some students or if they simply obscure low levels of performance by reporting growth without indicating the actual level of achievement. Growth models also can be out of alignment with a standards-based emphasis if they have no tie to specific academic content. Recommendations Students with disabilities have benefited from a focus on grade-level content, regardless of the assessment in which they participate. The recommendations provided here retain a focus on the benefits of accountability for students with disabilities, while considering potential positive and negative consequences of growth approaches for accountability. Retain a standards-based approach. Standards-based education has a foundation in content standards of what students should know and be able to do, as well as performance standards that indicate how well students demonstrate their knowledge and skills. A focus only on growth can lose sight of the content standards and establish "growth standards" that have no connection to any particular content. Maintain grade-level focus. Approaches to measuring growth can retain their connection to content standards but still fall into below grade-level testing (and instruction). Below grade-level approaches can lower learning expectations and poorly reflect progress in grade-level content. Include all students. Ways to include students with disabilities who participate in assessments other than the general assessment need to be identified. Similarly, a growth model should treat students the same regardless of where they are in the performance continuum. Students with disabilities excluded from a growth model are likely to suffer the negative consequences of reduced access to standards-based instruction, limited focus on grade-level content, and lowered expectations. Ensure that all students are visible in the results. Growth approaches must retain transparency for the performance of all student groups. Progress of students with disabilities must not be masked by the performance of other students. Similarly, performance of students with only one year of data cannot be ignored for accountability. Policymakers and their measurement, special education, and curriculum partners should analyze proposed options for using growth for accountability, against these recommendations. Results of this analysis should be articulated publicly and clearly, so that any option that is selected can be defended in light of these recommendations. If this is not done, there is a risk of unintended negative consequences for students with disabilities. Resources Implementer’s Guide to Growth Models. CCSSO. (2008). Washington, DC. Author. "Student Absenteeism," in The Condition of Education, 2006. Table 24-2. NCES. Washington, DC: U.S. Department of Education. Going to School: Instructional Contexts, Programs, and Participation of Secondary School Students with Disabilities. Wagner, M., Newman, L., Cameto, R., Levine, P., & Marder, C. (2003). Menlo Park, CA: SRI International. Top of page |
||||||||||||||||||||
