Thirty-five regular and unique states joined the 12^th teleconference on May 1, 2006, "Investigating the Effects of Accommodations on Use of Student Scores: What Empirical Understanding Means for NCLB Accountability Purposes."

Presenters were Steve Sireci, professor, University of Massachusetts, and Director of the Center for Educational Assessment; Sharron Hunt, Assessment Director, Georgia Department of Education; and Melissa Fincher, Assistant Assessment Director, Georgia Department of Education.

Steve Sireci addressed the ways accommodations may alter valid score interpretation by either improving it (removing construct irrelevant variance) or undermining it (introducing construct under-representation). He also provided an overview of the accommodation literature, noting a lack of empirical studies with individuals with disabilities.

The Sireci, Scarpetti, and Li review (2005) concluded that accommodations being used are sensible on the whole (e.g., extended time, oral accommodation in math). Results suggest that accommodated test administration does not appear to change a test’s structure—or construct being assessed. Although studies show that there is a small performance boost among students not usually accommodated when allowed accommodations, there is a larger boost for students with disabilities and English language learners, indicating that barriers are being removed when accommodations are allowed.

Sireci finished with a challenge to states to conduct their own research, encouraging a focus on experimental design studies while looking at student preference in using specific accommodations. He encouraged meta-analyses based on practice (e.g., data warehouses to address issues in the field), and collecting or merging larger data samples to enable analyses on DIF and educational gains.

Sharron Hunt and Melissa Fincher presented background on an empirically designed modification research study currently being conducted. Georgia wanted to know which students were struggling to access the general test and what accommodations and modifications might help.

The study involves the state’s divisions of assessment, exceptional student, and curriculum and instruction. It looks at the effects of modifications on the reliability and validity of the Georgia Criterion-Referenced Competency Test (CRCT) and the Georgia High School Graduation Test (GHSGT) in reading and math, focusing on grades 4 and 7. Conditions include (1) a traditional administration with standard accommodations, (2) the use of a resource guide (sheet for a content area with key definitions, examples, etc.), and (3) a read aloud with a teacher reading passages and questions verbatim or calculator use for the math test.

A survey of student characteristics is also being administered. Data analysis will examine whether performance differs within and across groups, by administration condition, and by test. Item and test form level information will be included.

The next teleconference at the end of August or early September will continue the accommodations theme. Jim Shriner will address IEP decision making as it relates to accommodations. Sandra Warren from ASES SCASS will co-host, and state presenters will discuss training and monitoring for decision-making and the instruction-assessment links.

In lieu of a pre-session to the CCSSO large-scale assessment conference, NCEO and its technical assistance and research partners are offering an intensive seminar in October. The seminar will be offered in two locations, Denver on October 10-11 and Washington DC/Baltimore area on October 24-25.

Seminar Purpose
States are required to fully include students with disabilities in their assessment and accountability systems. There are many options for appropriately assessing these students, particularly students with the most significant cognitive disabilities. However, states must document and defend the technical quality of whatever assessment option they choose. Alternate assessments based on alternate achievement standards are necessarily less standardized in several key aspects because of who the students are and how they learn and demonstrate their skills and knowledge. Addressing this inherent flexibility and documenting the technical quality has been a tremendous challenge for states because there are few resources to help guide states in this work.

This seminar is designed to assist states in recognizing what aspects of their system should remain flexible and what might be more standardized. The seminar will provide a framework for helping states conceptualize the evaluation of technical quality and help provide an understanding how this flexibility interacts with such evaluation.

Format
Researchers, state practitioners, and technical assistance presenters will conduct this collegial seminar on the options for assessing students with disabilities and implications for technical quality and educational soundness using tools and strategies developed by measurement, curriculum, and special education expert teams over the past 18 months. There will be an interactive working session that blends presentation of research findings, state examples of how these findings were derived and then applied, and periodic checkpoints where state staff can work on how these findings and examples apply to their own situation, using tools and processes facilitated by the technical assistance providers in each region. Over the 1.5 day session, participants will have a chance to interact with all partners, including other participating states, in a supported environment.

Who should attend?
This seminar is designed for state-level leadership responsible for development of the state assessment system for purposes of NCLB. This may include leadership from assessment, curriculum, special education, and other sections. It will be helpful for teams to attend, but far more productive to have a cross-section team attend than to have 2-3 people all representing the same disciplinary background. Also invited are the new regional Comprehensive Centers staff. Seminars will be structured around specific state responsibilities and requirements, and states will sit together in working teams. Enrollment will be limited for meaningful participation. The brochure will be out soon. Please register early to guarantee that your state attends its preferred session. Participants will pay a registration fee to cover meals, meeting space, and materials; seminar partners will cover costs for curriculum development, expert and state presenter stipends and expenses, and for participant materials. Participants will pay their own travel expenses.

Seminar Partners
National Center on Educational Outcomes (NCEO), National Alternate Assessment Center (NAAC), Assessment and Accountability Comprehensive Center (AACC), all Regional Resource and Federal Centers, K-8 Access Center, National Drop-Out Prevention Center for Students with Disabilities, Council of Chief State School Officers (CCSSO), National Association of State Directors of Special Education (NASDSE), National Association of State Title I Directors (NASTID). The seminar will use expertise from the National Center for the Improvement of Educational Assessment, Inc. (NCIEA). It is also sponsored by the New Hampshire Enhanced Assessment Initiative.