WHO WE ARE
|
PUBLICATIONS | PRESENTATIONS | PROJECTS
| RELATED SITES | STAFF
| SITE MAP
| SEARCH
| WHAT'S NEW
| HOME
WHO WE ARE
|
PUBLICATIONS | PRESENTATIONS | PROJECTS
| RELATED SITES | STAFF
| SITE MAP
| SEARCH
| WHAT'S NEW
| HOME
Matching State Goals to a Model of Outcomes and Indicators for Grade 8
by Patricia Seppanen, Rod Schaefer, and Nicole R. Julian
Published by the National Center
on Educational Outcomes
in collaboration with St. Cloud State University and the National Association
of State Directors of Special Education
October, 1995
This document has been archived by NCEO because some of the information it contains is out of date.
Any or all portions of this document may be reproduced and distributed without prior permission, provided the source is cited as:
Seppanen, P., Schaefer, R., &Julian, N. R. (1995). Matching state goals to a model of outcomes and indicators for grade 8 (Technical Report No. 16). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Retrieved [today's date], from the World Wide Web: http://education.umn.edu/NCEO/OnlinePubs/Technical16.html
Overview
In 1990, the President and governors of the United States agreed upon six national education goals. Their purpose was to help improve the quality of education by setting high standards and focusing on how well our society is able to achieve them. The original six goals (and two others) have become part of education reform law and at least ten different standards-setting groups have been working to set out guidelines of what U.S. students should know and be able to do. The passage of the Goals 2000: Educate America Act, along with other education reform initiatives such as the School to Work Opportunity Act and the Improving America's Schools Act (the former Elementary and Secondary Education Act) are designed to further stimulate standards-based assessment and reform in schools across the nation.
States have been following closely on the heels of these national reform initiatives. Within six months of announcing the national educational goals, 18 states had announced their own versions of the goals, and within one year 44 states had done so. Many states have gone on to articulate learner outcomes, objectives, performance standards, and benchmarks/indicators. And, building on the Goals 2000 work, most states are now using language that includes all students in their educational reforms, including students with disabilities.
At the same time that these reforms were initiated, the National Center on Educational Outcomes for Students with Disabilities (NCEO) began its work by identifying a conceptual model of outcomes and indicators appropriate for all students, including students with disabilities (Figure 1). Using a multi-attribute, consensus-building approach (Vanderwood & Ysseldyke, 1993), hundreds of stakeholders from a variety of perspectives (including national reformers, special educators, school administrators, teachers, parents, measurement experts, legislators, and representatives of advocacy groups) contributed to the articulation of eight major outcome domains.
The model articulates outcomes and indicators at key stages in a student's development: age 3, age 6, grade 4, grade 8, school-completion, and post-school. In Figure 2, the specific outcomes within each domain are provided for the grade 8 level. Possible indicators of each outcome have also been identified. The overall design, from domain to outcomes to indicators, is shown in Figure 3 on the following page.
One of NCEO's activities is to check the extent to which there is correspondence between state articulated student outcomes and the outcomes specified in the NCEO conceptual model. This matching activity also gives us the opportunity to present an inventory of the outcomes and indicators that have been articulated by each state at the grade 8 level. We believe this information will be useful to state and local level practitioners involved in the articulation of educational goals, performance standards, assessments, and curriculum frameworks at different age and grade levels.
Figure 2. NCEO Outcome Domains and Outcomes for Grade 8
| OUTCOME DOMAIN | OUTCOME |
|
|
Method
The process of matching the educational goals, outcomes, and standards adopted by states to NCEO's list of outcomes and indicators included three distinct stages.
Stage 1: Obtaining State Documents
During the Spring of 1994, we mailed letters to all Commissioners of Education or State Superintendents requesting copies of their states' most recent student outcomes, standards, or goals document(s). In the Summer of 1994, we sent out a second letter to states from which we had not received responses. At this point, we asked nonrespondents to verify whether these documents (a) have not been published at the state level, or (b) are under development. A total of 48 states (including the District of Columbia) responded, either submitting documentation or verifying that the documentation is not available or is currently under development and not available for review. Thirty-six states submitted some type of documentation. Of the states submitting documentation, 30 included information related to goals, outcomes, standards, or indicators that could be compared to the NCEO conceptual model at grade 8.
Stage 2: Selecting Documents to Match at the Grade 4 Level
States have developed various documents related to state articulated education goals, outcomes, and standards. We selected the state documents that most specifically reflected learner goals, objectives or standards, and indictors, without delving into curriculum-level materials or state assessment test items. When states submitted multiple types of documents, we considered them for inclusion in the mapping activity in the following priority order:
Statements of learner goals, objectives, outcomes, performance standards, benchmarks, and/or indicators that typically were related to state assessment systems;
Statements of curriculum standards or frameworkds that include specific statements of learner goals, objectives, performance standards, benchmarks or indicators;
Statements of state education goals;
Statements of educational program standards or opportunity-to-learn standards.
Only a few states target educational goals toward specific ages or grades of students. A number of states have a single set of goals that cover kindergarten through grade 12; others have clusters of age or grade related goals (e.g., K-3, 5-8, and 9-12). In many states, the grades or ages included in the cluster vary by subject or domain area.
As a result, two NCEO staff independently reviewed the documents submitted by each state to (1) select the type of document that would be used in the matching activity, and (2) specify the age or grade levels that would be matched to the grade 8 level of the NCEO model. Discrepancies between the reviewers were resolved by group consensus, and/or review by a third individual. The document used as part of the matching activity is listed and briefly described at the beginning of each state list of goals in Chart 4.
Stage 3: The Matching Process
NCEO's model is presented in three levels that become increasingly more specific: Domains, Outcomes, and Indicators. Matching was done at each of these levels in Charts 1-3. In addition, we present a listing of each state's goals that we used in the matching process in Chart 4. More specifically, the following sequence was used to complete the matching process.
State Articulated Goals: States' goals were first listed using their format as much as possible (see Chart 4). We then matched the NCEO domains, outcomes, and indicators to these state goals. Matches were first established at the domain level. If the state goal fit within the NCEO domain, a "deeper" match at the outcome and indicators levels was sought. The deepest possible match to the NCEO model is recorded in a space next to the state goal.
The Indicator Level: Using the information from Chart 4, we then reversed the process and matched the state goals to the NCEO model at all three levels: Indicator, Outcome, and Outcome Domain. If possible, matches were made first at the indicator level. If this was not possible, we then looked to match a state goal with an outcome, and then a domain. Chart 3, which shows the results of this process, contains an "X" at the deepest level of match. Thus, when an "X" appears at the domain or outcome level, the match is generally less precise than if it were at the indicator level.
The Outcome Level: If the state has one or more goals that fit under one of the NCEO outcomes (at the indicator or outcome levels), we put an "X" in the outcome box and also in the broader domain box (see Chart 2).
The Domain Level: If the state has one or more goals that fit under a specific NCEO domain (at any level), an "X" was put in the box for that domain (see Chart 1).
As is often the case in content analyses, the concepts included in state articulated goals do not provide a one-to-one correspondence with the concepts included in NCEO's domains, outcomes, or indicators. Thus, several decisions had to made by the reviewers. The following decisions provide an illustration of the reasoning used in the matching process.
The degree of specificity in the states' goals and the NCEO model are not always the same. Since the intent of our review was to examine the overall correspondence between state goals and the NCEO model, we sometimes match specific goals listed in the state document to an NCEO domain. A match with an NCEO domain, therefore, does not necessarily indicate the state has embraced all the NCEO outcomes and indicators within that domain.
The state goals sometimes contained more than one concept and seemed to fall under more than one NCEO domain, outcome, or indicator. In these instances, we matched the state goal to as many domains, outcomes, or indicators as seemed appropriate. Thus, the state goal Students will participate in problem-solving activities so they can use concrete models to develop an understanding of concepts of addition, subtraction, multiplication, and division matches to three NCEO outcomes or indicators: (A2a) Percent of time students participate actively in a variety of meaningful learning activities and routines in general education classrooms, (F2a) Percent of students who demonstrate problem-solving and critical thinking skills, and (F3a) Percent of students who demonstrate competence in math to function in home, school, and community environments.
NCEO's outcome indicators are written in the form of finding a percent of the number of students that meet a particular indicator. An example of an indicator is Percent of students who meet individualized standards of physical fitness. Most state goals are not written using this language. Although the form of measurement for the state goal may not be the same, the two were matched if the same general concept were discussed in both.
General Findings
The following general findings emerged when state goals were matched to NCEO's model at the domain level:
All 30 of the states that have articulated goals for students at grade 8 included statements that correspond to the NCEO outcome domain, Academic and Functional Literacy.
Between one-half to three-quarters of the states we examined specified goals that corresponded to the following NCEO domains:
Personal and Social Adjustment (22 out of 30)
Few states identified goals that were matched to two NCEO domains: Accommodation and Adaptation (four out of 30) and Satisfaction (two out of 30)
We also matched state goals to the NCEO model at the outcome level. This analysis takes us one step "deeper" (or more specific) into the NCEO model. We examined the general degree of match between the states' goals and the overall group of outcomes within each NCEO domain. The key question we asked is: To what extent do states identify student goals that correspond to the outcomes specified under each domain of the NCEO model? To answer this question, we determined the proportion of states that articulated goals that correspond to the outcomes specified under each domain of the NCEO model. General findings include:
A high proportion of states (more than 75%) specified goals that correspond to most of the outcomes under the NCEO domain, Academic and Functional Literacy.
All four of the outcomes under the domain Personal and Social Adjustment were matched to goals of at least one-half of the States we examined
A moderate proportion of states (50-75%) articulated goals that generally corresponded to at least one outcomes under the following domains:
Contribution and Citizenship (outcome: Complies with school and community rules)
Given the poor correspondence at the domain level, it is not surprising that few states (less than 25%) have articulated goals that correspond to outcomes under the following two NCEO domains: Accommodation and Adaptation, and Satisfaction
The NCEO model includes a number of indicators for each outcome statement. We grouped states in terms of the degree of correspondence of goals with NCEO indicators. Strong matches represent more than 75% of the states. Moderate matches represent 50-75% of the states, while weaker matches represent less than 50% of the States. General findings include:
There was a strong match (greater than 75%) for most indicators under the domain, Academic and Functional Literacy
The following indicators (followed by their domain and outcome in parenthesis) received a moderate number of matches (50-75%)
Percent of students who engage in productive group work (Domain: Personal and Social Adjustment; Outcome: Gets along with other people)
The remaining NCEO indicators were only weakly matched to state goals. This overall lack of correspondence, however, may be due more to the level of specificity being used by States to articulate goals than a lack of conceptual congruence.
Reference
Vanderwood, M.L., & Ysseldyke, J.E. (1993). Consensus building: A process for selecting educational outcomes and indicators (Outcomes and Indicators Number 2). Minneapolis, MN: National Center on Educational Outcomes.
WHO WE ARE | PUBLICATIONS | PRESENTATIONS | PROJECTS | RELATED SITES | STAFF | SITE MAP | SEARCH | WHAT'S NEW | NCEO HOME
SPECIAL TOPIC AREAS: ACCOMMODATIONS | ACCOUNTABILITY | ALTERNATE ASSESSMENTS | GRADUATION REQUIREMENTS | LEP STUDENTS | OUT-OF-LEVEL TESTING | PARTICIPATION | REPORTING | STANDARDS | UNIVERSAL DESIGN
National Center on Educational Outcomes Web
site: http://education.umn.edu/NCEO
© 2006 by the Regents of the University of Minnesota.
The University of Minnesota is an equal opportunity educator and employer.
Online Privacy Statement
This page was last updated on Friday, October 27, 2006.
This Web site is produced by the National Center on Educational Outcomes through a Cooperative Agreement (#H326G050007) with the Research to Practice Division, Office of Special Education Programs, U.S. Department of Education. Additional support for targeted projects, including those on LEP students, is provided by other federal and state agencies. Opinions expressed in this Web site do not necessarily reflect those of the U.S. Department of Education or Offices within it.