|
Investigators at the Universities of Minnesota,
Kansas, and Oregon launched the Early Childhood Research
Institute on Measuring Growth and Development (ECRI-MGD) in
October, 1996, to produce improved methods for continuously
measuring the skills and needs of individual children with
disabilities (or at-risk for developing a disability) from birth
to eight years of age.
ECRI-MGD's
Purpose
One important reason for this project lies in the
ever-increasing demands for greater levels of accountability from
early childhood practitioners, represented in part by Goal 1 of
the National Education Goals (i.e., by the year 2000, all
children in the United States will start school ready to learn.)
and by the Head Start Reauthorization Act of 1998, among other
sources. Without such accountability, national constituencies
have difficulty gauging how well early childhood practitioners
are achieving programmatic objectives and how much more can be
done to optimize services and child outcomes. This is especially
true for young children with disabilities, who have been
traditionally excluded from typical measures of educational
accountability.
Both "consumers" and "providers" of early
childhood services want tools that will increase knowledge of
program effectiveness for individual children and groups of
children. In a survey we conducted early in the project, parents
of young children with and without disabilities and professionals
in early childhood and early elementary education rated the
importance and adequacy of developmental information available to
families as their young children receive educational services.
While roughly 85% of the parents who responded to our survey
attached great importance to information about their children's
development both before and after the children enrolled in
school, only 45% of them indicated the information they received
qualified as "very adequate." In a similar vein, 78% of
the professionals who responded to our survey felt clear,
easy-to-understand information about individual children's
development was "very important" to share with
families. However, only 29% of them indicated they had "very
adequate" information to share.
Another reason for initiating this project centers on the
current state of early childhood assessment and the need for
tools to monitor the growth of young children continuously toward
important, long-term outcomes. Instruments are available that
allow educators to screen young children's skills to determine if
additional assessment is needed. Other instruments assist
educators in determining a child's eligibility for special
education services. Once a child qualifies for special education
services, additional assessment tools may be used to generate
relatively short-term goals by identifying specific skill
deficits in need of intervention. However, none of these
instruments have been designed to monitor young children's
developmental growth efficiently over both short and long periods
of time as these children progress toward broad, valid outcomes,
such as fluency in expressive communication, proficiency in
reading, and skill in positive social interaction. Additionally,
many early childhood assessment instruments have been developed
without paying enough attention to use with the diversity of the
nation's young children who receive educational services,
including those with disabilities, those who speak a language
other than English as their first or primary language, and those
who live in socioeconomically difficult circumstances. Staff of
ECRI-MGD seek to create an assessment system that will fill an
unfilled "niche" in the toolbox of early childhood
assessors, measuring a wide range of young children's growth
toward general outcomes in efficient yet effective ways.
ECRI-MGD's
Scope
Work on ECRI-MGD has focused on three major elements, each of
which will be described in more detail: (a) a set of general
growth outcomes describing important developmental progressions
for children between birth and age eight; (b) growth and
development indicators for monitoring the progress of individual
young children (i.e., individual growth and development
indicators, or IGDIs); and (c) "Exploring Solutions"
Assessments (i.e., ESAs) allowing families and early childhood
and early elementary educators to identify features of classroom
and home settings they can change to improve children's
developmental outcomes.
These elements are used together as part of an ongoing,
decision-making model to enable educators to meet the needs of
individual children. In this model, we use general outcomes and
IGDIs to monitor a child's developmental growth and decide if we
need to intervene in any way to boost the child's trajectory. We
use ESAs to generate options for intervening on the child's
behalf and deciding which options to implement as potential
solutions. Finally, we continue to use outcomes and IGDIs to
evaluate attempted solutions and monitor the child's ongoing
developmental progress.
General
Growth Outcomes
In the project's first two years, we relied on current
research to craft a set of "common" developmental
outcomes describing children's growth between birth and eight
years of age, and then conducted a mail survey of parents and
education professionals to determine the accuracy and
acceptability of these outcomes. It was important to us to craft
these outcomes with four principles in mind
- These outcomes should describe children's growth from
birth through age eight using the same terms, even if
specific behaviors differed across the age range. For
example, we know an infant points to objects to
"tell" an adult she wants it. A preschooler may
simply ask an adult to give it to him. Although the
behavior differs between the infant and preschooler, both
of them use communication to convey their desire to
obtain the object.
- It was important to strike a balance between outcomes
that globally described young children's growth from
birth through age eight while maintaining a manageable
number of statements. Too many specific statements would
make it impossible to design an efficient system of
indicators to monitor children's growth toward the
outcomes. By shaping broadly stated outcomes, we
facilitated creation of indicators that both described
children's development across the entire early childhood
range and would likely not overwhelm teachers and others
who eventually use the indicators.
- We crafted outcomes to capture children's growing
proficiency in reaching developmental endpoints rather
than just the endpoints themselves. That is, we did not
want to identify skills only an eight-year-old child
could perform, even if those skills represented the
long-term outcomes we expect all young children to
attain. Instead, we wanted to make sure these outcomes
described children's changes across time, facilitating
the creation of indicators sensitive to growth.
- It was important to craft outcomes that could be measured
with indicators directly administered to children or
their families, including observations of children in
their natural settings (i.e., home, classroom,
community). It was also important for the outcomes to be
measurable in repeated and efficient ways, again allowing
educators to monitor young children's growth on an
ongoing basis.
Once we crafted these outcomes, we sent them to parents of
children with and without disabilities and professionals in the
fields of early childhood and early elementary education
(including special education) to solicit their feedback about the
accuracy and acceptability of the outcomes. Representatives of
national, child-focused organizations such as the Council for
Exceptional Children (CEC), ZERO-TO-THREE, the National
Association for the Education of Young Children (NAEYC), and the
National Association of School Psychologists (NASP) provided us
with names of members from all regions of the country. Parents
and professionals selected from these lists were asked to rate
the importance of each outcome statement, and they were invited
to offer comments for revising the outcomes. Overall, we received
feedback from 351 parents and 672 professionals. Roughly 70% of
parents and 50% of professionals rated the outcomes as critically
important. Additionally, qualitative feedback from parents and
professionals indicated these outcomes did not require major
revisions to apply to all children, regardless of disability or
other subgroup label. Finally, the vast majority of professionals
(79%) told us a system of indicators that easily and directly
helps them monitor young children's rates of development toward
these outcomes, and helps them plan changes in instruction, would
be "very useful" to them.
Individual
Growth and Development Indicators (IGDIs)
Individual Growth and Development Indicators resemble a
"thermometer" used by a pediatrician to gauge a child's
general health status effectively, efficiently, and quickly. High
readings from a thermometer will alert you that a child has a
fever without telling you why he or she has a fever or what you
can do to reduce the fever. Additional assessment will be
required to understand the source of the fever and to generate
ideas for reducing it. Similarly, our indicators will provide a
relatively quick but effective and efficient reading of a child's
status and growth within multiple developmental areas, alerting
educators and families to the need to intervene on behalf of
children whose trajectories do not meet expectations.
Our team has set multiple criteria for determining if a
prospective indicator qualifies as an IGDI. First and foremost,
the indictor must tell educators the status of an individual
child's skills relative to peers (either within the same
classroom, school, district, state, or from a national
perspective) within the developmental domain of interest (e.g.,
communication, early literacy). Although administration of the
indicator will require a brief period of time, just as with a
thermometer, the indicator must sample enough of a child's
behavior to gauge his or her skill level on the outcome. It is
also important for each indicator to provide valid comparative
information for as many subgroups of children as possible,
including children with disabilities, children from diverse
ethnic and socioeconomic backgrounds, and children who speak a
language other than English as their first or primary language.
Second and relatedly, the indicator must sensitively measure a
child's change in status (or, growth) over time toward the
outcome, again for as wide a range of children as possible. That
is, the indicator must also be able to detect changes in a
child's skills, either positive or negative, over relatively
short periods of time, such as weeks or months. By detecting a
change in a child's skills over such short periods of time,
educators may continuously monitor the child's developmental
progress and intervene on his or her behalf in a more timely way
if the child's growth trajectory does not meet expectations,
either compared to the child's previous pattern of growth or
compared to peers. By extension, indicators must also be
sensitive to the effects of intervention. If an educator decides
to change intervention as a result of observing a child's
insufficient growth with the indicator, the educator will then
depend upon the indicator to tell if any adjustments made
actually lead to a change in the child's growth trajectory.
Third, acceptable indicators must be easy and efficient for
educators to use. Educators typically have limited time to assess
an individual child or groups of children, especially repeatedly
over time. For progress monitoring indicators to succeed across
most classrooms, they must be short in duration, the materials
needed to administer them must be inexpensive and easy to obtain,
and the procedures for administering them must be clear to a wide
range of adults who may use them, including teachers,
paraprofessionals, and parents, among others. To enhance the
efficiency of a system based on these indicators, it may not be
necessary to administer the indicators to all children in a
classroom, for instance. Administration of these indicators to
all children may only need to be accomplished three times a year,
as part of a benchmarking process, with more repeated
administrations to children who appear at risk (e.g., whose
scores fall within the lowest quartile at each benchmarking
administration).
Research
Evaluations
All of the IGDIs described here, as well as new ones under
development, are evaluated in a series of studies conducted in
preschool and day care programs throughout the Twin Cities. While
the details of these research evaluations will not be reviewed
here (but can be found at
http://cehd.umn.edu/ceed/projects/ecri/dissem.html),
results of this research are quite promising. In general,
ECRI-MGD investigators and graduate students have found that
IGDIs correlate with children's age and show growth over time
(evidence of their developmental sensitivity). Further, IGDIs
correlate with other standardized measures within their
respective domains, like the PPVT-3 or the Test of Phonological
Awareness (evidence of IGDIs' validity). While this research
continues, early evidence suggests the value of these measures
for assessing developmental progress.
Examples
of Individual Growth and Development Indicators for Preschoolers
Over the past three years, we have developed and tested
prospective IGDIs for preschool-aged children in three outcome
areas: expressive language, early literacy, and social
interaction. While our research continues to evaluate the
adequacy of these indicators, the following examples appear to
meet the qualifying criteria we have set for IGDIs.
Expressive
Language
Picture
Naming
The format of this indicator entails presenting a child
photographs and detailed, color line drawings of objects commonly
found in preschoolers' natural environments (i.e., home,
classroom, community), one at a time, asking a child to name the
pictures as fast as possible. Categories of objects used in this
format include animals, food, people, household objects, games
and sports materials, vehicles, tools, and clothing. Each
photograph and line drawing is printed on an 8" x 5"
index card. After providing a set of sample items, the examiner
asks the child to look at each card and name it as quickly as
possible. After exactly one minute has expired, the examiner
stops the activity and counts the total number of pictures named
correctly.
Early
Literacy
Alliteration
In this format, adapted from the work of Lonigan and his
colleagues (1998), we identified a set of words commonly known by
preschoolers and obtained photographs or color line drawings of
these words. We assembled stimulus cards with a target photo or
line drawing at the top of each card (e.g., rain) and a set of
three photos/drawings in a row at the bottom of each card (e.g.,
house, rake, pig), one of which starts with the same sound as the
target picture. After providing a set of sample items, the
examiner asks the child to look at each card and point to one of
the three pictures at the bottom of the card with the same
initial sound as the fourth, target picture. The task continues
for a total of two minutes. The score generated from this format
is the number of pictures the child correctly identifies within
two minutes.
Rhyming
Also adapted from Lonigan and his colleagues (1998), this
format is similar to the Alliteration format. Again, after
identifying words commonly known by preschoolers, we constructed
stimulus cards with a target photo or line drawing at the top of
each card (e.g., bees) and a set of three photos/drawings in a
row at the bottom of each card (e.g., pants, gate, cheese), one
of which rhymes with the target picture. After providing a set of
sample items, the examiner asks the child to look at each card
and point to one of the three pictures at the bottom of the card
that sounds the same as (or rhymes with) the fourth, target
picture. The task continues for a total of two minutes. The score
generated from this format is the number of pictures the child
correctly identifies within two minutes.
Segment
Blending
We have assembled a set of words that can be segmented in one
of three ways: by word (e.g., tooth < brush), syllable (e.g.,
pup -py), or phoneme (e.g., r < a < m). The examiner states
aloud a word in its segmented form and asks the child to blend
the segments to verbally produce a complete word. The task
continues for a total of two minutes. The score generated from
this format is the number of words the child blends correctly
within two minutes.
Social
Interaction
Picture
Prompt
In this format, the examiner presents the child with a
photograph of the child's classroom, playground, or other
familiar natural setting. The examiner asks the child to imagine
playing in the setting with a friend. The examiner instructs the
child to say everything he or she could do with a friend in the
setting. The activity lasts for a total of three minutes. One of
the scores generated from this format is the number of novel
activity ideas the child states for playing with a friend in the
pictured setting.
Social
Play
In this format, a child plays with a peer using one of two
types of materials (only one of which is used for each
administration of the format): (a) a "silly face" game
in which children use plasticized pieces of human faces to create
a design on a blank template; and (b) a set of 12-piece puzzles
with puzzleboards. Children participate in this activity for a
total of five minutes. One of the scores generated from this
format is the duration (in seconds) of joint play in which the
target child is engaged. A second score from this format is the
number of interactive responses (or, social turns) the target
child elicits from the peer.
General Growth Outcomes for
Children Between Birth and Age Eight
The child uses language to convey and comprehend
communicative and social intent
- Child uses gestures, sounds, words, or
sentences (including sign language and augmentative and alternative
communication) to convey wants and needs or to express meaning to others.
- Child responds to others' communication with
appropriate gestures, sounds, words, or word combinations (including sign
language and augmentative and alternative communication).
- Child uses gestures, sounds, words, or
sentences (including sign language and augmentative and alternative
communication) to initiate, respond to, or maintain reciprocal
interactions with others.
The child takes responsibility for his/her
behavior, health, and well-being, even in the face of
challenge or adversity.
- Child engages in a range of basic self-help
skills, including but not limited to skills in dressing, eating,
toileting/hygiene and safety/identification.
- Child meets behavioral expectations (such as
following directions, rules, and routines) in home, school, and community
settings.
- Child appropriately varies or continues
behavior to achieve desired goals.
The child negotiates and manipulates the
environment.
- Child moves in a fluent and coordinated
manner to play and participate in home, school, and community settings.
- Child manipulates toys, materials, and
objects in a fluent and coordinated manner to play and participate in
home, school, and community settings.
The child initiates, responds to, and maintains
positive social relationships.
- Child interacts with peers and adults,
maintaining social interactions and participating socially in home,
school, and community settings.
- Child appropriately solves problems in
his/her interactions with others.
- Child shows affect appropriate to the social
context.
The child uses cognitive skills to explore the
environment, reason, and solve problems.
- Child demonstrates an understanding of
age-appropriate information.
- Child demonstrates recall of verbal and
non-verbal events.
- Child understands and uses concepts related
to early literacy and math skills.
- Child solves problems that require reasoning
about objects, concepts, situations, and people.
Field
Testing of IGDIs: Heartland AEA, Iowa
In late 1998, staff at the Heartland Area Education Agency
(AEA) 11 in central Iowa contacted Institute investigators to
discuss the possibility of forming a collaborative relationship.
Specifically, Jeff Grimes, Heartland's Coordinator of Innovation
and Best Practices, Jerry Gruba, Coordinator of Early Childhood
and Literary Services, asked if they could integrate our
Individual Growth and Development Indicators into their current,
decision-making model for identifying and serving young children
with disabilities. Heartland AEA serves approximately 120,000
students, including roughly 13,000 students receiving special
education services, across 56 public and 34 non-public school
districts. Agency staff do not rely on traditional methods for
identifying and serving children with disabilities (i.e., based
on extensive use of standardized tests). Instead, they place more
emphasis on using functional assessment, progress monitoring
methods, data-based decision making, and trends in individual
child performance to make high-stakes decisions about children's
educational needs. Based on the close fit between Heartland's
approach to serving children with disabilities and the
Institute's conceptual foundations, we agreed to work out a
collaborative agreement for testing Individual Growth and
Development Indicators within Heartland's problem-solving model.
Meetings between Heartland and Institute staff were held in
May, September, and December of 1999 to provide Heartland with
detailed information about the purpose and goals of the
Institute, and specific training in the use of expressive
language IGDIs for infants, toddlers, and preschoolers. Heartland
staff in attendance at these meetings included three individuals
hired prior to the start of the 1999-2000 academic year
specifically to help the agency implement tools and procedures
from our Institute. This team consists of a school psychologist
(Susan Ward), an early childhood consultant (Stacey Dun), and a
speech/language pathologist (Kim Thomas).
Ongoing contacts between Heartland and Institute staff have
led to setting the following two priorities in our collaborative
efforts: (a) extending the normative data collected thus far on
the validity, reliability, sensitivity, and utility of Individual
Growth and Development Indicators for as wide a range of young
children as possible; and (b) extending the Institute's research
on the sensitivity of IGDIs to the effects of intervention for
individual children within a decision-making model.
Exploring
Solutions Assessments
What can an educator do to generate ideas for intervening on
behalf of children whose growth trajectories, as measured by
IGDIs, do not appear to meet expectations? Staff of ECRI-MGD have
explored the use of three different types of tools to assist
educators with this endeavor, each of which may be used alone or
in combination with each other, depending on preliminary analyses
of child-related concerns.
- We have identified critical program features of natural
settings in which young children play and learn, features
that have been empirically linked with children's
developmental growth of their expressive language and
early literacy skills across time. These features have
been organized by age group (i.e., birth to
three-years-old, three-to-five-years-old, and
five-to-eight-years-old).
They have also been organized as an observation form for
educators to use to rate the broad quality of settings in which
children participate. Ratings by teachers or external observers
may lead to general changes in setting arrangements, availability
of materials, or curricular goals, benefiting a group of children
rather than just one child.
- We have begun to develop an activity-based assessment for
more fine-grained observation of an individual child's
developmental strengths and needs. This assessment will
utilize typical, everyday activities in a child's routine
(e.g., pretend play, snack) to create structured,
standardized scenarios in which an adult (or adults)
interacts with the child in scripted ways to elicit
behavior from the child across developmental domains.
Observation of the child's behavior in these structured
contexts will enable educators to evaluate the child's
skills and needs in greater detail than will be possible
through administration of IGDIs, which in turn should
lead to specification of areas to target through
intervention.
- We have utilized ecobehavioral assessment to evaluate the
effects of interactions between a setting's ecological
factors (e.g., type of activity, materials available,
group structure) and adult behavior on a child's
behavior. Ecobehavioral data allow us to identify in
minute detail the ways in which specific combinations of
ecological factors and adult behavior appears to
accelerate or decelerate a child's behavior (e.g.,
providing opportunities or lack thereof and reinforcement
for the child's use of expressive language). By
identifying specific combinations that accelerate desired
behavior for a child whose growth trajectory does not
meet expectations (based on IGDI data), we may begin to
mold an intervention strategy.
- to better meet the child's needs. Three computerized
instruments are available to conduct ecobehavioral
assessment, all of which were developed at the Juniper
Gardens Children's Project at the University of Kansas:
the Code for Interactive Recording of Caregiving and
Learning Environments (CIRCLE) for observation of infants
and toddlers, the Ecobehavioral System for Complex
Assessments of Preschool Environments (ESCAPE) for
observation of preschoolers, and the Code for
Instructional Structure and Student Academic Response:
Mainstream Version (MS-CISSAR) for observation of early
elementary-aged students.
Next
Steps
During the 2000-2001 school year, ECRI staff will develop and
test IGDIs for monitoring the progress of young children's motor
(or, movement) and adaptive skill development. Studies of the
adequacy of prospective indicators, using the criteria earlier in
this report, will involve working with a diverse range of
children, including children with disabilities and children
enrolled in Head Start classrooms.
Additionally, we will initiate two new projects at the
University of Minnesota. The first new project focuses on
development of a Web-based application for helping teachers
obtain IGDIs and related measures, manage data they collect using
these measures, and collaborate with others in designing and
evaluating interventions for children whose growth does not
appear to meet expectations. Named Get It, Got It, Go!, this Web
tool will provide educators with information on using measures
such as IGDIs to monitor the ongoing progress of young children.
Detailed administration procedures, stimulus materials (e.g.,
picture cards), and scoring forms will be available for teachers
to download from the site and use after a minimum of self-study.
Once teachers begin to use IGDIs or related measures to collect
data from individual children, they will be able to enter such
data into a password-protected, secure area of the site. In
return, teachers will receive reports of each child's status and
growth over time, compared to the child's previous performance
and to results from groups of peers in the same classroom,
school, district, or in other programs. Finally, teachers will be
able to "invite" fellow teachers, parents, University
faculty, and others to participate in a password-protected
threaded discussion for interpreting individual child reports,
planning interventions, and evaluating the effects of
interventions.
The second new project, entitled Improving Preschoolers'
Reading Outcomes through Measurement and Intervention in
Classroom Environments (I'PROMICE), has two major goals. First,
we will examine the connections between expressive language/early
literacy IGDIs for a diverse group of preschoolers and comparable
measures for kindergartners and first-graders. This effort may
enable us to begin to identify "typically developing"
children as young as three who may experience difficulties
becoming proficient readers by the end of third grade. Second, we
will collect detailed observations of preschoolers participating
in regular classroom activities related to their expressive
language and pre-literacy skill development. These observations
will serve as the foundation for elaborating an intervention
program for use with preschoolers whose growth of expressive
language and early literacy skills may not appear to meet
expectations.
Investigators
Center for Early Education and Development, University
of Minnesota
* Scott McConnell
* Mary McEvoy
* Jeff Priest
* Carol Leitschuh
Juniper Gardens Children's Project, University of
Kansas
* Judy Carta
* Charles Greenwood
* Dale Walker
* Gayle Luze
* Deb Linebarger
School Psychology Program, University of Oregon
* Ruth Kaminski
* Roland Good III
* Karen Rush
* Sylvia Smith
Related
Courses at the University of Minnesota
EPsy 5200-03 Hierarchical Linear Modeling
EPsy 5231 Introductory Statistics and Measurement in
Education
EPsy 5616 Behavior Analysis and Classroom Management
EPsy 5849 Observation and Assessment of the Preschool Child
EPsy 8706 Single Case Designs in Intervention Research
|
Fall 2000,
Volume 27, Issue 2