Long Term Effect of Teacher Enhancement: Scope, Sequence
& Coordination (SS&C):
Overview of Evaluation
Scope, Sequence & Coordination
(SS&C) is a national teacher enhancement and curriculum development project
committed to developing activities that help students become more scientifically
literate as defined by the
National Science Education Standards (NRC, 1996). The SS&C project is
guided by the following principles:
-
every student should study every science subject every year,
-
science should explicitly take into account students' prior knowledge
and experience,
-
students should be provided with a sequence of content from concrete
experiences and descriptive expression to abstract symbolism and quantitative
expression,
-
concepts, principles, and theories should be revisited at successively
higher levels of abstraction, and
- learning should be coordinated in the four science subjects so as
to interrelate basic concepts and principles.
SS&C was funded by the National Science
Foundation to develop and implement 9th and 10th grade science activities
and this evaluation was designed to document the effect of the SS&C project
in relation to the NRC standards.
The evaluation used a post-test only, quasi-experimental design. In this
approach the performance of 9th and 10th grade students who received the
treatment (in this case the SS&C activities) was compared to the performance
of 9th and 10th grade students who did not receive the treatment. The comparison
groups were the previous year students in the same schools as those students
who received the SS&C activities. In essence, it was a time-lag design where
the prior year's ninth and tenth grade science students were compared to
the following year's ninth and tenth grade science students.
Thirteen high schools were originally included in the project. They were
selected by the National Science Teachers Association (NSTA)
to be representative of the diversity in the US in terms of geographic areas,
population and race/ethnicity. Three schools were located in Houston, TX;
one in Sacramento, CA; one in Riverside, CA; two in a suburban area of IA;
one in northeast NC; one in Kalispell, MT; one in White Plains, NY; and
one in Washington, DC. There were approximately 50 ninth and tenth grade
science teachers at these schools and over 2,000 students being taught by
these teachers each year.
Data were gathered at the eleven school sites using ten instruments developed
by the evaluation team. Contextual variables were assessed using a classroom
observation schedule, principal, teacher and student interview protocols,
a teacher questionnaire, a student questionnaire, and a course content survey.
Because the major measure of the effectiveness of the SS&C project was student
achievement, it was examined from several different perspectives. Students
answered multiple choice and open ended science content items, participated
in a five station hands-on laboratory skills test and designed and conducted
an experiment. Standardized administration and scoring protocols were developed
to ensure consistency and objectivity. Descriptions of all assessment instruments,
the instrument development process, and the psychometric properties of the
instruments are included in Appendix of the ninth grade evaluation report.
At each school data were collected from all participating ninth and
tenth grade students and teachers. In addition three classes at each
school were targeted for more comprehensive data collection. All the
data were collected by the evaluation team during site visits, except
for the science literacy test and the course content survey which were
mailed to the schools near the end of the school year. During spring
visits, the school principal and the teachers were interviewed, and the
three target classes were observed. In addition to the interviews and
observations, most of the students were given a questionnaire to
complete. Those not taking the questionnaire were 6-12 of the students
in each target class who were randomly selected to take the performance
tests administered by the evaluators. One student from each target class
was also interviewed.
The evaluation team at the University of Minnesota functioned separately
from the design and implementation of the SS&C project to maintain objectivity.
Evaluators did not provide formative feedback on the design and implementation
to project participants and all evaluation data were provided anonymously
to project directors. As a further check of validity the evaluation was
continuously monitored by a meta-evaluator, Dr. Wayne Welch, an expert in
both science education and program evaluation.
Four questions guided the SS&C evaluation:
-
Is the learning environment in the SS&C science classes different from
the learning environment in comparison science courses?
-
Are SS&C students more motivated about science than comparison students?
-
Do SS&C students have a better understanding of science concepts than
comparison students?
- Are there differences between SS&C and comparison science classes
when data are analyzed by school and sex of the students?
Current Evaluation Efforts
The evaluation team at the University of Minnesota has received funding
from the National Science Foundation to conduct a long-term follow-up study
of the SS&C project. We are currently following up at five of the original
thirteen schools: Fox Lane High School, Kalispell High School, Pleasant
Valley High School, Sacramento High School, and Yates High School. We are
in the process of examining two different aspects of the SS&C project:
-
The long-term effect of the SS&C project on future cohorts of 9th grade
science students will be examined. We will compare the original 9th grade
implementation of SS&C, which occurred during the ‘95-’96 school year,
to 9th graders in 1998, 1999, and 2000. The results of these comparison
will be available in the fall of 1998, 1999 and 2000, respectively.
- The long-term effect of the SS&C project on 12th grade students who
took SS&C in the 9th grade. We will compare 12th grade students who took
SS&C to those 12th grade students in the same schools who did not take
SS&C. The results of this evaluation will be available in the fall of
2000.
|
