Research/Practice - Fall 1999 - Achieving the Science Standards (CAREI)

Ms. Moore's life science class was investigating the response of earthworms to various environmental conditions. She had a general idea of what they would find out, but was not exactly sure what interesting things students would find on their own. She had placed a variety of materials on the back shelf for students to use. She welcomed the students as they came in and asked them to sit in their lab groups. She stated the topic for the day's class and then asked student what they already knew about it. She wanted to know if anything from the life science portion of the class related to this topic. Finally, she asked the students a question and asked how they might go about finding the answer. After listening to a few suggestions, she directed the students to use the materials in the back to determine an answer. While the students worked in their groups, Ms. Moore helped as necessary when the groups could not answer their own questions. When all the groups had finished collecting their data, Ms. Moore asked them to report what they found to each other. When a disagreement arose between two of the groups, Ms. Moore suggested they check their answers by doing some more lab work the next day. The students left the class debating their answers.

This example highlights some of the unique science classroom features created by the National Science Teachers Association as part of the SS&C project (Scope, Sequence & Coordination). The SS&C project created a new high school science curriculum that coordinated the content in the four basic sciences (life, earth, physics and chemistry) to allow students to study every science every year. The curriculum sequenced activities to encourage teachers to use inquiry-based instruction where students engage in hands-on activities before teachers define concepts.

The typical U.S. high school science curriculum is layer-caked with a different science taught each year. Generally, physical science is taught in grade 9, biology in grade 10, chemistry in grade 11, and physics in grade 12. With graduation requirements of only two years of high school science, many students end up studying only physical science and biology, never taking courses in chemistry or physics. In addition, typical U.S. science classes rarely emphasizes the more inquiry-oriented techniques recommended in the new NRC National Science Education Standards (NRC, 1995). The Standards recommend that students engage in full scientific inquiries including formulating questions, designing investigations, gathering evidence, and communicating results. In response to the problems associated with the traditional layer-cake approach to science teaching and the lack of inquiry-based science instruction, the National Science Teachers Association designed and implemented the high school science curriculum called Scope, Sequence and Coordination (SS&C).

To examine the impact of SS&C, researchers at CAREI designed a comprehensive study comparing students who took SS&C science in 9th and 10th grade to students who did not take the new course. The study used a time-lag design which compares the prior year's science students to the present year's science students. The purpose of the study was to closely examine the effect of the standards-based curriculum on both the classroom learning environment and on students' achievement in the sciences. Thirteen schools implemented the new science course. The schools were located in California, Iowa, Montana, New York, North Carolina, Texas and the District of Columbia and included more than 4,000 ninth graders and 2,500 tenth grade science students.

Based on the premise that the learning environment affects student achievement, researchers assessed both the learning environment and student achievement. Using a nested data collection design, the evaluation team developed ten instruments to gather information at each school (See Figure 1). The team collected data at each school from all participating ninth and tenth grade students and teachers. In addition three ninth and tenth grade classes at each school were targeted for more comprehensive data collection, including observations and interviews with teachers and students. The evaluators also selected a random sample of students in each target class to take a science performance test.

Figure 1: Data Collection Instruments and
Sources of Information

Evaluators assessed the learning environment using a classroom observation schedule, principal, teacher and student interview protocols, a teacher questionnaire, a student questionnaire, and a course content survey. Because student achievement was the primary measure of the effectiveness of SS&C science, evaluators examined it from several different perspectives. Students answered multiple choice and open ended science content items, completed a hands-on laboratory skills test, and designed/conducted a hands-on experiment test.

The student questionnaire developed by the authors assessed student perception of the science classroom learning environment (Huffman, Lawrenz & Minger, 1997). The instrument included six different scales designed to measure the inquiry nature of the class. The student questionnaire also included items about the frequency of class activities, and student attitude and motivation about studying science. The four science achievement tests used in this study were designed to measure students' achievement of the National Science Education Standards. All of the items on the tests were selected from existing national sources such as the National Assessment of Educational Progress (NAEP), International Assessment of Educational Progress (IAEP), and the Second International Science Study (SISS). Almost half of the items required higher-level reasoning at the application level of understanding or higher. The hands-on laboratory skills test included five different laboratory stations, one for each content area (earth science, life science, physics, and chemistry) and one on the use of science instruments. The hands-on full investigation measured a student's ability to design, conduct and draw conclusions from a scientific experiment. It was modeled after the test developed by APU. In this test, students were given approximately 25 live sowbugs, and asked to conduct an investigation to determine which of four environments the sowbugs preferred: damp/dark, dry/dark, damp/light, or dry/light. Overall, the achievement test provided a wide range of information about students understanding and skills; all the way from students' ability to answer written questions about science concepts to students' skills at actually designing and conducting experiments.

Results

The results from the student questionnaire and observations of classes were quite impressive.

Figure 2: Frequency of Class Activities

(1 = Never; 2 = < once a week; 3 = About once a week; 4= Several times a week)

However, there were no differences between groups on the items related to studying more science in the future. Observations made by the research team confirmed the results on the student questionnaires. SS&C classes were observed to be more student-centered and less teacher-centered than the comparison classes. In other words, observers found that students engaged in more inquiry-oriented, hands-on activities and spent less time using traditional lecture oriented instruction. (See Figure 3). The achievement results were not quite as positive as the learning environment changes, however, there were indications that the SS&C course has affected students' laboratory skills. The results on the multiple-choice test were essentially identical between the two groups, but on the lab skills and experimental design test there were some positive differences at several of the sites.

Figure 3: Observations of percent of Class Time Spent on Different Types of Instruction

Changes to Classroom Come With Both Costs and Benefits

The evaluation results provide some interesting insights into teacher enhancement efforts and raise several important questions. These data show that the classroom environment was significantly changed by the SS&C Project. The learning environment in the SS&C classes was more in line with the types of classes recommended in the NRC standards with more student interaction, more hands-on activities, more inquiry, and less lecture. These differences were corroborated by observations, by teacher self report and by student surveys. The SS&C teachers produced class environments that explicitly took into account students' prior knowledge and provided a sequence of content from concrete experiences to conceptualization just as expected from the curriculum. The classes also covered all four areas of science, life, earth, chemistry and physics with some degree of coordination among them.

However, there were costs and benefits to these changes to the classroom environment. The student interviews and surveys showed that the students were quite pleased with their classes. The students thought the classes were fun, not too hard and that the best things were the hands-on activities. The teacher interviews, however, revealed mixed feelings about the implementation effort. The issues coalesced into three areas, materials, presentation method and content.

The teachers felt somewhat overwhelmed by the SS&C curricular requirements. Organizing and selecting materials for all the hands-on activities created quite a time burden. The teachers felt they had to spend unreasonable amounts of time preparing hands-on laboratory materials for the students to use. Furthermore, teachers often reported having to buy materials on their own or having to use materials that were not quite right.

While the SS&C hands-on approach and asking students use the materials before the concepts were presented were good ideas, teachers felt it was just too difficult to do as often as the curriculum recommended. Many teachers felt that the students needed more help to understand the concepts. They wanted to explain the concepts to students and believed that the SS&C sequence of teaching would result in the students not really understanding the concepts and not learning the necessary facts and vocabulary.

Teachers found coordination among the sciences difficult. They felt the students got a little bit of earth science and then were off to something else in life science or physical science. In situations where one teacher was teaching all of the sciences, some of the teachers felt that they did not have enough background knowledge to teach all the different science disciplines. On the other hand, in sites where the SS&C classes were taught by several teachers with the appropriate content expertise, some of the teachers felt that they did not get to know their students well enough and that it was difficult to coordinate among the science disciplines with different teachers. Clearly there were some interesting tradeoffs. School districts considering a switch to more inquiry-oriented science need to be well aware of the added workload requirements.

The most immediate effect of more stimulating environments are not necessarily improved student attitudes toward science. The students in the first year of SS&C did seem to think their classes were more motivating, but by the time the students were in the second year of SS&C, they were somewhat blasé about it. This seems somewhat unusual given the more involving classes, but perhaps the positive first year results occurred because SS&C was so different from standard science classes. The students were slightly more interested, but this novelty effect wore off by the second year. This interpretation is supported by results of the old ISCS materials in the 1960's that were completely lab based. The students really loved the approach for a while. However, that after some time students were asking their teachers to lecture to them just for a change of pace. SS&C seems less one sided than ISCS, since it involves group work and class discussions, but the questions about novelty and motivation remain the same.

Student Gains Take Time

A causal model of teacher enhancement would suggest that changes in the learning environment should lead to changes in student achievement. However, so far only small achievement changes have been found. There were no major differences in achievement as measured by the variety of assessment formats even when the students had taken SS&C for two years in a row. It must be kept in mind that the achievement tests were not tied to the specific content covered in the SS&C curriculum. Tests were tied to the NRC standards and therefore were a very broad look at student achievement not a look that was designed to determine if a student learned what was specifically taught.

It is difficult to explain the small changes given the strong changes in the learning environment. It is possible that the learning environment has less effect on student performance than we would like to believe. This notion is supported by the fact that there were no differences in achievement even when students had studied a particular science area for the entire year. For example, in the grade 10 comparison group, almost all of the students took life science for the whole school year, yet they did not score higher in life science than the SS&C students who only had life science part of the time. Also the SS&C students who had studied earth sciences did not perform better than the comparison group students who had not studied those areas.

Perhaps the real key to student outcome is within the student. Certainly this fits with motivational and social context theory. It may be unreasonable to expect a teacher enhancement effort to have such broad sweeping effects on student outcomes. Perhaps teachers should be assessed only on whether they changed their behaviors and whether the students learned material specific to the enhancement.

On the other hand, it might take a significant amount of time to see changes in learning outcomes. Two years may not be enough time. Some educators believe that in large curriculum projects first order effects, (like changes in the learning environment) are reasonable to expect, however, second order effects (like changes in achievement) are much more difficult to produce and may take much longer to actually appear.

To consider these alternative explanations, CAREI is in the process of conducting a longitudinal study at selected SS&C schools. The purpose of this long-term study is to examine the impact of SS&C over several years. We are looking at two different groups of students. We are: 1) following 9th grade SS&C students all the way into 12th grade to examine the impact of SS&C on this cohort of students, and 2) examining future cohorts of 9th grade SS&C students. It is possible that the SS&C classes will eventually have a positive impact on achievement and this longitudinal study will seek to examine this possibility. We are also looking more closely at individual sites using a case study approach. Some sites did show positive changes in student achievement and we are following up with particular sites to better understand why SS&C may have worked at some sites but not at others. The results of this continuing study will help educators better understand the impact of an inquiry-based, hands-on science curriculum.

References

Huffman, D.; Lawrenz, F. & Minger, M. (1997). Within-class analysis of ninth grade science students' perceptions of the learning environment. Journal of Research in Science Teaching, 34 (8), p. 791-804.

National Research Council (1995). National Science Education Standards. Washington, DC: National Research Council.

	Return to: U of M Home
	One Stop \| Directories \| Search U of M

Achieving the Science Standards: A National Study of Inquiry-Based Instruction in High School Science