Grade-Level Standards-Based Science Outcomes for English Language Learners and Language Minority Students: A Review of the Literature

In this state, language minority students and ELLs, along with native English speaking students, would need to learn the vocabulary words "domain", "kingdom", "phylum", "class", "order", "family", "genus", and "species" in order to correctly classify organisms.

In state 3 the following standard not only requires students to learn particular vocabulary items, but also asks them to internalize a Western scientific way of thinking about catastrophic events and to communicate that Western explanation.

Some language minority students and ELLs may come from a culture where natural disasters are explained as acts of an angry god or punishments for human misbehavior rather than scientific phenomena (Lee, 2003). For such students, switching to a Western scientific view of these phenomena requires more than simply learning facts, it requires a shift in values and beliefs as well (Lee, 2003). In addition, this standard also requires students to understand and use scientific discourse patterns (e.g., using evidence to support predictions and conclusions, making inferences, etc.) as part of demonstrating the knowledge of content.

Looking at samples of science standards provides important context for reviewing studies of standards-based science outcomes. These selected state standards illustrate the kinds of areas in which language minority students and ELLs need to demonstrate academic achievement and provides examples of some of the specific content and skills they may need to learn.

A multi-step search procedure identified literature on science outcomes or language minority students and ELLs. First, a library meta-search engine that could search multiple databases at the same time was used to look for articles in databases such as ERIC, Educational Full Text, PsycInfo, and Linguistics and Language. Search terms included combinations such as science and English language learners, science and language minority students, science and bilingual students, science and senior high, science and junior high, science achievement and student outcomes, etc. Second, reference lists of identified articles were scanned for additional resources. Finally, an Internet search using combinations of search terms similar to those used with the meta-search engine was used to verify that the relevant articles had been located.

The following criteria were used to determine which articles to review:

From this search process, 22 articles were identified that met all of the study criteria (see Appendix A for a complete listing). They were all reviewed, regardless of whether outcomes were standards-based or not, in the belief that the total group of studies would have something to say about the priorities and interests of researchers in the field. A study coding sheet was used to highlight the research design of each study along with the theoretical and conceptual frameworks underlying the research and the links between the research and standards-based instruction or outcomes.

The 22 studies included in this review represent diverse research perspectives, and a multitude of research designs (see Appendix A for a complete listing of studies, research methods, measures, study samples, and findings). These studies described a variety of outcomes, and represented varying degrees of connection to state standards. Some studies followed rigorous research procedures while others did not. Six studies that included ELLs or language minority students, along with other students, in the study sample did not provide disaggregated data for ELLs or language minority students, limiting the conclusions that could be made based on the data (Bouchey, 2004; Bouchey & Harter, 2005; Hadi-Tabassum, 1999; Lau & Roeser, 2002; Maheady, Michielli-Pendl, Harter & Mallette, 2006; Varelas, Pappas, Kane, Arsenault, Hankes & Cowan, 2007). As a result of this tremendous variability, it is difficult to make strong statements about the conclusions that can be drawn from this small but growing body of literature.

Links to State Standards

Appendix B shows each of the 22 studies included in this review, the outcomes associated with each study, and whether those outcomes had a connection to state standards. A number of studies were linked to either national science tests such as the National Assessment of Educational Progress (Lau & Roeser, 2002; Lee et al., 2008b; Lee et al., 2008c; Lee et al., 2005; Luykx et al., 2007) or to the national science standards published by different groups (Lynch et al., 2002). Because state and national science standards may be very different, this review begins with a look specifically at the peer-reviewed literature pertaining to state standards. A total of nine studies addressed outcomes related to state standards. These were:

Cognitive Outcomes

Standards-based measures of outcomes were usually associated with the cognitive domain. These outcomes typically were in the form of scores on state tests (Amaral et al., 2002; Lee et al., 2008b; Lee et al., 2008c; Rivera & Stansfield, 2003-2004; Torres & Zeidler, 2002) or researcher-developed tests based on curricula aligned with state standards (Lee, Deaktor, et al., 2008b, Lee et al., 2005; Maheady et al., 2006).

Two studies linking curricula with state test scores (Amaral et al., 2002; Lee et al., 2008c) indicated that test scores went up after implementation of an inquiry-focused curriculum based on state standards. However, only one of these studies (Amaral et al., 2002) reported state science test scores. Lee et al. (2008c) reported increases in math test scores because there was no state science test at the time of their study and the science curriculum that was implemented for the study incorporated math skills. While documented increases in state test scores are positive signs of improved classroom practices and student learning, neither study made clear how far ELLs and language minority students still were from passing state tests because the studies only focused on the amount of gain students made.

Of the three studies that reported scores on researcher-developed assessments aligned with state standards (Lee et al., 2008b, Lee et al., 2005; Maheady et al., 2006), Maheady et al. (2006) did not break down test results to show specifically the performance of the ELLs in the sample. The two related studies by Lee and colleagues (Lee et al., 2008b; Lee et al., 2005) showed a significant increase in standards-related test scores for ELLs and language minority students after implementation of the same inquiry-based curriculum across multiple years. In addition to explicit teaching of inquiry skills, the curriculum included a scientific language component that the researchers believed to be successful. In spite of the increase in test scores, ELLs and language minority students still scored well below other groups of students in the study. It was unclear whether their level of performance may, perhaps, have been well below what standards required for that grade level. The authors did not address the issue. The gaps in performance levels between ELLs or language minority students and other students tended to narrow a bit for students who had participated in the standards-based curriculum for two years, but gaps still remained.

Rivera and Stansfield (2003-2004) studied the effect of simplification of the language of state science test items on the performance of ELLs and non ELLs in one state. There were so few ELLs enrolled in K-12 schools in the state, and so few participating in the state science test, that no definitive conclusions were made about the effect of linguistic simplification for these students.

Studies that only measured affective outcomes such as perceptions of science-related creativity or math and science ability, and perceptions of adult support for science learning, tended not to link results to state science standards. Maheady et al. (2006) was the only exception. These researchers measured student attitudes toward a particular format of cooperative learning called Numbered Heads Together (NHT) with, and without, a behavioral incentive package. While the intervention was implemented using a standards-based science curriculum, the focus of the intervention was on the effectiveness of NHT. Every student, including ELLs, performed better on standards-based classroom assessments with some form of NHT, but the addition of the behavioral incentive had the greatest effect on scores. It also positively affected students’ attitudes toward science instruction.

Two studies that focused on other forms of outcomes (Cuevas et al., 2005; Lee et al., 2006), namely the ability to conduct science inquiry tasks, were related to the same grant project that implemented a standards-aligned curriculum to directly teach inquiry processes. Over time, the students took a greater responsibility for performing inquiry tasks, after they had gained skills. Both studies (Cuevas et al., 2005; Lee et al., 2006) found that the inquiry skills of ELLs did increase after direct instruction. Lee et al. (2006) discovered that ELLs had some of the largest gains in their ability to perform inquiry tasks. However, students became more effective only on certain parts of the inquiry tasks, such as their abilities to support theories with data and controlling specific variables (e.g., amount of water used, the length of time and the location of an experiment). They continued to struggle with controlling other variables (e.g., the size and shape of containers, heat, location and length of time) (Lee et al., 2006). Lee et al. (2006) attributed some of the difficulty with inquiry-based learning to students’ cultural backgrounds and values.

The researchers believed that developmental readiness to perform inquiry tasks was also a factor that caused 4^th graders to perform better than 3^rd graders in the study. Lee et al. (2006) provided an important implication for teaching inquiry science to ELLs. Furthermore, as student understanding of and skill with inquiry procedures grew, they often struggled to express that understanding with their developing English skills.

The picture that emerged from these nine studies was that a handful of researchers examined ways to increase the grade-level standards-based science achievement of ELLs and language minority students. However, these students may still have performed below the level that was required to pass standards-based assessments and to graduate within a standards-based system. The studies did not provide much information on which to judge how far ELLs and language minority students have to go, and in what areas or aspects of science, to be successful. With limited and incomplete information it is difficult to use the research literature as a guide to plan instructional improvements.

Given that there is so little research with clear data on state standards-based science outcomes specifically for ELLs and language minority students, perhaps the most important issues that arise from this review relate to the state of the field and recommendations for the future. These issues and recommendations are based on all 22 studies, not only those related to state standards.

Relationship of Studies to Model

Referring back to the Revised Theory of Educational Effectiveness under Standards-Based Reform in Figure 2, how did the studies reviewed match up to the elements of the figure? As Table 2 shows, the majority of reviewed studies addressed either student factors (n=13) such as attitudes, cultural background and language proficiency levels, or classroom factors (n=11), particularly curriculum implementation and associated teacher training. Four of the studies (Duran et al., 1998; Hampton et al., 2001; Lee et al., 2005; Lynch et al., 2002) examined factors at both the student and the classroom levels. Amaral et al. (2002) reported the effects of a science curriculum on students in an entire school district, and, for the purposes of this review, is included with the school level. Only one study (Rivera & Stansfield, 2003-2004) addressed national/local policy context factors in the form of a state test accommodation not commonly allowed in state testing policies.

Because the intent of standards-based reform is to assist all students in achieving grade-level content by improving the quality of classroom conditions that support learning, an emphasis on classroom factors in the research literature seems important and appropriate. By focusing on the classroom, researchers have placed the responsibility for increasing student outcomes primarily on educators and schools, not on the students. Further studies on classroom factors are important. In addition, the 22 studies reviewed here also emphasize the role of student factors, such as attitude and motivation, in learning science. These studies illuminate the ways in which students are unique. They have different attitudes toward science, differing levels of support for science learning, and differing motivations to succeed in the science classroom. Differences are important to study so long as the emphasis remains on stimulating teachers to teach standards-based content differently in order to better meet student needs.

In a previous review of studies on science instruction for diverse students, Lee and Luykx (2007) reported that few studies provided concrete, quantitative outcomes (p. 177). One possible reason for this finding may be that many studies in the field come out of disciplinary traditions and theoretical or conceptual frameworks with non-cognitive types of emphases. Table 3 highlights that half of the total group of studies reviewed (n=11) fall into this category. The few studies in Table 3 that included cognitive outcomes, such as Bouchey (2004), also included other types of outcomes as well. Footnotes to the table describe terms used to refer to theoretical or conceptual frameworks, as used by article researchers.

Two examples from the table provide illustrations of the way in which aligning research traditions with theoretical or conceptual frameworks, and thus with research questions, can lead to emphases on varying outcomes. First, Duran, Dugan, and Weffer (1998) chose to study science and language minority students from the perspective of social constructivism, which is associated with the tradition of social psychology. Duran et al. (1998) define social constructivism in the following way:

Because this study focused on social speech as a mediator of individual student learning, as expressed through language, it is logical that the researchers collected interview and observational data on outcomes related to student production of meaning in classroom interactions.

A second example of the way disciplinary traditions and theoretical or conceptual frameworks align with science outcomes can be found in Beghetto’s (2007) study. The researchers examined science through the perspective of Achievement Goal Theory, which comes out of the field of psychology. As Beghetto (2007) explained:

As a result of this focus, Beghetto collected data on students’ perceived efficacy, perceived science competence, achievement goals, and perceived teacher support. Cognitive outcomes were not included in Beghetto’s study. Again, in the context of this particular disciplinary tradition and theoretical framework, outcomes relating to student attitudes and motivation make sense.

The remaining studies reviewed focused solely on cognitive outcomes such as test scores and grades. Six of these studies, several from the same research project, incorporated test scores to measure the effectiveness of implementing new curricula (Amaral et al., 2002; Cuevas et al., 2005; Lee, Deaktor et al., 2008; Lee, Maerten-Rivera et al., 2008; Lee et al., 2006; Lee & Luykx, 2006). Studies like those of Duran et al. (1998) and Beghetto (2007) add richness to our understanding of the context of science outcomes. However, the model of educational effectiveness under standards-based reform (Figure 2) does emphasize standards-based cognitive outcomes as an end goal of instruction. Other types of outcomes may be important contributors to cognitive outcomes (e.g., levels of academic English proficiency, attitudes toward science), but these are not the end goal of instruction.

The results of this research review indicate that there is a small but growing research base that addresses science outcomes for language minority students and ELLs. The purpose of this review was to focus more specifically on what is known about outcomes related to state science standards for ELLs and language minority students. Of the 22 studies that addressed science outcomes for these students, many were related to either national science assessments or to national science standards. A smaller proportion was related to state standards. The small number of studies available combined with the diversity of topics studied and the multitude of viewpoints makes it difficult to distill findings from this body of literature. For example, theoretical and conceptual frameworks researchers chose for the larger group of studies influenced the way outcomes were conceptualized and measured. Some studies conceptualized outcomes in purely cognitive terms such as scores on tests, while others addressed affective or other types of outcomes.

There were only nine studies linking science outcomes to state standards in some way. Four of these nine studies were based on an implementation of one inquiry-based science curriculum used in the same grant project (Lee et al., 2008b; Lee et al., 2008c; Lee et al., 2005; Luykx et al., 2007). It is not possible to state definitive research findings with such a small number of studies, but two ideas contained in those eight studies stand out for further consideration.

First, scientific inquiry skills, which are often part of state science standards, are important to explicitly teach to language minority students and ELLs, but may be challenging for students to fully learn. Because of differing cultural backgrounds and different levels of English proficiency, educators should not assume that linguistically and culturally diverse students will implicitly have the knowledge or skills to perform inquiry tasks without being taught how to do so. A few studies in this review found that ELLs and language minority students did, in fact, perform significantly higher on inquiry-based tasks and test items after receiving instruction on the different inquiry elements plus scientific language. However, ELLs in particular still struggled with certain inquiry concepts and skills after the interventions.

Second, curriculum-related interventions had differential effects for language minority students and ELLs. While ELLs displayed a great deal of growth in science learning after the implementation of these interventions, they still scored below their language minority peers on content-related assessments and inquiry tasks. Researchers observed that a sizeable gap in performance levels between ELLs and other students was still evident. One major challenge in summarizing science outcomes for linguistically and culturally diverse students is that the available literature did not state how far below grade-level standards the performance of ELLs and language minority students was.

Both of these key points in the literature highlight the challenges inherent in teaching academic English language skills specific to science simultaneously with both science content knowledge and science processes in the mainstream classroom. ELLs and language minority students who have recently exited English as a Second Language services have to learn more knowledge and skills in order to achieve at grade level in the science classroom than do their native English speaking peers. The work of Lee and colleagues (Cuevas et al., 2005; Lee et al., 2008a; Lee et al., 2008b, Lee et al., 2008c; Lee et al., 2006; Lee et al., 2005) highlighted the need to incorporate scientific language instruction into the teaching of scientific inquiry skills while also focusing on content acquisition. It is vitally important to conduct more studies of this kind relating content, language, and skill integration in the science classroom.

Considering the extremely small number of studies relating specifically to standards-based science outcomes for ELLs and language minority students, the strongest statements that can be made about the 22 studies reviewed relate to current emphases in the field of science education for linguistically and culturally diverse students and possible research directions.

At this point in time, the research literature simply does not provide enough information about state standards-based science outcomes for ELLs and language minority students to guide educational improvement efforts. This lack of research may be, in part, because state accountability for standards-based science outcomes, in the form of mandatory statewide achievement tests, is relatively new. Policymakers, educators, and researchers are just beginning to turn their attention to the instructional implications of standards-based science assessment scores for all students and largely have not yet begun to focus on outcomes for ELLs and language minority students. Nevertheless, the available research suggests some promising directions for future research efforts.

References

Amaral, O., Garrison, L., & Klentschy, M. (2002). Helping English language learners increase achievement through inquiry-based science instruction. Bilingual Research Journal, 26(2), 213-239.

Beghetto, R. (2007). Factors associated with middle and secondary students’ perceived science competence. Journal of Research in Science Teaching, 44(6), 800-814.

Bouchey, H. (2004). Parents, teachers, and peers: Discrepant or complementary achievement socializers. New Directions for Child and Adolescent Development, 106, 35-53.

Bouchey, H., & Harter, S. (2005). Reflected appraisals, academic self-perceptions, and math/science performance during early adolescence. Journal of Educational Psychology, 97(4), 673-686.

Creemers, B. P. M., & Kyriakides, L. (2008). The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools. New York, NY: Routledge.

Cuevas, P., Lee, O., Hart, J., & Deaktor, R. (2005). Improving science inquiry with elementary students of diverse backgrounds. Journal of Research in Science Teaching, 42(3), 337-357.

Duran, B., Dugan, T., & Weffer, R. (1998). Language minority students in high school: The role of language in learning biology concepts. Science Education, 82(3), 311.

Hadi-Tabassum, S. (1999). Assessing students’ attitudes and achievements in a multilingual, multicultural science classroom. Multicultural Education, 7(2), 15-20.

Hampton, E., & Rodriguez, R. (2001). Inquiry science in bilingual classrooms. Bilingual Research Journal, 25(4), 417-434.

Lau, S., & Roeser, R. (2002). Cognitive abilities and motivational processes in high school students’ situational engagement and achievement in science. Educational Assessment, 8(2), 139-162.

Lauer, P., Snow, D., Martin-Glenn, M., Van Buhler, R., Stoutemyer, K., & Snow-Renner, R. (2005). The influence of standards on K-12 teaching and student learning: A research synthesis. Mid-Continent Research for Education and Learning (MCREL).

Lee, O. (2003). Equity for linguistically and culturally diverse students in Science education: A research agenda. Teachers College Record, 105(3), 465-489.

Lee, O. (2004). Teacher change in beliefs and practices in science and literacy instruction with ELLs. Journal of Research in Science Teaching, 41(1), 65-93.

Lee, O. (2005). Science education with English language learners: synthesis and research agenda. Review of Educational Research, 75(4), 491-530.

Lee, O., Adamson, K., Maerten-Rivera, J., Lewis, S., Thornton, C., & LeRoy, K. (2008). Teachers’ perspective on a professional development intervention to improve science instruction among ELLs. Journal of Science Teacher Education, 19, 41-67.

Lee, O., Buxton, C., Lewis, S., & LeRoy, K. (2006). Science inquiry and student diversity: Enhanced abilities and continuing difficulties after an instructional intervention. Journal of Research in Science Teaching, 43(7), 607-636.

Lee, O., Deaktor, R., Enders, C., & Lambert, J. (2008). Impact of a multiyear professional development intervention on science achievement of culturally and linguistically diverse elementary students. Journal of Research in Science Teaching, 45(6), 726-747.

Lee, O., Deaktor, R., Hart, J., Cuevas, P., & Enders, C. (2005). An instructional intervention’s impact on the science and literacy achievement of culturally and linguistically diverse elementary students. Journal of Research in Science Teaching, 42(8), 857-887.

Lee, O., & Luykx, A. (2006). Science education and student diversity: Synthesis and research agenda. New York: Cambridge University Press.

Lee, O., & Luykx, A. (2007). Science education and student diversity: Race, ethnicity, language, culture and socioeconomic status. In S. Abell & N. Lederman (Eds.), Handbook of research in science education (pp. 171-197). Mahwah, NJ: Lawrence Earlbaum.

Lee, O., Maerten-Rivera, J., Penfield, R., LeRoy, K., & Secada, W. (2008). Science achievement of English language learners in urban elementary schools: Results of a first-year professional development intervention. Journal of Research in Science Teaching, 45(1), 31-52.

Luykx, A., Lee, O., Mahotiere, M., Lester, B., Hart, J., & Deaktor, R. (2007). Cultural and home language influences on children’s responses to science assessments. Teachers College Record, 109(4), 897-926.

Lynch, S., Kuipers, J., Pyke, C., & Szesze, M. (2005). Examining the effects of a highly rated science curriculum unit on diverse students: Results from a planning grant. Journal of Research in Science Teaching, 42(8), 912-946.

Maheady, L., Michielli-Pendl, J., Harper, G., & Mallette, B. (2006). The effects of numbered heads together with and without an incentive package on the science test performance of a diverse group of sixth graders. Journal of Behavioral Education, 15(1), 25-39.

Marzano, R., & Kendall, J. (1998). Awash in a sea of standards. Denver, CO: Mid-Continent Research for Education and Learning (MCREL).

McClure, P. (2005). Where standards come from. Theory into Practice, 44(1), 4-10.

Medina, M., & Mishra, S. (1994). Relationships among Spanish reading achievement and selected content areas for fluent- and limited-Spanish-proficient Mexican Americans. Bilingual Review, 19(2), 134-142.

National Research Council (1999). Testing, teaching, and learning: A guide for states and school districts. Committee on Title I Testing and Assessment, Richard F. Elmore and Robert Rothman, editors. Board on Testing and Assessment, Commission on Behavioral and Social Sciences and Education. Washington, D.C.: National Academy Press.

Paige, R. (2003, March). Remarks by Secretary Paige to the Commonwealth Club of California. Speech presented at the Commonwealth Club of California.

Rivera, C., & Stansfield, C. (2004). The effect of linguistic simplification of science test items on score comparability. Educational Assessment, 9(3&4), 79-105.

Saderholm, J., & Tretter, T. (2008). Identification of the most critical content knowledge base for middle school science teachers. Journal of Science Teacher Education, 19, 269-283.

Shaw, J. (1997). Threats to the validity of science performance assessments for English language learners. Journal of Research in Science Teaching, 34(7), 721-743.

Stoddart, T., Pinal, A., Latzke, M., & Canaday, D. (2002). Integrating inquiry science and language development for English language learners. Journal of Research in Science Teaching, 39(8), 664-687.

Torres, H., & Zeidler, D. (2002). The effects of English language proficiency and scientific reasoning skills on the acquisition of science content knowledge by Hispanic English language learners and native English language speaking students. Electronic Journal of Science Education, 6(3). Available: http://ejse.southwestern.edu/original%20site/manuscripts/v6n3/articles/art04_torres/torres.pdf

Varelas, M., Pappas, C., Kane, J., Arsenault, A., Hankes, J., & Cowan, B. (2007). Urban primary-grade children think and talk science: Curricular and instructional practices that nurture participation and argumentation. Science Education, 65-95.