State Systemic Initiatives (SSI) Program - Project Description (CAREI)

The NSF Statewide Systemic Initiatives were designed to broaden the impact, accelerate the pace and increase the effectiveness of improvements in science and mathematics education at the K-12 level. These initiatives followed closely on the heels of the development of national standards for science and mathematics and were grounded in the belief that significant change would be most likely in a system that was supportive of change. In the past NSF initiatives have been more individual project based with emphases on teacher enhancement, curriculum development or research. The SSIs were designed to produce an environment where all the components of the educational system in a state were coordinated to produce change. The expectation was that teacher enhancement efforts, curriculum development and implementation, underrepresented student recruitment and retention, teacher preparation programs, informal science and mathematics education institutions, business and industry, colleges and universities and state and local policies would all come together to achieve a common goal. The underlying aim was to elevate teaching and learning standards based on the assumption that all students can learn challenging content.

Previous studies of the SSIs have resulted in the definition of six drivers of reform. These represent a framework that encapsulates both the components of systemic reform and an accountability regimen. The six drivers are:

The project proposed will incorporate these drivers and a theoretical framework of school change into an empirical model to help explain the impact on student achievement that can be accomplished through a systemic reform effort.

Although the vision for science and mathematics education incorporated into the SSIs is unique to each state, they are all grounded in the national standards for science and mathematics (NCTM, 1989; NRC, 1996; AAAS, 1993). These Standards espouse the philosophy of high quality science and mathematics for all learners, that all students can succeed in learning mathematics and science and that all should be given appropriate opportunities to do so. The goal of the Standards is to have students learn science and mathematics as communities of scholars using their hands and minds to understand concepts by engaging in science and mathematics in ways that are developmentally appropriate. Students will learn to ask their own questions of mathematics and science and to design investigations to answer their questions. They will learn to connect their ideas to the larger body of knowledge developed by scientists and mathematicians. An overwhelming body of research clearly establishes that learning environments where subject matter is personally relevant, where students are actively engaged in learning, and where the discourse is focused on inquiry about important problems is associated with a variety of desirable educational outcomes (Fraser, 1994). This project will mesh these images of science and mathematics education with the drivers already identified and the factors associated with high performance learning communities into indicators of accomplishment. These indicators and others already developed (e.g., NSF, 1996; Porter, 1991) will then be translated into data gathering devices to obtain the information directly relevant to the SSIs.

A Research-Based Model of High Performance Learning Communities in SSI Schools

Recent research (Fullan and Stiegelbauer, 1991; Scheerens, 1992) suggests that we still know very little about how to drive an externally developed reform initiative into the classroom. Most research on comprehensive school improvement efforts suggests that teaching and learning practices have changed only modestly (Tyack & Cuban, 1995; Elmore, 1995; Fullan, 1995). Even in schools that are "well along" on the path to reform, "authentic instruction" and in-depth presentation of demanding curricula is not typical (Newmann and Associates, 1997). Recent educational research tied to the literature on business learning environments suggests a number of characteristics of high performance learning communities that are highly adaptable to creating more effective science and mathematics education in schools (Simsek & Louis, 1994; Tjepkema, 1994; Marks & Louis, in press). In the proposed study we will place the six SSI drivers in a broader context of what is known about how to effect improvements in student achievement.

Although the school improvement literature has assumed that the most appropriate unit for change is the school itself (van Velzen, et al, 1984), the focus of what is to be improved must be within the classroom, where students and teachers create the most powerful learning communities. In order to obtain classroom effects in a school, there must be supportive conditions throughout the organization and in the local environment. Our model thus assumes that school and other contextual conditions do not typically have a strong direct effect on student learning, but largely an indirect effect by enabling improvements in the functioning of classrooms. See Figure 1.

Effective Pedagogy

Learning cannot take place without a knowledge base (Louis & Raywid, 1994). A knowledge base may come from several sources: (1) individual knowledge that is brought by both professionals, parents, students and community members; (2) knowledge that is "imported" from experts and the experiences of other schools; and (3) knowledge that is created by members of the school community to address specific questions or problems. Any effort to improve science and mathematics education must identify and share knowledge from each of these three critical sources. In the case of SSIs, the knowledge base is derived from the professional association standards for each discipline, but must be adapted to each state’s curriculum standard. Local adaptations are also inevitable (Fullan and Stiegelbauer, 1991).

Our model draws on the base of school effectiveness/teaching effectiveness research that has been conducted both in the U.S. and in many other countries over the past two decades (Louis and Miles, 1990). At the classroom/teaching level, the broad patterns of effective instruction include the elements that Newmann and Associates (1997) summarized in their dimensions of "authentic pedagogy" which incorporates both findings from the effective teaching research, and also models constructivist teaching:

We would add Stallings’ (1980) emphasis on allocated academic time to the above list to produce a five point basis for instructional improvement in middle/secondary schools.

Key School Context Factors

Classroom practice is not the only factor affecting student learning. Many studies have identified significant school effects (Scheerens, 1992; Stringfield, 1995). Creating high performance schools first requires professional communities that are focused on student learning (Kruse, Louis, & Bryk, 1995, Bryk, Camburn & Louis, 1996). School capacity for development of professional communities is dependent on a number of factors, including (1) the availability of meaningful standards for student development and achievement; (2) the availability of research knowledge about practices and structures that will improve students achievement, and (3) the development of stronger collaborative relationships within the school. Professional community is demonstrably related to student achievement (Louis and Marks, 1998).

Developing a high performance learning community in schools demands different forms of leadership (Murphy & Louis, 1994; Leithwood, 1994). Decentralized, facilitative leadership is particularly important in creating the consensus that characterizes a highly reliable organization that does not accept failure (Stringfield, 1995; Louis and Kruse, 1995). Another factor is a supportive structure including cultural features such as an orientation to innovation and problem solving, adequate professional development, the effective use of time to permit teachers to work in innovative ways both in and outside the classroom, interdependent teaching roles, etc. (see Louis, Marks and Kruse, 1996; Louis and Kruse, in press; Stringfield, 1995). In addition, there is a need to confront the structural barriers to increased classroom performance, including traditional schedules, limited of teacher and administrator contact with each other and research, limited interdependencies in teaching roles, etc. Finally creating high performance learning communities requires that local accountability standards be developed and internalized and that relationships be forged with significant local constituencies, including parents, district, and relevant local leaders.

Summary of Model

The literature shows that to create high performance learning communities interventions should:

These elements form the core components of a theory driven framework, within which the SSI drivers are embedded. Although the school reform paradigm has developed largely outside of the research that led to the identification of the SSI drivers, it is, not surprisingly, highly compatible. The SSI drivers are defined in terms of policy-manipulable dimensions – e.g., levers that are amenable to intervention by state and national policy. The High Performance Learning Community model for school improvement, on the other hand, has emerged primarily as a result of studying "reform in situ" – research focusing on schools who are making concerted efforts to change in the context of state and national policies (see, for example, Newmann & Associates, 1997). Table 1 shows the overlap between the "top down" SSI Drivers model and the "bottom up" high performance learning community model.

Table 1: SSI Drivers and High Performance Learning Community Factors

SSI Drivers
HPLC Factors	Standards-Based Curriculum	Coherent Policies	Convergence of Resources	Support from Constituents	Evidence of Achievement	Achievement among Underserved Populations
Knowledge Base focused on Core Concepts	XX
Knowledge Utilization Process		XX
Effective pedagogy	XX
Professional Community			XX
Expanded Leadership			XX
Local Accountability Standards					XX	XX
Linkages with Community				XX

Educational Significance

This study will accomplish several specific objectives in its investigation of the relationships between drivers, school improvement factors and student achievement. Ultimately the impact of our investigation would be to provide the information necessary to produce more effective models of systemic change, and to improve student achievement.

Methodology

Determining the impact of SSIs is very difficult because of the high degree of confounding that occurs when examining extensive systematic implementation efforts. Any impacts that are found might be ascribed to several different initiatives of which the SSI is only one. Therefore it is crucial that the causal lines be carefully tied to the SSI and that other effects are "controlled for" –matched or equated or removed as much as possible.

To accomplish this we propose to conduct quasi-experimental studies of the relationships among the suggested drivers and student achievement. The relationships among the drivers and student achievement will be determined for treatment and comparison groups separately and then compared. The "treatment" group will be schools with a high level of contact with an SSI. There will be two different types of "comparison" groups. One will be matched schools in an SSI state that have had only a low level of contact with the SSI. In other words, schools in a state with high levels of involvement (treatment) will be contrasted with schools with low levels of involvement (comparison). This will allow the identification of the differing relationships among the drivers and student achievement that occur with different levels of involvement with an SSI within an SSI state. The other "comparison" group will be composed of schools in non SSI states. These schools will be representative of what happens in the absence of an SSI altogether.

The investigation will use several methods to increase the level of control over the confounding variables. Furthermore since collection of statewide achievement data is beyond the scope of this initiative, only states with existing student assessment systems will be included. One method of controlling confounding will be to have an "insider" and an "outsider" be part of the impact study team. This will allow us to examine the complex relationships from both external and internal perspectives. In this way the confounding variables within the existing context can be better understood and accounted for. We are proposing to use four states in our study: Louisiana, Montana, Colorado and Illinois. Louisiana, Montana and Colorado were selected because they are SSI states with comprehensive student achievement data and successful SSIs. Illinois was selected as a representative state without an SSI but as one with comprehensive student achievement data. The research design calls for the use of relationships among drivers and student achievement determined in IL as a comparison set for the relationships determined in the SSI states. In addition within LA, MT and CO two sets of schools will be identified to provide within state comparison groups. One set will have had extensive involvement with the SSI and another will have had limited involvement. Finally we will incorporate a "replication" study. Data would be collected the first year only from MT, LA and IL since their statewide student assessment systems are already in operation, while data from CO would be gathered during the next year (2000-2001) because that is when their statewide student assessment system would be operational. During the summer and fall of 2000 the data from the first three states would be analyzed and preliminary hypotheses formulated. Any unanticipated questions or emerging theories would be incorporated into the data collection for CO. In other words the CO data gathering and modeling effort would serve as a replication and verification of the information gathered during the first year. (See Table 2 for a description of research design.)

Table 2: Research Design

Grade Level	Within SSI States (LA, MT, CO replication)		Across States (SSI vs. non-SSI)
	Treatment	Comparison
8^th Grade (Math or Science)	High SSI Contact (LA, CO replication)	Low SSI Contact (LA, CO replication)	IL
11^th Grade (Math)	High SSI Contact (MT)	Low SSI Contact (MT)	IL

The student achievement data within each state will be used as the outcome measure and will be obtained from the various state assessment offices by school. Because of the differing natures of the SSIs in the selected states, we will focus on students from different grade levels within the states. Because elementary school science and mathematics was a focus in CO and LA, we will concentrate on middle school student assessment (grade 8) in those states. Because MT emphasized high school mathematics we will look at high school student assessment (grade 11). Information about students from both grade levels and from both mathematics and science will be gathered in IL to provide comparisons. Using these grade levels will allow us to examine the corresponding NAEP (1997) and TIMSS (1996) data. Also both of these grade levels are critical times for students to make career choices (NSF, 1993). Finally, but most important, selection of these students in the SSI states which have been in operation for several years will allow us to gather achievement data from students who have been affected by the SSI for several years, i.e., through grades 9,10 and 11 or through grades 5,6,7,and 8. This will allow us to assess the accumulated effectiveness of the SSIs. Student achievement often does not show up until after the students have been exposed to the changes over a number of years. By selecting these students and tracking their involvement in SSI affected courses, we will be able to investigate longitudinal effects of participation. Our system of comparison groups will also allow us to separate the effect of the SSI from other initiatives.

Three primary methods will be used to guarantee the validity of the information collected by the project. First, information on the selected indicators will be obtained from different information sources and triangulated. Second, different data collection methods will be used so that any bias inherent in a collection method will be counteracted (Patton, 1990; Shadish,1993). Third, the instruments developed (surveys, observation protocols, and interview formats) will be reviewed by external experts in science, mathematics and systemic change, and pilot tested and revised before use. We have used the Local Systemic Change Initiative instruments and will use items from these whenever they fit within our model (Horizon Research Incorporated, 1997). This will allow us to tie our information about the SSIs to the data being collected about the LSCs.

In order to accomplish the triangulation, different information sources will be tapped. The sources will include at the least teachers, students, principals, superintendents and state curricular leaders. Sets of the same questions will be asked of, for example, teachers and students so that the teachers’ perceptions of what is going on in the classrooms would be verified by the students’ perceptions. As another example, the teachers’ perceptions of the value of school policies in improving science and mathematics education would be verified by questions asked of the principals and superintendents. The three different confirmatory data collection methods would be surveys, observations and interviews. We plan to survey a large sample and then make visits to selected sites to conduct interviews and observations to validate and expand on the survey data. To maximize insights and cross site consistency, site visits would be conducted by a team of at least two: one member from the state and one member from outside the state. The site visits will also allow the construction of case studies of individual school’s responses to the change process. These rich descriptions will provide insights for the development of more effective models for systemic change and more information on the effectiveness of the SSIs at selected sites.

Sampling Plan

We propose collecting information from at most 60 schools in each SSI state; 30 schools affected continuously by the SSI and 30 schools not directly affected by the SSI. The final number of schools selected will depend upon the individual state and its operation of its SSI. Thirty schools from each grade level with similar characteristics would be selected in IL. A sub-sample of students within each school will be selected. Site visits would be made to six schools within each state. In the SSI states this would be three schools with high levels of interaction with the SSIs and three schools with low levels of interaction with the SSIs. In IL three high schools and three middle schools will be visited.

Data Analyses

The student achievement data will be obtained from the individual state offices with the help of the state team members. The achievement information will be combined into databases with the information on the drivers and factors obtained from the students, teachers, principals and other respondents. We propose a multi-level data analysis approach, including univariate and multivariate analyses. These analyses will range from the simplest descriptive statistics of the schools and students in our sample, to the most complex modeling procedures. Our treatment and comparison group design will allow us to compare and contrast the impact of drivers and factors in both SSI schools and non-SSI schools within each state and across states. We will also be able to conduct analyses in both science and mathematics and at both 8^th and 11^th grade. A range of data analyses is important to better understand not only the impact of each individual driver and factor by itself, (i.e. the univariate analyses) but to also better understand the combined impact of all the drivers and factors when they are analyzed together (i.e. the multivariate analyses).

At the simplest level we will produce basic descriptive statistics on the states, districts, schools, students, and teachers. We will also examine the results according to subgroups in our sample such as ethnic background, gender, socio-economic status, and geographic location. As part of these preliminary analyses, we will examine relationships between individual drivers and factors, and student achievement in science and mathematics. The results of the univariate analyses will be used to better understand how the drivers and factors fit together in a system to inform our proposed multivariate analyses.

At the multivariate level of analysis, we propose using hierarchical linear model procedures to examine the relationships among the variables and student achievement. The research based model of high performance learning communities (proposed in Figure 1) is hierarchical in nature and will allow us to compare and contrast the impact of school change factors in all of the treatment and comparison conditions outlined above.

Finally, we propose using path analysis to better understand the interactive nature of the relationships between and among the SSI drivers. We have organized the SSI drivers into a conceptual model of how each driver interacts to affect student achievement. (See Figure 2). This model provides a preliminary framework for understanding how the drivers affect student achievement and will be empirically tested through the data collected for this project. The above univariate analyses will also inform our model and possibly lead to modifications. Ultimately, we hope to be able to test this model using structural equation modeling to better understand the impact of SSI’s on student achievement. We will develop instrumentation with the intent of creating a measurement model sufficient to test a complex structural equation model. As with the above analyses, we will conduct path analysis for each group separately resulting in two different models for each SSI state and a model for IL. The relationships outlined by these models will be compared to determine any differences attributable to the involvement of the schools in the SSI.

Figure 2: Conceptual Model

Dissemination

As stated in the introduction, the final goal of the SSIs is the improvement of the diverse educational systems so that they promote better understanding and appreciation of science and mathematics. The proposed project will produce information on what elements and combinations thereof, are the most effective in enhancing student understanding of science and mathematics concepts. It is imperative, therefore, that the insights gained through this project be disseminated widely to all of the stakeholders of systemic change. Presentations by various team members will be made at professional meetings and since the team is representative of a variety of interest groups, several different professional communities will be informed. However, the team expects to do more than the traditional presentation of findings at professional meetings. We anticipate setting up a web site and providing reply services (this capacity is already available in CAREI). We plan on producing reports in a variety of formats that will be targeted to different audiences, e.g., technical research reports, colorful community informational sheets, how-to guidelines, public media presentations, dialog pieces in professional and popular journals, etc. Finally if possible, we will join with the other projects funded under this initiative to hold national workshops designed to widely disseminate the results of the SSI Impact Studies.

Time Line

A detailed timeline has been developed to help communicate the variety of tasks that need to be completed for this study. The timeline includes details ranging from the preparation and planning of the study to the final analysis and dissemination of results.

1. Finalized Sample: Final identification of schools will take place in collaboration with the within state team members and the State SSI director. Districts will be contacted and permission to conduct the survey will be obtained.

Management Plan/Qualifications

The project will be located at the Center for Applied Research and Educational Improvement in the College of Education and Human Development at the University of Minnesota. The University is a major center of education, creative scholarship, research, and service. It is one of the largest public universities in the United States, with a total enrollment of approximately 75,000 full-time and part-time students.

The College of Education and Human Development is home to six academic departments, fourteen research centers and 130 faculty members. The College is consistently ranked among the top ten schools of education in the United States. Faculty and staff are on the forefront of interdisciplinary research in many areas, including: educational outcomes and accountability; curriculum studies, including science, mathematics, and reading; school reform and improvement; child and youth development; special education and disability studies; exercise science, health, recreation and recreation therapy.

The Center for Applied Research and Educational Improvement (CAREI) is a collaborative organization that brings the resources of the College of Education and Human Development and the University of Minnesota to bear on educational issues in Minnesota and across the nation. Located in Peik Hall on the Minneapolis campus, CAREI occupies approximately 3,500 square feet with 10 offices and two conference rooms. CAREI is able to draw on the expertise of faculty and staff from across the College of Education and Human Development and other centers and departments within the university to conduct research, evaluation and policy studies for local, state, and federal education and human service organizations. This includes working with large, national research and evaluation projects. For example, CAREI worked in collaboration with RPP International and the Institute for Responsive Education to conduct the first national study of charter schools for the U.S. Department of Education. This four-year study involved an annual telephone-assisted survey of all charter schools, onsite field work to a sample of schools from across the country, the collection of data related to student performance, and special policy studies. CAREI also received a field-initiated grant from the U.S. Department of Education to examine the impact of block-scheduling on students, schools, and communities, and a grant from NSF to examine the long-term impact of science teacher enhancement on student achievement.

The co PI’s will be Drs. Lawrenz, Huffman and Louis. They have extensive experience in conducting evaluations of science and mathematics programs and in managing large projects. As an additional component, Lawrenz and Huffman are responsible for a new evaluation fellows program where Ph.D. level science, mathematics, engineering and technology (SMET) evaluators are trained. Dr. Lawrenz was also a member of the assessment team in the development of the science standards and has written two summaries of science and mathematics curriculum in NSF’s Indicators of Science Education Source Books. Dr. Huffman has a degree in engineering and a Ph.D. in science education with a minor in program evaluation. He has taught middle and elementary school science and conducted several evaluations of SMET programs. Dr. Karen Seashore Louis is a professor of educational policy and administration at the University of Minnesota and an expert in education policy with a specialization in school based change. She has published extensively on school change and provides an important perspective to the research team. One other member of the team at the University of Minnesota is Ernest Davenport, an associate professor of educational psychology and an accomplished statistician familiar with the NAEP science and mathematics databases. Remaining members of the team will come from the involved states. Dr. Carl Frantz from LA is a political scientist with experience evaluating both LA’s SSI and CETP. Dr. Carol Thoresen from MT has her degree in educational administration and has worked as both a school principal and a chemistry teacher as well as a program evaluator. Dr. Jeffery Gliner from CO has an extensive background in evaluation and research design and is the evaluator for his state’s CETP. The collaborator from IL, Ken Travers, is an internationally known mathematics expert who has worked on the international mathematics assessments and served as the director of the evaluation section of the Science Education Directorate at NSF. In addition to the team members mentioned, we will be employing scientists and mathematicians as consulting meta-evaluators of the project, e.g., Lawrence Rudnick, U of MN physicist and coPI of the Minneapolis LSC and Peter Braunfeld, Professor Emeritus of mathematics, U of IL at Urbana-Champaign presently at EDC, Boston. This will help to keep us focused on important science and mathematics content. Clearly this team provides a rich and diverse background from which to study the impact of the SSIs.

The project would be managed as a team effort with final decisions being made by the PI/co-PIs. The team members communicate easily via email and conference phones would be used to hold virtual meetings of the entire team. All members of the team are familiar with and supportive of collaborative research and all have worked with at least one other member of the team before. Dr. Lawrenz has worked with all members of the team on projects in the past and will be able to facilitate the interaction necessary for a successful project. The individual state specialists would be mostly involved in their own state’s data collection but crossover to help inform comparisons would be built in with site visit teams of in and out state collaborators. We also plan to have at least one face-to-face meeting a year, perhaps in conjunction with a professional conference.

Final Project Summary

This highly collaborative project will bring together a comprehensive team of researchers to help to understand the relationships underlying successful systemic change. The team is composed of people with expertise in all relevant areas, i.e., science, mathematics, science and mathematics education, evaluation, policy development, school reform, measurement, and data analyses, and are from a variety of SSI and non SSI states. The team’s expertise and experience with schools and systemic change provides the diverse and theoretical background necessary to conceptualize realistic and research based models of systemic reform. Furthermore the composition of the team will allow for the successful implementation of the complex treatment and comparison group design proposed. The anticipated analyses will provide empirical verification of the proposed theoretical models. Comparison of the different models will provide information on the unique contribution of SSIs to educational reform. The plans for dissemination will guarantee that the results of the project will be widely known by diverse audiences. This information will hopefully promote the implementation of more productive models for systemic change. Ultimately this project will not only inform the knowledge base in the field of mathematics and science education reform, but will also aide all of the stakeholders in systemic reform in helping students achieve world class standards in science and mathematics.

References Cited

American Association for the Advancement of Science. (1993). Benchmarks for Science Literacy. New York: Oxford University Press.

Bryk, A., Camburn, E., and Louis, K.S. (1996). Promoting school improvement through professional communities: An analysis of Chicago elementary schools. Paper presented at the annual meeting of the American Educational Research Association, New York.

Elmore, R.F. (1995). Structural reform and educational practice. Educational Researcher, 24 (9), 23-26.

Fraser, B.J. (1994). Research on classroom and school climate. In D. L. Gabel (Ed.), Handbook of Research on Science Teaching and Learning. New York:Macmillan.

Fullan, M. (1995). The School as Learning Organization: Distant Dreams. Theory into Practice; v34 n4, p230-35.

Fullan, M., and Stiegelbauer, S. (1991) The new meaning of educational change. New York: Teachers College Press.

Horizon Research Incorporated. (1997). Local Systemic Change Core Evaluation Data Collection Manual. Chapel Hill, NC: Horizon Research Incorporated.

Kruse, S., Louis, K.S. and Bryk, A. (1995). A framework for analyzing professional community. In K Louis and S. Kruse and associates Professionalism and Community: Strategies for Reforming Urban Schools. Thousand Oaks, CA: Corwin Press.

Leithwood, K. (1994). Leadership for school reform. Educational Administration Quarterly, 4, 498-518.

Louis, K.S. and Kruse, S.D. (1995). Professionalism and community: Perspectives on reforming urban schools. Thousand Oaks, CA: Corwin.

Louis, K.S. and Kruse, S. D. (in press). Creating community in reform: Images of organizational learning in urban schools. In K. Leithwood and K.S. Louis (Eds.), Organizational Learning and School Improvement Linkages and Strategies.

Louis, K.S. and Marks, H.M. (1998). Does professional community affect the classroom? Teachers work and student work in restructuring schools. American Journal of Education.

Louis, K.S., Marks, H.M. & Kruse, S.D. (1996). Teachers’ professional community and school reform. American Educational Research Journal.

Louis, K. S., & Miles, M. (1990). Improving the urban high school: What works and why. New York: Teacher's College Press.

Louis, K.S. and Raywid, M. (1994). A tale of two schools: Knowledge, skill, collaboration and organizational learning. Paper presented at the annual meeting of the American Educational Research Association.

Marks, H. & Louis, K.S. (in press). Does teacher empowerment affect the classroom? Educational Evaluation and Policy Quarterly.

Murphy, J., & Louis, K.S. (Eds.). (1994). Reshaping the principalship: Insights from transformational reform efforts. Thousand Oaks, CA: Corwin.

National Assessment of Educational Progress (1997). NAEP 1996 Mathematics Report Card for the Nation and the States. Washington, DC: National Center for Educational Statistics.

National Assessment of Educational Progress (1997). NAEP 1996 Science Report Card for the Nation and the States. Washington, DC: National Center for Educational Statistics.

National Council of Teachers of Mathematics. (1989). Curriculum and Evaluation Standards for School Mathematics. Reston, VA: NCTM.

National Research Council. (1996). National Science Education Standards. Washington, DC: National Academy Press.

National Science Foundation. (1993). Indicators of Science and Mathematics Education, 1993. Arlington, VA: National Science Foundation.

National Science Foundation. (1996). Indicators of Science and Mathematics Education, 1995. Arlington, VA: National Science Foundation.

Newmann, F. & Associates (1997) Authentic achievement: Restructuring schools for intellectual quality. San Francisco: Jossey Bass.

Patton, M.Q. (1990). Qualitative Evaluation and Research Methods, 2nd Edition, Newbury Park, CA: Sage.

Porter, A.C. (1991). Creating a system of school process indicators. Educational Evaluation and Policy Analysis, 13, 13-29.

Shadish, W.R. (1993). Program evaluation: A pluralistic enterprise. In New Directions for Program Evaluation. No. 60, San Francisco: Jossey Bass.

Simsek, H., & Louis, K.S. (1994). Organizational change as paradigm shift. Journal of Higher Education.

Stallings, J. (1980). Academic learning time revisited, or beyond time on task. Educational Researcher, 8, 11-16.

Stringfield, S. (1995). Attempting to enhance students’ learning through innovative programs: The case for schools evolving into high reliability organizations. School Effectiveness and School Improvement, 6, 67-96.

Third International Mathematics and Science Study. (1996). A Splintered Vision: An Investigation of US Science and Mathematics Education.

Tjepkema, S (1994). Kenmerken van de lerende organisatie. Enschede, NL: University of Twente.

Tyack, D., & Cuban, L. (1995). Tinkering toward utopia: A century of public school reform. Cambridge, MA: Harvard University Press.

Van Velzen, W., Miles, M., Hamayer, U., Ekholm, M., & Robin, D. (1984) Making school improvement work. Leuven, Belgium: Acco Press.

	Return to: U of M Home
	One Stop \| Directories \| Search U of M

Project Description

Introduction