Rational Number Project Home Page

Lesh, R., Hoover, M. & Kelly, A. (1992). Equity, Technology, and Teacher Development. In I. Wirszup & R. Streit (Eds.), Developments in School Mathematics Education Around the World: Volume 3 (pp. ). Reston, VA: National Council of Teachers of Mathematics.

 

Equity, Assessment, and Thinking Mathematically:
Principles for the Design of Model-Eliciting Activities

Richard Lesh
Educational Testing Service

Mark Hoover
Educational Testing Service

Anthony E. Kelly
Rutgers University

 

Traditional approaches to mathematics assessment have favored the design of problems that are easy to score, but that deny individual students the opportunity to demonstrate mathematical problem solving. In this way, traditional problems may be serving to screen out students who are poor at computation and memorization, but may have mathematical abilities that are not being properly elicited. The focus of this paper will be a description of six principles for the design of what we will call model-eliciting activities, whose goal is to promote greater equity in mathematics assessment by spurring, nurturing, and supporting mathematical contributions from a larger pool of students.

According to the Mathematical Sciences Education Board, characteristics which distinguish mathematics from other domains of knowledge can be summarized as follows: (i) doing "pure" mathematics means investigating patterns (or systems) for their own sake, by constructing and transforming them in structurally interesting ways, and by studying their structural properties, and (ii) doing "applied" mathematics means using patterns (or systems) as models (or structural metaphors) to describe, explain, predict, or control other systems. Yet, when studies have investigated the alignment of nationally significant standardized tests with the NCTM Standards, their conclusions have been consistent and discouraging (National Research Council, 1990; Romberg, Wilson, & Khaketla, 1991), When new conceptual and procedural tools are used for new purposes in new types of problem solving situations, past conceptions of mathematical ability are often far too narrow, low-level, and restricted to use as a basis for identifying students whose mathematical abilities should be recognized and encouraged. To illustrate the contrasts between traditional and more current views of assessment in mathematics, consider the following table.

 

Assumptions Underlying Alternative Approaches to Assessment
 
 
Traditional Testing
Performance Assessment
The Nature of
Mathematics
Mathematics is considered to be nothing more than a list of mechanistic condition-action rules )facts, skills, definitions)... plus perhaps a few higher-order "metacognitive" rules to decide which lower-level rules should be stored and retrieved when they are needed. Many of the most important cognitive objectives of mathematics instruction are descriptive or explanatory systems (or mathematical models) which are used to generate predictions, constructions, or manipulations in real life problem solving situations ... or whose patterns can be explored for their own sake.
  According to the Mathematical Sciences Education Board (1990): Two outdated assumptions are that: (i) mathematics is a fixed and unchanging body of facts and procedures, and (ii) to do mathematics is to calculate answers to set problems using a specific catalogue of rehearsed techniques. (p.4).
The Nature of
Problem
Solving
Problem solving is characterized as getting from givens to goals when the path is not obvious. And, in practice, problem solving is generally assumed to involve mainly answering questions posed by others, within situations described by others, to get from givens to goals specified by others, using strings of facts and rules which are restricted in ways that are artificial and unrealistic ... so that responses can be evaluated by making a simple comparison to the one(s) expected by the authority (the teacher). Many of the most important aspects of real life problem solving involve developing useful ways to "think about" the nature of givens, goals, and possible solution paths; solutions typically involve several "modeling cycles" in which descriptions, explanations, and predictions are gradually refined and elaborated. Therefor, several levels and types of responses are nearly always acceptable (depending on purposes and circumstances); students themselves must be able to judge the usefulness of alternative models.
  Whereas problems in textbooks and tests tend to emphasize the ability to create meaning to explain symbolic descriptions, real problems tend to emphasize the ability to create symbolic descriptions to explain (manipulate, predict, control) meaningful situations. For example, for a mountain climber, the problem is to understand the terrain of a given mountain or cliff; once the terrain is understood, the activity of getting from the bottom to the top is simply a (strenuous, complex) exercise.
The Nature of
Experts
Good students (or teachers, or experts) are interpreted as people who flawlessly remember and execute factual and procedural rules ... and who are clever at assembling these facts and rules in ritualized settings. Experts are people who have developed powerful models for constructing, manipulating, and making sense of structurally interesting systems, and found ways to adapt, extend, or refine their models to fit new situations.
  If humans are characterized as information processors, and if knowledge is characterized as a list of condition-action rules, then learning tens to be viewed as a cumulative process of gradually adding, deleting, and debugging mechanistic rules (definitions, facts, or skills). If the precise state of knowledge is known for and expert (E) and for a given novice (N), then the difference between these two states is portrayed as the subtractive comparison (E-N).
The Nature
of Learning

Mislevy (1992), a leading psychometrician at the Educational Testing Service, states:

  • The test theory that dominates educational measurement today might be described as the application of twentieth century statistics to nineteenth century psychology. (Mislevy, 1991, p. 234).
  • The essential problem is that the view of human learning that underlies standard test theory is not compatible with the view rapidly emerging from cognitive and educational psychology.

Humans are considered to be model builders, theory builders, and system builders; the models that are constructed develop along the dimensions such as concrete-to-abstract, particular-to-general, undifferentiated-to-refined, intuitive-to-analytic-to-axiomatic, and situated-to-decontextualized. Also, cognitive development often involves discontinuities and conceptual reorganizations (such as those which occur when students go beyond thinking WITH a given model to also think ABOUT it). Consequently, experts not only know more, they also know differently.

 

Romberg, Zarinnia, and Collis (1989) emphasize impacts of the following outdated beliefs. (i) Reductionism assumes that, if you have something you want to explain, you take it apart until you identify its simplest parts. In this way, mathematics is partitioned into fragmented lists of behavioral objectives that are each treated as an end in itself. (ii) Mechanism assumes that once you break something apart, you build it up again based on simple chains of condition-action rules.

Resnick and Tucker (1991) emphasize similar outdated beliefs. (i) Decomposability likens thought to a simple machine; first, learn isolated facts and skills in isolation, then simply link them together to learn higher-order concepts and principles or to solve more complex problems. (ii) Decontextualization assumes that teaching (or testing) a skill out of context is the same as teaching (or testing) it within a realistic and meaningful situation.

The Nature
of Teaching
Teaching is considered to involve mainly: (i) demonstrating relevant facts, rules, skills, and processes, (ii) monitoring activities in which students repeat and practice the preceding items, and (iii) correcting errors that occur. According to the Mathematical Sciences Education Board (1990): The teaching of mathematics is shifting from an authoritarian model based on "transmission of knowledge" to a student-centered practice featuring "stimulation of learning" (p. 5). ...Teachers should be catalysts who help students learn to think for themselves. They should not act solely as trainers whose role is to show the "right way" to solve problems. The aim of education is to wean students from their teachers (p. 40).
 

At the national level, our foremost problem is not to screen talent; it is to identify and nurture capable students. As long as we continue to collapse all achievements and abilities into a single score or letter grade, discrimination is inevitable, since many talented students will be misclassified (Lesh & Lamon, 1992).

Research in mathematics education offers overwhelming evidence that: (i) there are many alternative types of mathematical talent, (ii) many different kinds of personalities, knowledge, and capabilities can lead to success, (iii) many different types of success are possible, and (iv) most people have irregular profiles of expertise, with strengths in some areas and weaknesses in others. In ethnographic studies investigating the mathematical capabilities of shoppers, tailors, carpenters, street vendors, and others (e.g., Lave, 1988; Saxe, 1990; Carraher et al., 1985), it has become clear that most people's "school math" abilities operate relatively independently from their "real math" abilities, and that failure or success in one area does not guarantee failure or success in the other. For example, Resnick & Resnick (1987) focus on the following reasons why traditional textbooks, teaching, and tests have been inconsistent with "real life" problem solving and decision making. School learning (i) emphasizes individual cognition, while learning in everyday contexts tends to be a cooperative enterprise; (ii) stresses "pure thought," while the outside world makes heavy use of tool-aided learning; (iii) emphasizes the manipulation of abstract symbols, while non-school reasoning is heavily involved with objects and events; (iv) tends to be generalized, while the learning required for on-the-job competency tends to be situation specific. They conclude that "... school work draws on only a limited aspect of intelligence, ignoring many of the intelligences needed for vocational success, especially in the more prestigious vocations" (p. 21).

Mathematical Problem Solving Involves Modeling Cycles

Our study of actual problem solving in mathematics suggests that the more traditional "get-one-answer-in-one-minute-or-less" tasks do not capture the problem solving that we wish to promote. To illustrate the types of modeling cycles which tend to occur when problem solvers engage with model-eliciting problems, consider a problem that was discussed in one of our teacher groups:

 

THE MATH PLACEMENT PROBLEM

Problem A: Imagine that you are the teacher at a middle school that has developed its own performance assessment program. The tests are not multiple-choice, and students' achievement scores reflect not only test performance but also teachers' classroom observations and evaluations of students' work portfolios. You've been given the following assignment.

Your school offers three sections of 9th grade mathematics classes. Your assignment is to work with the school counselor and another teacher to develop a policy for assigning students to one of these three sections. Write a letter to the principal describing the policy that you recommend for assigning students to the available courses. Then, demonstrate how your policy should be applied to the students whose test scores are shown in the following data sheet.

 

Math and Reading Achievement (Grade Level Equivalent Scores)

 
3rd Grade
4th Grade
5th Grade
6th Grade
7th Grade
8th Grade
 
Math
Read
Math
Read
Math
Read
Math
Read
Math
Read
Math
Read
Al
2.1
3.0
2.9
3.5
3.3
5.0
3.6
6.2
5.9
7.8
8.6
8.2
Barb
3.8
3.1
3.8
3.1
4.5
4.8
4.8
5.5
5.9
6.1
5.8
6.1
Carl
4.8
5.0
5.7
6.2
6.8
7.2
7.6
8.0
8.8
9.1
10.8
9.2
David
4.8
4.9
5.0
5.8
5.5
7.8
6.1
9.6
7.5
10.8
8.8
12.6
Edith
5.0
5.9
6.8
7.1
8.0
8.8
10.2
11.0
10.8
12.1
11.0
12.2
Fran
5.0
5.3
5.8
5.9
6.6
6.6
7.2
7.3
7.5
7.8
8.0
8.1
Greg
1.5
2.3
2.6
3.5
4.2
4.5
4.8
5.0
7.0
7.5
8.5
8.8
Hank 
  
 
 
 
 
 
2.3
1.5
5.8
6.0
9.8
8.8
Ida
3.3
3.1
4.5
4.8
5.6
5.5
6.8
6.9
7.8
8.1
8.9
9.2
Jan
5.6
5.1
7.9
7.0
9.0
7.5
9.6
8.0
10.8
8.3
8.0
7.8

 

COMMENTS FROM PREVIOUS TEACHERS

Al Al works hard. He always turns in homework and he even comes in after class for help. But, math has been difficult for him . . . The projects in his portfolio are not inspired; but they show his dedication
Barb Poor attendance. Often late for class. Since her mother died two years ago, Barbara has had to take care of her younger brothers and sisters at home. Her homework is rarely finished.
Carl Charles has consistently worked hard and is a very productive contributor to class discussions. Jed seems to know a great deal more than he has been able to demonstrate on tests.
David David is gaining confidence in himself, His success in sports seems to be rubbing off on other activities.---David is a leader.
Edith Edith is a gifted student. Recently however, she seems to have lost interest in her work. Although she finishes assignments on time, she doesn't seem to devote much time to them
Fran Fran is working to the best of her ability; but she needs work on basic skills. She spends too much time on low-level skills, and still low level mistakes lend to hurt her performance on tests and projects.
Greg Greg's projects are the best in the class. He is also good at creative types of problem solving - especially in geometry. Greg's immature behavior has interfered with his progress this year and has occasionally disrupted the whole group. He is the class clown.
Hank Hank only spoke Spanish when he moved here three years ago. He has shown tremendous improvement this year, but his previous training has been rather weak so he still has a lot to learn. If his remarkable improvement continues he could become the best in the class.
Ida Ida has a lot of natural ability but she hasn't worked up to her potential because of failure to complete assignments and to pay attention in class.
Jan Jan has discovered boys. She can be a good student when she wants to; lately she seems to have lost interest. She often falls asleep in class.
 

The solution that follows illustrates the kind of "modeling cycles" that one typical group of teachers went through while dealing with the problem.

The First Modeling Cycle. The group's first idea was to try to calculate a single "average score" for each student. They spent ten minutes reading and recording the numerical data in a spreadsheet table. One member of the group read the data; another member typed; and, the third member monitored and checked for data entry errors. Then, one member showed the others how to calculate an average for each row of scores.

The Second Modeling Cycle. After the preceding calculations were completed, the teachers began sorting students into groups based on the averages that were given. But disagreements arose about where some students should be assigned; in the course of these discussions, the group began to think more deeply about what the preceding scores meant in terms of whether a student was doing well or not doing well at a given grade level. For example, a score of 7.0 at the sixth grade means something quite different from a score of 7.0 at the eighth grade. As a result of these discussions, the original data were converted to "grade level equivalent" scores (as shown below):

 

Math and Reading Scores Compared to Actual Grade Level
 
3rd Grade
4th Grade
5th Grade
6th Grade
7th Grade
8th Grade
 
Math
Read
Math
Read
Math
Read
Math
Read
Math
Read
Math
Read
Al
-0.9
0.0
-1.1
-0.5
-1.7
0.0
-2.4
0.2
-1.1
0.8
0.6
0.2
Barb
0.8
0.1
-0.2
-0.9
-0.5
-0.2
-1.2
-0.5
-1.1
-0.9
-2.2
-1.9
Carl
1.8
2.0
1.7
2.2
1.8
2.2
1.6
2.0
1.8
2.1
2.8
1.2
David
1.8
1.9
1.0
1.8
0.5
2.8
0.1
3.6
0.5
3.8
0.8
4.6
Edith
2.0
2.9
2.8
3.1
3.0
3.8
4.2
5.0
3.8
5.1
3.0
4.2
Fran
2.0
2.3
1.8
1.9
1.6
1.6
1.2
1.3
0.5
0.8
0.0
0.1
Greg
-1.5
-0.7
-1.4
-0.5
-0.8
-0.5
-1.2
-1.0
0.0
0.5
0.5
0.8
Hank 
  
 
 
 
 
 
-3.7
-4.5
-1.2
-1.0
1.8
0.8
Ida
0.3
0.1
0.5
0.8
0.6
0.5
0.8
0.9
0.8
1.1
0.9
1.2
Jan
2.6
2.1
3.9
3.0
4.0
2.5
3.6
2.0
3.8
1.3
0.0
-0.2
 

Next, average scores were again calculated for each student, and the teachers again began to sort students into three groups, a high ability group, a middle ability group, and a low ability group. But disagreements arose again about where some students should be assigned. One of the teachers pointed out that "It doesn't make sense to use reading scores to sort students into math groups!"

The Third Modeling Cycle. As a result of the preceding discussions, a new table was created in which the reading scores were deleted (as shown below). Then, because the spreadsheet was able to create graphs quickly and easily, the following graph was created.

 

Math Scores (Only) For All Grade Levels
 
3rd
4th
5th
6th
7th
8th
Average
Al
-0.9
-1.1
-1.7
-2.4
-1.1
0.6
-1.1
Barb
0.8
-0.2
-0.5
-1.2
-1.2
-2.2
-0.7
Carl
1.8
1.7
1.8
1.6
1.8
2.8
1.9
David
1.8
1.0
0.5
0.1
0.5
0.8
0.8
Edith
2.0
2.8
3.0
4.2
3.8
3.0
3.1
Fran
2.0
1.8
1.6
1.2
0.5
0.0
1.2
Greg
-1.5
-1.4
-0.8
-1.2
0.0
0.5
-0.7
Hank 
     
-3.7
-1.2
1.8
-1.0
Ida
0.3
0.5
0.6
0.8
0.8
0.9
0.7
Jan
2.6
3.9
4.0
3.6
3.8
0.0
3.0
 
 
The Fourth Modeling Cycle. Again, when the group tried to use the preceding information to sort students into groups, everyone began to doubt that scores from the primary grades should count as much as scores from later grades. So, gradually, the idea arose to look at TRENDS rather than AVERAGES ... as shown in the graph that follows.
 
 

The Fifth Modeling Cycle. Finally, when the group began to use the information based on trends, they also began to make use of the qualitative information (i.e., the teachers' comments) which had been ignored in their earlier interpretations of the data. For example, they noticed that Hank's scores in math were probably influenced by his early lack of proficiency in speaking English. Therefore, when such factors began to be considered, the teachers concluded that the school should abandon its policy of sorting students into low, middle, and high ability groups. Instead, they recommended that: (i) three equivalent math groups should be created, (ii) students who appeared to need special attention should be distributed equally among the three sections, and (iii) all of the case histories (e.g., the trend information and teachers' comments) should be made available to the students' teachers.

Summary. The preceding solution illustrates the following characteristics which are common in "real life" situations in which mathematics is used: (i) coordinating and communicating efforts is often as important as the efforts themselves, (ii) analyzing plans (or processes, or results) is often as important as generating them, and (iii) justifying and explaining decisions is often as important as simply making decisions. Further, since powerful technology-based tools were available, more time was spent discussing the meaning of the procedures that were used; and, because of this process, one-rule/single-step solutions often emerged as being overly simplistic. Finally, because both "too much" and "not enough" information were available, and because some of the relevant information involved patterns of data rather than simply isolated pieces, an explicit model was needed to filter, organize, interpret, "fill holes," and "project beyond" the information that was available.

If the solution process itself is examined, some other common characteristics of "real life" problems also become apparent. First, the overall solution process involved a series of "modeling cycles," each of which involved somewhat different interpretations of givens, goals, and available solution steps. Second, the earliest interpretations of the problem made a number of unjustified assumptions that were not given, and (at the same time) failed to recognize a number of significant pieces of information that were given. Third, the final solution involved a significant reformulation of the original conception of the problem itself; in addition it also went beyond providing a solution to a single isolated situation to provide a conceptual framework for interpreting a wide range of other structurally similar situations. Fourth, even though many levels and types of "correct" responses were possible, the problem solvers themselves knew when one model was better than another, and they were also able to identify in which directions a given model should be modified in order to improve.

After discussing the characteristics of a variety of problem solving situations of this type, the teachers who participated in our projects agreed to adopt the following first-round assumptions about the "real life" mathematical activities that they were trying to develop.

  • Solutions should require approximately 5-50 minutes to construct, and should provide powerful prototypes for dealing with issues that are important to the students or others they would like to impress.
  • Contexts should be the kind that might reasonably occur in the everyday lives of students or their friends or families.
  • Issues should fit the interests and experiences of targeted students, and should encourage students to engage their personal knowledge, experience, and sense-making abilities.
  • Solution procedures should allow students to use realistic tools and resources, including calculators, computers, consultants, colleagues, and "how to" manuals.
  • Evaluation procedures should recognize more than a single type and level of correct response.
  • Overall activities should contribute to both learning and assessment ... because students simultaneously learn and document what they are learning.

Beyond the preceding characteristics, however, the most important attribute of the problems is that they should involve inventing (constructing, extending, refining) a conceptual system that was both mathematically significant and practically important... without explicitly teaching this system to them. In other words, the problem should be model-eliciting.

Principles for the Design of Model-Eliciting Problems

The remainder of our paper will focus on examples to illustrate the following six principles for developing model-eliciting activities aimed at helping students (simultaneously) develop and document "real life" problem solving abilities.

  • The Model Construction Principle. Does the task create the need for a model to be constructed, or modified, or extended, or refined? Does the task involve constructing, explaining, manipulating, predicting, or controlling a structurally significant system? Is attention focused on underlying patterns and regularities rather than on surface-level characteristics? The Simple Prototype Principle. Is the situation as simple as possible while still creating the need for a significant model? Will the solution provide a useful prototype (or metaphor) for interpreting a variety of other structurally similar situations?
  • The Model-Documentation Principle. Will the response require students to reveal explicitly how they are thinking about the situation (givens, goals, possible solution paths)? What kind of system (mathematical objects, relations, operations, patterns, regularities) are they thinking about? . The Self-Evaluation Principle. Are the criteria clear for assessing the usefulness of alternative responses? Will students be able to judge for themselves when their responses are good enough? For what purposes are the results needed? by whom? when?
  • The Model Generalization Principle. Does the model that is constructed apply to only a particular situation, or can it be applied to a broader range of situations?
  • The Reality Principle. Could this really happen in a "real life" situation? Will students be encouraged to make sense of the situation based on extensions of their own personal knowledge and experiences?

The Model Construction Principle

The goal is not simply for students to produce an "answer" to a question. Students should also construct mathematically significant systems that can be used to describe, explain, manipulate, or predict a wide range of "real life" experiences. Therefore, in order for a problem to be model-eliciting, one of the activity author's main goals is to confront students with the need for a model. If the need for a model is clear, then students tend to produce one. How can activities create the need for a model? Authors should ask themselves "What kinds of situations create the need for anyone (myself, other adults) to create models, whether they are working in mathematics, in science, in business, or in everyday life?" Answers to this question include the following.

  • Models are needed when it is necessary to make predictions based on underlying patterns or regularities, e.g., to anticipate real events, to reconstruct past events, or to simulate inaccessible events.
  • Models are needed when constructions or explanations are requested explicitly, e.g., for the purpose of describing hypothesized patterns and regularities, or for the purpose of describing decision-making situations in which "too much" or "not enough" information is available.
  • Models are needed when it is necessary to justify or explain decisions, e.g., by describing underlying assumptions, conditions, and alternatives.
  • Models are needed to resolve interpretation mismatches, e.g., between hypotheses and reality (or between two competing interpretations, predictions, or explanations of a given situation).
  • Models are needed when it is necessary to re-create and critically analyze conclusions or descriptions generated by others.

From the point of view of instruction, the preceding kinds of activities lead to three positive outcomes. First, the models that students construct often involve mathematical ideas that are far more sophisticated than those associated with their prior failures in traditional textbooks, tests, and teaching. Second, they enable teachers to recognize students' baseline abilities and understandings (or misunderstandings), so that follow-up instruction can build on students' strengths and address their weaknesses. Third, the solutions that students produce explicitly reveal the diverse ways that students think about mathematically rich situations. Therefore, they help teachers recognize and reward a broader range of mathematically capable students.

Consider the following running problem (for students), which is based on a recent newspaper article and is similar to the math placement problem (for teachers) that was described earlier. It involves making predictions that extrapolate beyond the given information; it also involves producing materials to explain the predictions that are made.

 

The Fast Track Activity

The editor of the high school newspaper is writing an article entitled "The Fast Track." The article will predict and compare the speeds for women and men in the 200 meter events for future World Olympics. The editor needs your help to predict the speeds for the next 50 World Olympics (the next 200 years).

Write a report of your predications and conclusions for the editor. The editor needs to justify the predictions in the article. Therefore, include any graphs, charts, or other materials that would help the editor understand the reasoning for the predictions.

 
Women runners threaten to overtake men

If women's running times continue to improve, top women may soon catch up with the best men.

In fact, women may even outrun men someday, according to two scientists. This is based on the rate at which women’s race times have been improving. Since 1920, women's times have improved much more than men's.

to do this," said Dr. James Downing, an exercise physiologist at the University of Texas. "I'd agree that there's a way to go yet in women’s performance, but if they're suggesting that women will approach men, that's silly." But the two researchers said that the women's trend has been too constant to ignore.

"Before looking at the data, I would have thought it was impossible

fastest man, according to Whipp

In the marathon, women's time have improved about 61 percent since 1955. Men's performance has improved only 18 percent. There's no question that women have come a long way in catching up with men, but there is a long way yet to go.

The fastest female runners today would not even qualify for the men's

looks good, but I've been to 100 marathons around the world, and I've seen all the women runners. Women will never pass men. Never, never."

Joan Benoit Samuelson, a runner who was the 1984 Olympic marathon champion and holds the record for American women, said that women might get closer to men’s times, but would never beat them.

"Men have had a lot

The researchers say that the best female runners have been getting faster at such a rapid pace that they should run marathons as quickly as men by 1998. Women should catch up with men in shorter track events by the middle of the next century.

These predictions are based on a comparison of trends in men's and women's world records over the past 70 years.

more time to evolve in the sport, and since they've got that jump start, they'll be hard to beat now," she said. "You also have the male ego to consider, and that's going to keep men going."

According to Downing, who won three Olympic

gold medals in the 1960s for running, addition, men have more red blood cells, which means that they can get more oxygen to their muscles. Downing

Based on these patterns, projections are made into the future.

The results were published by Dr. Brian J. Whipp and Dr. Susan A. Ward in the British journal, "Nature." The two scientists teach at the University of California in Los Angeles.

Other experts do not believe Whipp and Ward. "I wonder why they're even bothering

said Whipp, "But if the trend continues, women will have the same times as men by the next century." He added, "This is not me talking. It's the facts." When Roger Bannister became the first man to run a four minute mile in 1954, Diane Leather was the first woman to run a five minute mile. If they were in the same race, she would have finished 320 meters behind Bannister.

Today, the top female runner would only finish 180 meters behind the

track events in the Olympics. In the marathon, the men's world record is 2:06.50, while the women’s is 2:21.06. By marathon standards, this is a huge difference.

Many people doubt that women will ever catch up with men.

"Women will never, ever catch up to the men in Olympic running events," said Frank Uttle, president of the New York Road Runners Club. "Maybe on paper this

thinks that the women's improvements are due to social changes. "Finally, women are starting to get out and do the same things as men."Patti Sue Plumer, the number one ranked woman runner, does not think that physical advantages are that important. "As an athlete, I've learned that the mind plays a much stronger role than anything physical," she said.

Perhaps the debate will only be settled by time. For female runners, the race has only just begun.

 
Gold Medalists in the Men's
200 Meter Event
Gold Medalists in the Women's
200 Meter Event
Year Name, Country Time
in
seconds

Speeds
in mph

Year Name, Country Time
in
seconds

Speeds
in mph

1988 Joe DeLoach,
United States
19.75 22.6 1988 Florence Griffith-Joyner,
United States
21.34 20.9
1984 Carl Lewis,
United States
19.80 22.5 1984 Valeri Brisco-Hooks,
United States
21.81 20.5
1980 Pietro Mennea
Italy
20.19 22.1 1980 Barbel Wockel,
E. Germany
22.03 20.3
1976 Donal Quarrie,
Jamaica
20.23 22.1 1976 Barbel Eckert,
E. Germany
22.37 20.0
1972 Valeri Borzov,
USSR
20.00 22.3 1972 Renate Stecher,
E. Germany
22.40 19.9
1968 Tommie Smith,
United States
19.83 22.5 1968 Irena Szewinska,
Poland
22.5 19.8
1964 Henry Carr,
United States
20.3 22.0 1964 Edith McGuire,
United States
23.0 19.4
1960 Livio Berruit,
Italy
20.5 221.8 1960 Wilma Rudolph,
United States
24.0 18.6
1956 Bobby Marrow,
United Stated
20.6 21.7 1956 Betty Cuthbert,
Australia
23.4 19.1
1952 Andrew Stanfield,
United States
20.7 2.6 1952 Marjorie Jackson,
Australia
23.7 18.8
1948 Mel Patton,
United States
21.1 21.2 1948 Francina Blanker-Koen,
Netherlands
24.4 18.3
1936 Jesse Owens,
United States
20.7 21.6        
1932 Eddie Tolan,
United States
21.2 21.1        
1928 Percy Williams,
Canada
21.8 20.5        
1924 Jackson Scholz,
United States
21.6 20.7        
1920 Allan Woodring,
United States
22.0 20.3        
1912 Ralph Craig,
United States
21.7 20.6        
1908 Robert Kerr,
Canada
22.6 19.8        
1904 Archie Hahn,
United States
21.6 20.7        
1900 Walter Tewksbury,
United States
22.2 20.1        
 
If an activity is truly model-eliciting, then authors should be able to give clear answers to the following questions about the systems that students are being challenged to construct. (i) What kind of mathematical "objects" do students need to consider? (Possible answers include ratios, trends, and coordinates.) (ii) What kind of relationships among objects do students need to consider? (Possible answers include equivalence relationships, ordering relationships, and invariance under transformations.) (iii) What kind of combinations or interactions among objects do students need to consider? (Possible answers include additive combinations and multiplicative interactions.) Therefore, if we are to write effective model-eliciting activities, the goal is to create the need for students to construct a description or an explanation that involves interesting mathematical objects, relations, and operations.

Of course, when an activity creates the need for a description or an explanation, there is nothing to guarantee that the system that students construct will be identical to the one the teacher had in mind. This is because choices are nearly always available about issues such as: (i) which representation system to use (e.g., graphs versus equations), (ii) which types of units to emphasize (e.g., trends involving ratios, or relationships involving trends), or (iii) which level of accuracy and precision is most appropriate. For example, in the case of the running problem, one recent sample of middle schoolers generated results which reflected the following ways to think about the situation.

Different Ways to Think about the Running Problem

Units of Analysis (Simple vs. Composite): What are the units people think about when working on this problem? Sometimes people use small simple units such as one year, one Olympics, individual running speeds, or individual running times. Other times they use larger, composite units, such as blocks of data, patterns, or trends.

 

 

Differences vs. Ratios: How do people think about change? Sometimes people think of change in terms of differences (absolute change). Other times they think more in terms of percentages or ratios (relative change). Change can be relative to time intervals (e.g., years), or change can be relative to running speeds or running times. Complex ratios can be relative to both time intervals and running speeds/times.

Numerical Patterns vs. Graphical Patterns: How do people think about data? Sometimes people think in terms of numerical data, such as numbers, sequences of numbers (e.g., lists or tables), or composites of numbers (e.g., sums or averages). Other times they think in terms of visual data, such as points, sequences of points (e.g., patterns or graphs), and composites of points (e.g., slopes).

Independent Data vs. Comparative Data: How do people think about more than one set of data? Sometimes people respond to men's and women's data separately, project into the future, then compare the two. Other times they consider differences between the two sets of data, then project the differences.

Little-Picture vs. Big-Picture: How do people think about making predictions based on past performances? Sometimes people think about the problem by projecting from little-picture information what a next one will be, then a next one, a next one, and so on. Other times they extrapolate from big picture information what some future situation will be and then attempt to fill in the holes.

 

 

Linear vs. Non-Linear: How do people think about trends? Sometimes people think about the problem in a linear fashion. They see a constant rate of increase and project it in a steady-state fashion. Other times people think about the problem in non-linear ways. They think in terms of a limiting factor or a dynamic rate of change. This may be expressed as leveling off, maxing out, or peaking.

The important point to notice about the preceding ways of thinking about the problem is that the various types of objects, relationships, and operations could be combined in many ways to produce acceptable responses. Also, just as in the case of the math placement problem for teachers, students' solution procedures for the running problem could involve using many different types of tools, and a given student could play many different roles during the solution process. Nonetheless, in each case, when students construct a structurally interesting model, they actually invent (or modify, or refine, or extend) an important mathematical system.

The Simple Prototype Principle

Situations that serve as good prototypes (or good structural metaphors) tend to be elegant. This is why the goal of the simple prototype (or parsimony) principle is to emphasize activities that are as simple as possible, while still serving as useful prototypes for thinking about significant mathematical systems. Yet, when teachers first begin to try to write model-eliciting activities of the type described in this paper, simplicity is not one of the terms that they are most likely to use to describe the kind of activities they want to emphasize. Instead, they use terms such as realistic, applications, complex, and difficult. There is some truth in each of the preceding terms. But they also tend to reflect the following common misconceptions:

  • Realistic situations (at least for students who are not viewed as college bound) are seldom portrayed as management-level decision making issues. Yet even in leading graduate schools of management, the models that are emphasized are seldom based on anything beyond middle school mathematics.
  • Applications are often considered to be appropriate after all of the relevant mathematical ideas have been "mastered." Yet it is well known that mathematical ideas tend to develop along dimensions such as concrete-to-abstract, or intuitive-to-formal, or situated-to-decontextualized.
  • Complex situations tend to be thought of as those that involve lots of intricate calculations. But because model-eliciting activities should serve as prototypes for thinking about significant conceptual systems, their procedural complexity should be minimized whenever possible.
  • (A faulty "rule of thumb") If it's difficult for me (the teacher), then it will be ten times more difficult for my students. Our experience has shown that when teachers first begin to work on realistic model-eliciting activities, they rarely perform better than their average students.

One reason this last phenomenon occurs so frequently is that teachers, much more than their students, are often reluctant to put aside their "school math minds" and use their "real math minds." Yet many skills and beliefs that contribute to success in school are counterproductive in most "real life" situations. For example, in school, being caught thinking tends to mean that you were caught not knowing; taking more than three seconds to respond means you'll probably be passed over by the teacher; getting an answer that isn't a whole number is a clue that you're doing something wrong; and using more than a single rule means that you probably aren't doing it the "right" way. By contrast, in many "real life" situations, it is often the inappropriate responses that are associated with quickly generated answers and single-rule "canned" solution procedures.

The Model Documentation Principle

One way in which the running problem differed from the math placement problem was that for many groups of teachers who worked the latter, it was necessary to watch videotapes of their solution processes in order to discover how they had interpreted the situation. This is because the stated goal of the math placement problem focused mainly on making decisions (or on giving answers) rather than on descriptions or explanations of the decisions that were made.

From the point of view of the author of the running problem, it was important to state the goal of the problem in such a way that the results students produce reveal as much as possible about: (i) how they are thinking about the situation, and (ii) information that otherwise would have become \apparent only by watching a videotaped record of the solution process. Of course, not everything that is apparent on a videotape can be captured in students' final results. For example, the final results that students produce seldom provide much information about: (i) the "modeling cycles" that students went through to arrive at these results, or (ii) alternative models that students developed and rejected, or (iii) the roles that different students played during the solution process. Nonetheless, if care is taken, it is usually possible to state the goals of model-eliciting problems so that they capture as much information as possible about how students think about the situation. Then other relevant information can usually be recorded by teachers using classroom observation forms, or using other data gathering tools and procedures.

One straightforward implication of the model-documentation principle is that traditional kinds of mathematics questions are not acceptable if the results they ask students to produce consist of nothing more than simple numerical answers (e.g., 30 feet, 5 dollars) or simple decisions (e.g., yes-no, multiple choice). Even though it is possible for such problems to elicit a model, they provide little information about the nature of the model that is elicited. Effective model-eliciting activities should enable students to simultaneously learn and document what they are learning.

The Self-Evaluation Principle

If students are unable to make judgments about whether (or in which directions) current solutions need to improve, or about which of several alternative solutions is most useful, then it is unlikely that they will ever go beyond first-round solutions to problems. The direction of the task (as described by information about what is to be produced, when, why, for whom, and under which conditions it is to be produced) is especially important in the case of model-eliciting activities for groups of students, because outstanding solutions generally involve several modeling cycles. The model produced at each cycle requires that in the statement of the problem there exist some criteria for judging its adequacy.

Effective model-eliciting activities should be stated in such a way that students themselves can assess the usefulness of the results that they produce. Therefore, when students ask "Are we done yet?", the appropriate response from teachers should be to refer students back to the statement of the problem, where guidelines should be available (implicitly or explicitly) about issues such as: useful to whom? useful for what purpose? useful under what conditions?

To emphasize the preceding points, some teachers who participated in our research have written "letters from the editor" (see the example below) to address some of the student responses. Other teachers have chosen to use such letters to provide guidelines for whole-class discussions, where the goal is for various groups of students to compare alternative responses to the problem and to assess the strengths, weaknesses, and directions for improvement for each.

The Model Generalization Principle

The following two types of generalization are not the same. First, one can ask whether a given student is able to generalize a particular piece of mathematical knowledge; second, one can ask whether a student is able to construct a model which generalizes to a whole class of situations. In the first case, generalizability is attributed to the student; in the second case, generalizability is attributed to the model. In the first case, generalizability tends to be quite difficult to verify; but in the second case, it is usually a straightforward task to verify it. It is the second case to which the model generalization principle applies. To understand the intent of the Generalization Principle, consider the Don't Drink & Drive Problem.

 

FAST TRACK ACTIVITY
Feedback Letter: Student Group Two

Dear Students,

I looked through the materials that you gave me. I must say, they are rich in ideas! Unfortunately, in their present form, none of the ideas is complete enough for me to use for my article.

You have some nice ideas on the first page. The idea that men have been running longer and have leveled off is interesting. Couldn't you follow through with that idea and make some predictions based on that view? I realize that nay predictions we make are not exact, but what is the best that we can do?

The graph of predictions that you gave through 2040 is nice, but I need predictions for the next 50 Olympics and it is unclear to me where the predictions come from. Besides, it sounds as if you do not believe the predictions.

Your "Increase Graph" seems to be based on yet another set of ideas. Why should the percentage increase remain constant? If the percentage increase is constant, then the increase would skyrocket rather than level off )as you claimed on the first page).

In addition, I am glad that you checked out the data, but I did not understand your arguments that the data is faulty. Speed and time are inversely related, but this does not mean that their quotients are constant.

The pieces don't fit together. The text doesn't fit with the graphs. The graphs don't fit with each other. It looks as if there are some good ideas here, but that the ideas are not yet sorted out. If you revise this work, please re-submit it to me. Good luck!

Thanks,            
The Editor        

 
New drunk driving law takes effect
Goal is to cut down number of accidents

In 1990, 46,300 people were killed in automobile accidents. Hundreds of thousands more were injured.

As many as half of these automobile accidents were caused by drunk drivers. States are cracking down on drunk driving. Across the country, laws are being passed which make drunk driving a very serious crime.

It is important for those people who choose to drink to understand what their blood alcohol level is. The higher the level, the more impaired a person's abilities are. A 12 ounce can of beer, a five ounce glass of wine and a one and a half ounce shot of hard liquor all contain the same amount of alcohol. Each of these drinks should be counted as one drink.

How much an alcoholicdrink affects someone depends on that person's weight. Thus the effect of a glass of wine is much more on a light person than on a heavy person.

Once a person stops drinking, their blood alcohol level drops about 0.015 percent every hour.

For example, a 140 pound

person with a blood alcohol level of 0.10 percent would have a blood alcohol level of 0.085 percent after one hour and 0.070 after two hours.

While a person can legally drive at blood alcohol levels of 0.04 percent or lower, experts warn that impairment is still possible. To be safe,the best policy is never to drink and drive.

A new local law goes into effect December 31. This law defines drunk driving as anyone who has a blood alcohol level of .05 percent or higher. A person caught driving while legally drunk can face up to $500 in fines and a possible jail sentence on the first offense.

If one drinks, one should at least have a "designated driver." A designated driver goes out with the other people, but does not drink. At the end of the night, the designated driver can take everyone home safely. Other ways for drinkers to be safe are to call a taxi or stay overnight at the place where they are drinking. These alternatives may seem inconvenient-but getting in an accident or going to jail is worse yet.

 

Don't Drink & Drive Hotline Activity

A community group wants to start the Don't Drink and Drive Hotline. The group asked your class for help. The hotline needs a method to estimate a caller's blood alcohol concentration (BAC). Develop useful tools for estimating BAC. (These might include tables, graphs, computer software, etc.) Your method should work for the wide range of cases the hotline might receive. Write a description of your tools and method to assist hotline employees in using them.

 

Early versions of the Don't Drink & Drive Problem were similar to many problems in textbooks and tests. For example, students were asked to estimate the blood-alcohol index for a person whose weight or level of consumption fell between or beyond the data given in the table. But no information was given about who was asking the question, or for what purpose the question was being asked. Consequently, there was no way for students to make judgments about issues such as how accurate the answer should be. Also, if the question involved only a single instance (i.e., a single person, and a single set of circumstances), then it usually was possible for students to generate "acceptable sounding answers" without really doing much mathematics. On the other hand, when the task that was posed involved using a graph or a formula to describe (and generate results for) a whole class of situations, then the responses students generated naturally focused on significant mathematical ideas.

The Reality Principle

The goal here is to encourage students to develop mathematical models based on extensions of things that they already understand and can do. Therefore, if students' legitimate ideas are dismissed or not taken seriously (even though their points might be valid in real situations), then the result is that the students usually "tune out" even if the topic appeared to be one that would "turn them on." To illustrate the Reality Principle consider a teacher's rewriting of the following problem, which was included in the NCTM's Standards for Curriculum and Assessment in School Mathematics (NCTM, 1989).

 

The Original Softball Problem

 

The table gives the record for Joan Dyer's last 100 limes at bat during the softball season She is now coming up to bat. Use the data to answer the following questions:

What is the probability that Joan will get a home run? What is the probability that she will get a hit? How many times can she be expected to get a walk in her next 14 times at bat?

 

Home runs
9
Triples
2
Doubles
16
Singles
24
Walks
11
Outs
38


Total
100

 

One Teacher's Analysis Of The Softball Problem

Critique: On the surface, this problem appears to be embedded in a real- world situation: Joan is coming up to bat, and the problem description gives some data about her prior performances But, in a real situation, it wouldn't be sensible for someone (other than a math teacher) to want to know the answer to the questions as they are stated (concerning probability of a home run, or the probability of a hit) In fact, simply computing this "probability" using the intended rule depends on ignoring common sense and/or practical experience In reality, the probability depends on who is pitching (Are they left handed or right handed?), on field conditions or the weather, and on a lot of other factors that people who play softball are aware of (Who is in a slump or on a streak? Who is good under pressure) Furthermore, since we don't know who is asking the question, or why, we can't know what to take into account, or how accurate the answer needs to be, or what the risks or benefits might be. Therefore, the criteria for judging the quality of answers are not implicit to the situation; and, solutions must be judged according to whether they conform to the calculation that was expected, rather than according to whether they succeed in any practical or meaningful sense of the word,

Overall, I do not think such a problem is realistic, or that it promotes authentic performance Real softball players would actually have to turn off their "real life" knowledge and experience.

Analysis: In this problem, it is not particularly necessary to have Joan coming up to bat right now. In fact, for the third question we might wonder what coming to bat, at this particular moment in time, has to do with expectations about walks for the next 14 times at bat, which may not even happen in the same game, Is there some reason, now that Joan is coming up to bat, to want to know her expected number of walks for the next 14 times at bat? Why not the next 15 times at bat? Is it possible that having Joan at bat in the first place was only the awkward result of an well-intentioned effort to create a life-like context (for a problem with essentially one appropriate solution path that leads to a single "right" answer)? Or was it a superficial gesture to suggest that even girls play sports? If not for either of these reasons, why it is there?

Possible Improvements: Here is a suggestion for improving this problem, Notice that the "math answer" is not an end in itself, It is a means to an end (or a tool for informing actions, decisions, and judgments), If Joan really is coming up to bat now, the mathematics should address a plausible question that might occur in that situation, I think that the revised item asks a more authentic question in the sense that it asks for a decision that might be required in the context.


The Improved Softball Problem

You are the manager of a softball team It is the bottom of the ninth inning, two outs gone, and no one is on base Your team is one run behind You plan to send in a pinch hitter in hopes of scoring the tying run. Your possibilities are Joan, Mary, and Bob, Their batting records are given in the table below, who would you choose to bat? Explain your reasoning.

 

 
Joan
Mary
Bob
Home runs
9
15
6
Triples
2
5
3
Doubles
16
11
8
Singles
24
34
18
Walks
11
20
12
Outs
38
85
36

 

 

As the preceding analysis makes clear, a "real" problem is not simply one that refers to a real situation. The question that is asked should also make sense in terms of students' "real life" knowledge and experiences.

The topics that work best tend to be those that fit the current local interests of specifically targeted groups of students; thus, teachers should "localize" problems to fit the current interests and experiences of specific groups of students. Not all students should have to demonstrate their competencies in the same situation. Clones, or structurally isomorphic problems, should be available that involve a variety of contexts. The goal is to provide all students with as many "low pressure, high interest" opportunities as possible to demonstrate their abilities and achievements within contexts that are familiar and comfortable. Therefore, teachers and students should work together to select whichever task fits their interests and experiences.

Summary

Even in the case of everyday problems that involve little more than addition or subtraction, the mathematics that is used is functioning as a model of the situation. But unfortunately, students in schools are rarely allowed to see situations in which the model that is being used is problematic-even though such situations abound within "real life" situations.

Thinking mathematically means, more than anything else, interpreting situations mathematically. Yet, most of the word problems in traditional textbooks and tests attempt to minimize (or eliminate) the interpretation phases of problem solving, often because of the misguided belief that thinking mathematically consists mainly of rule following, and because of the corollary that mathematical problems usually have only one correct answer.

Mathematical ability does not simply consist of the ability to flawlessly remember and execute intricate sequences of rules; and when doing mathematics includes generating descriptions, explanations, manipulations, and predictions, there is rarely only one correct answer. Choices about which answer is most useful generally depend on conditions (e.g., time constraints, resource constraints) and on purposes (e.g., risk constraints, generalizability constraints). Furthermore, when powerful conceptual and procedural tools are available, many more options are available concerning modes of responses and solution paths. When a broader range of mathematical ideas and tools are used to address a broader range of problem solving situations, a broader range of mathematical abilities emerge as resources for success-and a broader range of students emerge with higher potential.

References

Carraher, T., Carraher, D. & Schliemann, A. D. (1985). Mathematics in the streets and the schools. British Journal of Developmental Psychology, 3, 21-29.

Lave, J. (1988). Cognition in practice: Mind, mathematics and culture in everyday life. New York: Cambridge University Press.

Lesh, R. & Lamon, S. (1992). Assessments of authentic performance in elementary mathematics. Washington, DC: American Association for the Advancement of Sciences.

Mathematical Sciences Education Board. (1990). Reshaping School Mathematics: A Philosophy and Framework for Curriculum. National Research Council. Washington, DC: National Academy Press.

Mislevy, R. J. (1992). Foundations of a new test theory. In N. Frederiksen, R. J. Mislevy, and I. Bejar (Eds.), Test theory for a new generation of tests. Hillsdale, NJ: Lawrence Erlbaum Associates.

National Council of Teachers of Mathematics. (1989). Curriculum and Evaluation Standards for School Mathematics, K-12. Reston, VA: Author.

Resnick, D. P. & Resnick, L. B. (1985). Standards, curriculum, and performance: A historical and comparative perspective. Educational Researcher, 14(4), 5-21.

Resnick, L., & Tucker, M. (1991). The standards project: Creating a national examination system that prepares students for the challenges of the 21st century. (Unpublished overview). Pittsburgh, PA: Learning Research Development Center, University of Pittsburgh.

Romberg, T. A., Wilson, L., & Khaketla, M. (1991). The alignment of six standardized tests with the NCTM standards. Madison, WI: University of Wisconsin.

Romberg, T. A., Zarinnia, E. A., & Collis, K. F. (1989). A new world view of assessment of mathematics. In G. Kulm (Ed.), Assessing higher order thinking in mathematics (pp. 21-38). Washington, DC: American Association for the Advancement of Sciences.

Saxe, G. B. (1990). Culture and Cognitive Development: Studies in Mathematical Understanding. Hillsdale, NJ: Lawrence Erlbaum Associates.

Steen, L. A. (Ed.). (1990). On the Shoulders of Giants: New Approaches to Numeracy. National Research Council. Washington, DC: National Academy Press.

Biographical Notes

Richard Lesh is a Principal Research Scientist at Educational Testing Service, where he is currently serving as the Director of ETS's Research Program on Technology and Assessment. For more than fifteen years he was a Professor of Mathematics and Psychology at Northwestern University.

Dr. Lesh's research has included studying children's abilities to use elementary mathematics in everyday problem-solving situations, where realistic tools are available and responses often require 30-60 minutes to construct. Current NSF-funded projects include three in which "real life problem-solving activities" are being used to focus on issues involving equity, teacher development, and the influence of technology-based tools in on-the-job problem-solving situations.

He has contributed much to software development, and has authored books for children, teachers, teacher educators, curriculum developers, and researchers. He is co-author of Assessments of Authentic Performance in Elementary Mathematics, published by the American Association for the Advancement of Science which is particularly relevant to the current national curriculum reform effort.

Mark Hoover is an Associate Research Scientist who divides his time between computer science test development for the GRE and AP programs and research in mathematics and computer science education. He has an M.A. in mathematics and an M.A. in anthropology from the University of North Carolina at Chapel Hill and a Ph.D. in computer science from the University of New Mexico at Albuquerque.

Dr. Hoover's early research was in educational anthropology, theoretical computer science, and computational graph theory. He is currently conducting research on personality types in mathematics education and instruction/assessment in mathematics education.

Anthony E. Kelly serves as Assistant Professor at the Graduate School of Education of Rutgers University. He received his Ph.D. from Stanford University in psychological studies in education. Dr. Kelly's research interests include student modeling techniques for intelligent tutoring systems as they relate to mathematics and science, as well as authentic assessment.

(top)