Evaluating the Impact of Educational Reform in Statistics: A Survey of Introductory Statistics Courses

Final Report for NSF Grant REC-9732404

Principal Ivestigator: Joan Garfield
Department of Educational Psychology
University of Minnesota
Email: jbg@umn.edu

Overview

Many people are familiar with the calculus reform movement that aimed to transform the teaching of calculus in high schools and colleges. Less well known is a similar movement within the statistics community that recommends major changes in the introductory statistics course (e.g., Cobb, 1992; Hogg, 1992). The National Science Foundation funded numerous projects designed to implement aspects of this reform (Cobb, 1993). Moore (1997a) describes the reform in terms of changes in content, pedagogy, and technology. Scheaffer (1997) sees more agreement today among statisticians about the content of the introductory course. However, Moore (1997b) points out that many people teaching introductory statistics are not statisticians. In fact, far more sections of introductory statistics are taught in mathematics departments or in other disciplines than by statisticians in statistics departments. Two important questions to consider, which led to this study, are:

• How has the reform movement in statistics education affected the teaching of introductory statistics courses?
• How different is the teaching of statistics in different departments and institutions?

The original project proposal outlined a three-stage evaluation to answer these questions. The evaluation would provide empirical evidence regarding the extent and impact of this reform and examine the teaching of statistics across disciplines and departments. This study would also lay the foundation for future studies that might assess student outcomes to ultimately determine the success of the reform efforts.

The project was to include three phases. The first phase was to conduct a large-scale survey of all mathematics and statistics departments. The second phase was to conduct a similar survey based on samples from other departments that teach statistics. The third phase was to conduct case studies of a few exemplary departments that have successfully implemented reform efforts in their introductory statistics courses.

Results of the evaluation would provide reports containing summaries of how statistics is currently being taught in the different disciplines, to what extent the reform has affected the teaching of statistics, what efforts have been made to implement the reform in departments, and what are viewed as the outcomes, in terms of faculty perceptions of their “reformed” courses and their students’ learning.

Changes to the original proposal

Based on recommendations from the project advisory group (see Appendix for the list of advisors) as well as the Committee on Applied and Theoretical Statistics (CATS), the project was modified in two ways. First, a Phase 0 was introduced to gather data on the infrastructure of introductory statistics courses. Second, a stratified random sample was taken of mathematics departments based on highest degree offered. Rather than sending a survey to all mathematics and statistics departments, surveys would be sent to all statistics departments and to about 400 mathematics departments.

Phase 0: Gathering data on the infrastructure of introductory statistics courses

A mailing was sent to a sample of 506 chairs of mathematics and statistics departments, as shown in Table 1. Two different cover letters were used. David Moore, President of the American Statistical Association, wrote a cover letter for the mailings sent to statistics departments, urging faculty to participate in the survey. George Cobb, Chair of the MAA-ASA Joint Committee on Statistics, wrote a similar cover letter for the mailings to mathematics departments. A second letter, written by the project director, described the forthcoming Fall Survey and its purpose (to gather baseline data on the current teaching of the introductory statistics course) and asked the department chair to indicate which structure best describes his/her department. The brief survey included in this first mailing provided the name of the department, the type of school, and the type of structure used to teach introductory statistics (non-calculus based, often terminal, introductory applied statistics course). Department chairs were also asked to indicate a person or people to whom the Fall Survey should be sent along with their mail and e-mail addresses.

Follow-up phone calls were made to all non-respondents and in some cases the survey was given orally on the phone. In other cases, copies of the survey were faxed to departments to complete and return. The numbers of departments surveyed, listed according to highest degree offered, as well as survey response rates are shown in Table 1. In general, the highest response rates (70% or higher) were from mathematics and statistics departments offering graduate degrees and the lowest response rates were for mathematics departments in two-year or four-year colleges. Fifty-seven percent of the mathematics departments responded to the survey and 76% of the statistics departments responded to the survey.

The data gathered on the types of introductory statistics course offerings are shown in Table 2. The most typical structure for two and four-year college mathematics departments, as well as for mathematics departments offering the MS as their highest degree, was one, common, introductory course. Forty-nine percent of all the departments surveyed selected this category on the preliminary survey. Thirty-five percent of the departments surveyed indicated that other departments at their institution also teach statistics, and 14% responded that they do not offer a non-calculus based introductory statistics course. Eighteen percent of the departments offer multiple, introductory courses. Most of the departments in this category were from Ph.D. granting statistics departments.

Phase 1: Fall Survey on teaching introductory statistics in mathematics and statistics departments

Using the names and email addresses of faculty identified in the preliminary survey, email messages were sent to statistics instructors in mathematics and statistics departments. Again, the purpose of the survey was described and faculty were invited to participate. The letter emphasized that this study was focused only on the non-calculus based, often terminal, undergraduate level, introductory statistics course. Faculty were offered the option of completing the fall survey via e-mail, a web survey, or a hard copy sent in the mail, and were asked to indicate whether they would like a copy of the final report.

Two different versions of surveys were used. Form A was sent to faculty who taught a common, departmental introductory course (those who selected response “a” in the preliminary survey). Form B was sent to faculty teaching one of several versions of introductory statistics courses (responses “b, c, e or g” on the preliminary survey). The difference between the two surveys was that Form A asked faculty to describe their common, departmental course and the impact of the reform on this course, while Form B asked faculty to describe the particular course they teach and the impact of the reform on themselves. A regular mailing of a cover letter and paper copy of the appropriate survey was mailed to faculty for whom no email addresses had been given, or for whom email addresses appeared to be incorrect. E-mail reminders were sent about three weeks after the initial mailings to increase the response rate.

Phase 2: Spring Surveys

A list of departments (other than mathematics and statistics) that teach introductory statistics courses was generated using information gathered in the preliminary survey. Email messages were sent to faculty in these departments, representing many of the same colleges that participated in the Fall survey. However, several of the individuals contacted stated that their course was either calculus based (e.g., economics), was not an introductory or undergraduate course, or was otherwise not appropriate for inclusion in this study. A total of 31 people completed the Spring Survey, representing six different disciplines (sociology, psychology, business, anthropology, economics and biology).

The number of Fall and Spring surveys received broken down by department and highest degree offered is displayed in Table 3. A total of 216 Fall Surveys were returned: 89 Form A and 125 Form B, 147 from mathematics departments and 65 from statistics departments. A comparison of departments for faculty who responded to the different forms of the survey indicates that respondents using Form A (one common introductory course) are more likely to represent mathematics departments in two-year colleges or mathematics departments in smaller colleges that do not offer graduate degrees in mathematics. Respondents using Form B (multiple introductory courses) are more likely to represent statistics departments that offer graduate degrees in statistics.

Analysis of the Survey Data

The “Survey of Introductory College Statistics Courses” was divided into four sections: information about the introductory statistics course, changes in the introductory course, faculty reactions to statistics education reform efforts, and information about the department. After first analyzing the surveys separately for respondents using each form, it was decided to group the surveys together across forms but to break down responses according to type of department.

The next analyses compared responses of mathematics departments to statistics departments, to “other” departments. However, due to the variability of responses within the Math Departments, it was decided to break down responses for math departments into three categories: 2-year colleges or those not offering degrees, colleges offering four year degrees but not graduate degrees, and those departments offering graduate degrees in mathematics. Because the number of respondents from “other” departments was so small, these data were kept together in one group rather than breaking down their responses into the different disciplines. Therefore, the final set of analyses compared five groups: Math 2YEAR (n=56), Math 4YEAR (n=57), Math 4YEAR+ (n=34), STAT (n=65) and OTHER (n=31). It is important to note that these numbers may not necessarily represent all the statistics courses taught in these departments and that results may be biased as teachers supportive of the reform recommendations could have been more likely to respond.

Despite these cautions, it is still possible to describe the teaching of statistics in the courses of those instructors who completed the survey.

For these courses, the following results were observed:

1. The number of sections of introductory statistics courses is greater in STAT, 4YEAR+, and 2 YEAR colleges. These types of schools also tend to have larger class sizes. Students taking courses in STAT departments are most likely to have textbooks written by Moore, Moore and McCabe, McClave and Dietrich, or Freedman et al. Students in 4YEAR courses are likely to have textbooks by Moore (or Moore and McCabe), or Bluman. Students in 2YEAR courses are most likely to have Triola as their text, followed by Moore, and Weiss. Data were not tabulated for students in OTHER courses as there were no consistent trends across the courses. (See Table 4)
    
2. Most students in the courses surveyed are required to use some type of technology, although students in 2YEAR courses are more likely to use graphing calculators (for computations using small data sets) and about one-third of the instructors in OTHER courses require students to learn a spreadsheet such as EXCEL. About one-half of the faculty surveyed have students use a statistical software program, typically Minitab, although SPSS is often used in the OTHER courses. More in-class demos are used in courses other than 4YEAR+. More out of class assignments are used in STAT, 4Year and OTHER courses. Most instructors report using computer printouts of statistical analyses in their classes. For those not using technology, reasons given for 2 Year instructors were more likely to be because of the costs, for 4Year, 4 YEAR+ and STAT, the data sets don’t require computing. OTHER faculty reported that students need practice in computing by hand. Web resources such as data sets, applets, and discussion groups, are used more often by 4YEAR, STAT, and OTHER instructors. (See Tables 5-9)
    
3. The most frequent teaching method used is the lecture, although most instructors incorporate some type of demonstration or experiment, discussions of statistics in the media, or case studies. The main users of videos (such as the Against All Odds series) are STAT instructors. Small group activities and student presentations are used more often in 2YEAR and 4YEAR courses, and writing to learn activities are used more in 4Year and OTHER departments (See Table 10).
    
4. Exams, homework, and quizzes are the most frequently used assessment tools, although some teachers use team projects, lab activities, and critiques of articles in this role. Projects and take-home exams are used more often in courses outside of statistics departments. 2YEAR courses use the widest variety of assessment techniques. OTHER courses use more multiple-choice exams, and fewer quizzes, compared to STAT, 2YEAR, and MATH instructors. 4YEAR teachers use more out of class assignments. Minute papers are more likely to be used for feedback to the instructor in 2YEAR, STAT and OTHER courses. (See Table 11and 12).
    
5. Courses are often being revised. More than two-thirds of the faculty surveyed reported making moderate to major revisions in their course over the past few years. The most common changes include the increased use of technology (67-81%, across the five groups), followed by teaching methods (50-66%), course content (43-70%), and assessment (24-34%). For most instructors, these changes are often due to the increased availability of technology and software, by their own dissatisfaction with the course, and to a lesser extent, due to students’ dissatisfaction with the course. More STAT instructors reported being influenced by suggestions from influential colleagues in their institution or elsewhere. More math instructors reported that they were influenced by recommendations from statistics education articles or presentations. (See Tables 13 and 14).
    
6. Instructors’ reactions to changes made in their courses appear to be mostly positive, despite the increased demands on their time that these changes require. Most report that their students appear to be enjoying the course more (55-76%), work harder or the same as before (but not less), learn more content, and learn somewhat different content. Most faculty enjoy teaching more, share ideas more, and need more time for preparing for their classes. (See Tables 15 and 16).
    
7. The majority of colleagues of the instructors surveyed are aware of reform efforts and have made changes, but many faculty report colleagues who are not in favor of reform recommendations, especially those in 4YEAR departments, and may or may not be supportive of reform recommendations. However, many of the instructors report increased involvement in statistical education activities. STAT faculty reported more seminars and guest speakers on teaching statistics, and more sharing of materials on educational reform, and have participated in faculty development opportunities. 2YEAR instructors are more likely to enroll in mini courses and participate in other faculty development efforts to improve their teaching. (See Tables 16 - 18).
    
8. A large percentage of the faculty surveyed (87-92 %) anticipate more changes to be made in the use of technology, and a majority also anticipate changes in teaching methods (61-67%). Fewer respondents project changes in course content (44-59%) or assessment (23-47%). (See Table 19).

The results of this survey suggest that major changes are being made in the introductory course, that the primary area of change is in the use of technology, and that the results of course revisions generally appear to be positive, although they require more time of the course instructor. Results were surprisingly similar across departments, with the main differences found in the increased use of graphing calculators, active learning and alternative assessment methods in courses and the reasons cited for why changes were made. The results are also consistent in reporting that more changes will be made, particularly as more technological resources become available.

While it is difficult to compare the content covered in the courses taught by the instructors surveyed, the textbooks used in these courses give an indication of the extent to which course content is more traditional or more in alignment with reform recommendations. The textbooks by Moore fall into the “reform” category and are the most frequently used books in introductory courses offered in statistics departments and in mathematics departments offering four year or advanced degrees. However, the favored textbook in mathematics departments in two-year colleges is Triola’s, which is considered to be a more traditional text.

Phase 3: Case Study of Statistics Instructors

To better understand the process of changing one’s course, and to provide a more detailed picture of what some “reformed” courses look like, the last phase of this project was a case study of a select group of statistics instructors, representing the different types of departments and courses. A small group of teachers (n=14) were interviewed who appear to be teaching innovative courses incorporating reform recommendations. Interviews were conducted either by e-mail or by telephone. Instructors were asked to describe the key features of their introductory course, how it differs from a “traditional” course, the process that led them to develop their course, what types of support they received, and how the course will continue to be revised in the future.

The results reveal surprising differences from course to course and illustrate the complexities of teaching in different institutions and departments. Although all instructors were implementing some reform recommendations, the nature and extent of the implementation varied quite a bit, sometimes due to available resources, sometimes due to the characteristics of students at a particular institution, and often due to the instructor’s experience and beliefs about teaching.

When asked how their course differs from a traditional course, the responses included:

• I teach statistics as a language course, and try to help the students develop literacy about statistics.
• I have students keep journals of both statistical problems and reactions to the course.
• There is no memorization required of students. On exams, I give credit for effort and explanation.
• I use a mastery exam (scored but not graded), which students must past, like a drivers’ test, before they are allowed to carry out a real statistical investigation.
• I co-teach the course with someone from a different discipline, and we often have arguments during class.
• I use lots of pairs and group work.
• I emphasize data production and simulation.
• Students have many opportunities for self-assessment.
• I create an interactive learning environment.
• I use two types of technology tools in my class; Minitab for Homework and projects, Fathom for illustrating and developing concepts.
• I use the PACE model to create a highly interactive learning-centered classroom. PACE stands for Projects, Activities, Cooperative learning in a Computer-based classroom environments, and reinforcement through Exercises.

Despite the differences listed above, there was also a common theme among many instructors who stated that they focus more on concepts and big ideas and on data analysis and interpretation and less on computation, formulas, and theory.

The process that led these instructors to their current course often included conversations with other statistics educators, reading articles in the Journal of Statistics Education or listening to presentations at professional meetings, and trial and error testing of new techniques. Challenges faced along the way included lower teaching evaluations due to problems that arise when trying new techniques for the first time, the lack of rewards for effort applied to teaching (as opposed to research), student resistance to changing from passive to active learners (where more is demanded from them), and colleagues who want to see introductory statistics courses with more math, rigor, and probability.

The interviews revealed that these instructors have spent a great deal of effort thinking about their courses, and have dedicated huge amounts of time to improving and revising their courses. Although generally pleased with the results, most shared ideas they have for how they will continue to make changes and indicated that their courses are still being developed. Most feel that they are “moving in the right direction” but still “have a ways to go.” Some report that each time they teach it’s a different course. Others commented that the first time they taught a “reform course” was difficult but that they felt things would go better the next time.

One instructor commented on the changing population of students who work more hours at fulltime or part time jobs, do not read newspapers, and have less interest and motivation. In order to find topics that interest her students she reports being “pretty much down to weather and cell phones and fast food.” Another remarked that “students struggled in the course but many learned a lot and were able to retain a fair bit. Often they didn’t appreciate that they were learning more until they saw how much other students struggled in later courses. Students often began to appreciate the prevalence of statistics in everyday life, and how much more cautious we should be using statistical statements and interpretations”. Some instructors have been pleased to see much better quality in student projects that are well written and use appropriate graphs and analyses. Others note increases in student satisfaction and attitudes about statistics. One instructor commented that “a large majority of my students see the course as a positive experience.”

A number of instructors indicated that they have been able to devote their time and effort to teaching because of having tenure and academic freedom. Some have enjoyed freedom to experiment with their course because no one in their department knows or cares about how they are teaching the courses (one instructor referred to this a ‘benign neglect’). A few instructors have appreciated the support of a department chair or colleagues or have received internal or external funding to support their efforts. A consistent result is that most of the faculty studied cite colleagues from outside of their institution as their main source of teaching support, particularly those they see at their professional meetings. One instructor commented: “I’ve received absolutely no support on campus. Although we teach many sections of introductory statistics at my university, the instructors never get together to share ideas or discuss problems. Consequently we don’t know what others do in the course.”

Summary and Recommendations

The results of this study suggest that many statistics instructors are aligning their courses with reform recommendations regarding technology, and to some extent, with teaching methods and assessment. Only about one-fifth of the instructors surveyed appear to be teaching a traditional statistics course, relying primarily on the lecture method, not incorporating technology, and using non-reform textbooks. Most instructors indicated that the type of content students are learning is different than in past courses, most students taking the introductory course are required to use some type of technology, which is typically a statistical software package. In-class exams, homework assignments, and quizzes are the most frequently used assessment methods. However, many faculty use student projects, in class group activities or labs, out –of-class group assignments, and critiques of new articles in addition to the traditional assessment methods.

A large percentage of respondents describe changes made in the past few years, with the most frequent change being in the use of technology, followed by teaching methods and course content. Reform efforts appear to be affecting many introductory statistic sources, along with the increased availability of technology resources. Most faculty reported positive outcomes regarding changes made: more student satisfaction, more student learning, increased faculty enjoyment, and more sharing of ideas and methods with colleagues. However, most faculty reporting changes also cited the increase in time required to prepare for class or to grade student materials.

The ways that individual faculty have implemented reform recommendations have varied quite a bit, and have often been possible only due to that person’s convictions and tenured position, rather than because of departmental support. These individuals have devoted large amounts of time to studying educational resources, networking with like-minded colleagues, and preparing for class, which is usually only possible if tenure and promotion have already been already been awarded.

Despite the positive findings of reform recommendations being implemented, and instructors’ perceptions of positive outcomes, this study suggests the need for some high quality assessments to use to determine how well the “new” courses are preparing students to think, reason, and communicate, using statistics. An examination of assessment results might indicate that more changes are needed beyond the increased use of technology to achieved desired course outcomes. These assessments would also allow for comparison of the effectiveness of different activities and materials in helping students develop statistical thinking.

More opportunities should be offered to provide support for statistics instructors. These might include workshops like the MAA STATS workshops of the last several years. Especially important are support groups like the “Isolated Statisticians” and their regional meetings, and the new MAA special Interest Group for Statistics Education (formerly the Isolated teachers of Statistics). Collegial support is often lacking at their own institutions, and faculty who are successfully teaching “reform” courses need more outlets through which to share information on what they are doing and how well it is working with students, and to provide detailed examples of student outcomes. Finally, more programs are needed to help prepare future teachers of statistics, particularly while they are in graduate school.

References

Cobb, G. (1992). Teaching Statistics, in Heeding the Call for Change, MAA Notes, 3-43.

Cobb, G. (1993), ‘Reconsidering Statistics Education: A National Science Foundation Conference’, Journal of Statistics Education 1 (1).

Garfield, J., Hogg, R., Schau, C., and Whittinghill, D. (2000). Best Practices in Introductory Statistics. Paper prepared for the ASA Undergraduate Statistics Education Initiative, Indianapolis, IN.

Hogg, R. (1992) Report of Workshop on Statistics Education, in Heeding the Call for Change, MAA Notes, 34-43.

Moore, D.S. (1997a). New pedagogy and new content: the case of statistics. International Statistical Review, 65, 123-137.

Moore, D.S. (1997b). Response. International Statistical Review, 65, 162-165.

Scheaffer, R.L. (1997). Discussion. International Statistical Review, 65, 156-158.

Appendix

Project Advisory Group

George Cobb, Mt. Holyoke College
Jon Cryer, University of Iowa
Jackie Dietz, North Carolina State University
Marilyn Mays, North Lake College
Gary McClelland, University of Colorado
J. Laurie Snell, Dartmouth College
Judy Tanur, State University of New York, Stony Brook
Ann Watkins, California State University, Northridge
Dex Whittinghill, Rowan University

NRC Committee on Applied and Theoretical Statistics




Table 1: Response rates for Preliminary Surveys

 Table 2: Data on Infrastructure of Introductory Statistics Courses
 

Dept./Degree	n	a	b	c	d	e	f	g	h
Math Ph.D.	28	8 (28%)	3	3	10	0	12	0	3
Math M.S.	32	19 (59%)	6	6	1	1	22	1	0
Math B.S.	79	48 (61%)	15	3	11	2	33	1	2
Math A.A.	79	52 (66%)	8	7	8	2	13	0	5
Math, no degree	9	6 67%)	1	0	1	0	3	0	1
Stats Ph.D.	68	14 (20%)	2	32	11	4	18	2	4
Stats MS	10	4 (40%)	1	2	2	1	6	0	1
Stats No degree	3	1 (33%)	0	2	0	0	0	0	0
Totals:	308	152 (49%)	36 (12%)	55 (18%)	44 (14%)	10 (3%)	107 (35%)	4 (1%)	16 (5%)

Code:
     a one, common, departmental course
     b one course, but different versions
     c multiple introductory courses
     d no introductory course
     e one common course and also tailored courses
     f courses are also taught in other departments
     g an interdisciplinary course is taught
     h other

Table 3: Numbers of Surveys received
 

Department /degree offered	#Sent	Survey Form			Response Rate
		A	B	total
Math Ph.D.		7	9	16
Math M.S.		7	11	18
Math B.S.		31	26	57
Math A.A.		34	19	53
Math, no degree		1	2	3
Total Math depts:	207	80	67	147	71%
Stats Ph.D.		8	51	59
Stats MS		1	5	6
Stats BS		0	0	0
Stats No degree		0	0	0
Total Stats depts:	81	9	56	65	89%
No information on school			4	4
Other departments	78		31	31	40%
Total:	366	89	158	247	67%

Form A: for faculty in departments that offer one, departmental course
Form B: for faculty in departments that offer multiple introductory statistics courses.

Table 4: Information about the statistics courses
 

		Math 2 Yr (56)	4 Yr (57)	4 Yr+ (34)	Stat   (65)	Other (31)
Number of sections of introductory statistics taught per year in the department
	Range	1-65	1-20	1-90	1-90	1-44
	Median	6	6	7	9	5
Average number of students per section
	Range	5-100	2-50	3-175	10-380	8-300
	Median	24	25	30	33	36
Textbook(s) used this year in the introductory course: (more than one could be listed)
		%	%	%	%	NA
Moore/Moore & McCabe		12	16	32	17
Triola		34	2	6	8
Weiss		12	0	0	4
Bluman		9	13	3	4
Johnson & Kuby1		0	5	6	2
McClave, Dietrich		4	5	6	10
Freedman, PPA		0	5	3	10
Devore & Peck		0	2	0	4
Other		29	24	29	25

Table 5: Technology use required of students
 

Students are required to:		Math			Stat	Other
		2 Yr (56) %	4 Yr (53) %	4 Yr+ (33) %	(65) %	(31) %
Use a graphing calculator		48	13	18	3	16
Use a computer spreadsheet program (e.g., Excel)		19	13	9	13	30
Use a statistical software program (e.g., Minitab)		46	61	42	56	61
Software packages available for teaching introductory statistics
	Minitab	41	56	61	61	19
	Excel	21	23	0	14	25
	SPSS	2	21	29	20	45
	SAS	0	2	9	9	0
	JMP	4	5	23	4	10
	Splus	0	0	9	4	3
	Other	41	10	6	20	45
	None:	20	16	29	19	3

Table 6: Computer use
 

Computers are used for:	Number of times used per course
	0-1 %	2 %	3 %	4 %
In-class demonstrations of concepts (e.g., how sampling distributions behave)
Math (2 Yr)	31	43	8	18
Math (4 Yr)	35	27	16	22
Math (4 Yr+)	36	36	21	6
Stat	32	35	10	24
Other	39	29	6	26
In-class demonstrations of how to use statistical software to analyze data
Math (2 Yr)	27	43	10	20
Math (4 Yr)	40	18	5	36
Math (4 Yr+)	44	31	9	16
Stat	41	22	14	23
Other	23	13	17	47
Out of class homework assignments
Math (2 Yr)	35	21	13	31
Math (4 Yr)	34	11	13	42
Math (4 Yr+)	45	18	3	33
Stat	28	11	14	48
Other	23	7	27	43
In-class lab activities
Math (2 Yr)	46	21	10	23
Math (4 Yr)	50	17	7	26
Math (4 Yr+)	62	22	6	9
Stat	55	18	5	23
Other	45	21	7	28
One or more out -of-class student labs
Math (2 Yr)	54	20	10	16
Math (4 Yr)	64	13	9	13
Math (4 Yr+)	71	19	3	6
Stat	59	11	8	21
Other	57	11	7	25
One or more out-of-class student projects
Math (2 Yr)	37	53	4	6
Math (4 Yr)	43	40	13	4
Math (4 Yr+)	56	38	3	3
Stat	60	27	11	2
Other	39	29	14	18
Printouts of analyses to be used in class
Math (2 Yr)	32	32	19	17
Math (4 Yr)	41	20	17	22
Math (4 Yr+)	44	31	16	9
Stat	30	13	21	36
Other	40	10	17	33

Table 7: Graphing calculator use
 

Graphing Calculators are used for:	Number of times used per course
	0-1 %		2 %	3 %	4 %
Basic computations on small data sets
Math (2 Yr)	36		10	2	52
Math (4 Yr)	52		2	2	44
Math (4 Yr+)	68		4	4	24
Stat	84		4	0	11
Other	63		5	0	32
Computations for sets of data that are too large to do by hand
Math (2 Yr)	45		12	6	37
Math (4 Yr)	63		11	4	21
Math (4 Yr+)	71		7	0	22
Stat	91		0	2	7
Other	74		5	5	16
Performing simulations
Math (2 Yr)	55		21	11	13
Math (4 Yr)	73		15	8	4
Math (4 Yr+)	82		15	4	0
Stat	98		2	0	0
Other	95		0	5	0
Making transformations of lists
Math (2 Yr)	68		17	6	9
Math (4 Yr)	79		19	2	0
Math (4 Yr+)	85		15	0	0
Stat	96		2	0	2
Other	95		5	0	0
Making graphical displays
Math (2 Yr)	40		14	12	34
Math (4 Yr)	73		4	13	10
Math (4 Yr+)	74		15	0	11
Stat	93		2	4	0
Other	95		5	0	0
Constructing confidence intervals and doing hypothesis tests without going through all the computational steps
Math (2 Yr)	57		6	16	20
Math (4 Yr)	74		9	6	11
Math (4 Yr+)	74		4	4	19
Stat	91		7	0	2
Other	95		0	0	5
How many students use them in class and/or on homework?
	Math			Stat	Other
	2 Yr %	4 Yr %	4 Yr+ %
none	8	8	24	19	26
some	32	38	40	53	26
many	11	18	8	12	5
most	15	18	16	12	11
all	34	14	12	5	32

Table 8: Use of web resources
 

Web resources are used in class or to produce materials to bring to class:	Number of times used per course
	0-1 %	2 %	3 %	4 %
Data sets to analyze in class
Math (2 Yr)	55	31	12	2
Math (4 Yr)	60	23	8	9
Math (4 Yr+)	41	41	14	3
Stat	44	35	9	12
Other	41	19	19	22
Data sets for students to analyze for projects
Math (2 Yr)	65	29	6	0
Math (4 Yr)	68	21	2	10
Math (4 Yr+)	59	31	10	0
Stat	61	34	5	0
Other	48	24	8	20
News articles to discuss that contain statistical concepts
Math (2 Yr)	41	45	12	2
Math (4 Yr)	49	32	8	11
Math (4 Yr+)	41	45	14	0
Stat	36	36	19	10
Other	48	33	11	7
Applets to illustrate or test concepts
Math (2 Yr)	94	4	2	0
Math (4 Yr)	87	11	0	2
Math (4 Yr+)	82	14	4	0
Stat	70	18	5	7
Other	72	16	8	4
Web-based discussions
Math (2 Yr)	90	8	0	2
Math (4 Yr)	96	4	0	0
Math (4 Yr+)	96	4	0	0
Stat	82	13	4	2
Other	72	16	8	4

Table 9: Reasons for not using technology
 

Main reasons for not using technology:	Math			Stat	Other
	2 Yr	4 Yr	4 Yr+
Number or respondents	14	10	9	11	6
	%	%	%	%	%
They are not readily available.	29	0	0	36	0
They are too expensive to require for students.	57	38	44	17	17
The students need practice computing by hand.	21	44	22	18	83
The data sets used in this course don't require much computing power.	29	60	56	64	50
I have not received adequate training in using them.	7	13	22	18	0

Table 10: Teaching Methods
 

Use of different teaching methods:	Frequently Used %	Sometimes Used %	Not used at all %
Lectures
Math (2 Yr)	89	7	4
Math (4 Yr)	81	19	0
Math (4 Yr+)	97	3	0
Stat	97	0	3
Other	100	0	0
Demonstrations/experiments
Math (2 Yr)	34	59	7
Math (4 Yr)	19	75	5
Math (4 Yr+)	35	50	15
Stat	39	48	13
Other	48	38	14
Case studies
Math (2 Yr)	18	46	36
Math (4 Yr)	4	61	36
Math (4 Yr+)	12	56	32
Stat	18	57	25
Other	14	52	34
Videos
Math (2 Yr)	9	39	52
Math (4 Yr)	4	13	84
Math (4 Yr+)	0	32	68
Stat	11	40	49
Other	3	28	69
Small group activities
Math (2 Yr)	25	56	18
Math (4 Yr)	25	51	25
Math (4 Yr+)	21	35	44
Stat	22	34	44
Other	31	31	38
Oral presentations by students
Math (2 Yr)	0	38	63
Math (4 Yr)	0	36	64
Math (4 Yr+)	0	15	85
Stat	2	19	80
Other	0	23	77
Writing-to-learn activities
Math (2 Yr)	9	47	43
Math (4 Yr)	15	49	36
Math (4 Yr+)	12	41	47
Stat	11	30	59
Other	20	43	37
Discussions of statistics in the media
Math (2 Yr)	23	63	14
Math (4 Yr)	19	67	14
Math (4 Yr+)	15	38	18
Stat	22	66	13
Other	10	77	13

Table 11: Assessment Methods used to provide grades or feedback to students
 

Assessment methods used:	Frequently Used %	Sometimes Used %	Not used at all %
Individual student projects
Math (2 Yr)	30	48	21
Math (4 Yr)	20	50	30
Math (4 Yr+)	15	38	47
Stat	11	34	55
Other	36	29	36
Homework assignments
Math (2 Yr)	65	22	13
Math (4 Yr)	63	25	13
Math (4 Yr+)	79	6	15
Stat	82	8	11
Other	90	0	10
Group projects
Math (2 Yr)	19	41	41
Math (4 Yr)	16	42	42
Math (4 Yr+)	12	35	53
Stat	14	22	65
Other	23	27	50
Posters or presentations
Math (2 Yr)	4	26	70
Math (4 Yr)	5	18	77
Math (4 Yr+)	0	3	97
Stat	0	12	88
Other	3	10	87
In-class group activities/labs
Math (2 Yr)	27	55	18
Math (4 Yr)	32	41	27
Math (4 Yr+)	26	38	35
Stat	28	28	44
Other	28	34	38
Out of class group assignments
Math (2 Yr)	13	37	50
Math (4 Yr)	7	48	45
Math (4 Yr+)	6	29	65
Stat	13	31	56
Other	13	27	60
Students working at the board
Math (2 Yr)	7	29	63
Math (4 Yr)	5	23	71
Math (4 Yr+)	0	18	82
Stat	0	11	89
Other	0	13	87
Portfolios of students’ work
Math (2 Yr)	0	15	85
Math (4 Yr)	5	13	82
Math (4 Yr+)	3	12	85
Stat	0	8	92
Other	3	7	90
Multiple-choice exams
Math (2 Yr)	4	40	57
Math (4 Yr)	9	16	75
Math (4 Yr+)	12	29	59
Stat	29	31	40
Other	52	10	38
In-class exams
Math (2 Yr)	87	11	2
Math (4 Yr)	84	14	2
Math (4 Yr+)	85	15	0
Stat	83	9	8
Other	87	3	10
Take-home exams
Math (2 Yr)	7	31	62
Math (4 Yr)	7	30	63
Math (4 Yr+)	0	30	70
Stat	0	14	86
Other	17	17	67
Quizzes
Math (2 Yr)	45	36	20
Math (4 Yr)	32	41	27
Math (4 Yr+)	38	32	29
Stat	40	32	28
Other	40	17	43
Critiques of news articles
Math (2 Yr)	6	53	42
Math (4 Yr)	7	33	60
Math (4 Yr+)	0	32	68
Stat	2	37	61
Other	7	39	54

Table 12: Assessment methods used to provide feedback to the instructor
 

Assessment methods used:	Frequently Used %	Sometimes Used %	Not used at all %
Minute papers
Math (2 Yr)	10	25	65
Math (4 Yr)	8	18	74
Math (4 Yr+)	0	19	81
Stat	4	31	65
Other	0	36	63
End-of-course evaluations of instruction
Math (2 Yr)	65	25	9
Math (4 Yr)	82	14	4
Math (4 Yr+)	91	9	0
Stat	95	3	2
Other	97	3	0

Table 13: Changes in the Introductory Course
 

To what extent their teaching of introductory statistics has changed the past few years:	Math			Stat %	Other %
	2 Yr %	4 Yr %	4 Yr+ %
no appreciable change	6	7	9	11	17
minor changes 22 23 26 17 13
moderate changes 44 44 35 34 43
major revisions 28 26 29 38 23
Areas in which substantial changes have been made:
in teaching methods 66 60 50 65 57
in course content 49 52 70 60 43
in use of technology 81 67 73 73 81
in assessment methods 34 33 24 25 30

Table 14: Reasons for changes made
 

Reasons for changes made:	Major reason %	Minor reason %	Not a reason %
Student dissatisfaction with the course
Math (2 Yr)	15	47	38
Math (4 Yr)	7	59	34
Math (4 Yr+)	8	63	29
Stat	4	67	28
Other	6	94	0
Instructor’s dissatisfaction with the course
Math (2 Yr)	32	47	21
Math (4 Yr)	68	25	7
Math (4 Yr+)	56	37	7
Stat	54	31	1
Other	85	15	0
Requests from other departments
Math (2 Yr)	10	57	33
Math (4 Yr)	7	43	50
Math (4 Yr+)	17	58	25
Stat	12	43	45
Other	8	92	0
Recommendations in statistics education articles or presentations on changing the introductory course
Math (2 Yr)	51	29	20
Math (4 Yr)	43	38	19
Math (4 Yr+)	46	42	12
Stat	29	49	22
Other	29	71	0
An influential colleague or colleagues
Math (2 Yr)	28	39	33
Math (4 Yr)	34	37	29
Math (4 Yr+)	33	54	13
Stat	44	22	33
Other	8	92	0
Low student success rate
Math (2 Yr)	19	55	26
Math (4 Yr)	15	40	45
Math (4 Yr+)	5	67	29
Stat	12	53	35
Other	21	79	0
Increased availability of computers and software
Math (2 Yr)	70	15	15
Math (4 Yr)	58	26	16
Math (4 Yr+)	58	33	8
Stat	52	31	17
Other	70	30	0

Table 15:Results of changes made on students
 

Results of changes made: Note: Respondents to the following section consisted of: Math 2 Yr n=38, Math 4 Yr n=38, Math 4 Yr+ n=22, Stat n=48, and Other n=19
Compared to courses taught in the past (before changes were made):			more %	same %	less %
Students appear to enjoy the statistics courses:
Math (2 Yr)			76	21	3
Math (4 Yr)			76	24	0
Math (4 Yr+)			55	45	0
Stat			67	31	2
Other			63	32	5
Students appear to be working/studying:
Math (2 Yr)			45	50	5
Math (4 Yr)			37	63	0
Math (4 Yr+)			36	59	5
Stat			33	62	4
Other			26	68	5
The amount of content students appear to be learning is:
Math (2 Yr)			58	34	8
Math (4 Yr)			53	34	13
Math (4 Yr+)			52	48	0
Stat			44	47	9
Other			37	58	5
The type of content and skills students are learning is:	Math			Stat %	Other %
	2 Yr %	4 Yr %	4 Yr+ %
about the same	29	22	14	28	21
somewhat different	58	62	77	57	58
very different	13	16	9	15	21

Table 16: Results of changes made on faculty
 

Results of changes made:	Math			Stat %	Other %
	2 Yr %	4 Yr %	4 Yr+ %
The changes made have had no real impact	11	2	6	3	3
Compared to courses taught in the past (before changes were made):	more %		same %		less %
Faculty enjoy teaching statistics
Math (2 Yr)	75		23		3
Math (4 Yr)	69		31		0
Math (4 Yr+)	64		32		4
Stat	68		28		4
Other	70		30		0
Faculty are sharing ideas and methods with colleagues:
Math (2 Yr)	67		33		0
Math (4 Yr)	49		51		0
Math (4 Yr+)	68		32		0
Stat	67		33		0
Other	60		40		0
The time required to prepare for teaching is
Math (2 Yr)	79		18		3
Math (4 Yr)	62		36		3
Math (4 Yr+)	65		23		12
Stat	70		28		2
Other	67		24		5

Table 17: Faculty response to reform efforts
 

Response to statistics reform: Note: Responses in the following section were from people who indicated that they were not the only one teaching statistics in their department: Math 2Yr n=33, Math 4 Yr n=36, Math 4 Yr+ n=30, Stat n=50, and Other n=18
Are aware of the reform movement but are not in favor it.	none %	some %	many %	most %
Math (2 Yr)	45	45	6	3
Math (4 Yr)	6	28	53	13
Math (4 Yr+)	34	52	10	3
Stat	40	48	6	6
Other	29	57	14	0
Are aware and have made some changes
Math (2 Yr)	9	55	12	24
Math (4 Yr)	8	56	14	19
Math (4 Yr+)	13	50	27	10
Stat	8	56	16	20
Other	17	67	11	6
Are aware and have made major changes
Math (2 Yr)	22	50	19	9
Math (4 Yr)	37	40	6	14
Math (4 Yr+)	28	62	10	0
Stat	16	67	7	11
Other	17	67	17	0

Table 18: Faculty Involvement in Reform Activities
 

Participation in:		none %	some %	many %	most %
Workshops such as STATS or CHANCE
Math (2 Yr)		51	44	0	5
Math (4 Yr)		63	38	0	0
Math (4 Yr+)		69	28	3	0
Stat		52	44	2	2
Other		71	19	5	0
Mini-courses focused on teaching the introductory course
Math (2 Yr)		45	53	3	0
Math (4 Yr)		55	45	0	0
Math (4 Yr+)		68	25	7	0
Stat		67	31	0	2
Other		65	30	5	0
Other faculty development opportunities to improve teaching
Math (2 Yr)		24	42	13	21
Math (4 Yr)		30	63	3	5
Math (4 Yr+)		34	55	10	0
Stat		27	62	5	5
Other		38	43	14	5
Departmental activities:	Math			Stat %	Other %
	2 Yr %	4 Yr %	4 Yr+ %
Offered seminars on teaching statistics	5	7	6	33	4
Brought in guest speakers on this topic	8	14	16	38	4
Distributed materials on the reform	38	21	26	43	39

Table 19: Future Plans
 

Changes are anticipated over the next few years:	Math			Stat %	Other %
	2 Yr %	4 Yr %	4 Yr+ %
Changes in the use of the technology	90	77	90	85	92
Changes in teaching methods	60	67	61	64	65
Changes in the course content	44	56	55	59	56
Changes in assessment methods	47	45	42	41	23

PRELIMINARY SURVEY OF INTRODUCTORY STATISTICS COURSES

This survey concerns the introductory statistics courses taught on your campus. Introductory statistics is a class taught to various majors and typically does not use calculus. Please do not include information on the mathematical statistics courses typically taught to upper division majors. You are asked to provide a name or names of instructors who will be sent the Survey on Introductory Statistics next fall. If there are many people who teach the course, please list up to three names and e-mail addresses, selecting those who are most likely to participate in the survey and who are most knowledgeable about the teaching of this course.

Your name (if different than on the mailing label)
Your phone:     E-mail address:     Fax:

The introductory, non-calculus based, survey course is taught in the following structure:

__a. One introductory, undergraduate statistics course, a departmental course, using one common textbook.
Name and e-mail address of individual to contact regarding Fall Survey:

__b. One introductory course but taught differently and/or using different texts, depending on the instructor.
Names and e-mail addresses of up to three key people to be contacted regarding the Fall Survey:

__c. Multiple introductory, undergraduate courses designed for different audiences, taught by this department.
Names and e-mail addresses of up to three key people to be contacted regarding the Fall Survey:

Please turn over

__d. An introductory, undergraduate course IS NOT taught by this department but is taught in other departments (please list departments:_______________________)

__e. There's a common introductory course and also individual tailored courses in this department.
Name and e-mail address of person to be contacted regarding common course for the Fall Survey:

Names and e-mail addresses of up to three key people to be contacted regarding individual, tailored courses, for the Fall Survey:


Please also check any of the following that describe your introductory course(s):

__f. One or more introductory, undergraduate courses are taught by this department AND by other departments (e.g., psychology, economics, etc.).

__g. An introductory, undergraduate course is taught as an interdisciplinary courses with another department:_________.
Name and e-mail address of individual to contact regarding Fall Survey:

__h. Other (please describe and list names and e-mail addresses of up to three key people to be contacted regarding the Fall Survey).


Please return in the self-addressed stamped envelope to:
Joan Garfield, Dept. of Educational Psychology
178 Education Sciences Building
56 East River Road
Minneapolis, MN 55455
FAX : 612-624-8241
e-mail : jbg@umn.edu

Survey [.pdf]
1998 Survey of Introductory College Statistics Courses