Podcast: Guido Schwerdt talks with Ed Next about his new study.
An unabridged version of this article is available here.
------------
In recent years, a consensus has emerged among researchers that teacher quality matters enormously for student performance. Students taught by more-effective teachers learn substantially more over the course of the year than students taught by less-effective teachers. Yet little is known about what makes for a more-effective teacher.
Most research on teacher effectiveness has focused on teacher attributes, finding that readily measurable characteristics such as experience, certification, and graduate degrees generally have little impact on student achievement. Relatively few rigorous studies look inside the classroom to see what kinds of teaching styles are the most effective. We tackle this underexplored area by investigating the relative effects of two teacher practices--lecture-style presentations and in-class problem solving--on the achievement of middle-school students in math and science.
Ever since John Dewey explored hands-on learning at the University of Chicago Laboratory School more than a century ago, lecture-style presentations have been criticized as old-fashioned and ineffective. It is said, for example, that lectures presume that all students learn at the same pace and fail to provide instructors with feedback about which aspects of a lesson students have mastered. Students' attention may wander during lectures, and they may more easily forget information they encountered in this passive manner. Lectures also emphasize learning by listening, which may disadvantage students who favor other learning styles.
Alternative instructional practices based on active and problem-oriented learning presumably do not suffer from these disadvantages. But they may have their own shortcomings. Learning by problem-solving may be less efficient, as discovery and problem-solving often take more time than mastering information received from an authority figure. And incorrect or misleading information may be conveyed in conversations among students in middle schools.
Nonetheless, a number of small-scale studies have identified positive impacts of interactive teaching styles on student learning. As a consequence, prominent organizations such as the National Research Council and the National Council of Teachers of Mathematics, since at least 1980, have called for teachers to engage students in constructing their own new knowledge through more hands-on learning and group work. By the mid-1990s, in a study for the National Institute for Science Education, Iris Weiss could identify "some encouraging signs. The majority of elementary, middle, and high school science and mathematics classes worked in small groups at least once a week, and roughly one in four classes did so every day. Moreover, the use of hands-on activities had increased since the mid-1980s." Even so, more than a decade later, traditional lecture and textbook methodologies continue to be a significant component of science and mathematics instruction in U.S. middle schools. A rigorous, large-scale study has yet to resolve a question that has divided pedagogical thinking for generations.
In our study, we examine whether student achievement in the United States is affected by the share of teaching time devoted to lecture-style presentations as distinct from problem-solving activities. Employing information on in-class time use provided by a nationally representative sample of U.S. teachers in the 2003 Trends in International Mathematics and Science Study (TIMSS), we estimate the impact of teaching practices on student achievement by looking at the differential effects on the same student of two different teachers, using two different teaching strategies. We find that teaching style matters for student achievement, but in the opposite direction than anticipated by conventional wisdom: an emphasis on lecture-style presentations (rather than problem-solving activities) is associated with an increase--not a decrease--in student achievement. This result implies that a shift to problem-solving instruction is more likely to adversely affect student learning than to improve it.
Data and Methodology
Our research draws on data from the 2003 Trends in International Mathematics and Science Study (TIMSS). The TIMSS data comprise information on students in two grades in a number of countries, but we utilize only information on 8th-grade students in the United States. Our sample includes 6,310 students in 205 schools with 639 teachers (303 math teachers and 355 science teachers, of which 19 teach both subjects). In addition to test scores in math and science, the TIMSS data include background information on students' home and family life as well as data on teacher characteristics, qualifications, and classroom practices. School principals provide information on school characteristics.
Most important for our purpose, teachers were asked what proportion of time in a typical week students spent on each of eight in-class activities. The overall time in class apportioned to three of these activities--listening to lecture-style presentation, working on problems with the teacher's guidance, and working on problems without guidance--likely provides a good proxy for the time in class in which students are taught new material. We divided the amount of time spent listening to lecture-style presentations by the total amount of time spent on each of these three activities to generate a single measure of how much time the teacher devoted to lecturing relative to how much time was devoted to problem-solving activities.
A change in our measure of teaching style can be interpreted as a shift from spending time on one practice to spending time on the other, holding constant the total time spent on both practices. For example, an increase of 0.1 indicates that 10 percentage points of total time devoted to teaching new material are shifted from teaching based on problem solving to giving lecture-style presentations. We combined the other teaching activities (besides lecturing and problem solving) into a separate measure of the share of total teaching time devoted to other activities and control for this measure throughout our analysis. We also control for the total number of minutes per week that the teacher reported teaching the math or science class, as more total instructional time could have an independent effect on student learning.
Although it is difficult to determine from the TIMSS data exactly how much time is spent on lecturing as distinct from problem-solving activities, it appears that teachers generally follow the advice given by progressive educators. On average, they allocate twice as much time to problem-solving activities as to direct instruction. Specifically, teachers devote about 40 percent of class time to problem-solving activities (with or without teacher guidance); during roughly 20 percent of class time, students listen to the initial presentation of material to be learned. The remainder of the class time is allocated to such tasks as class management, reviewing homework, re-teaching the material, and clarifying content (see Figure 1).
Teachers who spent more time lecturing were more likely to be male and under age 50. Interestingly, they were also less likely to have the maximum number of years of teacher training registered by the background survey or to have taken pedagogical or content knowledge classes in the prior two years (see Figure 2).
A key challenge in studying the effects of teaching practices is that teachers may adjust their methods in response to the ability or behavior of their students. If teachers tend to rely more on lectures when assigned more capable or attentive students, this would generate a positive relationship between the amount of time spent lecturing and student achievement, even in the absence of a true causal effect. Similarly, there could be unobserved differences between students whose teachers rely more and less heavily on lecturing if, for example, teachers in schools serving low-income students adopt different practices than teachers in other types of schools.
To address these concerns, we exploit the fact that the TIMSS study tested each student in both mathematics and science. This allows us to compare the math and science test scores of individual students whose teacher in one subject tended to emphasize a different teaching style than their teacher in the other subject. In other words, we ask, if a given student's math teacher spent more (or less) time lecturing than his or her science teacher, does the student perform better or worse on the math test than on the science test?
Results
Contrary to contemporary pedagogical thinking, we find that students score higher on standardized tests in the subject in which their teachers spent more time on lecture-style presentations than in the subject in which the teacher devoted more time to problem-solving activities. For both math and science, a shift of 10 percentage points of time from problem solving to lecture-style presentations (e.g., increasing the share of time spent lecturing from 20 to 30 percent) is associated with an increase in student test scores of 1 percent of a standard deviation. Another way to state the same finding is that students learn less in the classes in which their teachers spend more time on in-class problem solving.
Importantly, the strength of the relationship increases when we restrict our analysis to the roughly one-third of students in the TIMSS sample who had the exact same peers in both their math and science classes. Among this group of students, a shift of 10 percentage points of time from problem solving to lecturing is associated with an increase in test scores of almost 4 percent of a standard deviation--or between one and two months' worth of learning in a typical school year (see Figure 3). This pattern increases our confidence that the overall result does not reflect differences in the peer composition of students' math or science classes. In fact, it suggests that peer effects may actually be leading us to understate the strength of the relationship between lecturing and student learning.
Do certain types of students benefit more from lectures than others? We find suggestive evidence that the relationship between lecture-style teaching and achievement is strongest among higher-achieving and more-advantaged students. For example, the positive effect is largest for students who report having more than one bookcase in the home, a rough indicator of the quality of their home environment. There is no evidence, however, that lower-achieving students or students from less-advantaged backgrounds learn less when their teachers emphasize lectures.
These patterns are consistent with the findings of a 1997 study by Dominic Brewer and Dan Goldhaber, which found that more in-class problem solving for American 10th-grade students in math is related to lower test scores on a standardized test. Because our results are based on comparisons of the same student in two different classes, however, they are less subject to the concern that teachers adjust their practices based on the students to which they are assigned. Furthermore, the other commonly investigated teacher characteristics (e.g., gender, experience, and credentials) do not show significant effects on student achievement in our analysis. This is in line with previous findings in the literature and underscores the importance of the statistical relationship between more lecture-style teaching and student achievement.
While the richness of the TIMSS data enables us to control for an unusually large set of teacher characteristics, our results could still be biased if teachers with different effectiveness levels are more likely to choose different teaching styles. For example, if more-effective teachers tend to spend more time lecturing because they are good at it and enjoy it, then our results could show a positive effect of lecture-style presentations, even if those teachers would have been even more effective had they devoted more time on problem-solving activities. Given the pedagogical emphasis on the use of problem-solving activities, it seems unlikely that the very best teachers would be using the less-effective teaching style (the only alternative explanation for our finding).
Still, it is important to keep in mind that our results are limited to student achievement as measured by the 2003 TIMSS test scores in 8th-grade math and science in the United States. Different results might be found for different subjects, grades, or tests. Depending on the teacher, the students, the content taught, or other factors, problem-solving activities could turn out to be the more effective style. Even though lecture-style teaching seems to be a more effective method in middle-school math and science, that does not mean it would be the preferable approach to elementary-school reading.
Also, our findings are based on student performance on the TIMSS math and science exams, which are designed to measure mastery of factual knowledge of the curricula that schools expect students to learn. Other tests intended to measure problem-solving ability and the competence to apply mathematical and scientific concepts in real-world settings (such as the Programme for International Student Assessment [PISA] administered by the Organization of Economic Cooperation and Development) might yield different results. Unfortunately, we are unable to ascertain whether this might be the case, as PISA did not ask teachers about their pedagogical approach.
Finally, our information on teaching practices, which is based on in-class time use reported by teachers, does not allow us to distinguish between different implementations of teaching practices. In other words, a certain teaching technique may be very effective if implemented in the optimal way. But the strength of our approach is that it examines which teaching style turns out to be effective, on average, for teachers in general. Optimal teaching methods that cannot be executed by teachers in general may do more harm than good.
Conclusion
Given the limitations of the data, our finding that spending increased time on lecture-style teaching improves student test scores results should not be translated into a call for more lecture-style teaching in general. But the results do suggest that traditional lecture-style teaching in U.S. middle schools is less of a problem than is often believed.
Newer teaching methods might be beneficial for student achievement if implemented in the proper way, but our findings imply that simply inducing teachers to shift time in class from lecture-style presentations to problem solving without ensuring effective implementation is unlikely to raise overall student achievement in math and science. On the contrary, our results indicate that there might even be an adverse impact on student learning.
Guido Schwerdt is a postdoctoral fellow at the Program on Education Policy and Governance (PEPG) at Harvard University and a researcher at the Ifo Institute for Economic Research in Munich, Germany. Amelie C. Wuppermann is a postdoctoral researcher at the University of Mainz, Germany.
An unabridged version of this article is available here.
Copyright Education Next Summer 2011