Apr 21, 2009

Mounds of Data on Education Interventions

by Tim Ogden

It should be no surprise that programs to improve education and schools are as poorly studied as most other philanthropic efforts. We’ve lamented the poor state of journalism on the topic before. But I’ve recently come across a number of papers documenting high quality studies of some of the most popular education interventions.

First, with funding from the U.S. Department of Education (as a result of the requirements of the No Child Left Behind Act) Mathematica Policy Research has conducted a multi-year randomized controlled trial of sixteen educational software programs (covering both reading and math) aimed at elementary and middle school students. The products selected were generally those that had at least some evidence of positive impact. After the first year of use, there was no measurable improvement among students using the software—even though the study verified that teachers were trained and were using the software as intended. The study continued for another year to determine if, after some experience, teachers were able to use the software more effectively. After the second year, there was still no measurable difference in reading skills; on the math side, students in the sixth grade using the software did worse than the first year, while students in algebra classes improved on the first year. Overall, it’s no stretch to say that the educational software didn’t make much difference.

Second, Michael Kremer of Harvard University and Alaka Holla of Innovations for Poverty Action* have produced a working paper that summarizes the results of dozens of randomized controlled trials of education interventions in the developing world. The studies cover the gamut of interventions including increasing school attendance, vouchers, motivating teachers, improving classroom materials and increasing parental involvement. There is good news here: of the 26 distinct interventions they identify, 18 show statistically significant improvements in performance (though only half of these show an improvement greater than 0.2 standard deviations). The cheapest by far? Deworming of school children.

Editor’s Note: Innovations for Poverty Action is a client of Sona Partners, the sponsor of Philanthropy Action