Sep 11, 2008

Familiarity Breeds Contempt?

by Tim Ogden

Is it a symptom or a cause that journalism about American schools is so routinely poor? This spring we gave our second “Worst of Journalism” award to a Wall Street Journal story that purported to explain the success of Finland’s educational system by visiting a model school for a few hours, while ignoring or glossing over crucial data on how the system actually works.

This month the cover story of the September/October issue of Good magazine, “School Wars” by Gary Stager, joins the dishonor roll by offering unsubstantiated critiques (the author cites no studies or verifiable data in the entire article) of virtually every current experiment in American education, offers platitudinal advice such as “creat[ing] productive contexts for learning in which the needs of each child are met as their talent, interest, curiosity, and passion are amplified,” all served up with unintentional irony as he begins the article by poking fun at those attempting to reform public schools by noting, “[t]hese days, anyone who attended school is an expert in education.”

There is plenty of need and opportunity to cast a critical eye on both the current state of American schools and proposals or experiments to improve them. One of the areas of current debate and study is the value of preschool. Two well-executed articles in the Wall Street Journal lay out the opposing sides of this argument by citing specific studies and data. First, Shika Dalmia and Lisa Snell of the Reason Foundation respond to the Obama campaign proposal to expand universal preschool. They note that current evidence suggests that preschool only benefits children from economically or socially disadvantaged situations. Further there is little evidence that the gains achieved by even these children persist. Several studies have found that by fourth grade, children who attended preschool are statistically indistinguishable from those who did not. States like Georgia and Oklahoma who have implemented universal preschool are not reaping gains in terms of improved academic achievement on the National Assessment of Educational Progress. The authors argue that given the existing data, it makes little sense to invest heavily in expanding universal preschool programs. A better use of funds would likely be to improve the quality of the elementary schools that disadvantaged children attend.

A week later, the Journal, to its credit, reported on a new study from England which delves into the impact of preschool on children through age 10. The study, first published in Science, followed 3000 children from 141 preschools. It finds that high-quality preschool does have a measurable impact, particularly as it pertains to children’s math skills. That being said, age 10 is typically when the gains disappear from other studies. It’s also important to note that the impact of an effective preschool is half that of the child’s mother’s education level and one-third less than the quality of the child’s primary school. So while the study does show a benefit to preschool, it would be hard to conclude that it justifies a large investment in expanding preschool access beyond disadvantaged children.

So what should we take away from all this? Simply that more experiments backed up with high-quality studies are needed, as well as more journalism based on data rather than opinion and ideology. The New York Times Magazine recently ran a story on the wide range of experiments happening in New Orleans. The upside of the destruction caused by Katrina is that all the limits to experimentation were swept away. In its place are a variety of models including vouchers, privately-run charter schools, publicly-run magnet schools and a variety of teacher training programs. Such a real world lab is a boon to the type of experimentation necessary. To the author’s credit, he documents the experiments and the rationale of each without issuing policy prescriptions or facile conclusions.

Unfortunately there does not appear to be a structure in place that will allow a normalized comparison of the various approaches. For instance, some charter schools are selective, others are not. Some schools accept students mid-year, others do not. While backward-looking efforts to try to control for the various factors that could affect outcomes are possible, they are not nearly as reliable as randomized controlled trials set-up from the beginning. Thus, we can expect that no matter what happens in New Orleans we will see lots of stories that purport to use the data there to support whatever position the writer has already come to. Keep that in mind as you see these stories emerge over the next few years.