Monday, August 25, 2008

How to Mislead with Statistics

"A study comparing ten-year-olds in a small, walkable Vermont town and youngsters in a new Orange County suburb showed a marked difference. The Vermont children had three times the mobility, i.e., the distance and places they could get to on their own, while those in Orange County watched four times as much television." (Jane Holtz Kay, Asphalt Nation)

On the surface, this looks like a pretty strong argument that, as the author would have us conclude, places designed for cars are hurting or kids, and anecdotally it is interesting. But, without more details, it is nearly worthless as evidence. The quote itself raises questions: Did the Vermont kids actually walk more, or did they just have more potential mobility? How does TV watching correlate to mobility? Could the Vermont TV times be low because those kids spent more time on the computer (rather than walking)?

Then there is the nebulous study itself. Even if it was well-designed, how could it really control for all the non-car differences between a small town in Vermont and a suburb in California? Controlling for population density, economic status, and weather is pretty obvious, but what about parental attitudes? People who raise families in a small town in Vermont tend to have different values than people who move to California to raise kids, and those values extend to modes of transportation. It is conceivable that the California kids lived in a very walkable neighborhood but that their parents don't encourage (or want) them to walk places for status or convenience reasons (it is easier to monitor your kids when they are at home in front of a TV rather than outside on their own). There are several other possible explanations for the differences cited, too; the author's conclusions may be valid, but there is no way to know from this snippet of relevant-sounding information.

No comments: