I’m aghast. Over on TechCrunch, a site known as much for its gossip-mongering as its tech reporting, John Biggs is reporting (sort of) on a study that he claims demonstrates children who get computers in developing nations get worse grades in school.
Here’s the link: Study: Mixing School-age Kids and Computers makes for Bad Stuff.
I’m a researcher and there are so many things wrong with John’s throwaway piece that my head is reeling…
The greatest mistake is that correlation does not equal cause. That’s the mantra of all researchers, because it’s typically impossible and illogical to connect cause and effect when you aren’t studying that specific relationship.
A classic example: smokers are less healthy than non-smokers. Smokers have ashtrays in their houses while non-smokers do not. Therefore, ashtrays make you unhealthy.
John quotes unpublished (e.g. non-peer reviewed) research:
“In a draft of an article that the Quarterly Journal of Economics will publish early next year, the professors report finding “strong evidence that children in households who won a voucher received significantly lower school grades in math, English and Romanian.” The principal positive effect on the students was improved computer skills.”
John explains his bias up front, at least, when he says “I’ve always believed that computers, in a general sense, are not a panacea for developing nations…” but in his zeal to have this unpublished research substantiate his position, he completely misses the point that it’s convenient but inaccurate to conclude that since the children whose families won vouchers for computers received lower grades it must therefore mean that the introduction of computers into those families caused the lower grades.
A possible alternative explanation: families that didn’t have computers gained a computer through the voucher program and the parents then spent more spare time playing with the computer than supervising their children’s studies. Ergo, their lower grades could have been caused by a change in their homework and study habits, not use of the computer itself.
Probable? Dunno. Possible? You bet. But in this sort of apparently sloppy research (and, sorry John, lazy reporting) it is ignored in the rush to use “research” to substantiate an existing perspective.
Yech.
John does share anecdotal qualitative data about his own behavioral patterns as a way to explain why he is (presumably) searching for this sort of data: “I believe computers are an important tool but without supervision they are a massive distraction. I am paid, in a sense, to putz around on the Internet and let me tell you that it changes brain chemistry and thought patterns. I’m more distracted, less polite, and unable to maintain a train of thought for more than an hour.”
I’ll just say: if you’re going to report academic research, please, please focus on what the researchers actually say in their research, and be skeptical of conclusions, particularly of correlational survey data of this nature.
You’ll note that I do not share my own views about this topic here. Why? Because they’re irrelevant to the topic I’m writing about. Op-Eds are not reporting…
A classic example of syllogistic conclusions based on where they want the conclusions to go. Thanks for calling them out Dave.
I am a big technology fan – and, yes those comments appear a bit shallow. As a parent, I know that the computer is an essential learning tool – and I also know that it is a huge distraction. Managing that screen time is essential. My guess is that the parents in the families that received vouchers were ill equipped to handle the complexities of managing their children’s time and habits on the new devices.
Dave, it’s true that retrospective studies show correlation and not necessarily causation. However, they are often all we’ve got because it is so often unethical or impractical to do a randomised, prospective study. I wouldn’t be so quick to dismiss the results of this study, which appears to match that of other studies reported on recently in the New York Times (one in North Carolina found that as more households in an area got broadband, grades and test scores went down; another was in Texas where large amounts of money were spent by the state providing computers to poor families–once again, grades and test scores went down, though there was an improvement in computer skills, as in Romania).
In any event, though, there is a logical flaw in the way you laid out your argument here. You wrote that it was “inaccurate to conclude that…the introduction of computers into those families caused the lower grades.” But then you went on to offer a “possible alternative explanation: families that didn’t have computers gained a computer through the voucher program and the parents then spent more spare time playing with the computer than supervising their children’s studies.” Perhaps so (though I doubt this is all of it). But even if this accounts for all of it, it is still the *introduction* of the computers which caused the problem–see?
That’s true, Alan, but I was offering that as a possible alternative explanation. I could just as easily offer that it was a seasonal issue or that there was some other factor unrelated to the computer introduction that caused the issue: jealousy of the other kids increased the stress of the students, something that happened because of the *voucher awards* and wouldn’t otherwise happen if they and their peers would all have been given the systems. Etc etc.