I constantly get queries from academics asking if I can help them disseminate information about a survey they’re doing on some subject or the other, often without indicating critical things like the topic of the survey or — for those of you that understand the nuances of research — the source of their funding.
This is another in that vein, a research project from the University of Connecticut exploring the impact of multimedia books. “The survey is for academic purposes only and the responses are only recorded in the aggregate. Respondents may opt-in to a drawing for iTunes gift cards.”
If you’re so inclined, go and respond before you read my commentary on the survey and its questions, because you likely won’t want to once you’ve read what I’ve written…
Before I posted this, I asked them about their funding, and received what I deem a typically ambiguous response:
“The study is funded by the University of Connecticut and the State of Connecticut for academic purposes.”
The lead researcher? She’s a Research Analyst at the Connecticut Center for Entrepreneurship and Innovation. Clearly they’re seeking different information than someone from, say, the early childhood education group would be trying to generate.
You might not know that I have two Master’s degrees, and have spent a fair amount of time studying research and survey methodologies. I even have a book about designing quantitative survey instruments on my shelf somewhere or the other. This piques my interest, to say the least, so I looked a bit closer…
The Danger of A Desired Outcome
Imagine that they’re doing research funded by a traditional children’s book publisher and the desired outcome of the survey is to identify that children who read multimedia books are more likely to spend lots of time on the computer and less likely to read printed books three months later. The publisher then promotes the research findings, explaining that they always knew those evil multimedia books were messing kids up.
Or, perhaps the research is funded by a multimedia book publisher (e.g., read-along CD or tape, or perhaps even an interactive book on the iPad or similar) and their desired outcome is to demonstrate that kids who engage with those type of “modern” books pick up computers faster and therefore the books are an important way that parents can help their children prepare for a high-tech future.
Surveys can absolutely lead the people answering to the results the researcher desires if you’re not very, very careful with how the researchers word questions, what order they’re in, the population from which respondents are drawn, etc.
Is this particular survey on one side of the fence?
Thought the survey is ostensibly about multimedia books, it’s curious that part way through the questions shift and we learn that…
“The following questions pertain to a handheld educational toy with one button (similar to the size of a TV remote) that recognizes images in a book and provides relevant audio for the text. (e.g. In a book about a trip to the zoo, the user would trigger the toy when prompted by an illustration and be asked questions about the animals)”
The survey asks about the price you’d pay for the item:
Doesn’t seem very objective. There are lots of different ways that multimedia books could be delivered.
Shortly thereafter the survey asks this too:
But perhaps a multimedia device that was actually an app for your smartphone? “I would be willing to allow my child to use my smartphone to interact with children’s book for extended periods of time….”
As far as I can tell, the purpose of the survey is to do advanced product research for a group on campus that’s considering introducing a direct competitor to the small handheld multimedia reader devices. If so, why not just say so?