Monday, December 22, 2008

Delusions of Quality

The Advertising Research Foundation is getting clients, vendors, and researchers together to talk about quality in online research. It sounds promising at first glance, but a closer look reveals the project's limited scope. It turns out all they want is to prove that research done with online panels produces valid data. Don’t get me wrong it’s a worthwhile project, but they are working with a very narrow definition of quality.

While the validity of data is unquestionably important, how can ARF make any conclusions about the quality of online research without looking at factors like panel recruitment methods, the quality of online survey interfaces, how we treat panelists, or the things we inherited from offline MR that don’t work online?

I applaud the idea of getting online MR stakeholders from all sides together to talk about quality, but not if it’s just a half-baked PR exercise to keep the money flowing.

Ford used to say Quality is Job 1 ... now they're one missed bailout installment away from bankruptcy. We aren't important enough to qualify for a bailout, so we can't afford to start believing our own spin machine.

0 comments: