Inspiration, anyone?

Somehow it seems fitting to end 2008 with the second video in Geert Desager's Bring The Love Back series. Inspiration, Anyone? is the sequel to The Break Up, and it's just as excellent.

I decided to make the last post of 2008 about this video for two reasons: 1) it's a perfect summary of industry's current attitude toward the web, and 2) it previews the conversations that many companies will finally start having in the coming year.

Happy 2009!

Do you know what your panelists are saying about you?

Most survey panels measure panelist satisfaction in one way or another, but are they doing anything with that data?

Here are the most complained about online survey panels, according to Survey Police.

Survey Company	Active	Unresolved	Resolved	Total
Lightspeed	15	15	75	105
GlobalTestMarket	6	1	47	54
MyView	5	15	26	46
Greenfield Online	6	10	21	37
PromoSquad	5	17	10	32

And here are the panels who have the most unresolved complaints.

Survey Company	Active	Unresolved	Resolved	Total
Zoom Panel	7	18	5	30
PromoSquad	5	17	10	32
MyView	5	15	26	46
Lightspeed	15	15	75	105
Greenfield Online	6	10	21	3

No matter which column you sort on, most of the names remain the same. Lightspeed, Greenfield Online, MyView, and PromoSquad seem to be the worst of the bunch.

When we look at the top-rated panels ...

1.	HotSpex	854	92%
2.	panelbase.net (UK)	115	92%
3.	PineCone Research	124	89%
4.	Opinion Outpost	89	86%
5.	ItsYourView (UK)	86	86%

...none of those names come up. I also noticed that only two panels have a 90%+ rating, and after the top 5 there is a steep drop into an ocean of mediocrity. There seem to be lots of bad panels, and very few good ones.

The fact that so many panelists are posting complaints on sites like SurveyPolice.com and GetSatisfaction.com tells me that we have failed to provide adequate support and now our dirty laundry is being aired all over the grapevine. There is no excuse for this. We are research companies. We get paid to tell our clients how to keep their customers happy, why aren't we turning the magnifying glass on ourselves? It's time for us to take a cold, hard look in the mirror and do something about this.

Astroturfing Update

Today I was suffering from post-Christmas-pre-New Year malaise and I needed something to snap me out of it. I decided it might be fun to poke around on Technorati a bit and see who has been astroturfing recently.

XPEED is the quintessential astroturfer. In a single post he mentions: American Consumer Opinion Panel, Surveysavvy, Ciao Surveys, Greenfield online(Gozing), Pinecone Research, Globaltestmarket, Opinion Outpost, NFO Mysurvey and Lightspeed Research, Synovate, NPD Research, NOP World (Surveys.com), ECN Research and Nielsen Netratings. He even advertises his services with "Hire me! Have me blog about you on this site" and "Buy reviews on leading blogs" links prominently displayed next to the post. 

Shane Dolby calls himself the Internet Business Expert, but he's just reposting XPEED's stuff ... or is it the other way around?

Sydney Hazelton is a big fan of GlobalTestMarket, but after reading one of her earlier posts where she explains how to get paid for writing reviews, I'm no longer convinced her praise is genuine.

SandynPaul from bigbigforums.com really loves the Ipsos i-Say Panel. i-Say must really love her back, because the affiliate ID in the link she posted would suggest they're paying her.

Surveyline is astroturfing for TNS mysurvey.com, Opinion Outpost, Ipsos i-Say, and several others.

Realm of Prosperity isn't even trying to come across as legitimate. Every single post is about how much money American Consumer Opinion Panel, Opinion Outpost, GlobalTestMarket, Pinecone Research, or one of their other sponsors might pay you for taking surveys. They also accidentally confess to astroturfing in one of their posts.

The Flash Advancer, UltraPing, Tradtional & Contemporary Marketing (no that's not a typo), EigenDecomposition, The Transmission Outfit, The Content Blender, and I'm sure many others I missed, all posted the exact same content on the same day promoting top-paidsurveys.org. Turns out top-paidsurveys.org is a front for several of the usual suspects: Ipsos i-Say, Synovate Global Opinion Panels, ECN Research, etc.

Astroturfing seems to be on the rise as email becomes less and less productive for lead gen companies. I hope astroturfing will not be as damaging to the blogosphere as spam was to email, but only time will tell. I will continue to post periodic updates as long as there is MR-related astroturfing to expose.

Survey Design Tip: Asking for Age & Drop Down Menus

Vovici says we should get rid of drop-down menus and age ranges, and collect only the respondent's year of birth in a text box. I agree.

Generally speaking, drop down menus become tedious for respondents when there are too many (more thant 10?) items to choose from. Most of the time you can fix this problem with a text input box and some validation logic.

Time is Precious

Argumentum ad verecundiam

This is the most problematic fallacy in online market research. Decision making authority rests with people who know a lot about market research, and nothing about the internet. There is a critical difference between an online market research company and a market research company that collects data online. Which one are you? If you said online market research company, I don’t believe it … but I secretly hope someone comes out of the woodwork to prove me wrong.

Online companies succeed because they utilize the qualities of the internet to make something remarkable. Traditional companies who try to migrate their offline business model online flounder and fail because they don’t understand that rules are completely different online.

While everyone is paralyzed by fear of the economic uncertainty that surrounds us, there is a tremendous opportunity for someone to win big. Start a new online company or division. Staff it with people who know a lot about the internet and almost nothing about market research. Let them break all the rules. When they come back with a bunch of brilliant but heretical ways to gather thoughts and opinions online introduce them to some open minded statisticians and market researchers who can help them harness these new tools to reinvent market research. Now comes the hard part; you have to convince clients to trust the new methods. Congratulations, you are now the only online MR company that matters!

Delusions of Quality

The Advertising Research Foundation is getting clients, vendors, and researchers together to talk about quality in online research. It sounds promising at first glance, but a closer look reveals the project's limited scope. It turns out all they want is to prove that research done with online panels produces valid data. Don’t get me wrong it’s a worthwhile project, but they are working with a very narrow definition of quality.

While the validity of data is unquestionably important, how can ARF make any conclusions about the quality of online research without looking at factors like panel recruitment methods, the quality of online survey interfaces, how we treat panelists, or the things we inherited from offline MR that don’t work online?

I applaud the idea of getting online MR stakeholders from all sides together to talk about quality, but not if it’s just a half-baked PR exercise to keep the money flowing.

Ford used to say Quality is Job 1 ... now they're one missed bailout installment away from bankruptcy. We aren't important enough to qualify for a bailout, so we can't afford to start believing our own spin machine.

Letting go / Tip of the Hat: Vovici

Online survey panels have a hard time recruiting new panelists. So much so, that in their desperation to reach recruitment quotas they have to make deals with the devil.

Many new members are lured into joining by false promises of money and prizes. They quickly realize the recruitment offer was misleading, and since the panel experience is not engaging, they want to leave. Not so fast! Most panel companies intentionally make unsubscribing confusing and difficult to deter panelists from leaving. This is part of their retention strategy; since we have trouble getting people in, we have to make sure the ones we have can’t get out. This is insane! Making it difficult for people to leave doesn’t solve the problem because if those people don’t want to be part of your panel they aren’t going to do surveys. 

If you want to retain more panelists focus on ways to improve their experience. Locking the doors and windows so they can't leave is just going to make them angry enough to say bad things about you in public.  

I was glad to see Vovici acknowledge there is a problem with how companies handle unsubscribe requests.  They even followed up by suggesting ways to improve it. For that, Vovici get’s a tip of my hat.

 

Essentialism is an Illusion

In his book Everything is Miscellaneous, David Weinberger debunks market segmentation based on essentialism.

Particular demographic properties are selected because the marketers believe they define a group susceptible to the same message; hence 18-24 year old suburban males get the “It'll make you tough” advertising while the females get the “Boys will like you” message ... Thinking that 18-24 year old suburban males exist as a market [online] – as something more than a random slice [of the population] – gets in the way of seeing the truly fascinating phenomenon: miscellaneous customers finding one another in the digital world and forming real social groups, not because they share essential demographic traits but because they're talking with one another. The markets that conversations make are real markets, not mere statistical clusterings.

Essentialism makes the world seem more manageable, but it can lead us to miss what's really going on.

Essentialism worked in the TV-industrial complex because mass media was able to isolate people and bombard them with archetypes until they become convinced they fit the moulds made for them by the marketers. The vast majority of teenage boys in the 90's didn't share any essential traits with Michael Jordan, but Nike kept telling them they “want to be like Mike” over and over and over again until they believed it. Once convinced, they expressed it the only way they could; by getting his $200 Nike Air Jordan shoes, the $100 Air Jordan training suit, etc. The market was created by the marketing.

The online world promotes individuality, miscellaneousness, infinite choice, and user control. It is an environment in which the old marketing strategies cannot succeed because interrupting people with marketing messages whenever and for however long you want is impossible. This means that marketers, and by extension market researchers, must fundamentally rethink their approach.

Wag of the Finger: LogicDepot.com

A wag of the finger for LogicDepot.com, who launched a survey so bad it was featured on failblog.org.

People don't trust companies

Forrester says people don't trust company blogs. I think they missed the bigger picture. While company blogs may be least trusted, every form of brand-to-consumer communication gets a failing grade. People don't trust companies; period! Since one of the main objectives of MR is to help those companies develop their marketing messages, I guess that makes us the lowest of the low.

Here's the kicker, people are right. We (companies) don't deserve their trust. For decades we have profited from developing ways to brainwash them into buying things. We didn't really care if the products were good of if these people needed them, we just wanted them to consume as much as possible.  No matter how many declarations of love we put in our advertising, we care a lot about the bottom line and very little about our customers ... and they know it. We didn't anticipate the internet would provide them with the tools to expose our hypocrisy and now we're caught with our pants down.

Some companies get it, and they're starting to do things about it. The solution is simple. Get real. Get a sense of humor. Upgrade your consumers to customers.  Open the doors and windows, and let your customers in. Hire employees who will be ambassadors, not sheepwalkers. Start having real, two way conversations with your customers. Basically, start doing what you claimed to have been doing all along: care about your customers and show them some respect.

We have to start treating our survey respondents better and our clients have to start treating their customers better. Our clients pay us for market insight, and this is a bit of insight they desperately need. Companies who can't or won't do these things deserve to wither away, and they will.

The best stats you have ever seen

I spend a lot of time pontificating on the need to improve data collections methods, but I think the same can be said for how we present data. In his TED talk about third-world myths, Hans Rosling shows us how great data visualizations can expose powerful new subplots in statistics we thought we understood. 

Warning: After seeing this you may no longer be able to use the standard Excel/PowerPoint charting tools without becoming acutely aware of how much they suck.

Generation Next

Robbie Cooper's Immersion project is fascinating stuff, and there is a message for market researchers.

These kids will begin trickling into the 18-34 demographic in 2 years. They are growing up immersed in ultra-realistic virtual worlds. Their hobbies include car jacking, disemboweling alien warriors, and ripping through hordes of enemy combatants with machine guns. How are you going to engage them with surveys composed of radio-buttons and check boxes?

Get a clue

For those among us whose clue-deficiencies manifest themselves daily as barriers to new ideas, please read this. It's time to get a clue and start making a positive contribution.

A long time ago The Cluetrain Manifesto inspired me to change my way of looking at the business world. It was a turning point in my life. In the last few days I have been painfully reminded that almost 10 years down the road Cluetrain's message is even more relevant but still ignored.

Cluetrain contains some damning implications for market reserch, and ignoring them is not an option any longer. G.I. Joe was right, knowing is half the battle, and it's the first half, so we better realize there is a problem if we hope to solve it.

If you haven't already, I encourage you to buy a copy. If you really can't part with the few dollars it costs, you can read it online for free. Read it and tell others about it. If you only have time for one clue this year, this is the one to get.