Customer Crossroads

Anne Crassweller + Frank Graves #NetGain8 - using IVR systems for research

I've been talking to Anne Crassweller of NADbank about research and the challenges faced by companies like hers, so I know this is going to be an interesting session.

NADbank collects the data about readership for the daily newspaper industry, how many people read a newspaper every day and throughout the week, and all the related data.

Frank Graves is a major figure in political polling, and has been a big advocate of IVR polling. This is where you pick up the phone, and get to respond to a very small number of polling questions through an interactive voice response. I kind of enjoy these, they are fast, and usually interesting. And faster than a real person, frankly.

Anne: It's all about the sample. But that doesn't mean we can do, or want to do things, the way we did them in the past. Consumers do not all use land lines, so land line methods are inherently biased now.

She says her stakeholders want the data really fast, overnight if possible. So they have been working with Frank Graves at Ekos to develop alternative approaches.

Frank Graves is now up.

[BTW - first presenters to use Prezi]

Online panels are not reflective of households, and have a gazillion other problems for sampling. Challenges for declining response rates, sampling, commercial pressures for faster and cheaper ... So, they have been playing around with IVR, "robo-calls."

They have been refining the methodology to get better and better samples.

Best lines of the day: "Now that's an exciting table!" "Prediction is really hard, particularly about the future."

Anne is back up, talking about how the HD-IVR delivers results for them.

• Gender results are a closer match to census than any of the other methods compared
• Age results are much closer to census than any of the other comparison methods. This is the only method to really get 18-24 people effectively
• Education is very close to the census, again much better than the other methods compared to.

Part of the challenge is that the CATI surveys are much longer, so they have been trying to figure out what questions to shift to IVR.

Cell-phone only households are the challenge, because so many people are getting rid of their landlines. The debate is, does cell-only define your media behaviour? Their tests have showed that, NO, your choice of being a cell-only household is not a major determiner of your media habits.

My quant buddies are probably following the stats quicker than me, but I think this is quite interesting from the standpoint of just getting good, real, data. There really is no substitute for actual facts, is there?

 

 

Caroline Fletcher on mobile self-ethnography at #NetGain8

I'm a big fan of having people take videos and pictures of events in their lives, whether that is using an ATM machine or walking their dog. And with smart phones, handhelds and tablets, all of this is just so much easier to do. So I'm of course interested to see what kinds of things Caroline Fletcher will be showing us.

Why use mobile video ethnography:

• genuine behavior
• creative expression
• cost effective
• pre-post task - review footage together

There are so many ways to approach qualitative research projects, when do these approaches make sense?

• when people would not want a researcher there. e.g. carving the Thanksgiving turkey, other private or special moments
• capturing pain points, like setting up HDTV, setting up home internet WiFi, IKEA furniture assembly

Saw a nice sample of some edited video.

 

#NetGain8 Tom deRuyck on Consumer Communities

I met Tom deRuyck at the speakers dinner last night. We had a lively discussion about what "community" means in the context of marketing research, and found that we largely agreed with each other!

He says his firm has done more than 500 communities in the last 8 years. Based on that, these are the things that companies need to survive:

• being open - embracing the empowered consumer
• being agile - moving quickly to respond to consumers

The average CEO finds this scary, and some bravery is needed to be open and agile. Most companies do not have two-way dialogues with consumers, even when they claim to. They are concerned about what will be said if they actually open up a dialogue.

deRuyck asserts that marketing (in campaigns) and research (ad hoc projects) are not responding to the continuous nature of experience. Silos in departments, and process-oriented marketing result in lack of agility. Projects are passed from silo to silo for each phase.

His recipe:

• work on project teams instead of departments
• stop working on ad hoc research
• stop working on ad hoc campaigns
• Bring consumers in to the business and put them at the heart of the business

Examples of companies that are doing this...

[1] His favorite cookie, PiM's, a Belgian cookie. He says he and other brand fans are more knowledgeable about the product than the brand manager who has been in the job a short period of time. "Your fans are emotional shareholders because they are emotionally connected to your brand."

[2] Ben and Jerry's wanted to develop a brand ambassador strategy. Found people who score 9 to 10 on a brand fan-ship scale. In a community, these people were the toughest audience. They became angry when a brand team did something wrong with "their brand." They truly want to help you out.

[Aside, @Tom de Ruyck has the best slides ever. Love the typographic design.]

A community - consumer consulting board - a closed online platform on which you bring together a larger group of people for a longer period of time. Closed = invitation only. closed = your secrets are safe. Efficiency of online. Most important charaacteristics - more people for more time. Allows for clashes of ideas, which can be very interesting. More time allows you to think along with consumers while you are developing a marketing campaign.

[3] Heineken - wanted to be known as a design brand. Did a year-long community. Did some mobile ethnography. Got immersed in the world of clubbing. This served as an immersion for a design team of 15 people. A year after the first day of the community, Heineken wanted to actually build the club of the future -- the ideal clubbing experience, as a pop up club in Milan.

After the first immersion, they started sketching. Then they got feedback on the concepts. They worked across silos, brought down the walls between different departments.

He suggests that communities be used to tackle all your qualitative questions. Use for all business objectives. Generating insights. Developing new concepts. Guiding marketers when they are launching a new product.

How to make it happen? Recipe for success:

• first element is getting the right people on board. the high performing community members are more into the brand OR more into the topic than other people are. It is NOT representative. Not a bad thing (this is qualitative). We work with people who are interested and interesting to listen to. You want people to inspire you!
• How many members? Bigger is not better. Their communities are small - they have studied this with Maastricht U. AFter 30 answers, you know what you need to know. In a short-term commuity, 50 people participating intensely will give you this. In a longer-term, more permanent community, you get 150 people. As you approach 150, it becomes more difficult for the moderator to interact and to probe.
• We brief people and create engagement. The incentive is not money -- it is recognition, feedback, and from time to time a small gift. e.g. from Heinz, every quarter they get a basket of products that they have worked on.
• Co-creation of the IKEA catalog around the world. People want to do this! Make it fun to engage people.
• Help keep people engaged. Gamification -- use the principles and psychology behind games to make a boring task more fun. At the question level, the individual level, and the community level. A question is turned into a challenge. e.g. have ten teams compete against one another, based on cities -- show us that your city is the coolest on Earth. (done for MTV globally) At the individual level, give people a status badge for quality and achievement. On a group level, organize battles of arguments -- create tensions, but do it in a game way. At the community level, you want to make the community act as a we, as a group. Set goals for whole community feedback

Chiquita smoothie example is kind of cool. They had regular fruit eaters and "unhealthy ones" switch their diets for a week. At the end of the week, they listed the benefits of healthy eating. Then they had to argue to convince people to change their diet.

Mobile tools are an obvious good fit for communities. But so is going into the social media where people already are. They created a Facebook page that only community members had access to, so they could do second screening during a TV program. So they could help people capture real time feedback (because they were going to be on FB anyway)

Interesting that they sought feedback on ther communities. Campbells' community had a professional moderator, and a participant co-moderator. this person helps to guide asking better questions. Co-moderated communities get 25% more on topic discussion.

Another thing they did was get analytical help from people to understand stories and other input that they don't understand. 20 to 40% additional insights, depending on the topic. Get more insights. The insights that they come up with, that you didn't find, those are the most interesting things.

Have also tried recruiting observers to just watch a community. Then they do first run of analysis, and run it by the observers. The observers provide feedback and correct the problems of the analysis. This freed the people from social pressure and cultural challenges. (This example was research done in China.)

He has a free book on the internet at www.consumerconsultingboard.com

 

 

Annie Pettit at #NetGain8 on Behavioral Economics

Love the title of this presentaiton - "Behavioral Economics: New, or just new to you?" You will be familiar with Annie Pettit from her @lovestats Twitter feed and terrific blog of the same name.

She thinks it's a whole lotta hooey. Interesting start! [Can't say I agree totally, but I am a little tired of people trying to explain all of human motivation through loss aversion tendencies.]

Definition: Behavioral economics is the application of insights and research from psychology to economics.

Basic tenet behind BE: humans are irrational. No surprise there.

Next up: she shows a number of historical figures dating from Adam Smith and covering the next 300 years. People who all documented how human behavior is not fully rational.

Her real argument here, I think, is that there is a lot of documented psychology (e.g. Asch conformity studies in the 50s, MIlgram obedience experiments, Zimbardo situational studies with randomly assigned guards and prisoners) that show how people do things that aren't fully rational, and the power of the pull to conformity. Situational ethics and norms.

She did a little experiment with us about the number of peanuts in a jar. We had to guess. But she planted the first dozen guesses, to show that we would tend to conform to the prior guesses.

Cognitive dissonance. Festinger and Carlsmith, also in the 50s, showed that we evaluate a task as boring or fun based on how much compensation we received. (If less compensation, evaluated as more fun)

Halo effect, Thorndike in the 20s. Our overall impression can be based on one trait. (This has been re-demonstrated in many fields thousands of times. e.g. effect of appearances on perceived intelligence of children, impact of name gender on resume, and many other approaches.)

Impact of all of this - her take:

• people want to give you the truth, but they can't always do that.
• they misunderstand their own behavior

So -- stop asking questions like:

• would you...
• why do you ...
• hypotheticals of any type

Instead, ask what I have done. Ask about past behavior. What did you buy, when, what coupons did you use, etc. THIS is GREAT ADVICE.

 Brian Singh, in his question, suggests that experimental design is something that is growing, because of wider acceptance of human irrationality.

 I think Annie's point is a good one overall -- that psychology wasn't just invented, and we need to pay more attention to the substantial actual science that exists, and has existed for years.

 

 

 

 

#NetGain8 - SAS's Chris Long

We're all fuelled up on chocolate-filled croissants and coffee, and ready to learn more. The gentleman beside me has an Elite luggage tag, which instantly made me wonder many things. Great badge value in those tags, isn't there? It says "I travel a lot," "My company pays for it" and "I always have lounge access." Such is the life of a qually, always observing and overanalyzing.

Next up, Chris Long from SAS, who is talking about marketing analytics and where that is going. Most of his slides are kind of complicated, and have a zillion logos on them, representing the MANY players in social media and marketing analytics.

I do like this graphic, a chart that explains social media.

• Twitter - I'm eating a donut
• Facebook - I like donuts
• YouTube - how to eat a donut
• Instagram - vintage-looking donut
• LinkedIn - my skills around donut eating
• Foursquare - where I eat donuts
• Pinterest - a donut recipe
• Last.fm - now listening to "donuts"
• G+ - I'm a google employee that eats donuts
• I might add, Quora - "how to build a donut business"

Mr. Long says Canada is 1.5 to 2 years behind the global technology curve on social media. From a business / marketing / analytics use perspective. Which suggests there will be more adoption of social media analytics by business, by mainstream media.

How companies will use social media: foresight, brand protection, marketing, sales.

He says we need to understand four dimensions of social media. Conversations, content, context, connections. For context, think Marshall McLuhan, "The medium is the message". Very true - seeing something on Facebook vs Twitter says something to you just because of the context.

My biggest takeaway from this presentation: the slide people will remember and talk about is the one I captured in the bullet points above. It's the one that tells a human story.

Questions on how to use social media data yield surprisingly obvious recommendations -- like monitoring and responding to what people say about your brand. It seems like you don't really need this kind of sophisticated tool to do something like this. My take: we are at the nascent stage in learning how to effectively use all this analytical horsepower.

 

#NetGain8 - Mark Michelson and mobile research

Mark is presenting virtually from Atlanta, which got snowed in yesterday. I know Mark as a fellow quallie, but he is current Exec Director of the Mobile MR Association. MMRA.

Smart phones are the big enabler of mobile MR, both surveys and qualitative. [I recently completed my first true mobile qual project, and it was great. Technology worked well, participants loved it, and we got a video diary of moments through a week in their life, plus Q+A discussion.]

Some other parts of the trend in mobile:

• voice possibilities, live video chats
• passive data collection possibilities
• location-based data
• app-centric communities
• QR and scanner codes to engage and classify
• real time and asynchronous
• Augmented reality - superimposed imagery in the real world (Vision Critical has done some of this apparently)
• gamification - survey questions inserted into the game in order to earn in-game rewards like lives

As wearable computing comes in, many more possibilities for data gathering will be possible, including biometrics.

The challenges now are actually in design, because you get a lot of data from studies of this type. Images, videos, etc.

Other challenges:

• responsive design to fit screen size
• user experience and on-boarding (training)
• keeping participants engaged - balance engagement with bothering people
• participant privacy
• validity of self-reported data, client trust in findings, and participant trust in using the apps
• device ID and interface optimization for specific devices -- there are so many different mobile devices

Relatively few companies are using mobile. General Mills is an industry leader.

MMRA has developed a new set of guidelines for privacy (full standards plus a short form that can be shared with participants in research). These integrate with existing ESOMAR standards, but expand them.

 Best practices:

• be transparent and respect respondents - share intent, expectations, in advance.
• avoid participant burn-out. Not too many questions. Respect time zone - don't send notices out in the middle of the night
• make the study appropriate for the screen size
• channel dictates question type (channel being native app, SMS, WAP, etc.)
• use micro-surveys, shorter length of questionnaires (e.g. 3 to 5 questions) Use data stitching techniques to bring these together.
• use mobile friendly tools - sliders, dials, graphic interface. Avoid typing where possible.

Clearly, the future of mobile research is great, and it will enable many more participants to engage than traditional online methods, especially when we consider global markets.

 

Mark's slides can be downloaded from mmra-global.org

 

 

NetGain8 - leading edge developments in Marketing Research

I'm at #NetGain8 today, and thought I would blog live. These may be fairly brief, but this conference is usually a great one. Plus, I have to moderate the final panel, so I do need to pay attention.

THE FUTURE OF RESEARCH - Jeffrey Henning

Jeffrey Henning from Researchscape International is talking about the future of the MR industry. bitly/NETGAIN8 has the slides, he says. He drew a beautiful metaphor about how roads and transportation have changed, but we still need to go from place to place faster than our feet will carry us. Methods change, fundamental needs do not.

Surveys are a victim of their own success. There is a lot of pressure on surveys, including length, badly written, too complex, declining response rates. And even more, behavioral economics has made it clear that direct questioning methods may not be very useful, because they rely on us being good observers of our own behavior and intentions.

However, some of the things driving growth are: mobile surveys, micro-surveys, DIY systems, automation, and cool features (think feature wars among software and solution providers) like the ability to capture emotions.

Proprietary analytics is one of the ways companies are competing in this environment. Better dashboards that reduce the need for human work to provide specific views, like single location results, for example. Automatic weighting, which is not very good now, is likely to become better as companies keep iterating. Automatic coding of open-end responses is getting better, especially where the answers are very short (e.g. what brand, what tv show). Statwing.com, for example, automatically applies the appropriate statistical method to the data, meaning someone without statistical knowledge can do good analysis. Crowd-shaped surveys change the questions based on early answers. Better weightings, text analytics of long verbatims, and a lot of aggressive innovation will characterize the supplier environment.

Things he thinks we should do to differentiate are interesting. Several of them are already being done by good qualitative people. e.g. infographic reports, posters, storytelling, running workshops to help internalize findings, and increased qualitative analysis. [Personally, I do a lot of these things, and many of my colleagues do as well. So my conclusion here is that the statistical part of the MR community is catching up to the qualitative side. Not sure if I like that trend, but it's still a good idea.]

 

Are you credible and relevant? Getting behind your assumptions

We see this question in surveys all the time. 

How important to you are ... followed by a list of items.

 

For example:

How important to you are the following?

• Networking opportunities with other professionals
• Professional development opportunities
• A free and democratic society
• Having clean socks in your drawer

[Okay, I admit it, I added the last two.]

Actually, all of these things are important to me. But I don't expect my professional associations to look after getting me clean socks. Or keeping my country free and democratic.

 

The challenge with this kind of thing is, the hidden assumption is that these are benefits that the questioner believes they are offering.

 

As marketers and business people, we are all trying to define a space, set priorities and get clarity. And there's not a thing wrong with that. But we also have to get beneath our assumptions.

 

It may be that some of the features and benefits you think are core to your offering are not core.

 

A free and democratic society is important to many people, but that does not mean they see your particular political party or lobby group as a credible or relevant standard-bearer in that mission. And even if you are, that does not mean they are willing to join, give money, provide volunteer service or anything else. 

 

To get behind these kinds of assumptions, we need deeper insights into the lives and concerns of our customers, and how else they meet those needs. If you are thinking that is not a simple question, I completely agree.

 

I'm picking on surveys again, because it is an easy way to illustrate the problem. It's not about how you ask the question. It's really about what questions you ask.

 

 

Related articles

Is loyalty a one way street?

Can you brainstorm over the phone? Yes!

Is loyalty a one way street?

We talk a lot about loyalty in marketing, and act like it means something to the organization. Like perhaps the organization should be loyal to customers, as well as customers being loyal to the organization. [Stop laughing, I'm being serious here!]

This seems often to be a missing ingredient to me.

Two recent cases in point illustrate, and they are related even.

The CIBC Aerogold credit card was long considered a leading credit card in Canada. For years, it was the card to beat. The insight that helped competitors turn the corner was this: people were frustrated with points programs where the points could not be used for much of anything.

So TD and RBC among others, launched credit cards that accumulated points that translated directly to dollars, and could be spent on pretty much any form of travel. Both now provide travel services as well, through partnerships with Expedia and Travelocity.

The latest development is that CIBC VISA has lost exclusivity with Aeroplan, and TD will now be offering this an Aeroplan card as well.

Here's where the loyalty issue gets interesting to me. They sold me.

The deal made provided for sale / transfer of a portion of the portfolio. My account was on the list. So how much loyalty does this show? I fully understand that this was legal, and always a possibility under the terms of the contract.

CIBC sent me a boilerplate letter that could have been written by a machine. TD has now sent me a welcome letter and a little treat -- a free lounge pass. The implicit assumption is that my loyalty for 10+ years was to the product and the points, not the organization or the brand.

That certainly got me thinking about whether or not this product will serve me well in the future.

It starts to get more interesting for me when I look at my current challenges in using the points. I'm heading to Budapest for the AQR-QRCA Worldwide Conference on Qualitative Research in the spring. And tacking on a little vacation in London on the way home.

With all these Aeroplan points I've been accumulating, I thought I might be able to do something on this trip. Airfare was a non-starter, I'd need a ton more points to make this work. But I thought perhaps a hotel.

843 hotels! Woo-hoo!

 The Air Canada site lists 843 hotels in London that I can choose from if I want to spend real dollars. Wow, that's a lot of choice! That's when I thought of using my Aeroplan points to reserve with one of these. On the Aeroplan redemption site, I discovered to my horror and amazement that there are exactly 5 hotels available in London on points. (There was one in Budapest. ONE!)

Now for years, Aeroplan has been pushing the notion that a mile, or a point, is just like cash. Except it isn't. And for your loyalty to this program, you are rewarded by restrictions of all sorts. If you are paying cash, Aeroplan knows their customers have many different needs and price points. But if you are on points, it's all ha-ha, you lose.

Really, shouldn't I just buy a new iPad and forget the whole travel thing?

Well, that's a murky area too. If you are buying miles to make up a shortfall, I think you will be paying $.03 (3 cents) a mile.(Surprisingly hard to find this information on their site.) You are paying retail.

But if you are selling miles in exchange for goods like an iPad 2, for instance, you will be selling them at less than a penny a mile. You are selling wholesale.

A lot of this whole house of cards relies on a lack of transparency. Something we know people are deeply resentful about. It just feels kind of unfair, doesn't it. But that's the loyalty business today, it's anything but loyal if you are on the customer side of the equation.

I do wonder how long this approach is sustainable. It's all powered by the desire to get stuff for free, and I guess we are all guilty of that.

What do you think? Is the points game a house of cards? Or does no-one do the math?

Related articles

What happened to the telephone? permission media redefined

Can you brainstorm over the phone? Yes!

CIBC to help TD, Aimia ease transition of Aerogold customers by mid-2014

TD, CIBC complete Aerogold deal