theIdeaStudio (Customer Crossroads)

How to save money on qualitative research

There's a great discussion evolving on the Greenbook blog, started by Jason Anderson with his article "The 7 BS Stories of the Market Research Business".

I don't agree wtih everything he says, however his perspective is likely reflective of what a lot of research buyers feel. And his vantage point from inside the video gaming industry (life doesn't get much faster moving than that) is certainly interesting and unique.

Be sure to check out the comments too -- some good value added material is posted by other researchers in response.

Is good research expensive? Sometimes it is. There are ways to ensure you are getting the most from your budget, however. I published an article on exactly this topic (link below).

Link to PDF: Good to the Last Drop: 10 Ways to Save Money on Qualitative Research, by Susan Abbott, published by AMA MarketingPower.com

Must read article: Rory Sutherland of Ogilvy UK on how to measure advertising

Every time I talk about advertising measurement as a 'black art' I get objections from people telling me how their normative metrics are well tested and very useful, blah, blah, blah. Let me be clear here: I do qualitative testing on advertising, and I'm not too positive on that approach either.

Why so negative? Well, there is a lot of good evidence that our actual preferences and behavior are not all that well related to what we say.

A brilliant article, "Questioning the Nature of Research" reports on an interview with Rory Sutherland, chair of Ogilvy UK, who makes a strong case that our current measurements are not strongly rooted in behavioral economic theory.

Sutherland's point is not that research is a bad thing in general, it's that the current norms for how research is done on advertising don't work.

...as you get closer to the point of decision, there are factors at work which never really appear in conventional market research. They are contextual factors, social factors. No-one in any research group would ever say, “If there are four brands of shampoo, I’ll buy the one that has most bottles on the shelf”

This is true, by the way, not just of survey research, but also of qualitative.

The best qualitative research approaches come at the topic from right angles -- indirectly. Like a magician's deception, we want to focus your attention on something so that you respond as naturally as possible. Or we try to mimic the actual circumstances of your decision. Still, no self-report in the world is as good as watching a thing happen. Or setting up experiments that show you what is actually happening.

What we have learned from behavioral economics research is this: "Your preference is a product of your behaviour and not the origin of it." If you customarily go to one drycleaner, you will be able to provide a rationale for that choice, (e.g. quality, service) which is likely to have little to do with the real reason (e.g.location).

A useful and thought-provoking article to get your synapses back to school and making new connections.

 

Are you capturing the insights of the Introverts?

Our society has a strong bias in favor of extraversion -- those who are energized by being around others, who can process information quickly, and blurt out opinions as fast as they can blink. But there is a sizable minority of the population that does not fit this pattern -- people with a preference for introversion.

I wrote recently on the NewQual Blog about the advantages of online asynchronous research methods for capturing the insights of Introverts. And boy, this really resonated with people!

A couple of people sent me private e-mails [as introverts would do, not wanting to be loud and out-there, for the most part].

It's not just qualitative research that suffers from this problem -- virtually all marketing research does. Consider the telephone survey, where you are asked a battery of questions, many with multiple choice answers or agree/disagree scales on relatively complex topics.And you need to answer pretty much immediately.

A lot of online shopping has time limits that test our patience -- we have to choose among options quickly, or the system "times out". But with many options to choose, even the speedy must find it challenging to react quickly enough.

For more on this topic, check out the other article here:

Introverts get their day: When discussion forums and bulletin boards can be more inclusive

What's your take?

Is the world stacked against introverts? Does anyone give you enough time to consider before expecting you to choose, react or opine?

To probe or not to probe: Active listening versus poking with sharp thingy

More and more qualitative research is happening online. One of my favorite tools for this is the discussion forum, also known as a bulletin board. This is an asynchronous environment -- everyone is not necessarily present at the same time. [I so wish there was a better word for asynchronous!]

It's not quite the same as face-to-face. In person, if I hear something intriguing, I might ask, "Can you say more about that", or something similar. This is known, in researcher parlance, as a "probe". Although we try to make it gentle -- drawing out more information and ideas -- sometimes I think it feels more like a sharp poky thing.

In the discussion forum environment (asynchronous), you definitely can probe and ask follow-up questions.  The technology support for making this easy has improved a lot – we can now flag things, send a mobile message, or e-mail to the respondent.

My approach on this has changed a lot over the years -- I try to be more of an active listener, and less of an inquisitor.

Treat participant energy as a scarce resource

I treat follow-up questions as if I have only a very limited number I can use, and I must use them only when really important.  Here’s why:  I really love it when participants start to engage with each other.  But they do this less if I’m constantly butting in.

Not only that, but asking people to go back to something they said before is sort of like trying to rewind a conversation by hours or days — people have moved on. The energy is now somewhere else.  So you spend a lot of participant energy asking them to go back.  When people do go back, their responses are often not worth the cost in energy.

The virtual head nod

Probing does have one clear benefit — it shows people you are listening.  And listening is hugely important.  Participants see what you say, and see whether you are listening and understanding them or not.

If I don’t have a definite need to follow-up for additional detail, I sometimes restate what I am hearing, just as I might do in a F2F discussion.  This approach generally results in feedback from the participants that they really felt “heard”, which is so important.  Who wants to talk to an empty room?

The virtual head nod is even shorter — just an indicator that I’m there, reading and listening.

The challenge

Clients distracted by other work may log in and read something intriguing that happened earlier in the week. Then they want me to post a follow-up.

To keep everyone happy, I tend to bring this kind of thing back in with a fresh topic later on, rather than ask people to go back.

I do make my presence regularly felt, as often as possible given the demands of eating and sleeping, but limit the number of probes.

There is another school of thought that you can run these projects with no probes at all — I think that would produce significantly reduced output. However, there are some new DIY qualitative platforms that essentially take this approach. There is no active listener.

What’s your view?

Does this resonate with anyone out there? Either as a participant in some form of interview or discussion, or as a moderator of same.

Portions of this were previously published on the NewQual Blog.

How to save on qualitative research

No matter how large your research budget, you probably wish you had more. Here's my list of ways to save money on qualitative research -- some more obvious than others. You can get a PDF of the full article, with more details on each method, here.

Here's my top 10 ways, in brief:

1. Keep a tight focus on objectives
2. Reduce the number of participants
3. Cut the travel
4. Manage the number of markets you touch
5. Manage back-room catering demands
6. Decide if you need transcripts, video and other goodies
7. Work with realistic deadlines
8. Remember that the screener is not a survey project
9. Consider your written report needs
10. Be a valued client, easy to work with

These are times when organizations need insight more than ever to compete, but every
expenditure is under scrutiny. A collaborative approach with a skilled researcher will help you
get more for your research dollars without sacrificing quality of insight.

Resources

This piece was published by AMA in their newsletter as "Good to the Last Drop: 10 Ways to Save Money on Qualitative Research".  You don't need a subscription to read the article, and there's a nice PDF here.

New Qualitative 101

Greenbook, in conjunction with my buddies at QRCA, have just come out with a directory of New Qualitative providers.

Quite apart from the fact that I am listed in it... you would find it useful for a few other reasons:

• Great basic articles telling you what you need to know about various methodologies. Well written by experts Betsy Leichliter and Jay Zaltzman
• Details and contact information for the major platform providers
• Listings for researchers experience in these methods

You can get a hardcopy for $29, but check out the digital version for free, here.

There's also a blog (I'm writing there too), which has some fantastic contributors, all people who've done a fair bit of heavy lifting in new methodologies, and will be sharing best practices, new ideas, and whatnot.

It's good news to finally have a resource in one place, and I'm sure it will grow as providers seem to be expanding continually. Many of these providers were simply lost in the giant directory that has everything from survey sample to focus group facilities listed.

So now we have resources in one place, plus information too. And that's just great news for practitioners and clients!

 

Conversation in slow motion: How to observe a bulletin board focus group

I'm currently moderating an extended online B2B project involving business execs in a discussion forum. It's somewhere between a bulletin board project and a community, because the same people have been involved for several weeks.

It's demanding for me, but hey, I designed it, so I only have myself to blame.

But extended online qualitative is really challenging for the clients, who have many demands on their time in a typical workday. When the data really starts to build, after a few days in a project, keeping up can be overwhelming.

It's all well and good for me to have insights and distill them into nuggets of wisdom. But they're as good as useless if the client doesn't learn anything. And the best place to start the learning is in the backroom, just as it is with F2F research.

Here's what I've learned that seems to help people engage:

Tips to help the team stay engaged, and help your moderator manage.

[1] Log in early and often

The first day, it often feels like nothing is happening. Team members who logged on expecting activity can get discouraged, and when they look at the transcript later, it can feel a bit depersonalized, and a lot overwhelming.

If you log in frequently, you will see the same ebb and flow the moderator sees. Some topics attract immediate interest, others are slower to build. It’s easier to see where participants are spring-boarding off others’ ideas. Even though the transcript is time-stamped, seeing the evolution is a richer experience.

[2] Schedule team meetings to review insights

One team scheduled daily debriefs in a conference room where they projected the discussion and talked, just like they would in a backroom setting. Snacks included.

[3] Divide up the work of observing

Assign individual team members to cover off one or two segments in a multi-group discussion forum, then report back to the team on their segment. Another approach is to divide up by days (or weeks, for longer projects).

One project team held a group meeting where they projected the evolving discussion and reviewed interesting things together, just as they would in F2F research in the backroom. What a great idea! 

[4] Add comments

Observers can usually add comments to the transcript that are invisible to participants. This is a great way to capture a budding idea, and can help the moderator see possibilities for probes.

[5] Ask for follow-up probes right away

The faster we can ask a follow-up question, the better. Going back more than one day in a discussion tends to sap the group’s energy, and often does not work anyway.  Have a plan in place for how you will direct these to the moderator, to prevent frustration on both sides.

As a moderator, I may make the decision to bring something back in later to the discussion -- even days later. Or I may know that the topic already came up earlier. So if the moderator doesn't respond to your request, try not to get frustrated -- they are likely not ignoring you, just trying to manage the flow in the best possible way.

Acknowledgements

Portions of this article were previously published on the NewQual Blog as Bulletin Boards from the Backroom: 5 Tips for Observing a Conversation in Slow Motion.

If you haven't seen the NewQual blog or directory, you should check out this new resource: NewQualitative  The directory (finally!) brings together all the providers of online research platforms, and it's a good place to discover other resources and content.

Is DIY research going to put us all out of work? The debate rages on!

There's an interesting debate raging around about Do-It-Yourself Research. I posted this comment earlier today on Power to the People, an article by Joshua Hoffman on the Research Access Blog.

As a quallie, I am well aware that lots of people think what i do looks not just easy, but dead simple. Relatively few clients want to do their own, although some do. The usual reason appears to be cost, from what I can see, as recruiting is no faster or slower for them or me.

I find client's analysis of qualitative is usually of the fast kind -- first impressions. If they did more analysis, many of them could come up with what i come up with.

The real issue for me is more one of raw capability. People who have the capability to do from an analysis perspective what I do certainly exist inside client companies, but they are typically senior managers and executives, and are primarily in a managerial or strategic role.  If client companies were prepared to hire people at a senior level to do this work, they could likely obtain great results.

A couple of other things are often missing when clients DIY on qualitative. The longer the person is in the job, the less breadth of exposure they get, the less ability to bring in contextual learning from a broad base of clientele. 

For some methodologies, it could be difficult to get people to tell you the unvarnished truth -- i say this because even as an independent, I (and my colleagues) often have to go to some lengths to assure people of true anonymity in their feedback. I struggle to see how this could be done by an employee.

I also -- as do many consultants -- spend a truly massive amount of time and money on my professional development every year. I attend conferences, I speak at conferences, I attend training events of various sorts, I participate in several professional organizations, I read widely.

My clients, on the other hand, are as busy as small mammals, scurrying on their wheel, often without the time to really think about anything in depth. Their voicemail says they are "in a series of meetings", and that menu rarely seems to change. Of course they have access to learning opportunities, but most are mandated by the company, not chosen by themselves for what they bring.

Even large research organizations have relatively few qualitative people involved in the world outside the corporation, which I would guess to be for cost containment reasons. They send one person to a conference with the idea that they will share the learning.

There is a lot to be said for people who strive to invent new things every year to compete. If I worked inside the client organization, I might not feel the same level of pressure to keep up and be inventive to get ahead. It would be easy to fall into patterns. As it stands now, I am constantly trying to figure out how to either add more value, or ratchet down the third party costs somehow.

Finally, I think the cost argument is upside down where qualitative is concerned. The senior people in qualitative would be prohibitively expensive to hire on a full-time basis, with the benefits and associated overhead.   Just think of the lost productivity in meetings alone!  

It is great to see the tools getting better to do technology-supported qualitative. I love it. I hope that the price drops, as surely it must if DIY is going to take off. Because feature-creep in online research has led to price creep, when it should be going the other way.

Some of the things raised in this article and others are relevant to the qualitative world as well.
* I wish that I could have more access to the learning from the CRM system when i am in the planning or reporting stages. I think clients themselves are struggling to integrate and effectively use all their many sources of data.
* I wish for better long-term collaboration. The best do it, the rest should.
* I wish for better semantic analysis tools -- not to replace me, but to make the work faster and cheaper.

But the good news in all of this is that clients want and need insights. And they don't want to pay 24 carat prices for fool's gold. And that's a good thing for those who can embrace the new reality.

What do you think about all this? I would love to hear your comments, and I know others would too.

[Hey, if nothing else, you can avoid writing that report. And that's gotta be good, right?]

mobile qualitative - ready for your phone today

This week I am participating in a demonstration of mobile phone texting as a method of gathering insights.

Not a new methodology, incidentally -- people have been using mobile methods for a few years now, but it's really starting to come into its own with the proliferation of smart phones.

In fact, I recall a large-ish student project of a few years ago, (bulletin board methodology), where i gave up using e-mail to communicate with students because they responded so much better to texts!

Back to the demo. Mobile methods for qualitative now come in many flavors, from simple Q+A gathered up into a spreadsheet at the end, to fully integrated texting in a discussion forum platform. 

Here's what I'm noticing, two days into the study:

- Far from being an irritant, I am finding the quick text messages a welcome interruption to my work a couple of times a day.

- I'm curious to see what the researcher is asking next. (Yes, it seems that a good and engaging question is as crucial here as it is anywhere, perhaps even more so!)

- I got a "great idea" response from the researcher that made me smile (the usual pats on the back we give to reinforce participation worked just fine on me too!)

I'll be interested to see the aggregated results of this demo study in a webinar later this week (put on by the Online SIG of QRCA.)

I love the idea of using this approach for a warm-up or follow-up to a more in-depth method. It's in the moment, very unscripted from the standpoint of the respondent, and reaches me where i am right now. The possibilities suggested are really wonderful!

If you have experience with this method -- either as a participant, or as a researcher / client, I'd love to hear your thoughts here in the comments.

Focus groups are a waste of time: or are they?

Researcher Chris Kann, (pictured left) and I spoke a while back about the reasons that people bash focus groups. The result of that discussion was an article that was published in Greenbook, called Don't Blame the Focus Group.

One of the most frequent negative examples trotted out about focus groups is the Herman Miller Aeron Chair story, a story of a very innovative product that was so foreign to consumers when introduced, that initial reactions were not all that positive. This story is often told as a cautionary tale about research -- it shouldn't be.

"That Herman Miller management chose to go ahead with a launch of the chair despite these findings and generated a massively successful product, is taken to mean the focus group did not do its job.

But perhaps this is a story of a focus group doing its job well, and management doing its job well too. Management knew what initial reactions to the new product would be and could plan accordingly. The history of the Aeron chair on the Herman Miller website in fact makes it clear that extensive consumer research, including focus groups, informed the development of the product."

We think it is pretty ironic that companies are invited to listen to consumers more, but only in certain ways.

I'm pretty sympathetic to the challenges of management in client organizations. So is Chris. The truth is, we've both been on the other side. Managers have to make a lot of tough calls, and they hope the research will help. And it often does help. But no research, regardless of method, can save leaders from the pain of making actual decisions, and some decisions will inevitably be wrong.

Bad research won't help your decisions much. But good research will save your hide over and over -- save you from making expensive mistakes, help you see the potholes in the road to success, and point to new opportunities.

Focus groups are not the be-all and end-all. But there's a reason they have stayed around, and it's because they are a good tool in the hands of a skilled researcher.

I'd love to hear your thoughts and experiences.

 

Resources:

Don't blame the focus group for leaders' shortcomings, by Susan Abbott and Chris Shields Kann, Greenbook, 2011.

About Bill Stumpf and Don Chadwick, designers of the Aeron Chair

Company history of the Aeron chair, including video with the designers.

Focus Groups: how can a table and some chairs cause such controversy?

Dan Pallotta created an interesting range of comments in response to his trashing of focus groups on his HBR blog post, Real Leaders Don't Do Focus Groups.

The comments are a much better read than the actual post, which coughs up the same tired mythologies you've already heard a hundred, if not a thousand, times. [e.g. Henry Ford and the "faster horse" soundbite, the "Apple doesn't do research" myth, and more of the same.]

Mr. Pallotta talks about his experience creating a successful charity fundraiser ride as if he created it in a complete vacuum, but it was successful because it was bold.

There are truths to be made on both sides of any controversy.

Truth #1:  Good ideas are sometimes killed with bad research. I've experienced this personally, although I'm bold enough to claim that I haven't contributed the bad research, just some good ideas that died at the hands of others.

Truth #2: Leaders who claim to not need insight research are either idiots, or they have a strong pipeline of connection to customers that they are calling "not research." Sometimes the connection is a simple one -- they are part of the target group personally. More often, in large organizations, a lot of people are actually gathering insights all the time. But they might not call it that.

Truth #3:  The focus group is only as good as the people using it. [aka: It's a poor carpenter who blames his tools.]

A few of the more insightful comments:

From Tony Flanders:

"The new model is --- engage, listen, dialogue, synthesize -- oops; that's actually the old focus group model." 

[Tony has posted an interesting article on focus groups on his blog, here]

From Nick Coster:

"The opportunities for innovation exist when we are able to strip what the market says it wants back to it’s underlying needs.This removes any given solution from the discussion (ie horse or internal combustion engine car) and abstracts it to I need to get from A to B in less that 1 hour."

From Richard Harris: 

"First, congratulations. I think you're the first person to post something on the HBR blog that pissed people off.
Second, I would have been curious to know what market-sensing mechanisms you do use to decide things like how bold to be. Something made you choose 7 days/600 miles/$2000 as your standard. It sounds bold, but you weren't so bold as to go for the round trip. How did you know how far to push the envelope?"

From Matt Towers:

"On the surface, it is sort of a flat Earth mentality that advocates that less information is better than more. It bespeaks a lack of confidence in understanding how to use information to make decisions."

And this gem: "Apple does do research, they just don't talk about it."

From Steve Willson:

"The key point that runs through the discussion is that to be successful an organization needs to listen to it's customers. They are talking, so you need to pay attention.

Whatever methods you use to engage them in a constructive dialog are better than shutting yourself off from the world and emerging with a set of features that are based on what you can build, not on what people want."

Care to add your own experiences? Please do.

Truth, lies and feedback (Part 4 final amazing installment)

I've been posting for a few days on this topic of how to get truthful feedback. Here are the links to Part 1 which poses a question, Part 2 about how to seek face-to-face feedback, and Part 3 which is about a questionnaire virtually designed to elicit falsehood.

Today, I want to look at the most challenging of all truth-finding situations, the police investigator.

This is a situation that makes the work of commercial interviewers of all sorts look like absolute child's play. People generally volunteer to talk to us, most times we actually pay them. The worst downside they are likely to experience is cognitive dissonance or perhaps embarrassment, possibly boredom.

The legal investigator, of course, is faced with someone highly motivated to lie for a number of reasons, social, cultural, psychological and rational. Everything is working against them.

This is why I found a recent presentation I attended by Donna M. Pence, a retired Special Agent who is now an educator, so totally absorbing.

If I was marketing her course, I'd probably call it something like impossible interviews, but she called her topic "When getting the truth is critical: Interviewing techniques used by law enforcement professionals".

It's not like on TV

One of the most interesting things Ms. Pence mentioned was the length of time spent preparing for, setting up, and actually interviewing.

There's the interview-before-the-interview, which is designed to develop a sense of who the subject is, how they respond to non-threatening questions, and even what their general comprehension level is. Then they seek to understand the answer style when under pressure, in a state of fear or anxiety. The protocol she teaches is called PEACE: Preparation, Engagement, Account, Closure, Evaluation.

She was very clear that every question asked has to have a defensible purpose that will withstand questioning in court. No "fishing expeditions" allowed.  This is a key difference from insight  research work -- our questions need to have a purpose, and do, but we are almost always going on a fishing expedition.

When you expect everyone to lie, you plan differently

Like a good insight seeker or facilitator, the good law enforcement interviewer sets the stage in terms of the environment, privacy, clothing, tone and manner, the actual interview space. They consider gender, ethnic and racial matching (matching or deliberately unmatching).

Unlike insight researchers, they are often seeking to disarm or destabilize. We saw her shift roles and accents like a skilled actor, and honestly, I was really glad I wasn't interacting with her professionally, so to speak!

The level of detail required to support criminal charges is, in her words, "excruciating". So one thing that is probably true from TV is the phrase "let's go over this again".

I truly loved some of her phrasing. Let me give you an example:

"I've never called anybody a liar. I say, 'I think you're not being as truthful as you could be.'  Or for a good ole boy, 'That dawg won't hunt'"

My top takeaways

These are approaches that we can all learn from, whether you are interviewing, or just trying to be a better communicator.

[1] Cognitive interviewing -- show the subject that you see them as an individual with a unique set of needs. In turn, present yourself as an identifiable person. Depersonalizing reduces the amount of information you will obtain/receive.

[2] Interact meaningfully -- don't just ask a list of predetermined questions.

[3] Communicate empathy -- see the situation from their point of view.

[4] Give your undivided attention -- convey that you want to listen and can be trusted. Give reassurance and support.

[5] Let them talk until they run out of steam.

Why does anyone talk to them?

The most amazing part, for me, was Ms. Pence's answer to my question of why anyone really talks to them, despite her persuasive methods. Her answer provided telling insight into the human condition.

For the most part, she says her tone is clear, calm, slow and matter-of-fact. She has often interviewed people who have violated foundational cultural taboos, such as child molesters, rapists, murderers.  She says things like, "You can't offend, embarass or surprise me", "I'm not going to be upset at anything you tell me. I've talked to many people in your situation".

While people are initially motivated to talk to get themselves out of trouble, if possible, they also want to give their side of the story, to provide an explanation for what happened. 

My take

We all live the existential problem -- we are alone in our own experience. The ability to share that inner world -- even a little bit -- is a powerful drug. I often wondered how law enforcement officers get people to tell them the truth, and I think this is the essence of it: empathy.

And on a less sobering note, if you meet someone like Ms. Pence in a police station, you are in serious big trouble, because she is one amazing interviewer.

Resources

Donna Pence recommends the book, Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage, by Dr. Paul Eckman

All of the graphics in this post were created by me, based on diagrams presented by Ms. Pence.

I heard Ms. Pence speak at the annual QRCA conference. If you love qualitative methods, you should join us sometime.

Common research mistakes

I logged in to Constant Contact a few minutes ago to start writing a newsletter. There was a button for completing a survey, so being a naturally curious [some might say snoopy!] person, I clicked.
Constant Contact now provides a survey platform. Sadly, they made several common mistakes. So egregious and common are these errors, it seemed essential to post about it and save others from the same mistakes.

This is a screen shot of the first page of the survey. I see three mistakes here:

First mistake -- ask for my name right up front. If you are going to make this non-confidential, you might want to ask for this information in the last question. Because then if I don't answer, you still get the rest of my data.

Second mistake -- non-exclusive categories. If I am in business 6 years, I have two choices. Same problem with 8 of the response categories. Oddly, the sequence changes from 8-9, and 10+.

Third mistake -- the person who wrote this survey has insufficient knowledge for the task. Two easy solutions. Either take a survey writing course -- there are lots of them around, and some are available online. Or, spend a small amount of money and hire a researcher to write the questionnaire for you. Seriously, this is not a complex questionnaire. It would be likely a half-day of work for an experienced person.

Or, you can just get poor data and look like a goof to the world.

You know, the thing is, the fact that I can use a power saw does not make me a carpenter.

Free e-book: Painless Insight Planning helps you set objectives, avoid wasting money on qualitative research

 

I've refined a number of tools to use in my work over the years. A few of them are tools designed to help clients in one way or another. This free e-book, Painless Insight Planning, is one of these tools with some narrative wrapped around it.

[Download Painless Insight Planning now]

One thing I've found that is often a challenge at the start of a project is getting clear objectives. People may have a problem or concern, but it is often not clearly articulated.

And I can tell you one thing for certain: a good way to waste money on research is not having clear objectives at the beginning.  I know this sounds perfectly obvious, but it is an issue more often than you would suspect.

Sometimes the problem is clear, but other elements are not. Such as -- who could we get insight from, otherwise known as "the research target."

I can't sit down with everyone individually and work through this, as I do with clients. So I thought I would put the whole thing into an e-book that you can download and use as much as you want. And then you'll have clear objectives for your research projects. And the world will be in a better place.

Because we really need insights these days, more than ever. And people inside companies can't afford to waste money or appear to waste money, get fired and be unemployed. Suppliers like me also can't afford to waste client's money. We all need to ramp UP on the insight quality and the innovation / ideation, but keep a lid on costs. 

What's it worth to get this answer?

An even harder thing for many clients is figuring out how much to spend. They ask me, what will it cost? 

WRONG QUESTION!  WRONG QUESTION!  WRONG QUESTION! 

The right question is "what is it worth to get good information on this situation?"

I have created a nice little rule to help figure this out. Not a complicated ROI on research thing. Just a back-of-the-envelope, cocktail-napkin kind of formula that I call the 10 Times Value Rule. It's a great place to start figuring out what you should consider spending to get some insight into your problem / challenge / situation.

[Download Painless Insight Planning now]

Easy worksheets

I hate filling in forms. Most people do. So there aren't forms in this book. There are easy worksheets designed to spur creative thinking to get you started. Here's an example.

Bonus content

If you download the book and actually read it, you'll find a link to some bonus content that will save you even more time. 

A small request - no strings attached

I'm simply putting this handy e-book out there for free with no strings attached. You can even add this e-book (with no changes) to your own site or blog for your own customers and prospects to use.  But I do have a small request.

If you think this e-book is of any value, please add this blog to your RSS feed reader. I also send out a newsletter about six times a year. You can add yourself to that list if you want to hear about other e-books I plan to release, or just get the newsletter. You can check out the previous issues here.

Thanks for reading. And good luck with your insights research projects!

[Download Painless Insight Planning now]

Listening right vs. listening wrong: Google Latitude and sampling

You've likely heard about a new service Google has launched called Latitude, that can enable you and your pals to track each other geographically via your mobile phones. There's a bit of controversy about the potential abuse of the service for stalking, but that's not what I want to talk about. At least not directly. I tested this concept a couple of years ago in research. Let me tell you about it.

About two years ago, I was planning a session with some grad students in a research program. You know the drill -- guest instructors who are practitioners come in and show their stuff. So I did an advance study with the group using an online bulletin board method. Then we talked about it in the class.

To try to find a topic of interest to all the students of various ages, I picked cell phones. Among other things, I cooked up a number of possible features and enhanced services that we concept tested. One of them I called "Friend Finder", which was essentially a product like Latitude.

I thought this Friend Finder concept was pretty clever, actually, and that if I tested such a product concept with a broad base of young adults, that it would likely be well received. I was really surprised when this group of research students reacted pretty badly to the idea, and were REALLY worried about privacy. Freaked-out and creeped-out about privacy, actually.

They also didn't like "Taxi Finder", something I personally would love. And they didn't like "Bus Finder". I don't think there was one concept that tested well.

So I pondered this finding ... and here's what I think.

This was not an average slice of humanity in any respect. Most of these people were into quantitative i.e. they enjoy statistics. A number of them had just finished a four-year degree and were running out of money. Several were mid-life career changers who were also pretty cautious about their spending. And they were in a small city on the edge of the Canadian Shield (lots of big rocks) and they told me cell coverage was often spotty. [Who knew? When I go up there to canoe, I'm pretty much ignoring my cell phone.]

And it was not a well targeted slice either.  A number of the students turned out not to have a cell phone at all, or used it very sparingly. In a real study, the client most likely would have been looking to gain insights into only those people who were already pretty keen on their cell phones and were already using some advanced features.

My colleague that day, Tania Heintzman, was personally VERY keen on Taxi Finder. But she uses taxis a lot and students don't. Off target.

I personally was very keen on Bus Finder, but the bus service in small cities is often terrible and so irrelevant to this group who all drove to campus. Off target again.

Basically, researching any number of the wrong people does not help. A bigger sample of the wrong people does not help. Statistical significance would not get you out of this issue either. The wrong sample is the wrong sample.

Sometimes only a few of the exactly right people can help you dive very very deep into powerful insight. So you really need to figure out who those right people are. Especially when you are trying to see into the future with new concepts for products.

If something like this happened in a real project, it would be a mistake to conclude that the concepts had no potential market. Because clearly, Latitude will find users. And all sorts of people are developing open-source products like my "Bus Finder" idea, even here in the frozen wastelands of Toronto. Soon it will be available via smart phone apps. 

On the other hand, some of the learning would have been valid. For example, the concern about privacy. That's an attitudinal and behavioral segment that you would want to understand. Even knowing about this issue would help you design a better service, and avoid the criticisms Google is getting now.

Innovation breakthroughs: now really IS the time or why recycling does not work for insights.

 

People are cutting back. [Oh, you heard that, too?]

I've had a couple of projects cancelled or postponed indefinitely in the last short while. One of them was to help an organization identify new markets so they could escape the commodity price-cycle trap of their current markets.

This is exactly the kind of project you definitely should not cancel -- projects that will help you crack new revenue sources. So what will this company do instead? I'm guessing, they will just push harder at existing markets, and find themselves in the same place a year or two down the road.

While having these thoughts, I bumped into a nice article in Marketing Profs. [Now that the economy is tanking, maybe we can catch up on our reading.] The quote from Jeffrey Immelt leapt off the page.

General Electric CEO Jeffrey R. Immelt has established a class of projects now well-known as "imagination breakthroughs," consisting of ideas that are "really hard or really important" and might generate significant revenues over a three-year period, the time that GE usually takes to implement a new idea. "Imagination breakthroughs are a protected class of ideas — safe from the budget slashers because I've blessed each one."

If it takes you a couple of years or more to really get an idea off the ground, then the fuzzy front end you cut now will leave you in even more serious glue in two to three years time. And if you kill the stuff already in the pipeline, you're not going to have anything new to launch in 2009 either.

I really wish clients would say instead: "Can you get me something substantive with less money?" That's a better challenge than dropping the idea creation process.

Caveat

The article this came from is filled with good ideas from the Zaltmans. But like many of their ilk, they firmly believe that no approach other than their technique (individual metaphorical interviews) can get to the good stuff. Not true, in my experience.

"These deeper ideas are not available from survey responses to statements such as "financial planning is difficult" or in focus groups where each participant has on average about 10 minutes of air time." 

OK, you are definitely not going to get these deep meanings from a survey. I agree. But individual interviews are not the gold standard either.

There may still be people running focus groups with 12 bodies in them. However, most of the kids I hang with have shifted to small groups when they are using a group discussion method. Small being five or six. And researchers frequently add advance activities such as blogging / electronic diaries / collage assignments / videos / you name it. And there's no rule about 90 minutes or two hours either -- you can keep people humming along for three hours quite easily. And yes, you can uncover metaphorical meanings in focus groups, in online research methods, and in many other ways.

In truth, the generation of insight has a lot more to do with the skill, knowledge and art inside the researcher's head than the method employed. There's no method going that guarantees insight.

As anthropologist Grant McCracken has said, the proof is in the pudding. If you got insight, you got insight. If you didn't, you didn't.

I have friends in the marketing world who have used Gerry's Zmet technique to great effect. And I believe all of methodological infighting and territory claiming must defer to this. If the method works, the method is good. The proof of the pudding has nothing to do with the theory of the pudding or the method of the pudding. The proof of the pudding is a client who says: "This was illuminating. I understand my consumer and my market in ways that I did not, could not before."

Back to the main point

This is no time to be cutting your future revenue streams by shutting down the insight machine. In fact, if you plan to be standing when things turn around, you should be cranking up the insight machine. Certainly, your customers and consumers brains are grinding away on many things right now. Not a good time to recycle some old insights.

Resources:

How to Foster "Workable Wondering", by Gerald Zaltman and Linday Zaltman. In marketing profs, reprinted from HBS.

The Zaltman Method, by Grant McCracken in his blog, 2006.

Cool new tools that enable feedback

I went looking for one thing online, and found a bunch of other stuff. Don't you love doing that on a Friday morning? So of course I wanted to share...

There are just such a variety of interesting online applications for assisting collaboration now. If you can add some others to the list, please do. If you have used any of these tools and have comments about them, please add your thoughts as well.

Mapanno Markup

This markup tool lets you invite comments from others on an image or a project. It feels a lot like Google Maps, where you can add an icon and comments, and stick it to a specific place on the document. Here's a screen shot that gives an idea...

Review Basics

I have actually tried out this tool on a demo basis at least a year ago, and it looks even better now. like Mapanno, it is currently free. But it has a broader range of functionality, and was designed more specifically with marketing folk in mind.

Here's a screen shot of what the markup environment looks like. There are bubbles that you can use to enter text and a variety of marking tools as well.

 

Review Basics also supports common video formats, as well as Word, PowerPoint, and PDF files.
I'm itching to try this out with a research project, but I think it could be really useful on a team internal basis as well. Right now, it's free, just like Mapanno.

Vyew
Vyew is one I just learned about, thanks to GeekBlog. I haven't had time to demo it at all, but it positions itself as a competitor to WebEx and GoToMeeting. In addition to those services, however, there is the potential for asynchronous involvement by multiple reviewers who can mark up documents, and view each other's comments.

Here's a comparison chart that may help.

There are some interesting case studies on the site as well, that make it clearer how one might use such a multi-featured application for things other than web conferencing. Like the other tools mentioned, it is currently FREE, and does not require software installation. There's a review by Guy Kawasaki

Concept Share

This service is specifically targeted to graphic designers. I haven't tried it out, but it looks pretty appealing. There is a free-premium model in place which affects the number of projects you can use, and the amount of online storage you get. Huge amounts of storage are available, which is needed for graphic files.

 

Basecamp

I've been using Basecamp for a few months now. Although it's not perfect, it's a very useful and low cost collaboration tool. I haven't figured out a way to use it for research projects yet. But you can store files, there are whiteboards, a really good messaging function that beats running things on e-mail, and some project management tools.

This stuff all just continues the flattening process where little tiny organizations have access to tools and technology as good as the stuff the big organization's have. Sometimes it's even better.

If you want to read about even more of these tools, here's a reference article from GeekBlog. And their handy comparison table.

Global Village Study on Teens and Technology

About this time last year I was helping to plan a global study on teens and technology use, called Global Village. This idea was cooked up by my colleagues Ilka Kuhagen (Germany) and Corette Haf (South Africa) as a way to demonstrate the power of using bulletin board research to gather global insights.

We sponsored this research as a group, and conducted the research with 12 groups. This was an impressive team of researchers, it was my honor to work with them. 

During the planning process, we had many logistical issues to work through. For example, our platform sponsor, Focus Forums, had to add Mandarin character capabilities to the bulletin board.

A decision was reached to focus our efforts on teens aged 14 - 16 years old. A challenging target in some ways, as ethical guidelines require parental permission to conduct research with this age group, which added to the logistics of the project. But we saw that this group is entering young adulthood with access to a level of technology that they have enjoyed since childhood. We wanted to see where things were heading with this group. So we found teens that had a cell-phone and used the internet relatively heavily.

The bulletin board was a fantastic tool for the project, and the teens loved the format. Age and gender barriers that would have been a major burden in a face-to-face setting evaporated, and we had high engagement and considerable openness and honesty.

In addition to this, the research team was also easily able to watch the research unfold, and share our own discussion about questioning strategies over the project schedule. This meant we could change and shift the later stages of the project based on earlier stages.

It was fascinating to see the style differences among the research team. As findings started to emerge, we were all impressed by how similar, in many ways, the attitudes of the teens were in each country. In picture sort exercises, for example, the same concepts -- and even the same images -- came up over and over again. We weren't really expecting this, it was a fascinating finding.

One thing is for certain -- these teens are joined at the hip to their cell-phones, and this is their life-line. If their house was burning down, it's one of the things they'd grab. The research team found text messaging a much more effective way to send out reminders to participate. (With an adult group, we would use e-mail). Almost none of the teens in the whole study had heard of Second Life. A few had been on Habbo Hotel, but these sites no longer held  -- they are way too busy forming real-world relationships to care too much about the virtual world. Yet.

Overall, a fascinating group of teens. The Canadian group were very much "the New Phoenicians" -- ethnically diverse, open to new ideas, enterprising and optimistic. I wanted to give them all a virtual hug, they made me feel proud to be Canadian. I have no doubt the other researchers felt the same about their own youth.

I'll be presenting on this project with my colleague Betsy Leichliter from New York at the ARF conference next week, and also in Winnipeg at the MRIA conference in May. Other team members are presenting at various conferences around the world, including the upcoming AQR/QRCA conference in Barcelona. The slide at the top came from a presentation done by Diva Oliveira in Brazil.
 

You can also read the write-up that Ilka and Corette did for the QRCA publication VIEWS, which is available free online. How Global is Your Village?  (the link spawns a PDF. Article is on pages 18 - 27)

You know advertising is hot when it's displacing IT among IT research houses

I always thought of Forrester as an IT-oriented research house. And then they were on stuff like supply chains and outsourcing. However, they are now getting talked about for their thought pieces on where advertising agencies are going.

This suggests two things to me:

1. Forrester is good at figuring out what's hot
2. Marketing, advertising, and brand building are hotter than they have ever been, but also under more scrutiny than ever.

I have other evidence of this trend.

The Globe and Mail, my daily must-read newspaper, never deemed marketing topics worthy of business coverage. [Well, any kind of coverage, actually.]  Not long ago -- within the last 18 months for sure -- they have added regular material on this topic. It's thin, but it's there.

Friday, I heard Terry O'Reilly speak at a conference on qualitative research. Mr.O' Reilly is a regular feature on CBC radio, and known for his expertise in advertising, as both a creator and a commentator. Here's one of the notable things he said on the topic of insight and advertising:

"Creative ideas born of intuition are the last legal means of gaining a competitive advantage over your competition."

I thought this was a) brilliant, and b) true.   

Forrester seems to think that advertising agencies are not helping marketing people connect with consumers and customers in a meaningful way. This is the summary of their recent research report, The Connected Agency: Partner With An Agency That Listens Instead of Shouts

"Today's agencies fail to help marketers engage with consumers, who, as a result, are becoming less brand-loyal and more trusting of each other. To turn the tide, marketers will move to the Connected Agency — one that shifts: from making messages to nurturing consumer connections; from delivering push to creating pull interactions; and from orchestrating campaigns to facilitating conversations. Over the next five years, traditional agencies will make this shift; they will start by connecting with consumer communities and will eventually become an integral part of them."

I have not read the whole report, but I am having trouble seeing this as news. And it's far too easy to just blame the agencies, who are under tremendous pressure to deliver results these days. But they can only do what they get a mandate and a budget to do from the marketing people. Surely marketers themselves must take responsibility for connecting with consumers.

And so must executives.

Being in the insight and intuition business myself, I can only agree with Mr. O' Reilly: we need better strategies. We need insights. We need innovation in marketing and advertising.

At the same conference where Mr. O' Reilly was a headliner, I presented with a client a case study on some innovative research we did together. When we got to the Q&A, the biggest question that came up -- which we brainstormed as a group -- was HOW to convince executives to do something less conventional than a focus group or a survey.

These researchers and marketers -- they want to deliver innovation, they want to deliver category-changing insight, but feel challenged to sell it into the organization.

We came up with a flip-chart full of ideas of how to do this. Things like: start small, start with intractable problems where there is motivation to try something different, and so on.

But as one of the research buyers said, "We want innovation, we want to do something new, and we want amazing insights, but we only want it once it's been done before. How do you sell this into the organization?"

Of course, we on the practitioner side have another challenge. Having promised these category-changing insights, we need to stand and deliver.

I only wish these insights that can change the landscape could be produced on demand. [I'll be hunting for a good one this week, wish me luck.]

Facebook Polling: a quick tour and a surprising finding

After holding forth about the future business applications of Facebook, I decided I should put my money where my mouth is, and do a quick little research study.
Facebook makes that easy for you with their polling feature.

Here's what I wanted to know -- what kinds of applications for business would interest people who are already on Facebook? So here's the question I asked:

What application for business would you be most likely to use on Facebook?
[a] A way to post RFPs / find suppliers?
[b] A way to bid on RFPs / find projects?
[c] A project team collaboration function / group?
[d] A way to sell content like e-books / e-learning?
[e] I would not use any of these

Facebook polling is so inexpensive, I thought I could afford to get 100 responses, which would cost about $25.00. [I do want to be clear that this is not a scientific poll, it's just a way to take a quick read on something.]

I couldn't select exactly the group I wanted -- freelancers and SMB people. So I went for age as a proxy, and asked for age 35 - 49. I was surprised that the oldest selection was age 49, actually, since surely there are many people using Facebook over the age of 49. I know several well, in fact.

I sat excitedly at my computer waiting for the colorful dashboard to show results. Which it did, looking like the chart above, with several other colorful charts breaking down my poll by age and gender.

Surprising Finding

When I had 49 responses, Facebook shut down the poll.

Before they did that, here's what I learned:

1. About half would be interested in one of the proposed functions
2. About half indicated no interest in any of them
3. The most popular of the proposed functions was [c], the project team collaboration function. This is the one I want too.

I received an e-mail from Facebook explaining that they had shut down my poll.

Hi, Susan,

Facebook Polls are intended for market research purposes. Facebook Polls created to promote products or services or that are otherwise deemed not appropriate are removed accordingly. Along these lines, Facebook has automatically removed your Facebook Polls (order #Facebook Poll).

To see the final results, go to: link 

Responses Desired: 100

Actual Responses Received: 40

Price per Response: $0.25

Your credit card has been charged an additional $10.00 This completes your order.

The Facebook team

I'm surprised that they thought this was promotion. So maybe it was deemed otherwise inappropriate. Perhaps the only people allowed to do research on Facebook itself are officially authorized. Perhaps using the word Facebook in the poll flagged my little project. As far as I know, they can't stop me from blogging about it.

Anyone have some insight into this?

I was thinking it would be cool to hire some savvy Elance developer to just make me the application I want, but now I'm reassessing.

Holiday notice

I'll be skiing in Banff. If you're there, say hi. I'll be the person doing the face-plant right in front of the lift line, wearing a fuzzy hat with ear flaps [that looks really cute, actually]. So I likely won't have the strength to post after a hard day of skiing. See you in the new year! And thanks for all the reading and the comments, which really make it fun to keep at it.

Jason Oke talks about what's wrong with qualitative research

Jason Oke is a planner for Leo Burnett -- a guy who spends a lot of time thinking about finding leverage points for the creation of great advertising. And is, incidentally, an excellent speaker. He thinks there's a lot of bad research being done, which has created a credibility problem.

Chief among his complaints is that there is far too much direct, literal Q&A going on, which we know -- I mean we really know this -- is not very useful. This kind of technique lacks so much subtlety, it is akin to "strip mining their heads," according to Mr.Oke.

By literal Q&A I mean approaches like asking people to tell us "what's important to you in a vodka", which will generate nothing that can help a vodka marketer. If the target group even knows, which is unlikely, they are hardly going to say things like, "I want to feel like part of an exclusive club" or "I want a wee boost to my social status."

This is not just true for products like vodka, by the way. I just completed a business study in which one group of people were clearly motivated by reducing their fear level. Now just how likely do you think they would be to [a] know that and [b] say it out loud? In fact at least one of them did know that, but she didn't say it, she wrote it down as part of an exercise. [Of course this creates an equally large challenge for the creative team, but that's another article.]

Jason Oke also believes that people are bored by the idea of sitting in a dimly lit room for two hours being poked at with inane questions and limp exercises. He thinks this is a major reason why it is getting difficult, and expensive, to recruit participants for focus groups. This was a new idea for me, and I think he's on to something.

Mr. Oke's recipe to turn the situation around:

[1] Take brain science into account.

People can tell us many things, but they don't have direct access to unconscious motivators. When we ask them questions depending on this access, they will tell us something, but it's unlikely to relate to the problem at hand. It may not even be true. But most people will generate an answer to a question like "what's important to you in a vodka" just because people are very good at generating reasons.

[2] Plan research activities that meet the needs of #1, and are also fun and engaging.

Mr. Oke's solution to the Vodka problem, for example, is to ask small groups of people to design a new vodka and how it will be marketed to people. Exactly. Generates great data, and is a lot of fun for the participants. He showed us clips from the tape, it was brilliant.

[3] Stand up to clients when they ask for bad research. Evangelize for good research.

The thing is, once clients have experienced really good research, they are left unsatisfied by the other kind. At least I think that's the case. I hope and pray that is the case. But it feels risky the first time you get on.

[4] Think of research as a brand touch-point.

Absolutely correct, and great advice.

Resources:

Jason Okes' blog is here

A related Malcolm Gladwell blog post is here

Clips from the Prelinger Archive film about marketing research, here

Book he recommends [I have not read, but it sounds great]: Herd by Mark Earls.

Mr. Oke presented to the qualitative division of MRIA at "Qual in the Fall" on Nov. 9, 2007 in Toronto.

Rethinking research

I'm off to the QRCA conference for the week in Vancouver this year. My mission, beyond catching up with friends and colleagues, is to try to really push the boundaries of how I think about qualitative research.
The reason is: fieldwork costs keep going up, and are going up faster than client budgets are. The whole infrastructure that surrounds qualitative research -- lists, panels, recruiters, facilities, catering, incentives -- is a large part of what is costing a fortune. It ain't me.

One of the worries about rising incentive costs is that this encourages people to become professional respondents, supplementing their income by participating in focus groups and interviews. Surely we have not tapped out the adult population of North America. There must be people who would like to share their opinions without being paid a huge sum to do it. Or are people so exhausted from constant requests to complete poor/badly written/excessively long surveys that qualitative is just one more intrusion?

I think there must be other ways of finding people -- perhaps Facebook? Is anyone finding respondents on Facebook?

I'm hoping to get inspired for a few days. Pick up a few new ideas. And challenge my own assumptions about how things should be done.

And, according to my own personal tradition, I'll keep you posted about anything interesting that I see or hear there.

Designing Experiences: prototype, pilot, tweak and adjust

James Rait posted some interesting commentary in Design-at-the-Edge in response to my Curves pieces from last week. Mr. Rait's wife is a Curves member, at a location where they sometimes give away doughnuts. [I know, it's hilarious, except it illustrates just how difficult it is to manage services with consistency to standards.]

I had outlined some possible backfires to the adoption of RFID tags to plan and track individual goals and progress. These were: [1] staff having to take their own initiative to interact with clients, instead of being forced to do it by the operational routine; [2] two classes of member emerging; [3] clients losing the happy ritual of taking their pulse rate together.

Mr. Rait, a guy with serious industrial design credentials, proposes some interesting alternative solutions to these issues in his post, here.

He also suggests that running some live market tests would have uncovered the potential issues I mentioned. I would be amazed if they had not done live market tests, actually. The execs at Curves have strong marketing backgrounds, and with 10,000-plus locations, I assume their budget supports quite a bit of research.

Sometimes it takes a while to understand what is going on. Plus the hype surrounding new technology can generate enough excitement to mask the longer-term impact. These phenomenon would have been unlikely to come to light in a short test. And if the test markets were conducted with the very best franchises -- which is usually the case -- these individuals may well have responded differently than the average.

I suspect the social evolution is not yet done. [There's lots of observing still to do. Yippee!]

Mr. Rait's post made some other interesting points about design in general, which I'll pick up tomorrow.

If you go back to read the Curves posts, note the passionate comments from one fan of the company. This kind of evangelism is a huge asset to a brand.

The photo came from a film in the Prelinger Archive, Exercise and Health, 1949.

New media measurement tools: Xuuk

Eye-tracking technology is cool, but it's always been a kind of headset-and-wires operation, not something you could use in a real advertising setting.

Xuuk has launched a gizmo that can track eyeballs for posters, billboards and displays. It's a low-end disruptor -- and although it is unlikely to have all the functionality of laboratory units -- it's extremely cheap at $999. And it potentially brings a whole new group of customers into the market.

Xuuk claims to be able to detect eye-balls looking at it from up to 10 meters away. It plugs into your USB port, so you can look at statistics.

Like many technology crossovers born in academia, Xuuk could use a little marketing support to make their message clearer. But don't let that stop you. This baby could be great tool to refine your merchandising and visuals for those times when you don't have a big budget available.

What we can learn from squirrels, and other research tips

I planted my garden over the last few weekends. A few wildflowers native to this part of the world (trillium, jack-in-the-pulpit), some herbs, and the usual impatiens, fuchsia, and such. [Yes, it's true, I do have a few domestic skills.]

For days after a new plant goes in, it's a certainty that the squirrels will dig it up. I replant every morning. This morning it was the hanging planter of morning glories.
Why do squirrels do this?

It came to me this morning. They do this because they can smell something in the freshly dug dirt. They know it has recently been disturbed. And this is a good place to look for snacks -- perhaps nuts that other squirrels have buried. It must be, otherwise why would they bother?

I've been annoyed with this behavior for years, but have never stopped to de-construct it. From a squirrel's point of view, it makes perfect sense. 

And so often this is what we need to  do when we study customers. Especially when their behavior seems irrational. Because behavior is not random. It is purposeful. If it is random, the randomness itself is purposeful. It may not be rational in the conventional sense, but it achieves a result, it tips the odds, or it has been rewarded in the past. There are reasons, there are patterns, even if the patterns are unconscious, or seemingly irrelevant.

We cannot change our customers. But we can understand them if we work at it.

More collaboration tools - maybe for research?

One of the joys of having a moderately successful blog is that I now get requests from people to check out their a) book b) new research application c) creative marketing or promotional strategy.
[One could easily do this full time, if said one were independently wealthy....]
In any event, some worthy candidates have passed by recently. Here's one I want to tell you about.

ReviewBasics

ReviewBasics is a feedback gathering platform which allows professionals across many industries to exchange ideas, gather thoughts, and review various types of creative, interactive, motion, and written content. We take a familiar, paper-based content review process and bring it online." [From a news release sent to me.]

I spoke to Tim Shih today, who is VP and sales manager for this new company. Mr. Shih says ReviewBasics is, "The next generation of tools for the online marketing research industry."

The company was started by a technology consultant and a product developer, both with experience in the e-learning space. They saw an opportunity for a rich feedback platform to support collaboration and research, and this is the result. The development team is in Russia, but Mr. Shih and team are based in New York.

Going for a test drive

I spent some time on their beta site, which is very nicely set up. I particularly liked their easy-to-follow graphic walk-through (see the screen shot below). I'd love to see more applications that do this.

This next shot shows the basic interface. You have a page where you put up an image, a graphic or whatever. This could be a screen shot of your web page, or creative concept, or a new packaging design.

You then invite some people to review what you have put up. Reviewers can mark items with arrows, text boxes, call-outs, emoticons and similar tools. There is also a space at the bottom right where they can provide generalized feedback in an open input box.

All this input is gathered together for the researcher to review, but the reviewers cannot see each others' comments.

Application in marketing research

For creative testing, we usually want to have a fair bit of control over the exposure time and conditions for the stimuli. So I don't see this being the starting point for that. But it could be a nice add-on at the end of an online questionnaire, for example.

I could also see using something like this to gather ongoing feedback from users of an application in something like B2B settings. 

I'm less sure about consumer research, but for the right project, maybe it would work. It has some of the features of a web-meeting interface, but it's asynchronous, so everyone doesn't need to show up at the same time.

What happens with this type of tool is that clever people figure out innovative methods once they get a look at it and start fooling around with it. As researcher Betsy Leichliter once said to me, and I'm paraphrasing: "A focus group room is just a room with some chairs. You can do a lot of things in a room with some chairs. Online tools are no different."

Breeze was started as an educational tool that could leverage Macromedia Flash for easy e-learning presentations. Now, the underlying Breeze platform is used for a variety of different types of applications, including some online qualitative marketing research tools.

You can still participate in the free beta test for ReviewBasics, by going to their site and clicking on the "RB – Click to Review Button” near the bottom of their homepage.

Getting Your Users to Tell You What to Do

Can your users tell you what to build / fix / enhance?
Sometimes.

We're seeing a lot more instances of some kind of contest or voting process where users submit ideas, and other users vote on their favorites. To see a current example of what I'm talking about, check out this contest being run by SNAP.

SNAP is a visual search engine that allows you to preview the search results without actually having to browse to the page. Pretty cool, actually. They are asking users to tell them what to build / fix / enhance in their next round of developments. And offering prizes for the best ideas, and for the best comments on ideas others have submitted.

Will we see more of this... i.e. Should I be worried?

Yes, I think we will see more of this. It's a neat way to really open up your development or innovation process to your user community. For a relatively small cost, you can enlist a lot of good ideas. 

I'm not sure what platform people are building this type of application on [comments anyone?], so I don't really know how hard the interface is to set up, but I'm guessing it's not worse than launching a bulletin board or some similar interactive application.

Why I'm not worried

This kind of approach makes a lot of sense when you have users that are fans, and that have enough knowledge to make suggestions. If you browse the contest, you will see that many of the individuals seem to have programming knowledge far beyond the average surfer. Personally, I didn't actually understand some of the suggestions.

There is a sampling issue here -- this kind of input only reaches the raving fan, or those that enter contests as a hobby. If these are your core users, then you are in great shape. If not, you will need to find other ways to tap into the customer experience.

Finally, this approach actually asks the users to suggest new features i.e. solve the problem. A more effective strategy for sustaining innovation is actually asking your customers to tell you about their problems, or tell you about their experiences, not ask them to solve the problem.  When your own development team has a deep understanding of customer experience -- including all the joys and the pain points -- they should be in a better position to articulate possible solutions.

The bottom line
I love triangulation -- where you look at a situation from as many angles as you can. And this kind of user-input is a great adjunct to anything else you might be doing to reach your users. As a nice side-benefit, it will likely also create some buzz, and engage the hearts and minds of at least some of your community. And that's all good.

Enquiring minds want to know
Like the Joker said of Batman, "Where does he get those toys!?", enquiring minds would like to know how to launch one of these contests with a vote. Is this kind of application available off the shelf, or is it a custom programming job?

Customer Driven Innovation

At a qualitative research conference a while back, I attended a session on using games to support innovation. The people you see above are other researchers, holding up their "product box". [Sorry the pics are so small, all I had was the camera phone. And it's taken me this long to get the darn pics out of the phone, but that's another story!]

This method was laid out in Luke Hohmann's book, Innovation Games, and this was a nice chance to see it in action. Tables were given lots of materials to work with, and a plain white box. The task was to create a box for an alarm clock that they would like to buy.

Participants came up with lots of interesting ideas. For example, at my table we wanted an alarm clock that knows what time it is, just like a cell-phone does. A smart clock.

Many people mentioned Apple inspired design -- features and controls so easy to use you don't actually need to read the manual, because it's intuitive. Can you tell that this group spends a lot of time in hotel rooms, trying to set yet another clock radio, or figure out what the current local time is?

What I found especially interesting about the exercise was that the room seemed to divide into two larger groups: those who were concerned about clock functionality, and those concerned about managing the morning mood and ambiance. Could these be segments, I wonder?

During the debrief, the facilitators asked multiple questions of each spokesperson, to uncover the needs and ideas around the benefits sought.

You'll find lots of people who will tell you that "focus groups can't lead to innovation", and other blah, blah of that nature. I say it depends a lot on who is running the event. If you ask customers literally "what should we do?", you may not get that much you can use. 

But the joy of this exercise was that it focused people on unmet needs, and benefits they would derive from the features they proposed. A development team could definitely learn a lot from the output of that.  And the act of creating the box helped all of us get past "design issues" and focus on what we'd really love to have.

References:
Summary of keynote speaker Paco Underhill from the same conference, talking about global trends in retail.

This method first came to my attention through Luke Hohmann's book, Innovation Games.

 

Qualitative Research Conference: Toronto October 2006

Why you need rubber soled shoes

It's cheaper and faster than ever to collect reams of information: it's wisdom in using this information that is in short supply. And you really should have some rubber soled shoes.

These were some of the things Paco Underhill talked about in his keynote last week at the joint QRCA/MRIA-QRD conference. Paco is the hugely successful analyst of shopping behaviors and author of very successful books on the subject, Why We Buy and Call of the Mall. 

Inner Context of the Shopper

One of the most interesting observations he made is that we all shop more often, even though we live in a state of accelerated time. We go into a store with a specific mission, and if we don't fulfil it, may leave empty handed.  That mission, even for the same person in the same store in the space of the same week, could have very different characteristics, based on the value system and emotion at play due to the inner context.

For example, shopping for a child's birthday cake, shopping for tonight's dinner, and shopping for the weekend when the relatives you dislike will be visiting. And in some respects, we need to treat each of those scenarios as if we were dealing with a different customer.

The Key Question

The key question, for Underhill, is always: "how do I respond to local market conditions and build same store sales?"  Depending where in the world you are, these challenges are vastly different.

When speaking of the global middle class, 2/3 of our expenditures are discretionary. We don't need to spend money on much -- someone has to entice us to spend.

In other parts of the world, money is a new and exciting toy, and that's where all the innovation is happening in retail.  Underhill shared some excellent examples of this.

Roppongi Hills, Tokyo

Underhill described Roppongi Hills as the "All Mall". It has hotels, an art gallery, residential towers, and a multitude of recreation facilities. One of the unusual elements is the large amount of public art that has been included throughout the facility, both inside and outside.

Creative use of space in Johannesburg

What you are looking at here is a drive-in movie theatre on the roof of a shopping mall (the Menlyn). A great use of space, and a good way to bring more traffic into the mall.

Another example cited by Underhill was a mall owner who built a stadium for student athletic activities on adjacent land, solely to drive traffic into the food court of the mall.

Forget about Parking in Sao Paolo

In order to bring traffic into the mall from the nearby office towers, this mall offers a free shuttle bus service that will pick people up and then run them back in complete safety so they can shop over their lunch hour.

Finding Happiness in Tokyo

Three Minutes of Happiness is the name of a chain of stores in Tokyo that sell inexpensive items for everyday use. Here's the description from Frommer's guide:

Bargain yen shops have opened all over Japan the past few years, but this is one of the best I've seen. It carries tableware, household goods, office supplies, cosmetics, watches, sunglasses, some basic clothing, and many other simple items, mostly in bright and happy colors like lime green and sky blue.

Although these shops seem very well designed, they are reminiscent of one of the fastest growing categories of shopping at home: the dollar store, and other deep discount formats, that are the antithesis of good design.

Dubai, Dubai

Underhill compared Dubai to the cantina scene in the original Star Wars movie, and called it the first place on the globe that is ready to receive the intergalactic traveller. You can get anything you want, and you can pay in any currency you choose.  Dubai apparently has fifty major shopping malls, in addition to the more traditional souks.

The Challenge

Underhill believes that landlords of shopping malls must go from being landlords to being placemakers. And the best ones are reaching for this. But there is still a lot of room for improvement.

His main recommendation, which we heartily endorse: get some good shoes, and get out the office to see what's going on.

Coming Events: Neuromarketing Webcast

Neuromarketing is the latest hot trend in research circles. Stuffing people into MRI machines and then showing them advertising, and that kind of thing.  If you want to find out what's up with this stuff, you can attend a free web-cast on this topic Nov. 2, sponsored by the Qualitative Research Consultants Association.

Neuroscience seems to be the latest buzz in the research world. There’s as much fear as there is wonderment about it. The purpose of this presentation is to define what neuroscience is and means in the research world, exploring the latest developments and their bearing on qualitative research. It will debunk the myths and set out a set of principles which can be used to inform and substantiate qualitative insight. One of the topics covered will be the neuro principle of ‘hardwiring’, which tells us why people so annoyingly always seem to remember old campaign messages despite millions of dollars being spent on later, more up-to-date campaigns – and what can be done about this.

Details:

"Brains and Brands: How Neuroscience is Impacting Marketing and Market Research"
November 2, 2006
12:00 – 1:30 p.m. ET (GMT -5)
Register HERE
(Note: registration is free, and will also give you access to the archive of other great web-casts, all sponsored by the association to educate our clients and the public. Don't you wish everyone did this?)

Paco's coming to town

If you are in the Toronto area next week, you can take in a good one-day conference on qualitative research, and also hear Paco Underhill (author of Why We Buy and Call of the Mall) speak on Friday, Oct. 20.  It's a joint event sponsored by MRIA and QRCA, about 200 people, and a nice location to boot. Details here.

I am volunteering on the conference evaluation committee, and we've done some pretty radical overhauling of the evaluation forms. (Some of it is even printed in color! What will they think of next!)  I'll show you what we did and tell you how it worked in a couple of weeks.

Survey Research: Are all the same people answering all the surveys?

Advertising Age has an interesting piece about survey research: Consumers Rebel Against Marketer's Endless Surveys: 30 Top Industry Execs Gather to Discuss 'Opinion Fatigue' Crisis

At a roundtable held among execs of large consumer products firms and their researchers in Chicago, there was serious handwringing about survey fatigue and non-responders.  Perhaps survey research is not as valid as it used to be, too small a sliver of the population is making money as professional respondents, etc. 

If you are hearing this kind of thing and wondering what to do in your own organization, my suggestions are these:

Treat your respondents with respect.

Ask smart questions. Ask only what you can act on, not what would be nice-to-know. Pre-test all surveys with a small sample of the target group and interview them. Then fix the problems with confusing or irritating questions.  This does cost a bit extra, but you will get much better results.

Out of a sense of obligation to the industry, I try to participate in all research I am not barred from. And I am frankly shocked at how much of it is frustrating to participate in. Maybe you've had this experience yourself, and it's disrespectful of you as a consumer or a customer or a voter or a taxpayer.

Be quick about it.
Challenge yourself to make your next survey half the length of the last one.

Use multiple methods.
Multi-method projects come together on the problem from several angles. This can add tremendously to your understanding. You'll also avoid being blindsided by a sampling or response bias issue from a single survey instrument. 
You could combine an internet survey with a site intercept questionnaire, with some depth interviews. If something doesn't seem to fit, test again.

Consider paying people for their opinions.
I know this is unpopular in research circles. And indeed, paying people to be on large research panels is part of the problem. But if you want people to spend some of their free time answering your questions, maybe you should reward them appropriately. After all, how do you yourself feel about this?

Spend the money for real research
Your research supplier should be a member of a professional research association in your country. These organizations have standards and ethics and auditing bodies. They conduct training and education programs, publish useful articles and journals, and generally try to maintain high standards. 

If your supplier isn't part of one of these organizations, you need to ask yourself why. Would you go to a doctor that didn't participate in continuing education? Even your car mechanic and hair stylist is likely to have recent educational certificates on display.

This problem is really the tragedy of the commons all over again. Because it is getting cheaper to do survey research, there's lots more of it, and the natural tendency of most people to co-operate and be helpful has been abused to the point where a lot of people just don't want to play anymore.

What's the story?

Reshma Anand, a qualitative researcher in Bangalore, has posted on a very innovative idea about how to enrich storytelling. She thinks that if we put ourselves in the present tense when we tell a story, we can add to the richness of the detail and enhance our recall. 

This makes complete sense to me, although I bet it takes some skill to get people to do this. If you try it, you'll see what I mean.

Instead of saying, "I stood at the reception desk for a while. No one was around, so I went to sit down." you are now saying "I am standing at the reception desk, waiting for someone to notice me. They don't seem to be around. I decide to sit down and wait." 

Interesting to remember that Einstein used just such a method to imagine what would happen to his streetcar if it accelerated to the speed of light. He tried to put himself in the moment on the streetcar, and then to put himself in the moment of someone observing the streetcar. [And people think qualitative isn't scientific...d'oh]

The nuances and recall help to enrich what is being said.

Kim Bercovitz told me recently about another innovative approach -- using actors to tell a story as a research method. It sounds a lot like the kind of improv used to entertain conference participants, except applied to get clarity around experience and emotions. The participants keep correcting the actors' performance of the thing until they get it right. And then you know you have all the ingredients, don't you? [I suspect there's a lot more to this method than I'm capturing here, sorry Kim.]

Even objects have stories to tell, as this post from ethnographer Grant McCracken shows. Grant's own stories give us insight into the mind of the researcher, the professional observer, the person who can never stop watching and thinking.

Reshma's post illustrates something else brilliantly: there is no big book out there of the right way to observe. A lot of people seem to think there is. There is trying, and testing and learning and improving. There is no one right way.

The rules are all nonsense, really, because your method is only as good as the insights you get.

You want to break out, you want to be better than everyone else? You do it by ignoring the conventional wisdom, or tweaking it, or trying something different.

Kisses and hugs to all the clients out there who help you make these moments of magic happen -- by leaving enough space to try something a little bit different. They are the ones who win the biggest prizes in insight. We bust our butts for those clients, to try to make the risk pay off. 

Choosing a Qualitative Method: handy-dandy decision tree you can download

We talk about research quite a bit on this blog, because you can't manage the customer experience or even really think about it meaningfully if you don't start diving deep into what is actually going on.  To do that, you need research.

Now that there are so many options in using new communications technology to support research, I get a lot of questions about which method is better, and that type of thing.

You can't really do it that way, though.  You need to think about the objectives of the research before you know which approach makes the most sense. 

Handy-dandy research decision tree to help you (Click image to enlarge.)

I've put together this handy-dandy decision tree with some diagnostic questions that should help you think this through.  You can even download the PDF and stick it up on your wall, and share it with friends and colleagues who will marvel at your amazing ability to dig out useful resources.

Download qualitative_choices_decision_tree.pdf

Although the graphic looks slick, the real value is in working through the diagnostic questions.  After you have done that, you'll have a clear idea of your options. Then it's all down to budget.

How much to spend?

You can get into calculating the ROI of your research, and there are formulas. 

I have one that's really simple: I think clients should get about 10 times the payback from a project, or we're wasting their money.  So if it's a $30K project, you'd better think that great insight and great strategy should be worth about $300K to the business, either in risk avoided, effectiveness of effort, or raw revenue.

I once got into a lengthy round of negotiations with a human resource department that wanted to research some employee problems, after already having surveyed the employees. I suggested that perhaps the real problem was that the executive -- quite rightly in my view -- wanted to spend the money FIXING the issue, instead of putting more $ into research. HR didn't agree, and kept pitching the research and asking me to find a way to do it cheaper, smaller, etc. Of course the unhappy end of this story is that they kept having the management team reject their request for money, and I wasted a bunch of time scoping a project going nowhere.  All of which could have been avoided by getting clear on what was needed and why. And sometimes, more research is not actually what you need.

So use the diagnostic questions, get clear, and be a star. What could be simpler?

Addendum Sept 22:

Roger Cauvin pointed out that I missed ethnography. What the heck was I thinking?

Innovative Methods in Online Qualitative Research

If you could use a break from the heat, stay in for lunch on Thursday and join me for a free webcast: When Online Qualitative Shines: Solving Business Problems.

I will be the guest speaker for a web-cast on the topic of innovative ways to use online qualitative research technologies to solve business problems.

I'll be joined by Betsy Leichliter of New York, who partnered with me in developing the background research for this presentation. Betsy and I interviewed more than thirty researchers worldwide to gather case studies.

We won't be talking about researcher methods -- this presentation is for clients who would like to know more about what other organizations are doing with the new tools.

As always, it's not about the technology, it's about how to use it to get results!

This interactive presentation format will give you access to a chat window, as well as the option to phone in with questions.  It should be lively and informative.

It's free.  You have to register for access, here.

These Q-Casts are non-commercial educational events sponsored by the Qualitative Research Consultant's Association.  Once you register, you will have access to the full archive of great presentations. 

Join us for a Web Cast on Online Qualitative Research

If you are interested in Qualitative Research, you may want to join me and colleague Betsy Leichliter for a webcast sponsored by QRCA on Thursday, August 3 12:00 Eastern Time.

When Online Qualitative Shines: Solving Business Problems

About a year ago, Betsy and I interviewed about thirty experts in the area of online qualitative research, and documented some illustrative case studies, which we presented at a conference in Beverly Hills last fall. Since then, we have had a lot of interest in the material. 

Unlike the conference, the web-cast is not only FREE, but also open to anyone.  We've restructured the material to focus on the themes most of interest to business people  - namely solving business problems. We won't be spending much time on logistics that only other researchers care about.

Q-Casts, as these are called, provide for interactive chat as well as phone questions, so I encourage you to join us and make it a lively event. 

More info about the topic of the session is here.

You do need to sign up in advance, but it's not a big deal, and you do it here.

Technorati Tags: Q-Cast, QRCA, online qualitative research, Leichliter

Qualitative Research: Storytelling

Once you have found the right people to talk to, covered in an earlier post, and you have determined what you actually need to know about, then you must determine how you will get that information. In short, what are you going to ask them?

Amateurs say things like: what do you think about X?  Do you think you would buy X?  How would you improve X?  Attempts to get directly at the information you seek are often of limited usefulness.  You may get some data, but not likely anything that is not already reasonably well known. And the conventional wisdom is seldom helpful to clients who want to differentiate themselves in the marketplace with leading edge products and services. 

It's crucial to get beyond the literal if you are going to find out anything meaningful. And good researchers will have hundreds, if not thousands, of alternative ways to do this.  Storytelling is one powerful method.

Storytelling
Stories are as old as humanity, and we can all relate something we have experienced. That experience could be using a product, shopping in a store, or managing an IT outsourcing project.  When you ask people to tell a story, they engage completely different parts of their brain. 

We don't tell stories that are boring, we add the details -- the human details -- that make it interesting to others. And in so doing, we relate much more of our full range of perceptions about the event, and even the whole category under discussion.

If invited to tell a story using a familiar template, most of us could do that too.  The template might be telling the story as a fairy tale, telling it as a news report, telling it as a country music song. Or just plain old, "tell us the story of..."

The skill of the facilitator is setting up the environment that makes it comfortable for people to tell a story in detail. 

Storytelling is not just good for research, by the way.  It's a very useful approach in change management and visioning exercises as well. 

All of the great religious traditions relied on stories like parables to convey deeply meaningful information in a simple way.  Listen carefully to the stories people tell and you can learn a great deal about what is in their hearts and minds.

Technorati Tags: qualitative research, storytelling, asking good questions

Qualitative Research: Asking the Right People

A lot of the art of good research is figuring out who to talk to.  It's never as straightforward as it looks. Let me give you an example from a recent project to illustrate.

One project I am now completing relates to a global company's brand as an employer. Initially, it seemed like we needed to talk to some soon-to-graduate students. And maybe a few people who had recently been hired.
After talking at length to the inside project team, we determined that we needed a much broader and also more specific base.  Here's approximately how we defined the group we ultimately included:

• Senior executives in the firm who are involved in talent pool recruitment and have clear ideas about the future needs of the organization and the culture.  Almost as important, in my view, is that they will need to support the outcome and recommendations of the project, and they are more likely to do this if they have been involved.
• Some recent hires into entry level positions. These are people who still remember the recruitment process, the other employment opportunities they had, and picked my client.
• Co-op and summer students who are interacting closely with the organization but are not yet permanent employees.
• Recent hires into senior positions.  These people have been working for several years somewhere else, and recently joined the organization, but not in entry level roles.
• Students. Ultimately, we decided we were looking for people from specific universities, in specific programs (some undergraduate, some masters level), who expressed an interest in specific career directions, and who were also getting good marks.

It would have made my life a lot simpler if we could just talk to anyone at all.  And finding these students has been a real needle-in-a-haystack exercise.  We got them, but it wasn't easy. But really, there would have been no point at all talking to students who were heading in a very different career direction, would there?

Every project has this problem

There are almost no projects where this kind of sampling exercise doesn't become an issue. A project currently in discussion involves a home health care product. Just imagine the possibilities... should we include doctors? Nurses? Chiropracters? Just people who used the product in the past? People who need the product but have never heard of it?

A huge part of good research is figuring out what you need to know and who you need to know it from.  If you just gather a bunch of people and stuff them in a room, give them cookies and ask them questions -- you'll definitely get data.  But you won't learn much that will help you, no matter how good your questions are.

Guiding questions to ask

Here are a few of the diagnostic questions I use to think these things through.  Please add your comments with your own useful guiding questions.

1. Who buys the product or service?
2. Who pays for the product or service?
3. Who uses the product or service?
4. Who influences the sale of the product or service?
5. Are there different segments of this market based on needs as we currently understand them?  What appears to distinguish these groups?
6. Could we distinguish repeat buyers from former buyers, from first-time buyers? 
7. Are there differences in product usage (e.g.light, heavy) that we should explore?
8. Do males and females approach this the same way, or perhaps not?
9. Are there important cultural influences that could be at work here? Ethnic groups? Language groups?
10. Is there an urban/rural dimension? A geographic dimension like west cost vs east coast, warm weather vs cold weather?
11. Are there different distribution channels involved?  On-line vs in-store vs. catalog vs. home-party vs. reseller etc?
12. What about after-sales service?  Should we talk to people who needed it? What about people who have complained?
13. What about the competitor's product or service? Should we talk to some people who prefer it?
14. Is there any way we can talk to people in the act of making the decision?  Or people who just made the decision?

How to distinguish Custom Research from a Poll
Figuring this stuff out is often what distinguishes custom research from a poll.  With a poll, you get a random sample of people in a given region or geography, like 1000 adults between 18 and 64. 

With custom research, the first part of the project often consists of developing a research strategy that will answer the business issues.

Now there's nothing wrong with polling.  But be sure that's what you need.

Technorati Tags: qualitative research, custom research, guiding questions to ask, deciding who to research, sampling

How Marketing Research is Changing: Angus Reid at MRIA 2006

Our last speaker today was Angus Reid, a leading figure, who founded the company now known as Ipsos-Reid, and who is now chair of Vision Critical, a panel management software solution.

Sorry about the blurry picture, but he was an energetic speaker!

Looking back 100 years, he pointed to three periods of major change and social upheaval, each of which was accompanied by significant shifts in marketing research.

The 1930's

• the new medium was radio
• people were uncoupling themselves from the great institutions of the day, and the depression was seen as a failure of society in many ways
• George Gallup invented the public opinion poll as we know it today.  They were doing face-to-face interviews at the door, and a poll took about 6 weeks to complete.

The 1960's

• the new media were telephone and television
• gender roles and the role of youth were redefined
• there was a renaissance in marketing research, as the shift to telephone polling made results available in as little as 24 hours, instead of 6 weeks.

Today

• the new media are cellphone, wireless and internet
• globalization is causing major shifts in society and the global economy, creating new winners and losers
• a new dynamic is taking place in the relationship between researchers, their clients, and consumers.
• it may not be the end of the era of the telephone survey, but it is definitely the beginning of the end

These shifts are always more subtle than we at first realize.  Radio was better suited to the big brassy sound of a big band than the subtle variations of the violin, often lost to static, resulting in a shift in musical tastes.

Online research has resulted in much faster, cheaper data -- but not always higher quality. Comscore networks has reported that 30% of internet surveys are completed by only .25% of the internet population. In short, there are massive measurement and sampling issues, which the industry is only just starting to grapple with.

Consumers want 'CPC' - community, participation, control

The desire for community -- something we have commented on often in this space -- is a big driver behind new trends in research.  Customers want more control, more input, more say, more access. They do not want to complete a 45 minute questionnaire on a product that looks like something that was produced in the 1930's -- but now it's online.

'Bleeding borders' between marketing and research

Reid says that some larger retailers are starting to see strong linkages between their panels and sales activity -- panelists tell their friends, take them into stores to see new product.  It is raising the question of how many of your customers you want on the panel -- maybe potentially all of them??

Like the early days of film

In the early days of film, a stage play was mounted and filmed, because people didn't yet know how to make a movie. Online research is a little bit past this stage, but you wouldn't know it to look at most online surveys.

He challenged the methodology people to start utilizing the new capabilities more creatively, instead of just porting old methods onto new platforms.

A fantastic talk.  It's so nice when an icon lives up to their reputation, as Mr. Reid did today.

[PS. Funny joke he told, on the difference between BC and Alberta: "In Alberta, the people hate the government; in BC, the government hates the people".]

Update from the MRIA Conference in Calgary:Shafeen Charania of Microsoft

This morning at the MRIA conference in Calgary I heard Shafeen Charania speak.  Here in the E-Tabs sponsored internet cafe, I thought I would share the highlights of his excellent presentation with you. 

Shafeen spoke of how Microsoft blundered past it's peak of brand coolness because of its culture in the 90's -- at the time of Windows 95 release. They used marketing research to understand market size, instead of understand customers. 

They are trying to change this now, by being much more outcome focused, much more self-critical.  He was frank in his admission of the many problems we all experience with technology. 

As evidence of their efforts to reform themselves, he showed a hilarious video (that Microsoft produced themselves) of how Microsoft would approach package design for the iPod.  If you haven't seen it, it's wonderful. 

They are also trying to find more ways to surface conversation that is "not on the party line, but still inside the party", and cited Scobleizer as one example. The other, which I hadn't heard of before, is Channel 9. When I checked it out just now, there were 40,000 on line there -- apparently all built virally. This is the insider-developer view of Longhorn development. 

They are starting to use more ethnography globally.  And learning some pretty cool stuff: 

• families share multiple cell phones in Kuala Lumpur
• males surf for porn in internet cafes, not on the family computer in Malaysia -- he called this "public is private and private is public"

His last observation really had me thinking.  That only 9% of the souls in the world use a computer somehow.  And they are looking for ways to engage with the other 91%.  Not a bad takeaway from a breakfast session.

Grant McCracken on Clotaire Rapaille: There's No Code to Crack, Just Good Research

When I saw the piece in Fast Company about Clotaire Rapaille, showman anthropologist, I wondered if my acquaintance Grant McCracken would comment on the article.  He has, and brilliantly. Both articles worth reading. Take particular note of Grant's definition of good research, which is bang on.

When I was a bank employee some time ago, I recall hearing about this Rapaille guy from my boss, who was enthralled. He made people lie on the floor and remember their first experience of a bank branch. His big insight was that people remembered pillars as part of bank branches, and that this solidity was an important part of bank branch design. Therefore, don't give up on solidity. At the time, this sounded so facile I thought I must be missing something. I was: this guy is a genius at marketing the Rapaille brand.

There is no single "code" for a culture. Do you think someone unfamiliar with Canada could talk to a bunch of us for a while and understand what makes us tick?  What would they make of the 300,000 people we import from the rest of the world every year?  And translate that code to marketing a product? Maybe, maybe not. We love Kraft Dinner, but think Red Lobster is boring. Frankly, we struggle to understand ourselves.

If you've been reading my blog recently, you know I was working in Jamaica last week. Jamaica is itself an amalgam of many cultures, including Chinese, Portugese, Lebanese, and many African nations, plastered over by the British, and now being re-engineered by Jamaicans themselves. "One Love, One Heart". It is not an easy culture to understand, even though they speak English better than I do, for the most part, and share many British traditions. I have definitely had some insights that have helped my client. But I've also been run into the boards quite painfully a few times for reasons I still don't fully understand, and continue to obsess about. [Run into the boards is a hockey analogy, BTW]

Let me give you one simple example.  My hotel put little cards on the pillow with Jamaican phrases in Patois and their meanings. Here's an example: Dawg mawgre but him head big. Translation on the card: no matter how much the dog's body shrinks, his head cannot shrink: some things cannot be changed.  My hosts at dinner one night said this was all wrong.  One thought the saying referred to someone you would underestimate who was smarter than they looked. Others thought it meant that even a mangy dog thinks well of himself. These were smart, well-educated, Jamaican-born executives who speak Patois as well as they speak English and THEY disagreed about the meaning of the phrase. Is this part of "the code"?

I often ask people what jokes they tell about themselves to try to understand what I don't understand. [What's a joke accountants tell about themselves, for example?  What's a joke Canadians tell about themselves? What jokes do the French tell about the Belgians?] What is most remarkable to me is that the jokes are almost never funny to an outsider.

In any event, I don't think there's one code. And this is the beautiful complexity of humanity. I know lots of elegant models, lots of psychology, lots of social science and I speak business reasonably well. And none of this stuff explains it all. And thank God for that.

Technorati Tags: Jamaican culture, cracking the code, Clotaire Rapaille, Grant McCracken, Anthropology

What is Neuromarketing? And Does It Matter?

Neuromarketing is the commercialized version of all that brain-behavior science you've been hearing about recently. 

The basic method is to put someone's head into an fMRI (functional Magnetic Resonance Imaging) machine, then give them various sensory stimuli.  This could be food to taste, a print image to look at, an advertising clip.

The subject's brain reacts to the stimulus, and various special parts of the brain light up.  Trained analysts can tell from these patterns of brain activity whether the subject likes the stimulus, is disgusted by it, etc.

I heard more about this from Dr. Karl Moore, a spokesperson for Neurosense, the company trying to commercialize this technology. (Speaking to a QRCA event in Toronto).

The most interesting thing that I heard was that this technology appears to be able to detect lies, which may ultimately be its most marketable benefit.  [When they get portable ones, maybe all they'll do at airport security is ask you if you are a risk to others, and they won't paw through your briefcase looking for nail trimmers.]

You may have heard of a study of Coke versus Pepsi done using this technology. When subjects were told they were drinking Coke their liking of the beverage increased. 

This should hardly be a surprise, since Coca-Cola has one of the most valuable brands in the world. Even people who don't use the product have strong brand associations. 

As we stay tuned for developments in this area, keep a couple of things in mind:

1. Liking and wanting are really different things.  So if your brain says you like something, that doesn't necessarily mean you want it.  And you can want things that you don't necessarily like (like addictive drugs). And neither of these things necessarily translates to purchasing behavior.  (I like Porsches, I kind of want one, but I'm not buying one anytime soon.)
   
2. Bigger brands are always more salient to consumers.  This is just a hard fact of life for small brands.  I think this may present a major problem when comparing brands using this method.
3. If you think focus groups are artificial, try a laboratory with a big machine in it. And it is very expensive relative to other types of qualitative research.

On the other hand -- If you are about to spend a ton of money on a new product launch, you could certainly add this to your research arsenal, and potentially take more of the risk out of your decision.  Make sure you get out there and observe some customers in their natural environment first -- you might be surprised how much you can learn with a low-tech approach.

Technorati Tags: neuroscience, research, marketing, neuromarketing

Getting Customer Feedback: Are You Asking the Right People?

Our land line phones were out of service for more than a day recently, a frankly shocking and distressing event.  Oddly, the VOIP still worked, and so did the Internet, intermittently.
There are two business people working in this house, and it's the other guy's job to deal with this kind of thing.  So he called the phone company on his VOIP line, to request service on my business land line. [Still with me?]

About a week later, I got a call on the now-functional business line for a post-service survey.  I said "you need to talk to the guy who handled this" and offered his number.  Survey company says "no, for validity of our survey we need to talk to the person at this number".  I explained.  They dug in.  Net result, they preferred not to get feedback rather than call the person who dealt with the problem. 

Please note this is a real business number that I pay real business rates for. So this survey would have been evaluating the business support or perhaps SME support, but not residential phone support. So how can this make any sense?

I found this whole thing rather baffling, but worse, it clearly invalidates their results in my view.  In an effort to ensure rigor, they have created tremendous sampling bias. Their validity process is invalid.  And both this big company and their research agency should know better! 

But now you know better, dear reader, and will not make this bush-league mistake. 

 

How Good is Your Support Site? Customer Problems and Service Recovery

I spent quite a bit of unproductive time on the weekend trying to resolve hot-synch problems on my Palm TX.  When I saw "do you have a minute to take a survey?" you can bet I clicked.  The last of about five questions was this: "would you recommend Palm Support?" 
You've got to be kidding, right?  Not "would you recommend Palm", but "would you recommend Palm Support". 

The "would you recommend" question is a classic loyalty question often used to determine an overall rating benchmark in satisfaction and loyalty research.  But putting it in this context seems like utter nonsense to me.

The only reason anyone would ever be on their support site is because they are HAVING problems. Even more important, the support site does not stand or fall alone -- it is part of the total brand experience.   

Here are some better questions that would make this short survey more relevant and useful:

1. Were you able to resolve your problem to your satisfaction?
2. How much time have you spent on this problem?
3. How easy or difficult was our support site to use in solving your problem?
4. Would you recommend a PALM PRODUCT to a friend or colleague?
5. Thank you for answering our questions.  Is there anything you would like to tell us?

Number 4 is the loyalty question that matters.  It's a short way of asking this:

Now that you have experienced some problems with our brand, how do you really feel?  Do you still like us?  Are you going to tell ten friends how irritated you are and advise them never to buy our product? 

Was your experience of our support site so great that you actually like us even more than before, and you'll be raving to five friends about how great we are? Are you left feeling that you were fairly and justly treated? Or did we act like no-one has ever had a "runtime-error-r6025-pure-virutal-function-call" problem before? 

When you have to replace this product, will you remember us as helpful and caring when the chips were down, or swear never to do business with us again?   

If you solve a customer problem well, your customers will like you more than they did before.  This is not just some squishy, feel-good marketing stuff, this phenomenon has been researched pretty thoroughly. 

But the reverse is also true.  You can make committed antagonists for your brand if you don't solve a problem effectively.

Yes, my Palm is now synching again.  But only because I removed some software. And I'm wondering if I should have listened to my tech guy who told me to buy the Dell.  If the TX can run flawlessly for about a year, I'll probably be loyal again.  I'll let you know.

Upcoming Presentation: Online Qualitative Research for B2B

---

Interested in Business to Business Research?  I'll be presenting at the MRIA's B2B learning series this month.  If you're nowhere near this geography, but would like to have a set of the slides, or participate in a demonstration of the methodologies, just send me an e-mail.  Or download the slides from this link:  [Please forgive this brief lapse into shameless self-promotion!]

Here's the official announcement...

GOING DEEP WITH ONLINE QUALITATIVE RESEARCH

Speaker, Susan Abbott, CMRP In February, the MRIA B2B Learning Series welcomes Susan Abbott to discuss the use of various Internet techniques for qualitative research. These techniques will be illustrated by mini case studies gathered by dozens of researchers worldwide. The presentation is based on one Susan made to the QRCA conference in Beverley Hills, California in 2005.

Live demonstrations of two of the methods will be created in advance of the February meeting session. If you would like to experience the technology first hand, please e-mail susan@abbottresearch.com or since@epic.to.

This session will take place in the Boardroom of the Duke of Westminster Pub, 1 First Canadian Place,Toronto, Wedensday February 15th, 6:00pm.

MRIA member registration is $7.00 pre-registered, $10.00 at the door; non-member is $10.00 pre-registered, $15.00 at the door. To register, call Branka Nikolovski at 416-922-1140 ext 125, or e-mail branks@rogers.com.

For full details on this upcoming Learning Series event, visit the B2B website.

Are Focus Groups Really Obsolete? Or Just One Tool out of Many?

Focus groups are being declared useless or obsolete all over the place these days.  They aren't really obsolete or useless -- what is happening is that the focus group is simply taking its place as one tool among many to understand customers and trends. 

The latest article was in the New York Times, Agencies Look Beyond Focus Groups to Spot Trends.

Leading advertising agencies such as Leo Burnett and DDB are trying new approaches to gathering trend indicators. For example, the article reports that Leo Burnett recently surveyed taxi drivers in ten Asian cities to get their views on effects of rapid economic development. 

Eva Steensig, a sociologist at DDB spoke about their new strategy of collecting market intelligence from their employees' observations worldwide, a program they call SignBank.  I thought that was also a really interesting idea, and a great way to leverage the insights of your staff at all levels.  But then she said this:

"Focus groups confirm what you already know... Talking about toilet paper for two hours in a room doesn't really help."

The author went on to say this:

The problem, Ms. Steensig said, is that most consumers are not experts on their own consumption patterns. They have other things to worry about - their families, or the weather. So in focus groups they are easily led and rarely come up with the kinds of original insights that a marketer needs to stay ahead.

Is surveying taxi drivers better than a focus group?

Americans worried about many things, including price of housing, political climate, job losses -- BIG Survey

This item came in from the Centre for Media Research today:

According to a recent BIGresearch survey, 8900 consumers were asked to respond to "issues that are on their minds". Some of the key responses are:

• I don t know how people on fixed income or living paycheck-to-paycheck are going to keep their heads above water.

• What will happen to all these people with adjustable rate mortgages that can not afford for their payment to go up?

• We cannot continue to be the world police and leave our own people to suffer.

• There is too much financial instability right now with the effects of these major hurricanes on the U.S. economy.

• Tired of the political bashing. Everything from the economy to natural disasters. People are placing blame depending on the political party they are associated with.

• The price of housing is excluding young families and seniors from owning homes and beginning to concentrate home ownership in the hands of landlords and the wealthy. The American Dream is in grave danger.

• When is the country going to act like a country that is at war?

• Our judges are trying to change what our constitution says by using their power to change laws. Which is not their job.

• With the economy the way it is, gas prices out the roof and grocery prices outrageous, it's getting very hard to get by!

• The world wide oil reserves are about to peak and we have no viable alternatives.

• The ready willingness of the media to present everything as if it is either the fault of the United States or the fault of our President.

• I am very concerned with the state of world affairs and the high number of natural disasters in recent months.

• Terrorist threats being made and carried out in the US. And the upcoming possible outbreak of Avian Flu world wide.

• Gas prices, homeland security, and lack of morality in our country and the world.

• Border security and our nations security... taking care of the US citizens needs instead of taking care of the world.

• All the recent tragedies that have been occurring world wide.

"Uncertainty about the future in this holiday climate of giving creates a paradox of self sacrifice; consumers told us in this survey that they will be spending less for themselves and co-workers while spending more on family and friends," said Joe Pilotta, VP of Research at BIGresearch, sponsor of the study.

[Ed. note: we're classifying this as Marketing Research, however it really is an opinion poll.]

Hispanic Marketing: Understanding 41 Million Consumers

Interested in Hispanic marketing or research?  You will want to make time for this one-hour live web-cast on November 2 with Rose Marie Garcia Fontana, a qualitative research specialist in this area.  Details below:

When: Wednesday, November 2nd, 12:00 - 1:00 pm Eastern Time

Q-Casts are a free educational service offered by Qualitative Research Consultants Association.  Register to attend at: http://www.qrca.org/QCastRegistration.asp

Hispanic Qualitative Research: Second language barriers and opportunities
by Rose Marie Garcia Fontana & Carlos Garcia

This presentation will explore the barriers and opportunities of adapting qualitative research to the US Hispanic market. This session will provide valuable insights about the unique issues surrounding research with and for the Hispanic market.

Cultural and language differences will be discussed during the Qcast, and suggestions for adapting the entire research process will be covered. Case studies will be used throughout the presentation to illuminate the importance of using culturally appropriate research techniques with Hispanic respondents. Information on how to adapt study design, market selection, screener development, recruiting, discussion guide development, moderating, analysis and presentation of results will also be covered.

Participants will leave with an understanding of ways to ensure that research results can be understood within a Hispanic frame of reference, and how to maximize opportunities to reach the growing market segment of 41 million Hispanics. The economic clout of U.S. Hispanics is significant- they are estimated to have more than $650 billion in purchasing power, and that figure is expected to top $1 trillion by 2008. (LA Times, Josh Friedman) Learning more about how to reach this important group of consumers is in the best interests of anyone involved in marketing, advertising, and market research.

Rose Marie Garcia Fontana, PhD. is an independent consultant, working in the areas of General and Hispanic Market Research. She has served as a senior research associate for Garcia Research Associates, a market research firm located in Burbank. As a freelancer, she has worked for a wide variety of advertising, branding, and market research firms. Dr. Fontana has over thirteen years of experience in General and Hispanic Market Research as a focus group moderator and researcher, conducting in-depth interviews and ethnographies for universities, school districts, community non-profits, and federal agencies.

Carlos Garcia is an accomplished professional in both qualitative and quantitative research and a successful entrepreneur. Prior to founding Garcia Research Associates, Inc. in 1990, he directed the quantitative divisions of two minority marketing research firms. Mr. Garcia also held the position of Research Manager with Maritz Marketing Research in their Los Angeles office.

QRCA Conference 2005 Diary: Packaging Insights aka The Cover Does Matter as Much as the Book

The graphic above was my drawing from memory of a bottle of mouthwash.  This is the kind of exercise that Lauren Goldberg of Perception Research uses to research package equity.  Lauren is an outstanding presenter, and I learned a lot from her session at the conference.  There are lessons here that translate broadly to customer experience, aside from being really interesting stuff.

The One Thing You Need to Remember
Ms. Goldberg's presentation was titled, "It's not a beauty contest".  It's really about how you cut through the clutter and get attention in five seconds or less:  her mantra is "Unseen is Unsold".

Good packaging is packaging that is noticed in a competitive environment, can hold attention long enough to communicate, facilitates the shopping process, and communicates points of differentiation.

Why Packaging is Becoming More Important
Ms. Goldberg noted a number of trends that are increasing the focus on packaging as part of the marketing mix:

• Increasing clutter at point-of-sale
• She says about 1/3 of the items on a shelf are ignored by consumers
• Impulse purchases will ultimately determine the success of the brand -- you'll never become a loyal customer unless you try it at least once
• Products can no longer differentiate on quality alone

[From my perpsective, these trends apply to most consumer and business purchase decisions, not just packaged goods]

What a Great Package can do for your Product / Brand
A few of the things a good package can do for you:

• Shelf-pop visibility – consistently break through visual clutter and get the product into the considered set
• Find-ability and shop-ability – help shoppers quickly locate your product, and the specific variety they want
• Solve consumer problems e.g. Duracell multi-battery pack – solves the customer problem of knowing which batteries are fresh, and which have been used before by providing storage for the new batteries
• Meet needs of different segments.  E.g. plastic squeeze ketchup bottle versus glass, different sizes, etc.
• Reinforce brand attributes, such as “high-tech” or “fun”
• Support premium pricing – tall versus short bottle, unique versus standard shapres, plastic versus glass
• Help people understand how to use or apply the product -- do you squeeze it out, pour it on, spray it...?
• Increase usage of the product – example is fridge packs for soft-drink cans, which have produced increased consumption

Ms. Goldberg says over 85% of new product introductions fail, probably in part due to weak design.  As she says, "Unseen is unsold".

Can People See It?
One of the most fascinating things to me was the data on shelf placement for visibility. 

Arm-level is the best.  Eye-level is 10% less visible. Above eye-level is 30% less visible. One shelf below arm-level is 10% less visible. The bottom shelf is 20% less visible.

[I have found this to be very true of brochures on display in bank branches.  The single best location is beside a line-up area.  If the brochures don't disappear from there, you are showing the wrong brochure.]

How to Research
Starting customer insight work at the pre-design stage can save a lot of grief, according to Ms. Goldberg.  This way, you can give some direction to the design team about  existing elements that have equity (like the Listerine bottle shape), and what is important in the shopping and purchasing decision, as well as usage considerations. 

When conducting qualitative research, Ms. Goldberg does not use creativity screening questions because she believes they bias the research. [I don’t either, but this is a common practice]

She reinforced several times that the challenge is to determine which packaging works best, not which packaging respondents like best.  Which means you need to do your testing in context, in an environment that is as cloase to reality as you can make it.

Some Packaging that Works
Examples of good package design mentioned:

• Seagram’s Crown Royal – the bottle is easy to spot on a shelf, because it has a different shape than the competitors, recognizable even in silhouette
• Simply Orange – this fresh juice bottle stands out from the crowd of 2 litre cartons in the fridge compartment.  It's been effective enough that it has attracted imitators

Packages that have good recognition even without the product name: Tide, Fantastic, Bayer Aspirin, Marlboro, Crackerjack, Pepto-Bismol, SunMaid Raisins.

Noteworthy Trends
You've probably noticed a few of these trends yourself, and they don't just affect packaging, they're actually pretty pervasive in the services world as well:

1. Packaging enhancements – glossy finishes, foil, etc.
2. Improved private label packaging
3. Globalization has increased the total number of brands
4. Increase in proprietary packaging structures
5. Development of optimum category layouts (example she mentioned – organics usually in the top right of a shelf display, that’s where consumers know to look for them)
6. Auditing existing designs to uncover signs of “wear-out” -- preventive maintenance
7. Observing behavior in actual environments across the full life-cycle of usage
8. Quantifying packaging and point-of-sale return on investment

My Take
Not all packaged goods marketing exports well to the customer experience world.  But there's a lot we can all learn.  Like attention to detail.  Like making sure people can find our service in a very noisy and cluttered world.  Remembering that what is unseen is unsold.  And that it's about what works for customers, not necessarily what is pretty to look at. 

QRCA Conference 2005: Frederick's of Hollywood Image Makeover

I thought this "No Cruising Zone" sign from Sunset Strip in West Hollywood was a good metaphor for us as we think about customer experience.  Nothing stands still for long; not the competition, and certainly not the customers.   

 Re-branding Frederick's of Hollywood

The biggest highlight of the QRCA conference for me this year was listening to Renee White Fraser talk about the process of re-branding Frederick's of Hollywood.

You will likely remember Frederick's as I did, as a rather trashy home of lingerie.  It's a relatively small operation compared to Victoria's Secret, and there is considerable customer overlap, wtih 80% of Frederick's customers also shopping at VS. 

Considerable qualitative research was used to dig deep with buyers and non-buyers, followed by quant, which ultimately led to a shift in the perception of the brand, and significantly improved profitability.

In addition to customer research, Fraser interviewed about 24 of the organization's executive team about their perceptions of the brand, competitors and the customer, by channel.  There was not good alignment among the executive, who in many cases held opinions and perceptions about Frederick's customers that ran contrary to the facts.  Executive alignment was essential to implement a new brand strategy across multiple channels (true words, often heard).

Much of the customer research was conducted in homes in groups of two or three people, not in a focus group room.  Recruits were asked to invite one or two friends over to their home, and bring their favorite piece of lingerie.  Exercises in the discussions included creating collages of the brand image. 

One of Fraser's interesting points was to suggest looking at what is omitted, not just what is included in things such as collages and discussions. 

The essence of the brand's DNA turns out to be about feminine empowerment: 

"Frederick's shows women what is sexy and fun and gives them permission to be that person"

Non-customers tended to associate Frederick's with stripper-wear.  Frederick's customers saw the brand more as fun and empowering. 

So the challenge in devising new imagery was to find images that brought in the prospects wtihout alienating the customers.  At this stage, the project tested images using quantitative methods (using mall intercept studies). 

One of the very interesting findings is that when the research team tested the same model-pose-outfit combinations with a Victoria's Secret brand logo versus a Frederick's brand logo, the Frederick's image tested "trashy" and the Victoria's Secret image tested "sexy".  (This type of finding would be present with a strong brand in any category, but it still really illustrates the power of brand in influencing expectations and perceptions.)

VS was seen as more passively sexy (feminine, soft, classy, subdued, nurturing), and Frederick's as more aggressive, fun and bold (fun, feathers, bold, confident, sexy, fearless).

This research was conducted in 2002, and subsequently Frederick's has shifted perceptions significantly, by changing their advertising and their imagery, as well as some of their product line.   Interestingly, Victoria's Secret has shifted as well:  they were perhaps a bit too soft when the research was done, but may have over-corrected on the sexy empowerment theme.  Check out the links and decide for yourself.

The Key Question for Assessing Loyalty

This month's Business 2.0 heralds a "radical new research technique".  Well it's not really all that new or that radical, but it is starting to catch on.
First popularized by Frederick Reichheld, author of The Loyalty Effect, the method is relatively simple:

1. Ask your customers "would you recommend us/this product to a friend?"
2. Subtract the number of strong positive responses from the number of Negative plus neutral responses
3. The resulting number is the "net promoter" score

If you don't have a positive number, you have work to do.

Correlates with revenue growth

Reichheld's research says this number -- the net promoter score -- is strongly correlated with revenue growth. Companies with high scores, such as SouthWest Airlines, have higher revenue growth. He says most US businesses have a net promoter score of 10% or lower.

Other reasons to ask

While the net promoter score is a good metric, there are other reasons to ask the "would you recommend" question.  A good research analyst can do a lot more with that question than you might think at first glance, if it is part of a larger questionnaire.  You can run all manner of correlations and similar analyses to determine what elements of the product/experience are driving the response to that question.  You can produce those great quadrant charts that are so helpful in seeing where you stand versus competitors.  You can determine if some groups/segments are more likely than others to like or not like your product/service. 

If you do customer surveys, you really need to include a question of this type, sometimes more than one, because it is so valuable in analysis.

One number that everyone can understand

Reichheld's measure can give you a single number that everyone can understand and mobilize around. This is a big advantage when dealing with front line staff, who have a lot of information coming at them. 

If you want to use this approach, here are a few more best practices to think about:

1. Sample customers as soon as possible after they have interacted with you.  This means within days or weeks of obtaining a new car loan, for example
2. Ask them a few questions related to their experience.  A good place to start is whether they experienced the standard of service you are targetting. (ie. did anyone call you by name?  Did we thank you for your business?  etc.)  Keep it short, maybe five questions or even fewer.
3. Organize to collect this information in a systematic way.
4. When you see trends you don't understand or don't like, use other methods to find out why.  For this, nothing beats a good qualitative study. 
5. If you want to do this on a shoestring budget, develop a script and have your area managers and head office executives make the calls. 

One of the things that Damon does not really acknowledge in the article is that this one question does not solve all your problems.  If you want to know why the number is low (or why it is high), you will actually have to ask a lot more questions.  And the companies that are cited as examples, such as GE, do a LOT of other kinds of customer research.

Credits, Resources:
The neat typeface comes from Misprinted Type
The Business 2.0 article: "The Only Question that Matters", Damon Darlin, September 2005