theIdeaStudio (Customer Crossroads)

Guerilla research: finding insights on a tight budget

I frequently find myself speaking on the phone to someone who wants to do some great research, but they don't have a budget.They are freaked out when they find out the costs involved, before they even pay the researcher! and who can blame them -- I find the costs a bit of a freak-out myself.

Today I was speaking to a very nice fellow working in a program to help recent immigrants get work in their field -- it happened to be conservation work (i.e. environment) so I wanted to help.

Here is more or less what I suggested to this individual:

• Do the recruiting yourself i.e. arrange to get the right bodies into the room at the right place and time

• Skip the official focus group facility If you're running a single group discussion, and you have few dollars to work with, you don't need it. You need a nice room with a table and some chairs. Or a living room (which is where focus groups started!)

• Live without a report. Take good notes, listen carefully to the moderators's debrief, or record the whole thing and have good notes typed up. Reports take time, and that time is expensive. On a small project, it might not be necessary.  Do I value analysis? Of course I do, and some of the best stuff only comes from wringing the source data until it begs for mercy. Do you need this for gathering simple feedback -- no, you don't.

• Treat the researcher's time like it was an audience with the Queen Have a well prepared set of objectives, don't have a lot of meetings, and make it simple for them to help you in an efficient way. That is the only way to keep costs under control. Unfortunately, small projects often work the opposite way -- there is more back and forth, more contact, more revisions, more time spent explaining things.

• Work with someone less experienced Everyone needs to get a start. If you are not creating new positioning for a major brand, you probably don't need massive amounts of experience anyway. 

• Finally, if you really have no budget at all, check out methodologies designed for lay people Appreciative Inquiry is my favorite. You will find a lot of great resources online and free about this method, and I think it works quite well. Can a skilled individual use these tools better than an amateur -- well yes, of course.[I'm not a carpenter, but I can still use a power saw. I don't try to make things beyond my skill level I go slow. I read the manual and do exactly what it says.]

My question for the world at large -- how do we help small organizations get the insights they need with the limited resources they have?

New Qualitative 101

Greenbook, in conjunction with my buddies at QRCA, have just come out with a directory of New Qualitative providers.

Quite apart from the fact that I am listed in it... you would find it useful for a few other reasons:

• Great basic articles telling you what you need to know about various methodologies. Well written by experts Betsy Leichliter and Jay Zaltzman
• Details and contact information for the major platform providers
• Listings for researchers experience in these methods

You can get a hardcopy for $29, but check out the digital version for free, here.

There's also a blog (I'm writing there too), which has some fantastic contributors, all people who've done a fair bit of heavy lifting in new methodologies, and will be sharing best practices, new ideas, and whatnot.

It's good news to finally have a resource in one place, and I'm sure it will grow as providers seem to be expanding continually. Many of these providers were simply lost in the giant directory that has everything from survey sample to focus group facilities listed.

So now we have resources in one place, plus information too. And that's just great news for practitioners and clients!

 

Conversation in slow motion: How to observe a bulletin board focus group

I'm currently moderating an extended online B2B project involving business execs in a discussion forum. It's somewhere between a bulletin board project and a community, because the same people have been involved for several weeks.

It's demanding for me, but hey, I designed it, so I only have myself to blame.

But extended online qualitative is really challenging for the clients, who have many demands on their time in a typical workday. When the data really starts to build, after a few days in a project, keeping up can be overwhelming.

It's all well and good for me to have insights and distill them into nuggets of wisdom. But they're as good as useless if the client doesn't learn anything. And the best place to start the learning is in the backroom, just as it is with F2F research.

Here's what I've learned that seems to help people engage:

Tips to help the team stay engaged, and help your moderator manage.

[1] Log in early and often

The first day, it often feels like nothing is happening. Team members who logged on expecting activity can get discouraged, and when they look at the transcript later, it can feel a bit depersonalized, and a lot overwhelming.

If you log in frequently, you will see the same ebb and flow the moderator sees. Some topics attract immediate interest, others are slower to build. It’s easier to see where participants are spring-boarding off others’ ideas. Even though the transcript is time-stamped, seeing the evolution is a richer experience.

[2] Schedule team meetings to review insights

One team scheduled daily debriefs in a conference room where they projected the discussion and talked, just like they would in a backroom setting. Snacks included.

[3] Divide up the work of observing

Assign individual team members to cover off one or two segments in a multi-group discussion forum, then report back to the team on their segment. Another approach is to divide up by days (or weeks, for longer projects).

One project team held a group meeting where they projected the evolving discussion and reviewed interesting things together, just as they would in F2F research in the backroom. What a great idea! 

[4] Add comments

Observers can usually add comments to the transcript that are invisible to participants. This is a great way to capture a budding idea, and can help the moderator see possibilities for probes.

[5] Ask for follow-up probes right away

The faster we can ask a follow-up question, the better. Going back more than one day in a discussion tends to sap the group’s energy, and often does not work anyway.  Have a plan in place for how you will direct these to the moderator, to prevent frustration on both sides.

As a moderator, I may make the decision to bring something back in later to the discussion -- even days later. Or I may know that the topic already came up earlier. So if the moderator doesn't respond to your request, try not to get frustrated -- they are likely not ignoring you, just trying to manage the flow in the best possible way.

Acknowledgements

Portions of this article were previously published on the NewQual Blog as Bulletin Boards from the Backroom: 5 Tips for Observing a Conversation in Slow Motion.

If you haven't seen the NewQual blog or directory, you should check out this new resource: NewQualitative  The directory (finally!) brings together all the providers of online research platforms, and it's a good place to discover other resources and content.

Focus groups are a waste of time: or are they?

Researcher Chris Kann, (pictured left) and I spoke a while back about the reasons that people bash focus groups. The result of that discussion was an article that was published in Greenbook, called Don't Blame the Focus Group.

One of the most frequent negative examples trotted out about focus groups is the Herman Miller Aeron Chair story, a story of a very innovative product that was so foreign to consumers when introduced, that initial reactions were not all that positive. This story is often told as a cautionary tale about research -- it shouldn't be.

"That Herman Miller management chose to go ahead with a launch of the chair despite these findings and generated a massively successful product, is taken to mean the focus group did not do its job.

But perhaps this is a story of a focus group doing its job well, and management doing its job well too. Management knew what initial reactions to the new product would be and could plan accordingly. The history of the Aeron chair on the Herman Miller website in fact makes it clear that extensive consumer research, including focus groups, informed the development of the product."

We think it is pretty ironic that companies are invited to listen to consumers more, but only in certain ways.

I'm pretty sympathetic to the challenges of management in client organizations. So is Chris. The truth is, we've both been on the other side. Managers have to make a lot of tough calls, and they hope the research will help. And it often does help. But no research, regardless of method, can save leaders from the pain of making actual decisions, and some decisions will inevitably be wrong.

Bad research won't help your decisions much. But good research will save your hide over and over -- save you from making expensive mistakes, help you see the potholes in the road to success, and point to new opportunities.

Focus groups are not the be-all and end-all. But there's a reason they have stayed around, and it's because they are a good tool in the hands of a skilled researcher.

I'd love to hear your thoughts and experiences.

 

Resources:

Don't blame the focus group for leaders' shortcomings, by Susan Abbott and Chris Shields Kann, Greenbook, 2011.

About Bill Stumpf and Don Chadwick, designers of the Aeron Chair

Company history of the Aeron chair, including video with the designers.

Free e-book: Painless Insight Planning helps you set objectives, avoid wasting money on qualitative research

 

I've refined a number of tools to use in my work over the years. A few of them are tools designed to help clients in one way or another. This free e-book, Painless Insight Planning, is one of these tools with some narrative wrapped around it.

[Download Painless Insight Planning now]

One thing I've found that is often a challenge at the start of a project is getting clear objectives. People may have a problem or concern, but it is often not clearly articulated.

And I can tell you one thing for certain: a good way to waste money on research is not having clear objectives at the beginning.  I know this sounds perfectly obvious, but it is an issue more often than you would suspect.

Sometimes the problem is clear, but other elements are not. Such as -- who could we get insight from, otherwise known as "the research target."

I can't sit down with everyone individually and work through this, as I do with clients. So I thought I would put the whole thing into an e-book that you can download and use as much as you want. And then you'll have clear objectives for your research projects. And the world will be in a better place.

Because we really need insights these days, more than ever. And people inside companies can't afford to waste money or appear to waste money, get fired and be unemployed. Suppliers like me also can't afford to waste client's money. We all need to ramp UP on the insight quality and the innovation / ideation, but keep a lid on costs. 

What's it worth to get this answer?

An even harder thing for many clients is figuring out how much to spend. They ask me, what will it cost? 

WRONG QUESTION!  WRONG QUESTION!  WRONG QUESTION! 

The right question is "what is it worth to get good information on this situation?"

I have created a nice little rule to help figure this out. Not a complicated ROI on research thing. Just a back-of-the-envelope, cocktail-napkin kind of formula that I call the 10 Times Value Rule. It's a great place to start figuring out what you should consider spending to get some insight into your problem / challenge / situation.

[Download Painless Insight Planning now]

Easy worksheets

I hate filling in forms. Most people do. So there aren't forms in this book. There are easy worksheets designed to spur creative thinking to get you started. Here's an example.

Bonus content

If you download the book and actually read it, you'll find a link to some bonus content that will save you even more time. 

A small request - no strings attached

I'm simply putting this handy e-book out there for free with no strings attached. You can even add this e-book (with no changes) to your own site or blog for your own customers and prospects to use.  But I do have a small request.

If you think this e-book is of any value, please add this blog to your RSS feed reader. I also send out a newsletter about six times a year. You can add yourself to that list if you want to hear about other e-books I plan to release, or just get the newsletter. You can check out the previous issues here.

Thanks for reading. And good luck with your insights research projects!

[Download Painless Insight Planning now]

How to Conduct a Panel Discussion: Not

I'm kind of down on attending panel discussions at conferences, because so often they are mini-presentations delivered in a linear fashion, that never actually obtain liftoff on anything that sounds like discussion. This is sad, really, because a good one can be trememdously stimulating.

Rick Wolfe sent me a video clip of one being conducted by Guy Kawasaki, who is clearly good at many things, but this isn't one of them.

Mr. Kawasaki is trying to liven up a conference presentation by bringing some of the target user group (young adults) into the room ⇒ a really good idea.

He puts the teens in a flat line at the front of the room  ⇒ not conducive to a discussion

He puts himself at a podium away from the rest of the group  ⇒ thereby setting up an unequal situation, also not good.

The first time he hears something funny he milks the entertainment value, not the learning value. [Start watching at 5:13] Everyone laughed at the girl whose parents cancelled her text messaging service when she sent 4,000 text messages in one month.  ⇒ The panelists are now either going to say something extreme, in order to get the desired laugh, or they are going to reveal as little as possible, in order to avoid saying the wrong thing and being laughed at.  And at 6:33, it's clear that this young man is nervously seeking the right answer to the question.

There are a lot of short, closed-ended questions. Each panelist is asked the same question.

Things start to get going for a minute at 8:37, as one panelist tries to explain text messaging to the audience. It's lovely and insightful. But not allowed to continue. Net result  ⇒ a serial interview and a lost opportunity.

Not long after that, the moderator starts polling people by getting them to raise their hands to questions like, "How many go to Blockbuster?" What a shame we couldn't just hear more talk from these very interesting panelists.

A better set-up might look like this:

• Put the moderator at the same table as the panelists
• Get rid of the "head table" feel, and seat the panelists so they can see each other
• Keep the mikes, get rid of the podium, and move the table into the room

If you're the one moderating the panel, try these tips:

• Get people comfortable talking by building an environment of trust. Gentle humour is good, laughing at panelists is not.
• Ask big, powerful questions to get things moving, then stop talking. Wrong way: "How many text messages do you send?" Right way: "What is text messaging all about?"
• Avoid setting up a norm of serial interviewing, by allowing a few awkward pauses, and forcing the panelists to take part of the responsibility for the conversation.
• Use a vote only to start a topic, or trigger discussion from different points of view, not as a poll. If you wanted statistics, you could have done a survey.

The whole idea is to get a conversation going that the rest of the audience can listen in on. Once it's going, you let it roll as long as it's on topic and productive, and no-one is taking over. 

Update on Zopa and Prosper, online un-banks

Zopa is an un-bank of sorts, a peer-to-peer finance operation. Zopa brings lenders and borrowers together in an online marketplace, letting the lenders (i.e. people like you and me) offer funds at a specified rate, for a specified term, with limits on the level of risk. Borrowers look at the funds on offer, and choose whether to take at that rate or wait for a better deal.  Zopa makes the market, provides administrative and matching services, and underwriting services, but it does not itself lend money.

Zopa has recently launched in the US, from it's UK origins, and decided to do some focus group research in advance of the launch. In the spirit of open-ness that seems to define this company, they have disclosed clips of the focus groups through a link on their blog, here. Or go directly to the clips here. (Quicktime required. And it takes a minute to download.)

The remark I found most interesting was the woman who wanted to help struggling artists, but couldn't figure out how to find them, and thought Zopa would be a way to lend them money safely. I guess the instinct to be a patron of the arts continues, even though most of us don't have the money of the Medici.

Zopa uses some clever marketing tactics, such as paying for referrals.  If you are an active Zopa lender and bring in another member who completes a transaction, you both get paid 30 pounds.

Zopa's blog invites guest authors to write.  Here's an example from Philipg27, a Zopa lender, who writes that Zopa is not a community, it is a market, and compares it to Lloyd's of London insurance. Philip27 takes the company to task for weaknesses in it's online functionality.

...most importantly for Zopa the message is that Zopa should be working towards making the market easy to use rather than attempting to foster a sense of community. The website is notoriously difficult to use, it should be as simple to use as my online-stockbroker, and provide me with enough information to make informed decisions on the most effective options to lend money. I don’t really need pictures of people jumping over mountains whilst baking bread. I need cash flow statements, better reporting and maybe even some idea of what’s happening to my loans being processed that are currently lost in the Zopa Triangle …

Prosper is the other major peer-to-peer lending operation, with a few differences from Zopa. 

Prosper permits lenders to form syndicates on their own, and choose their borrowers. Borrowers can also become part of a group, where the group's reputation rises and falls based on repayment patterns, which then affects their interest rates and ability to attract lenders. 

Prosper is a bit more wild and entrepreneurial in approach than Zopa, permitting lenders to take undiversified risks by lending a lot to one individual or group.  Zopa forces lenders to diversify.  Now that they are both operating in the same market, it will be interesting to see how things shake out.

I expect it will be a few years before we really see how these approaches work as a business.  Meantime, it's fascinating watching how both operations differentiate from banks in their communication approach.  If this approach really works, it seems obvious that a big player would try to enter the action, perhaps with a sub-brand, just as a few are now in the payday loans business using subsidiaries.

What is Good Qualitative Research?

I've decided there must be a lot of bad qualitative research going on.  Otherwise, why would so many people make disparaging comments about it?  I saw another one today, in an issue of Internet Retailer  -- a quote in an otherwise good article by Mary Wagner on listening to customers. 

One of the experts Ms. Wagner quotes provides a list of things marketers should monitor, like blogs, e-Bay, newsgroups, Flickr, etc.  It's a great list. Except for when he says you can learn things from Flickr "that would never be found in a focus group". 

Well, now just a minute there. You certainly can learn things from Flickr and from blog searches and e-Bay with a relatively small investment of your own time and resources. You might call that "Poor Man's Qualitative", and of course you should be doing it. Indeed, these are some of the things I do to brief myself when I start a new project for a client. But if you didn't get at least that much learning and much more from your paid research, you didn't get what you paid for.

So I think there is a lot of bad research going on, so much so that people no longer know what good qualitative research looks like and how to get it for themselves.  And getting good qualitative is a bit like understanding fine art -- there's a considerable amount of judgment involved. The good stuff has tremendous value. And you need a bit of education in the topic to be an informed buyer.   

Insider secrets will be revealed. Tips and tricks of the trade will be shared.  But I'm not worried about letting you in on all this. Because, oddly enough, the more informed you are, the less likely you are to think you can paint a masterpiece in your basement, and the more you'll want the good stuff. 

Here's the short list of topics that I plan to include, just to whet your appetite:

1. Asking the indirect question
2. Using imagery
3. Using storytelling
4. Talking to the right people
5. Going beyond rational and functional
6. Lying
7. How to find a good qualitative researcher – questions to ask, where to look
8. Homework
9. Anchoring
10. Listening at right angles
11. Having a conceptual framework to put the content into
12. It’s not just leading a group discussion
13. How to introduce bias, and how to avoid it
14. Putting people at ease
15. Research ethics

We probably won't need 15 articles to get there.  As I tell clients, "we will cover all these topics, but not necessarily in that order".

References:

Mary Wagner, Talk it up: The web turns up the volume on the consumer voice. Internet Retailer, May 2006 issue. link

Technorati Tags: focus groups, qualitative research