theIdeaStudio (Customer Crossroads)

Finding insight: what art projects have in common with insights projects

The art project, now documented as a book, Mapping Manhattan: A Love (And Sometimes Hate) Story in Maps is an interesting exercise in insight finding.

The artist, Becky Cooper, was initially asked to document the locations of public art in the city as a paid project, and realized how we create our own internal maps of places. How each space is formed of our memories and our experiences. So she created some maps on a printing press, and asked people to fill them in.

These three samples from the Tumblr site are wonderful examples of how people responded in highly diverse ways. As a researcher, I can tell you that this is EXACTLY the kind of thing I see when I ask people to produce a projective exercise* of this kind, such as a collage or a mind-map, or a video, or a story. 

You can see how much rich information is here to be mined, should one choose to do so. 

One could easily argue that all great art is about innovation and insight, actually. If you are ever in need of inspiration, look at what contemporary artists are doing. 

Resources:

I learned about this work via BrainPickings, a somewhat unfortunate name for my new favorite e-mail newsletter.

*Projective technique: has it's origins in Freud and Rorsach. Great definition from my buddy Monica Zinchiak: an "ambiguous stimulus designed to evoke inner thoughts and emotions".  The purpose is to start a different kind of conversation.

Are people gaming your customer satisfaction surveys?

We love our car dealer, we actually do. They look after us well. You could picnic on the floor of their service department, it's so clean. The organization prides itself on service, and wants to do well on the customer satisfacton index.

In fact, they are so focused on this that they help their customers anticipate and respond to the questionnaire even before it arrives!

"...please take the time to complete the survey with all 10's if we met and exceeded all your expectations"

It's hard to say what impact this approach is actually having on the feedback they receive. Certainly the letter provides a clear indication that they are trying hard.

One thing I do wonder about is this -- does their approach mean that their actual results are almost useless to them in terms of monitoring and improving? It might. You'd need a statistician to help you determine this based on comparing sets of data.

There's an interesting thread on LinkedIn discussing the challenges of actual cheating on this kind of survey, and how it can be detected using statistical methods.

My take:

If you want to know what's really going on with your customers, you probably need to go beyond this kind of automatic approach. Or at least supplement it with some telephone interviews or some good qualitative research.

Another problem with expectations is that they are always moving up, based on your past performance. So exceeding expectations becomes exceedingly difficult.

Related articles

Gathering customer feedback: the good, the bad, the ugly

Write Better Surveys Now: Secrets to Successful Surveys

We've just launched our newest e-book designed to help people write better surveys. It's written for average business people, consultants, freelancers, coaches, professional speakers, workshop leaders ... you get the idea. Anyone who is looking for a fast focused resource to help them right now, and save a ton of time.

This goes way beyond the FAQ in your online survey provider, I promise. Plus it's organized all in one place.

End the survey torture now - write better surveys.

I decided it was time to stop complaining and invest some time in a solution. There are just so many bad surveys out there.

So I decided to write a nice focused e-book  for the people who need it most. Not research professionals (who do sometimes write terrible surveys, of course, but should know better). I wrote it for all those people who are not in the survey business, but find themselves needing to do a survey anyway.

Consultants, mostly. But also facilitators, coaches, speakers, trainers. Volunteers on association committees and boards. Association executives. Analysts who haven't been sent on training yet. Managers outside the research function who find themselves doing a survey. In short, all those folks who have surveys thrust upon them and would like to do a better job at them.

There's lots more about the book here, including the table of contents. You can buy the e-book for immediate download using any credit card you want, from Gumroad, a nice simple service that just does this one thing very well. No excuses, it's seriously good value.

Research Tools for Consultants: The Survey

Think of it as buying us both a really large latte, and me telling you everything I know for about two or three hours. Well, as much as I could put into 42 pages, anyway, and still keep it readable, fast, and focused.

So please. Be part of the solution. Stop the survey torture. Read the book. Follow the simple rules and checklists. (And don't embarass yourself with any more awful surveys.)

Gathering customer feedback: the good, the bad, the ugly

I wanted to boost the usefulness of my iPad with a Bluetooth Keyboard, and ultimately got the sleek and delightful Logitech Ultrathin last weekend.

The Good

As it happens, I used BestBuy Canada's online option that allowed me to reserve the item for in-store pickup. You don't have to pay in advance, but you do have to provide basic contact info, including your e-mail address.

The purchase itself was uneventful. Go to store, pick up item, pay for item, head home. Worked well, I would do it again, except...

The Bad

Yesterday, five days after the purchase, I started receiving requests for feedback via e-mail.

On Thursday, I deleted 15 such requests. This morning, I see there are 11 more in my e-mail. EEK!

In self-defense, I decided I should answer the darn thing, provided by ForSee.

The Ugly Deconstruction of the Survey

The e-mail promises a 5 minute survey. This in itself seems crazy to me. It should be more like a 30 second survey, with about 3 questions. But wait till you see it.

There are 34 questions, and 32 of them are marked * meaning you can't finish the questionnaire unless they are answered. This is an odious practice in itself, because it's so one-sided. I might be willing to answer a few questions, but these questions are silly.

They've decided there's a model involving a lot of factors, and they're going to make me answer every question in the model.

Naturally, I am not going to get a thing for investing my time in this.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

I thought perhaps there would be a place for open-ended comments, which there is. However, the survey makes it clear that no one will follow up on my comments.

I love the part that says "If you would like us to contact you about your feedback... visit the contact us part of the web site."

There are several links at the bottom of the page, but none of them go directly to a contact page, or even better, spawn an e-mail or contact form.

Interestingly, as soon as I clicked through to the Forsee site, YET ANOTHER survey opened up!!

Yesterday, I posted a Tweet to @BestBuy about the multiple survey invites arriving. They actually did respond, and copied their response to @BestBuyCanada.

For the record: I am not answering your hideous questionnaire. And if you loved your customers, you wouldn't ask them to do this.It's a hideous process, even if it was working correctly.

Magic, sausages and insightful creation: perhaps best enjoyed from a distance

I've been thinking about cake-baking as a metaphor for the process of report writing. I'm convinced that the process looks fundamentally different from my point of view than it does from the client's.

To read the text that accompanies this picture, you need to bounce over to the new MRIA blog, where you'lI see the full article, called Magic, sausage making, and the baking of insight cakes: or, why I hate providing draft reports

Yes, Apple does research

There's a myth out there that Apple does not do marketing research. This is nonsense, of course, but seems to have tremendous staying power.

I saw something today that pretty much proves they conduct research -- they might call it product development, or usability, or ergonomic, or whatever, but it's research into what people want, what they like, what is comfortable for them. 

When you test 124 different prototypes on 600 people, that's research. Good companies do it. Including Apple.

Insights on a budget: How to do Marketing Research with no money: Part 2

In the previous post, I talked about how to do a few interviews on your own if you have a need for insights (and who does not need insights?) but no money. It's a common problem for small businesses and start-up businesses.

I do recommend avoiding some kind of high-tech thing in favor of just actually talking to people -- well listening to them mostly.

But, if you want to do this online, there are options. Some are designed for the DIYer, others say they can be used by anyone.

Here are a few that I know about. Not a complete list by any means. (Note that I am talking qualitative here.)

Option 1 -- use a free survey tool but ask only open-ended questions

This is actually a pretty good option. Think in terms of maybe five questions, and give people lots of space to answer. This is not about statistics, this is about using technology to get yourself a mini-qualitative study.(Seriously, forget statistics in this instance, just go for actual insights.)

If you think this sounds feeble, let me tell you that Jackie Huba (yes, that Jackie Huba) used exactly this method to conduct some of the advance planning for the Dell Customer Support Think Tank that I've been telling you about. She had about five questions, all open ended. It was a very effective approach.

Top tips: Be short. Rewrite. Rewrite again. Use plain language. Ask a "big" question. Leave a big space for an answer. 

Option 2 -- consider these other tools and resources

There are a few services out there designed to work for the DIY researcher. Full disclosure: I have not used all of these services, and I am not getting paid for this. (Maybe a drink at the next industry conference, she says hopefully?)

You can find a very good directory of online qualitative platforms on the Greenbook site -- the NewQual Directory, as well as lots of good articles about how this stuff works, written by some online experts, Jay Zaltzman and Betsy Leichliter.

Gut Check is a service designed to help you find respondents and complete -- fairly quickly -- small scale projects on fairly narrow topics, like testing an advertising concept. They will help with recruiting participants from their panel, or you can recruit your own. They provide a library of video tutorials and other forms of help.

ITracks is also a platform that provides leading edge professional grade tools to quallies. But they have a package called Reality Check that gives you the ability to do a quick concept test and get responses quickly. It's stripped down, but looks promising.

2020 Research has a suite of online qualitative tools. These are leading edge professional grade tools, but there's nothing stopping you from renting their platform, reading through their extensive resource base, and running your own project. There are a couple of articles on their site about the pros and cons of running your own projects.(I thought they had a tool specific to DIY, but I'm not seeing it there.)

BlogNog is an online research platform that has a lot of features, but they also promote it as a good tool for DIY research.

QualVu is an online video platform, that also promotes a DIY option. I'm not clear on how the DIY option is priced or handled differently than if someone like me was just to rent their platform.

IdeaScale is a tool that is designed to encourage the submission of ideas from a group of users, and let them vote up, vote down, comment and enhance the ideas. Depending on what you have in mind, it might work well for you.

There are MANY more tools, as you will see from the NewQual directory. Just like going to Home Depot, buying the power tools is actually the easiest part of the project. Just remember that carpentry adage: measure twice, cut once. 

Have you had experience with DIY? I'd love to hear from you on this. Was it a happy experience? Would you do it again?

Guerrilla marketing research: how to find information on a budget Part 1

My regular workout buddy is an outsourced marketing consultant. As we sweat it out in the cardio area we yak it up about many things. A frequent topic is the desire of her small business and start-up clients to get some kind of marketing research insights on essentially no budget.

Recently, she was telling me about a start-up who wants to get 30 interviews done across North America with various types of business owners, and has a budget of $5K for the whole thing. As you can imagine, a range of expressions crossed my face when asked for advice on how to do this!

It's a common problem, and I'm not unsympathetic. In fact, I keep thinking about putting out some kind of e-book on this topic, as I get the question so often. (What do you think about that, dear reader? Is this a good idea? What needs to be in it?)

What advice do I give?

For right now, here's what I usually say, or variations on this theme. (I'm thinking B2B here, because it comes up more in that context for some reason.)

1. A few really good in-depth interviews with the right people are better than a whole lot of interviews.

2. 30 interviews is not some kind of "random sample" or "more representative." The ONLY reason to do that many interviews would be to systematically cover a few people in many different categories. e.g. ensuring you have some people from each of six industries, for example, and subsets of "users" and "non-users" in each group. Or smaller and larger. Or more sophisticated / less sophisticated. Or some other dimension. (These are research design considerations.)

3. For business buyers in North America, regional location is only important because it affects industry, regulation, and access to solutions. Silicon Valley business is different than Chicago business because it's Silicon Valley industries, not because of the warm climate, unless you are talking about landscapers.

4. Research done with friends of the owner is dangerous, because they are anything but disinterested and unbiased. This is the worst kind of convenience sample (Yes, that's a real research term, and it means "people we could easily get hold of.")

5. You can get people to participate in research without a monetary incentive, but there really needs to be something in it for them. Examples of "something" include: a copy of the findings, a chance to network with peers and talk about shared concerns, a free meal in a nice setting, a chance to influence development of a new business concept, free stuff from the sponsor.

The reason professional researchers don't do it this way is that a) it's not fast/efficient enough and b) it's not reliable enough. It's not because it's impossible.

6. Assuming you have the right people to talk to, it's all about the questions you ask, and the environment you create, and how skilled you are at listening. You don't generally hire a carpenter just because they have tools -- they also know how to use the tools. If you decide to build your own deck, at least keep it simple. This leads to point 7 --> how to use the time in the interview.

7. Spend most of your time just learning about them, their business, their interests, what kind of solutions engage them. Do not start with a bunch of direct questions. Soak, soak, soak up the information about who they are and how they go about being successful. What are their pains? How do they deal with those pains today? This is always the key to developing good value propositions.

8. Plan to spend some time going over your notes in detail, and figuring out what the differences between people are, as well as what the similarities are. Going for only similarities is a very common newbie mistake that is made even by trained researchers.

9. Don't go all high tech, that won't substitute for a lack of finesse on 6 and 7. Just phone people up using that good old OLD tech, the telephone. If you are going to record the interview, tell them.

Planning help

I have a short e-book -- it's FREE -- that will help you think about what you are trying to figure out. Check it out. I get a lot of good feedback on it from people who take 30 minutes to work through the questions.

Your challenges

If any of this resonated with you, I would love to hear from you, either in the comments, or send me an e-mail with your thoughts -- susanabbottarc[at]gmail.com.

Research industry trends: under the covers in marketing with the GRIT report

Marketing is a function that runs on information much more than most people realize. From the outside, it looks like a beer commercial -- lots of cool people doing creative things. Under the covers, it's much more analytics driven.

Research is a big part of that analytics process -- so research trends tell us a lot about what's going on in the world of business.

Greenbook's Industry Trends Report (GRIT) comes out twice a year and tells the story. There's a wonderful infographic with the current issue, available here. The graphics below are extracted from same - the red circles are things I added.

Marketing wants results -- if innovation gets there, fine. If not, forget it

Innovation is on the list, but not if it compromises feasibility, budget and speed. Nobody in corporate life can afford to take a lot of risks these days -- so they're not trying out innovative methods unless they solve some other problem in a big way.

Social media analytics aren't just buzz -- it's happening

The longstanding buzz about social media measurement is finally warranted. People are using it. Which means that business is taking it very seriously indeed. Everyone is still learning, it's all still changing, but it's here to stay.

Budget isn't everything -- because research questions are getting tougher

Businesses are adopting some very expensive approaches, one of which is MROCs (aka insight communities). MROCs are an innovation that IS solving a problem for clients -- these communities provide more in-the-moment access to people as they go about their actual lives. But they require real skill to set up and manage, and they are anything but cheap.

They may be difficult and costly, but they are working and they are gaining traction. Clients are finding real value there, which is why their use is growing. The growth in MROCs also shows the need for deeper insights -- when you are swimming in data (as most large organizations are these days) what you need is deeper insights. Simple questions can be answered from all kinds of internally available data. What's left for research are relatively difficult questions.

I think Gamification is a trend that suffers from sounding "bigger" than it is -- it is really just making surveys more fun. Focus groups and other qualitative methods have been moving in this direction for a long time -- mostly we call this "projective techniques".

Mobile research is also coming of age, because it is solves a problem for clients (better insights in the moment) and leverages a technology that is ubiquitous. Unlike some of the other methods discussed, however, mobile research promises to be inexpensive (eventually). Mobile will include surveys, in-the-moment reactions on site ("how did we do today?") and mobile qualitative ("take a picture of what's happening right now, and send it to me with your thoughts about...")

This infographic is evidence of another important trend affecting everyone -- we all want information that is easy to consumer and easy to find.

Another great job from Lenny Murphy and the GRIT team.

 

How to save money on qualitative research

There's a great discussion evolving on the Greenbook blog, started by Jason Anderson with his article "The 7 BS Stories of the Market Research Business".

I don't agree wtih everything he says, however his perspective is likely reflective of what a lot of research buyers feel. And his vantage point from inside the video gaming industry (life doesn't get much faster moving than that) is certainly interesting and unique.

Be sure to check out the comments too -- some good value added material is posted by other researchers in response.

Is good research expensive? Sometimes it is. There are ways to ensure you are getting the most from your budget, however. I published an article on exactly this topic (link below).

Link to PDF: Good to the Last Drop: 10 Ways to Save Money on Qualitative Research, by Susan Abbott, published by AMA MarketingPower.com

Must read article: Rory Sutherland of Ogilvy UK on how to measure advertising

Every time I talk about advertising measurement as a 'black art' I get objections from people telling me how their normative metrics are well tested and very useful, blah, blah, blah. Let me be clear here: I do qualitative testing on advertising, and I'm not too positive on that approach either.

Why so negative? Well, there is a lot of good evidence that our actual preferences and behavior are not all that well related to what we say.

A brilliant article, "Questioning the Nature of Research" reports on an interview with Rory Sutherland, chair of Ogilvy UK, who makes a strong case that our current measurements are not strongly rooted in behavioral economic theory.

Sutherland's point is not that research is a bad thing in general, it's that the current norms for how research is done on advertising don't work.

...as you get closer to the point of decision, there are factors at work which never really appear in conventional market research. They are contextual factors, social factors. No-one in any research group would ever say, “If there are four brands of shampoo, I’ll buy the one that has most bottles on the shelf”

This is true, by the way, not just of survey research, but also of qualitative.

The best qualitative research approaches come at the topic from right angles -- indirectly. Like a magician's deception, we want to focus your attention on something so that you respond as naturally as possible. Or we try to mimic the actual circumstances of your decision. Still, no self-report in the world is as good as watching a thing happen. Or setting up experiments that show you what is actually happening.

What we have learned from behavioral economics research is this: "Your preference is a product of your behaviour and not the origin of it." If you customarily go to one drycleaner, you will be able to provide a rationale for that choice, (e.g. quality, service) which is likely to have little to do with the real reason (e.g.location).

A useful and thought-provoking article to get your synapses back to school and making new connections.

 

Are you capturing the insights of the Introverts?

Our society has a strong bias in favor of extraversion -- those who are energized by being around others, who can process information quickly, and blurt out opinions as fast as they can blink. But there is a sizable minority of the population that does not fit this pattern -- people with a preference for introversion.

I wrote recently on the NewQual Blog about the advantages of online asynchronous research methods for capturing the insights of Introverts. And boy, this really resonated with people!

A couple of people sent me private e-mails [as introverts would do, not wanting to be loud and out-there, for the most part].

It's not just qualitative research that suffers from this problem -- virtually all marketing research does. Consider the telephone survey, where you are asked a battery of questions, many with multiple choice answers or agree/disagree scales on relatively complex topics.And you need to answer pretty much immediately.

A lot of online shopping has time limits that test our patience -- we have to choose among options quickly, or the system "times out". But with many options to choose, even the speedy must find it challenging to react quickly enough.

For more on this topic, check out the other article here:

Introverts get their day: When discussion forums and bulletin boards can be more inclusive

What's your take?

Is the world stacked against introverts? Does anyone give you enough time to consider before expecting you to choose, react or opine?

To probe or not to probe: Active listening versus poking with sharp thingy

More and more qualitative research is happening online. One of my favorite tools for this is the discussion forum, also known as a bulletin board. This is an asynchronous environment -- everyone is not necessarily present at the same time. [I so wish there was a better word for asynchronous!]

It's not quite the same as face-to-face. In person, if I hear something intriguing, I might ask, "Can you say more about that", or something similar. This is known, in researcher parlance, as a "probe". Although we try to make it gentle -- drawing out more information and ideas -- sometimes I think it feels more like a sharp poky thing.

In the discussion forum environment (asynchronous), you definitely can probe and ask follow-up questions.  The technology support for making this easy has improved a lot – we can now flag things, send a mobile message, or e-mail to the respondent.

My approach on this has changed a lot over the years -- I try to be more of an active listener, and less of an inquisitor.

Treat participant energy as a scarce resource

I treat follow-up questions as if I have only a very limited number I can use, and I must use them only when really important.  Here’s why:  I really love it when participants start to engage with each other.  But they do this less if I’m constantly butting in.

Not only that, but asking people to go back to something they said before is sort of like trying to rewind a conversation by hours or days — people have moved on. The energy is now somewhere else.  So you spend a lot of participant energy asking them to go back.  When people do go back, their responses are often not worth the cost in energy.

The virtual head nod

Probing does have one clear benefit — it shows people you are listening.  And listening is hugely important.  Participants see what you say, and see whether you are listening and understanding them or not.

If I don’t have a definite need to follow-up for additional detail, I sometimes restate what I am hearing, just as I might do in a F2F discussion.  This approach generally results in feedback from the participants that they really felt “heard”, which is so important.  Who wants to talk to an empty room?

The virtual head nod is even shorter — just an indicator that I’m there, reading and listening.

The challenge

Clients distracted by other work may log in and read something intriguing that happened earlier in the week. Then they want me to post a follow-up.

To keep everyone happy, I tend to bring this kind of thing back in with a fresh topic later on, rather than ask people to go back.

I do make my presence regularly felt, as often as possible given the demands of eating and sleeping, but limit the number of probes.

There is another school of thought that you can run these projects with no probes at all — I think that would produce significantly reduced output. However, there are some new DIY qualitative platforms that essentially take this approach. There is no active listener.

What’s your view?

Does this resonate with anyone out there? Either as a participant in some form of interview or discussion, or as a moderator of same.

Portions of this were previously published on the NewQual Blog.

How to save on qualitative research

No matter how large your research budget, you probably wish you had more. Here's my list of ways to save money on qualitative research -- some more obvious than others. You can get a PDF of the full article, with more details on each method, here.

Here's my top 10 ways, in brief:

1. Keep a tight focus on objectives
2. Reduce the number of participants
3. Cut the travel
4. Manage the number of markets you touch
5. Manage back-room catering demands
6. Decide if you need transcripts, video and other goodies
7. Work with realistic deadlines
8. Remember that the screener is not a survey project
9. Consider your written report needs
10. Be a valued client, easy to work with

These are times when organizations need insight more than ever to compete, but every
expenditure is under scrutiny. A collaborative approach with a skilled researcher will help you
get more for your research dollars without sacrificing quality of insight.

Resources

This piece was published by AMA in their newsletter as "Good to the Last Drop: 10 Ways to Save Money on Qualitative Research".  You don't need a subscription to read the article, and there's a nice PDF here.

New Qualitative 101

Greenbook, in conjunction with my buddies at QRCA, have just come out with a directory of New Qualitative providers.

Quite apart from the fact that I am listed in it... you would find it useful for a few other reasons:

• Great basic articles telling you what you need to know about various methodologies. Well written by experts Betsy Leichliter and Jay Zaltzman
• Details and contact information for the major platform providers
• Listings for researchers experience in these methods

You can get a hardcopy for $29, but check out the digital version for free, here.

There's also a blog (I'm writing there too), which has some fantastic contributors, all people who've done a fair bit of heavy lifting in new methodologies, and will be sharing best practices, new ideas, and whatnot.

It's good news to finally have a resource in one place, and I'm sure it will grow as providers seem to be expanding continually. Many of these providers were simply lost in the giant directory that has everything from survey sample to focus group facilities listed.

So now we have resources in one place, plus information too. And that's just great news for practitioners and clients!

 

Conversation in slow motion: How to observe a bulletin board focus group

I'm currently moderating an extended online B2B project involving business execs in a discussion forum. It's somewhere between a bulletin board project and a community, because the same people have been involved for several weeks.

It's demanding for me, but hey, I designed it, so I only have myself to blame.

But extended online qualitative is really challenging for the clients, who have many demands on their time in a typical workday. When the data really starts to build, after a few days in a project, keeping up can be overwhelming.

It's all well and good for me to have insights and distill them into nuggets of wisdom. But they're as good as useless if the client doesn't learn anything. And the best place to start the learning is in the backroom, just as it is with F2F research.

Here's what I've learned that seems to help people engage:

Tips to help the team stay engaged, and help your moderator manage.

[1] Log in early and often

The first day, it often feels like nothing is happening. Team members who logged on expecting activity can get discouraged, and when they look at the transcript later, it can feel a bit depersonalized, and a lot overwhelming.

If you log in frequently, you will see the same ebb and flow the moderator sees. Some topics attract immediate interest, others are slower to build. It’s easier to see where participants are spring-boarding off others’ ideas. Even though the transcript is time-stamped, seeing the evolution is a richer experience.

[2] Schedule team meetings to review insights

One team scheduled daily debriefs in a conference room where they projected the discussion and talked, just like they would in a backroom setting. Snacks included.

[3] Divide up the work of observing

Assign individual team members to cover off one or two segments in a multi-group discussion forum, then report back to the team on their segment. Another approach is to divide up by days (or weeks, for longer projects).

One project team held a group meeting where they projected the evolving discussion and reviewed interesting things together, just as they would in F2F research in the backroom. What a great idea! 

[4] Add comments

Observers can usually add comments to the transcript that are invisible to participants. This is a great way to capture a budding idea, and can help the moderator see possibilities for probes.

[5] Ask for follow-up probes right away

The faster we can ask a follow-up question, the better. Going back more than one day in a discussion tends to sap the group’s energy, and often does not work anyway.  Have a plan in place for how you will direct these to the moderator, to prevent frustration on both sides.

As a moderator, I may make the decision to bring something back in later to the discussion -- even days later. Or I may know that the topic already came up earlier. So if the moderator doesn't respond to your request, try not to get frustrated -- they are likely not ignoring you, just trying to manage the flow in the best possible way.

Acknowledgements

Portions of this article were previously published on the NewQual Blog as Bulletin Boards from the Backroom: 5 Tips for Observing a Conversation in Slow Motion.

If you haven't seen the NewQual blog or directory, you should check out this new resource: NewQualitative  The directory (finally!) brings together all the providers of online research platforms, and it's a good place to discover other resources and content.

Is DIY research going to put us all out of work? The debate rages on!

There's an interesting debate raging around about Do-It-Yourself Research. I posted this comment earlier today on Power to the People, an article by Joshua Hoffman on the Research Access Blog.

As a quallie, I am well aware that lots of people think what i do looks not just easy, but dead simple. Relatively few clients want to do their own, although some do. The usual reason appears to be cost, from what I can see, as recruiting is no faster or slower for them or me.

I find client's analysis of qualitative is usually of the fast kind -- first impressions. If they did more analysis, many of them could come up with what i come up with.

The real issue for me is more one of raw capability. People who have the capability to do from an analysis perspective what I do certainly exist inside client companies, but they are typically senior managers and executives, and are primarily in a managerial or strategic role.  If client companies were prepared to hire people at a senior level to do this work, they could likely obtain great results.

A couple of other things are often missing when clients DIY on qualitative. The longer the person is in the job, the less breadth of exposure they get, the less ability to bring in contextual learning from a broad base of clientele. 

For some methodologies, it could be difficult to get people to tell you the unvarnished truth -- i say this because even as an independent, I (and my colleagues) often have to go to some lengths to assure people of true anonymity in their feedback. I struggle to see how this could be done by an employee.

I also -- as do many consultants -- spend a truly massive amount of time and money on my professional development every year. I attend conferences, I speak at conferences, I attend training events of various sorts, I participate in several professional organizations, I read widely.

My clients, on the other hand, are as busy as small mammals, scurrying on their wheel, often without the time to really think about anything in depth. Their voicemail says they are "in a series of meetings", and that menu rarely seems to change. Of course they have access to learning opportunities, but most are mandated by the company, not chosen by themselves for what they bring.

Even large research organizations have relatively few qualitative people involved in the world outside the corporation, which I would guess to be for cost containment reasons. They send one person to a conference with the idea that they will share the learning.

There is a lot to be said for people who strive to invent new things every year to compete. If I worked inside the client organization, I might not feel the same level of pressure to keep up and be inventive to get ahead. It would be easy to fall into patterns. As it stands now, I am constantly trying to figure out how to either add more value, or ratchet down the third party costs somehow.

Finally, I think the cost argument is upside down where qualitative is concerned. The senior people in qualitative would be prohibitively expensive to hire on a full-time basis, with the benefits and associated overhead.   Just think of the lost productivity in meetings alone!  

It is great to see the tools getting better to do technology-supported qualitative. I love it. I hope that the price drops, as surely it must if DIY is going to take off. Because feature-creep in online research has led to price creep, when it should be going the other way.

Some of the things raised in this article and others are relevant to the qualitative world as well.
* I wish that I could have more access to the learning from the CRM system when i am in the planning or reporting stages. I think clients themselves are struggling to integrate and effectively use all their many sources of data.
* I wish for better long-term collaboration. The best do it, the rest should.
* I wish for better semantic analysis tools -- not to replace me, but to make the work faster and cheaper.

But the good news in all of this is that clients want and need insights. And they don't want to pay 24 carat prices for fool's gold. And that's a good thing for those who can embrace the new reality.

What do you think about all this? I would love to hear your comments, and I know others would too.

[Hey, if nothing else, you can avoid writing that report. And that's gotta be good, right?]

mobile qualitative - ready for your phone today

This week I am participating in a demonstration of mobile phone texting as a method of gathering insights.

Not a new methodology, incidentally -- people have been using mobile methods for a few years now, but it's really starting to come into its own with the proliferation of smart phones.

In fact, I recall a large-ish student project of a few years ago, (bulletin board methodology), where i gave up using e-mail to communicate with students because they responded so much better to texts!

Back to the demo. Mobile methods for qualitative now come in many flavors, from simple Q+A gathered up into a spreadsheet at the end, to fully integrated texting in a discussion forum platform. 

Here's what I'm noticing, two days into the study:

- Far from being an irritant, I am finding the quick text messages a welcome interruption to my work a couple of times a day.

- I'm curious to see what the researcher is asking next. (Yes, it seems that a good and engaging question is as crucial here as it is anywhere, perhaps even more so!)

- I got a "great idea" response from the researcher that made me smile (the usual pats on the back we give to reinforce participation worked just fine on me too!)

I'll be interested to see the aggregated results of this demo study in a webinar later this week (put on by the Online SIG of QRCA.)

I love the idea of using this approach for a warm-up or follow-up to a more in-depth method. It's in the moment, very unscripted from the standpoint of the respondent, and reaches me where i am right now. The possibilities suggested are really wonderful!

If you have experience with this method -- either as a participant, or as a researcher / client, I'd love to hear your thoughts here in the comments.

Focus groups are a waste of time: or are they?

Researcher Chris Kann, (pictured left) and I spoke a while back about the reasons that people bash focus groups. The result of that discussion was an article that was published in Greenbook, called Don't Blame the Focus Group.

One of the most frequent negative examples trotted out about focus groups is the Herman Miller Aeron Chair story, a story of a very innovative product that was so foreign to consumers when introduced, that initial reactions were not all that positive. This story is often told as a cautionary tale about research -- it shouldn't be.

"That Herman Miller management chose to go ahead with a launch of the chair despite these findings and generated a massively successful product, is taken to mean the focus group did not do its job.

But perhaps this is a story of a focus group doing its job well, and management doing its job well too. Management knew what initial reactions to the new product would be and could plan accordingly. The history of the Aeron chair on the Herman Miller website in fact makes it clear that extensive consumer research, including focus groups, informed the development of the product."

We think it is pretty ironic that companies are invited to listen to consumers more, but only in certain ways.

I'm pretty sympathetic to the challenges of management in client organizations. So is Chris. The truth is, we've both been on the other side. Managers have to make a lot of tough calls, and they hope the research will help. And it often does help. But no research, regardless of method, can save leaders from the pain of making actual decisions, and some decisions will inevitably be wrong.

Bad research won't help your decisions much. But good research will save your hide over and over -- save you from making expensive mistakes, help you see the potholes in the road to success, and point to new opportunities.

Focus groups are not the be-all and end-all. But there's a reason they have stayed around, and it's because they are a good tool in the hands of a skilled researcher.

I'd love to hear your thoughts and experiences.

 

Resources:

Don't blame the focus group for leaders' shortcomings, by Susan Abbott and Chris Shields Kann, Greenbook, 2011.

About Bill Stumpf and Don Chadwick, designers of the Aeron Chair

Company history of the Aeron chair, including video with the designers.

Focus Groups: how can a table and some chairs cause such controversy?

Dan Pallotta created an interesting range of comments in response to his trashing of focus groups on his HBR blog post, Real Leaders Don't Do Focus Groups.

The comments are a much better read than the actual post, which coughs up the same tired mythologies you've already heard a hundred, if not a thousand, times. [e.g. Henry Ford and the "faster horse" soundbite, the "Apple doesn't do research" myth, and more of the same.]

Mr. Pallotta talks about his experience creating a successful charity fundraiser ride as if he created it in a complete vacuum, but it was successful because it was bold.

There are truths to be made on both sides of any controversy.

Truth #1:  Good ideas are sometimes killed with bad research. I've experienced this personally, although I'm bold enough to claim that I haven't contributed the bad research, just some good ideas that died at the hands of others.

Truth #2: Leaders who claim to not need insight research are either idiots, or they have a strong pipeline of connection to customers that they are calling "not research." Sometimes the connection is a simple one -- they are part of the target group personally. More often, in large organizations, a lot of people are actually gathering insights all the time. But they might not call it that.

Truth #3:  The focus group is only as good as the people using it. [aka: It's a poor carpenter who blames his tools.]

A few of the more insightful comments:

From Tony Flanders:

"The new model is --- engage, listen, dialogue, synthesize -- oops; that's actually the old focus group model." 

[Tony has posted an interesting article on focus groups on his blog, here]

From Nick Coster:

"The opportunities for innovation exist when we are able to strip what the market says it wants back to it’s underlying needs.This removes any given solution from the discussion (ie horse or internal combustion engine car) and abstracts it to I need to get from A to B in less that 1 hour."

From Richard Harris: 

"First, congratulations. I think you're the first person to post something on the HBR blog that pissed people off.
Second, I would have been curious to know what market-sensing mechanisms you do use to decide things like how bold to be. Something made you choose 7 days/600 miles/$2000 as your standard. It sounds bold, but you weren't so bold as to go for the round trip. How did you know how far to push the envelope?"

From Matt Towers:

"On the surface, it is sort of a flat Earth mentality that advocates that less information is better than more. It bespeaks a lack of confidence in understanding how to use information to make decisions."

And this gem: "Apple does do research, they just don't talk about it."

From Steve Willson:

"The key point that runs through the discussion is that to be successful an organization needs to listen to it's customers. They are talking, so you need to pay attention.

Whatever methods you use to engage them in a constructive dialog are better than shutting yourself off from the world and emerging with a set of features that are based on what you can build, not on what people want."

Care to add your own experiences? Please do.

Truth, lies and feedback (Part 4 final amazing installment)

I've been posting for a few days on this topic of how to get truthful feedback. Here are the links to Part 1 which poses a question, Part 2 about how to seek face-to-face feedback, and Part 3 which is about a questionnaire virtually designed to elicit falsehood.

Today, I want to look at the most challenging of all truth-finding situations, the police investigator.

This is a situation that makes the work of commercial interviewers of all sorts look like absolute child's play. People generally volunteer to talk to us, most times we actually pay them. The worst downside they are likely to experience is cognitive dissonance or perhaps embarrassment, possibly boredom.

The legal investigator, of course, is faced with someone highly motivated to lie for a number of reasons, social, cultural, psychological and rational. Everything is working against them.

This is why I found a recent presentation I attended by Donna M. Pence, a retired Special Agent who is now an educator, so totally absorbing.

If I was marketing her course, I'd probably call it something like impossible interviews, but she called her topic "When getting the truth is critical: Interviewing techniques used by law enforcement professionals".

It's not like on TV

One of the most interesting things Ms. Pence mentioned was the length of time spent preparing for, setting up, and actually interviewing.

There's the interview-before-the-interview, which is designed to develop a sense of who the subject is, how they respond to non-threatening questions, and even what their general comprehension level is. Then they seek to understand the answer style when under pressure, in a state of fear or anxiety. The protocol she teaches is called PEACE: Preparation, Engagement, Account, Closure, Evaluation.

She was very clear that every question asked has to have a defensible purpose that will withstand questioning in court. No "fishing expeditions" allowed.  This is a key difference from insight  research work -- our questions need to have a purpose, and do, but we are almost always going on a fishing expedition.

When you expect everyone to lie, you plan differently

Like a good insight seeker or facilitator, the good law enforcement interviewer sets the stage in terms of the environment, privacy, clothing, tone and manner, the actual interview space. They consider gender, ethnic and racial matching (matching or deliberately unmatching).

Unlike insight researchers, they are often seeking to disarm or destabilize. We saw her shift roles and accents like a skilled actor, and honestly, I was really glad I wasn't interacting with her professionally, so to speak!

The level of detail required to support criminal charges is, in her words, "excruciating". So one thing that is probably true from TV is the phrase "let's go over this again".

I truly loved some of her phrasing. Let me give you an example:

"I've never called anybody a liar. I say, 'I think you're not being as truthful as you could be.'  Or for a good ole boy, 'That dawg won't hunt'"

My top takeaways

These are approaches that we can all learn from, whether you are interviewing, or just trying to be a better communicator.

[1] Cognitive interviewing -- show the subject that you see them as an individual with a unique set of needs. In turn, present yourself as an identifiable person. Depersonalizing reduces the amount of information you will obtain/receive.

[2] Interact meaningfully -- don't just ask a list of predetermined questions.

[3] Communicate empathy -- see the situation from their point of view.

[4] Give your undivided attention -- convey that you want to listen and can be trusted. Give reassurance and support.

[5] Let them talk until they run out of steam.

Why does anyone talk to them?

The most amazing part, for me, was Ms. Pence's answer to my question of why anyone really talks to them, despite her persuasive methods. Her answer provided telling insight into the human condition.

For the most part, she says her tone is clear, calm, slow and matter-of-fact. She has often interviewed people who have violated foundational cultural taboos, such as child molesters, rapists, murderers.  She says things like, "You can't offend, embarass or surprise me", "I'm not going to be upset at anything you tell me. I've talked to many people in your situation".

While people are initially motivated to talk to get themselves out of trouble, if possible, they also want to give their side of the story, to provide an explanation for what happened. 

My take

We all live the existential problem -- we are alone in our own experience. The ability to share that inner world -- even a little bit -- is a powerful drug. I often wondered how law enforcement officers get people to tell them the truth, and I think this is the essence of it: empathy.

And on a less sobering note, if you meet someone like Ms. Pence in a police station, you are in serious big trouble, because she is one amazing interviewer.

Resources

Donna Pence recommends the book, Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage, by Dr. Paul Eckman

All of the graphics in this post were created by me, based on diagrams presented by Ms. Pence.

I heard Ms. Pence speak at the annual QRCA conference. If you love qualitative methods, you should join us sometime.

Common research mistakes

I logged in to Constant Contact a few minutes ago to start writing a newsletter. There was a button for completing a survey, so being a naturally curious [some might say snoopy!] person, I clicked.
Constant Contact now provides a survey platform. Sadly, they made several common mistakes. So egregious and common are these errors, it seemed essential to post about it and save others from the same mistakes.

This is a screen shot of the first page of the survey. I see three mistakes here:

First mistake -- ask for my name right up front. If you are going to make this non-confidential, you might want to ask for this information in the last question. Because then if I don't answer, you still get the rest of my data.

Second mistake -- non-exclusive categories. If I am in business 6 years, I have two choices. Same problem with 8 of the response categories. Oddly, the sequence changes from 8-9, and 10+.

Third mistake -- the person who wrote this survey has insufficient knowledge for the task. Two easy solutions. Either take a survey writing course -- there are lots of them around, and some are available online. Or, spend a small amount of money and hire a researcher to write the questionnaire for you. Seriously, this is not a complex questionnaire. It would be likely a half-day of work for an experienced person.

Or, you can just get poor data and look like a goof to the world.

You know, the thing is, the fact that I can use a power saw does not make me a carpenter.

Free e-book: Painless Insight Planning helps you set objectives, avoid wasting money on qualitative research

 

I've refined a number of tools to use in my work over the years. A few of them are tools designed to help clients in one way or another. This free e-book, Painless Insight Planning, is one of these tools with some narrative wrapped around it.

[Download Painless Insight Planning now]

One thing I've found that is often a challenge at the start of a project is getting clear objectives. People may have a problem or concern, but it is often not clearly articulated.

And I can tell you one thing for certain: a good way to waste money on research is not having clear objectives at the beginning.  I know this sounds perfectly obvious, but it is an issue more often than you would suspect.

Sometimes the problem is clear, but other elements are not. Such as -- who could we get insight from, otherwise known as "the research target."

I can't sit down with everyone individually and work through this, as I do with clients. So I thought I would put the whole thing into an e-book that you can download and use as much as you want. And then you'll have clear objectives for your research projects. And the world will be in a better place.

Because we really need insights these days, more than ever. And people inside companies can't afford to waste money or appear to waste money, get fired and be unemployed. Suppliers like me also can't afford to waste client's money. We all need to ramp UP on the insight quality and the innovation / ideation, but keep a lid on costs. 

What's it worth to get this answer?

An even harder thing for many clients is figuring out how much to spend. They ask me, what will it cost? 

WRONG QUESTION!  WRONG QUESTION!  WRONG QUESTION! 

The right question is "what is it worth to get good information on this situation?"

I have created a nice little rule to help figure this out. Not a complicated ROI on research thing. Just a back-of-the-envelope, cocktail-napkin kind of formula that I call the 10 Times Value Rule. It's a great place to start figuring out what you should consider spending to get some insight into your problem / challenge / situation.

[Download Painless Insight Planning now]

Easy worksheets

I hate filling in forms. Most people do. So there aren't forms in this book. There are easy worksheets designed to spur creative thinking to get you started. Here's an example.

Bonus content

If you download the book and actually read it, you'll find a link to some bonus content that will save you even more time. 

A small request - no strings attached

I'm simply putting this handy e-book out there for free with no strings attached. You can even add this e-book (with no changes) to your own site or blog for your own customers and prospects to use.  But I do have a small request.

If you think this e-book is of any value, please add this blog to your RSS feed reader. I also send out a newsletter about six times a year. You can add yourself to that list if you want to hear about other e-books I plan to release, or just get the newsletter. You can check out the previous issues here.

Thanks for reading. And good luck with your insights research projects!

[Download Painless Insight Planning now]

Listening right vs. listening wrong: Google Latitude and sampling

You've likely heard about a new service Google has launched called Latitude, that can enable you and your pals to track each other geographically via your mobile phones. There's a bit of controversy about the potential abuse of the service for stalking, but that's not what I want to talk about. At least not directly. I tested this concept a couple of years ago in research. Let me tell you about it.

About two years ago, I was planning a session with some grad students in a research program. You know the drill -- guest instructors who are practitioners come in and show their stuff. So I did an advance study with the group using an online bulletin board method. Then we talked about it in the class.

To try to find a topic of interest to all the students of various ages, I picked cell phones. Among other things, I cooked up a number of possible features and enhanced services that we concept tested. One of them I called "Friend Finder", which was essentially a product like Latitude.

I thought this Friend Finder concept was pretty clever, actually, and that if I tested such a product concept with a broad base of young adults, that it would likely be well received. I was really surprised when this group of research students reacted pretty badly to the idea, and were REALLY worried about privacy. Freaked-out and creeped-out about privacy, actually.

They also didn't like "Taxi Finder", something I personally would love. And they didn't like "Bus Finder". I don't think there was one concept that tested well.

So I pondered this finding ... and here's what I think.

This was not an average slice of humanity in any respect. Most of these people were into quantitative i.e. they enjoy statistics. A number of them had just finished a four-year degree and were running out of money. Several were mid-life career changers who were also pretty cautious about their spending. And they were in a small city on the edge of the Canadian Shield (lots of big rocks) and they told me cell coverage was often spotty. [Who knew? When I go up there to canoe, I'm pretty much ignoring my cell phone.]

And it was not a well targeted slice either.  A number of the students turned out not to have a cell phone at all, or used it very sparingly. In a real study, the client most likely would have been looking to gain insights into only those people who were already pretty keen on their cell phones and were already using some advanced features.

My colleague that day, Tania Heintzman, was personally VERY keen on Taxi Finder. But she uses taxis a lot and students don't. Off target.

I personally was very keen on Bus Finder, but the bus service in small cities is often terrible and so irrelevant to this group who all drove to campus. Off target again.

Basically, researching any number of the wrong people does not help. A bigger sample of the wrong people does not help. Statistical significance would not get you out of this issue either. The wrong sample is the wrong sample.

Sometimes only a few of the exactly right people can help you dive very very deep into powerful insight. So you really need to figure out who those right people are. Especially when you are trying to see into the future with new concepts for products.

If something like this happened in a real project, it would be a mistake to conclude that the concepts had no potential market. Because clearly, Latitude will find users. And all sorts of people are developing open-source products like my "Bus Finder" idea, even here in the frozen wastelands of Toronto. Soon it will be available via smart phone apps. 

On the other hand, some of the learning would have been valid. For example, the concern about privacy. That's an attitudinal and behavioral segment that you would want to understand. Even knowing about this issue would help you design a better service, and avoid the criticisms Google is getting now.

Innovation breakthroughs: now really IS the time or why recycling does not work for insights.

 

People are cutting back. [Oh, you heard that, too?]

I've had a couple of projects cancelled or postponed indefinitely in the last short while. One of them was to help an organization identify new markets so they could escape the commodity price-cycle trap of their current markets.

This is exactly the kind of project you definitely should not cancel -- projects that will help you crack new revenue sources. So what will this company do instead? I'm guessing, they will just push harder at existing markets, and find themselves in the same place a year or two down the road.

While having these thoughts, I bumped into a nice article in Marketing Profs. [Now that the economy is tanking, maybe we can catch up on our reading.] The quote from Jeffrey Immelt leapt off the page.

General Electric CEO Jeffrey R. Immelt has established a class of projects now well-known as "imagination breakthroughs," consisting of ideas that are "really hard or really important" and might generate significant revenues over a three-year period, the time that GE usually takes to implement a new idea. "Imagination breakthroughs are a protected class of ideas — safe from the budget slashers because I've blessed each one."

If it takes you a couple of years or more to really get an idea off the ground, then the fuzzy front end you cut now will leave you in even more serious glue in two to three years time. And if you kill the stuff already in the pipeline, you're not going to have anything new to launch in 2009 either.

I really wish clients would say instead: "Can you get me something substantive with less money?" That's a better challenge than dropping the idea creation process.

Caveat

The article this came from is filled with good ideas from the Zaltmans. But like many of their ilk, they firmly believe that no approach other than their technique (individual metaphorical interviews) can get to the good stuff. Not true, in my experience.

"These deeper ideas are not available from survey responses to statements such as "financial planning is difficult" or in focus groups where each participant has on average about 10 minutes of air time." 

OK, you are definitely not going to get these deep meanings from a survey. I agree. But individual interviews are not the gold standard either.

There may still be people running focus groups with 12 bodies in them. However, most of the kids I hang with have shifted to small groups when they are using a group discussion method. Small being five or six. And researchers frequently add advance activities such as blogging / electronic diaries / collage assignments / videos / you name it. And there's no rule about 90 minutes or two hours either -- you can keep people humming along for three hours quite easily. And yes, you can uncover metaphorical meanings in focus groups, in online research methods, and in many other ways.

In truth, the generation of insight has a lot more to do with the skill, knowledge and art inside the researcher's head than the method employed. There's no method going that guarantees insight.

As anthropologist Grant McCracken has said, the proof is in the pudding. If you got insight, you got insight. If you didn't, you didn't.

I have friends in the marketing world who have used Gerry's Zmet technique to great effect. And I believe all of methodological infighting and territory claiming must defer to this. If the method works, the method is good. The proof of the pudding has nothing to do with the theory of the pudding or the method of the pudding. The proof of the pudding is a client who says: "This was illuminating. I understand my consumer and my market in ways that I did not, could not before."

Back to the main point

This is no time to be cutting your future revenue streams by shutting down the insight machine. In fact, if you plan to be standing when things turn around, you should be cranking up the insight machine. Certainly, your customers and consumers brains are grinding away on many things right now. Not a good time to recycle some old insights.

Resources:

How to Foster "Workable Wondering", by Gerald Zaltman and Linday Zaltman. In marketing profs, reprinted from HBS.

The Zaltman Method, by Grant McCracken in his blog, 2006.

Cool new tools that enable feedback

I went looking for one thing online, and found a bunch of other stuff. Don't you love doing that on a Friday morning? So of course I wanted to share...

There are just such a variety of interesting online applications for assisting collaboration now. If you can add some others to the list, please do. If you have used any of these tools and have comments about them, please add your thoughts as well.

Mapanno Markup

This markup tool lets you invite comments from others on an image or a project. It feels a lot like Google Maps, where you can add an icon and comments, and stick it to a specific place on the document. Here's a screen shot that gives an idea...

Review Basics

I have actually tried out this tool on a demo basis at least a year ago, and it looks even better now. like Mapanno, it is currently free. But it has a broader range of functionality, and was designed more specifically with marketing folk in mind.

Here's a screen shot of what the markup environment looks like. There are bubbles that you can use to enter text and a variety of marking tools as well.

 

Review Basics also supports common video formats, as well as Word, PowerPoint, and PDF files.
I'm itching to try this out with a research project, but I think it could be really useful on a team internal basis as well. Right now, it's free, just like Mapanno.

Vyew
Vyew is one I just learned about, thanks to GeekBlog. I haven't had time to demo it at all, but it positions itself as a competitor to WebEx and GoToMeeting. In addition to those services, however, there is the potential for asynchronous involvement by multiple reviewers who can mark up documents, and view each other's comments.

Here's a comparison chart that may help.

There are some interesting case studies on the site as well, that make it clearer how one might use such a multi-featured application for things other than web conferencing. Like the other tools mentioned, it is currently FREE, and does not require software installation. There's a review by Guy Kawasaki

Concept Share

This service is specifically targeted to graphic designers. I haven't tried it out, but it looks pretty appealing. There is a free-premium model in place which affects the number of projects you can use, and the amount of online storage you get. Huge amounts of storage are available, which is needed for graphic files.

 

Basecamp

I've been using Basecamp for a few months now. Although it's not perfect, it's a very useful and low cost collaboration tool. I haven't figured out a way to use it for research projects yet. But you can store files, there are whiteboards, a really good messaging function that beats running things on e-mail, and some project management tools.

This stuff all just continues the flattening process where little tiny organizations have access to tools and technology as good as the stuff the big organization's have. Sometimes it's even better.

If you want to read about even more of these tools, here's a reference article from GeekBlog. And their handy comparison table.

Global Village Study on Teens and Technology

About this time last year I was helping to plan a global study on teens and technology use, called Global Village. This idea was cooked up by my colleagues Ilka Kuhagen (Germany) and Corette Haf (South Africa) as a way to demonstrate the power of using bulletin board research to gather global insights.

We sponsored this research as a group, and conducted the research with 12 groups. This was an impressive team of researchers, it was my honor to work with them. 

During the planning process, we had many logistical issues to work through. For example, our platform sponsor, Focus Forums, had to add Mandarin character capabilities to the bulletin board.

A decision was reached to focus our efforts on teens aged 14 - 16 years old. A challenging target in some ways, as ethical guidelines require parental permission to conduct research with this age group, which added to the logistics of the project. But we saw that this group is entering young adulthood with access to a level of technology that they have enjoyed since childhood. We wanted to see where things were heading with this group. So we found teens that had a cell-phone and used the internet relatively heavily.

The bulletin board was a fantastic tool for the project, and the teens loved the format. Age and gender barriers that would have been a major burden in a face-to-face setting evaporated, and we had high engagement and considerable openness and honesty.

In addition to this, the research team was also easily able to watch the research unfold, and share our own discussion about questioning strategies over the project schedule. This meant we could change and shift the later stages of the project based on earlier stages.

It was fascinating to see the style differences among the research team. As findings started to emerge, we were all impressed by how similar, in many ways, the attitudes of the teens were in each country. In picture sort exercises, for example, the same concepts -- and even the same images -- came up over and over again. We weren't really expecting this, it was a fascinating finding.

One thing is for certain -- these teens are joined at the hip to their cell-phones, and this is their life-line. If their house was burning down, it's one of the things they'd grab. The research team found text messaging a much more effective way to send out reminders to participate. (With an adult group, we would use e-mail). Almost none of the teens in the whole study had heard of Second Life. A few had been on Habbo Hotel, but these sites no longer held  -- they are way too busy forming real-world relationships to care too much about the virtual world. Yet.

Overall, a fascinating group of teens. The Canadian group were very much "the New Phoenicians" -- ethnically diverse, open to new ideas, enterprising and optimistic. I wanted to give them all a virtual hug, they made me feel proud to be Canadian. I have no doubt the other researchers felt the same about their own youth.

I'll be presenting on this project with my colleague Betsy Leichliter from New York at the ARF conference next week, and also in Winnipeg at the MRIA conference in May. Other team members are presenting at various conferences around the world, including the upcoming AQR/QRCA conference in Barcelona. The slide at the top came from a presentation done by Diva Oliveira in Brazil.
 

You can also read the write-up that Ilka and Corette did for the QRCA publication VIEWS, which is available free online. How Global is Your Village?  (the link spawns a PDF. Article is on pages 18 - 27)

You know advertising is hot when it's displacing IT among IT research houses

I always thought of Forrester as an IT-oriented research house. And then they were on stuff like supply chains and outsourcing. However, they are now getting talked about for their thought pieces on where advertising agencies are going.

This suggests two things to me:

1. Forrester is good at figuring out what's hot
2. Marketing, advertising, and brand building are hotter than they have ever been, but also under more scrutiny than ever.

I have other evidence of this trend.

The Globe and Mail, my daily must-read newspaper, never deemed marketing topics worthy of business coverage. [Well, any kind of coverage, actually.]  Not long ago -- within the last 18 months for sure -- they have added regular material on this topic. It's thin, but it's there.

Friday, I heard Terry O'Reilly speak at a conference on qualitative research. Mr.O' Reilly is a regular feature on CBC radio, and known for his expertise in advertising, as both a creator and a commentator. Here's one of the notable things he said on the topic of insight and advertising:

"Creative ideas born of intuition are the last legal means of gaining a competitive advantage over your competition."

I thought this was a) brilliant, and b) true.   

Forrester seems to think that advertising agencies are not helping marketing people connect with consumers and customers in a meaningful way. This is the summary of their recent research report, The Connected Agency: Partner With An Agency That Listens Instead of Shouts

"Today's agencies fail to help marketers engage with consumers, who, as a result, are becoming less brand-loyal and more trusting of each other. To turn the tide, marketers will move to the Connected Agency — one that shifts: from making messages to nurturing consumer connections; from delivering push to creating pull interactions; and from orchestrating campaigns to facilitating conversations. Over the next five years, traditional agencies will make this shift; they will start by connecting with consumer communities and will eventually become an integral part of them."

I have not read the whole report, but I am having trouble seeing this as news. And it's far too easy to just blame the agencies, who are under tremendous pressure to deliver results these days. But they can only do what they get a mandate and a budget to do from the marketing people. Surely marketers themselves must take responsibility for connecting with consumers.

And so must executives.

Being in the insight and intuition business myself, I can only agree with Mr. O' Reilly: we need better strategies. We need insights. We need innovation in marketing and advertising.

At the same conference where Mr. O' Reilly was a headliner, I presented with a client a case study on some innovative research we did together. When we got to the Q&A, the biggest question that came up -- which we brainstormed as a group -- was HOW to convince executives to do something less conventional than a focus group or a survey.

These researchers and marketers -- they want to deliver innovation, they want to deliver category-changing insight, but feel challenged to sell it into the organization.

We came up with a flip-chart full of ideas of how to do this. Things like: start small, start with intractable problems where there is motivation to try something different, and so on.

But as one of the research buyers said, "We want innovation, we want to do something new, and we want amazing insights, but we only want it once it's been done before. How do you sell this into the organization?"

Of course, we on the practitioner side have another challenge. Having promised these category-changing insights, we need to stand and deliver.

I only wish these insights that can change the landscape could be produced on demand. [I'll be hunting for a good one this week, wish me luck.]

Facebook Polling: a quick tour and a surprising finding

After holding forth about the future business applications of Facebook, I decided I should put my money where my mouth is, and do a quick little research study.
Facebook makes that easy for you with their polling feature.

Here's what I wanted to know -- what kinds of applications for business would interest people who are already on Facebook? So here's the question I asked:

What application for business would you be most likely to use on Facebook?
[a] A way to post RFPs / find suppliers?
[b] A way to bid on RFPs / find projects?
[c] A project team collaboration function / group?
[d] A way to sell content like e-books / e-learning?
[e] I would not use any of these

Facebook polling is so inexpensive, I thought I could afford to get 100 responses, which would cost about $25.00. [I do want to be clear that this is not a scientific poll, it's just a way to take a quick read on something.]

I couldn't select exactly the group I wanted -- freelancers and SMB people. So I went for age as a proxy, and asked for age 35 - 49. I was surprised that the oldest selection was age 49, actually, since surely there are many people using Facebook over the age of 49. I know several well, in fact.

I sat excitedly at my computer waiting for the colorful dashboard to show results. Which it did, looking like the chart above, with several other colorful charts breaking down my poll by age and gender.

Surprising Finding

When I had 49 responses, Facebook shut down the poll.

Before they did that, here's what I learned:

1. About half would be interested in one of the proposed functions
2. About half indicated no interest in any of them
3. The most popular of the proposed functions was [c], the project team collaboration function. This is the one I want too.

I received an e-mail from Facebook explaining that they had shut down my poll.

Hi, Susan,

Facebook Polls are intended for market research purposes. Facebook Polls created to promote products or services or that are otherwise deemed not appropriate are removed accordingly. Along these lines, Facebook has automatically removed your Facebook Polls (order #Facebook Poll).

To see the final results, go to: link 

Responses Desired: 100

Actual Responses Received: 40

Price per Response: $0.25

Your credit card has been charged an additional $10.00 This completes your order.

The Facebook team

I'm surprised that they thought this was promotion. So maybe it was deemed otherwise inappropriate. Perhaps the only people allowed to do research on Facebook itself are officially authorized. Perhaps using the word Facebook in the poll flagged my little project. As far as I know, they can't stop me from blogging about it.

Anyone have some insight into this?

I was thinking it would be cool to hire some savvy Elance developer to just make me the application I want, but now I'm reassessing.

Holiday notice

I'll be skiing in Banff. If you're there, say hi. I'll be the person doing the face-plant right in front of the lift line, wearing a fuzzy hat with ear flaps [that looks really cute, actually]. So I likely won't have the strength to post after a hard day of skiing. See you in the new year! And thanks for all the reading and the comments, which really make it fun to keep at it.

Jason Oke talks about what's wrong with qualitative research

Jason Oke is a planner for Leo Burnett -- a guy who spends a lot of time thinking about finding leverage points for the creation of great advertising. And is, incidentally, an excellent speaker. He thinks there's a lot of bad research being done, which has created a credibility problem.

Chief among his complaints is that there is far too much direct, literal Q&A going on, which we know -- I mean we really know this -- is not very useful. This kind of technique lacks so much subtlety, it is akin to "strip mining their heads," according to Mr.Oke.

By literal Q&A I mean approaches like asking people to tell us "what's important to you in a vodka", which will generate nothing that can help a vodka marketer. If the target group even knows, which is unlikely, they are hardly going to say things like, "I want to feel like part of an exclusive club" or "I want a wee boost to my social status."

This is not just true for products like vodka, by the way. I just completed a business study in which one group of people were clearly motivated by reducing their fear level. Now just how likely do you think they would be to [a] know that and [b] say it out loud? In fact at least one of them did know that, but she didn't say it, she wrote it down as part of an exercise. [Of course this creates an equally large challenge for the creative team, but that's another article.]

Jason Oke also believes that people are bored by the idea of sitting in a dimly lit room for two hours being poked at with inane questions and limp exercises. He thinks this is a major reason why it is getting difficult, and expensive, to recruit participants for focus groups. This was a new idea for me, and I think he's on to something.

Mr. Oke's recipe to turn the situation around:

[1] Take brain science into account.

People can tell us many things, but they don't have direct access to unconscious motivators. When we ask them questions depending on this access, they will tell us something, but it's unlikely to relate to the problem at hand. It may not even be true. But most people will generate an answer to a question like "what's important to you in a vodka" just because people are very good at generating reasons.

[2] Plan research activities that meet the needs of #1, and are also fun and engaging.

Mr. Oke's solution to the Vodka problem, for example, is to ask small groups of people to design a new vodka and how it will be marketed to people. Exactly. Generates great data, and is a lot of fun for the participants. He showed us clips from the tape, it was brilliant.

[3] Stand up to clients when they ask for bad research. Evangelize for good research.

The thing is, once clients have experienced really good research, they are left unsatisfied by the other kind. At least I think that's the case. I hope and pray that is the case. But it feels risky the first time you get on.

[4] Think of research as a brand touch-point.

Absolutely correct, and great advice.

Resources:

Jason Okes' blog is here

A related Malcolm Gladwell blog post is here

Clips from the Prelinger Archive film about marketing research, here

Book he recommends [I have not read, but it sounds great]: Herd by Mark Earls.

Mr. Oke presented to the qualitative division of MRIA at "Qual in the Fall" on Nov. 9, 2007 in Toronto.

Rethinking research

I'm off to the QRCA conference for the week in Vancouver this year. My mission, beyond catching up with friends and colleagues, is to try to really push the boundaries of how I think about qualitative research.
The reason is: fieldwork costs keep going up, and are going up faster than client budgets are. The whole infrastructure that surrounds qualitative research -- lists, panels, recruiters, facilities, catering, incentives -- is a large part of what is costing a fortune. It ain't me.

One of the worries about rising incentive costs is that this encourages people to become professional respondents, supplementing their income by participating in focus groups and interviews. Surely we have not tapped out the adult population of North America. There must be people who would like to share their opinions without being paid a huge sum to do it. Or are people so exhausted from constant requests to complete poor/badly written/excessively long surveys that qualitative is just one more intrusion?

I think there must be other ways of finding people -- perhaps Facebook? Is anyone finding respondents on Facebook?

I'm hoping to get inspired for a few days. Pick up a few new ideas. And challenge my own assumptions about how things should be done.

And, according to my own personal tradition, I'll keep you posted about anything interesting that I see or hear there.

Designing Experiences: prototype, pilot, tweak and adjust

James Rait posted some interesting commentary in Design-at-the-Edge in response to my Curves pieces from last week. Mr. Rait's wife is a Curves member, at a location where they sometimes give away doughnuts. [I know, it's hilarious, except it illustrates just how difficult it is to manage services with consistency to standards.]

I had outlined some possible backfires to the adoption of RFID tags to plan and track individual goals and progress. These were: [1] staff having to take their own initiative to interact with clients, instead of being forced to do it by the operational routine; [2] two classes of member emerging; [3] clients losing the happy ritual of taking their pulse rate together.

Mr. Rait, a guy with serious industrial design credentials, proposes some interesting alternative solutions to these issues in his post, here.

He also suggests that running some live market tests would have uncovered the potential issues I mentioned. I would be amazed if they had not done live market tests, actually. The execs at Curves have strong marketing backgrounds, and with 10,000-plus locations, I assume their budget supports quite a bit of research.

Sometimes it takes a while to understand what is going on. Plus the hype surrounding new technology can generate enough excitement to mask the longer-term impact. These phenomenon would have been unlikely to come to light in a short test. And if the test markets were conducted with the very best franchises -- which is usually the case -- these individuals may well have responded differently than the average.

I suspect the social evolution is not yet done. [There's lots of observing still to do. Yippee!]

Mr. Rait's post made some other interesting points about design in general, which I'll pick up tomorrow.

If you go back to read the Curves posts, note the passionate comments from one fan of the company. This kind of evangelism is a huge asset to a brand.

The photo came from a film in the Prelinger Archive, Exercise and Health, 1949.

New media measurement tools: Xuuk

Eye-tracking technology is cool, but it's always been a kind of headset-and-wires operation, not something you could use in a real advertising setting.

Xuuk has launched a gizmo that can track eyeballs for posters, billboards and displays. It's a low-end disruptor -- and although it is unlikely to have all the functionality of laboratory units -- it's extremely cheap at $999. And it potentially brings a whole new group of customers into the market.

Xuuk claims to be able to detect eye-balls looking at it from up to 10 meters away. It plugs into your USB port, so you can look at statistics.

Like many technology crossovers born in academia, Xuuk could use a little marketing support to make their message clearer. But don't let that stop you. This baby could be great tool to refine your merchandising and visuals for those times when you don't have a big budget available.

What we can learn from squirrels, and other research tips

I planted my garden over the last few weekends. A few wildflowers native to this part of the world (trillium, jack-in-the-pulpit), some herbs, and the usual impatiens, fuchsia, and such. [Yes, it's true, I do have a few domestic skills.]

For days after a new plant goes in, it's a certainty that the squirrels will dig it up. I replant every morning. This morning it was the hanging planter of morning glories.
Why do squirrels do this?

It came to me this morning. They do this because they can smell something in the freshly dug dirt. They know it has recently been disturbed. And this is a good place to look for snacks -- perhaps nuts that other squirrels have buried. It must be, otherwise why would they bother?

I've been annoyed with this behavior for years, but have never stopped to de-construct it. From a squirrel's point of view, it makes perfect sense. 

And so often this is what we need to  do when we study customers. Especially when their behavior seems irrational. Because behavior is not random. It is purposeful. If it is random, the randomness itself is purposeful. It may not be rational in the conventional sense, but it achieves a result, it tips the odds, or it has been rewarded in the past. There are reasons, there are patterns, even if the patterns are unconscious, or seemingly irrelevant.

We cannot change our customers. But we can understand them if we work at it.

More collaboration tools - maybe for research?

One of the joys of having a moderately successful blog is that I now get requests from people to check out their a) book b) new research application c) creative marketing or promotional strategy.
[One could easily do this full time, if said one were independently wealthy....]
In any event, some worthy candidates have passed by recently. Here's one I want to tell you about.

ReviewBasics

ReviewBasics is a feedback gathering platform which allows professionals across many industries to exchange ideas, gather thoughts, and review various types of creative, interactive, motion, and written content. We take a familiar, paper-based content review process and bring it online." [From a news release sent to me.]

I spoke to Tim Shih today, who is VP and sales manager for this new company. Mr. Shih says ReviewBasics is, "The next generation of tools for the online marketing research industry."

The company was started by a technology consultant and a product developer, both with experience in the e-learning space. They saw an opportunity for a rich feedback platform to support collaboration and research, and this is the result. The development team is in Russia, but Mr. Shih and team are based in New York.

Going for a test drive

I spent some time on their beta site, which is very nicely set up. I particularly liked their easy-to-follow graphic walk-through (see the screen shot below). I'd love to see more applications that do this.

This next shot shows the basic interface. You have a page where you put up an image, a graphic or whatever. This could be a screen shot of your web page, or creative concept, or a new packaging design.

You then invite some people to review what you have put up. Reviewers can mark items with arrows, text boxes, call-outs, emoticons and similar tools. There is also a space at the bottom right where they can provide generalized feedback in an open input box.

All this input is gathered together for the researcher to review, but the reviewers cannot see each others' comments.

Application in marketing research

For creative testing, we usually want to have a fair bit of control over the exposure time and conditions for the stimuli. So I don't see this being the starting point for that. But it could be a nice add-on at the end of an online questionnaire, for example.

I could also see using something like this to gather ongoing feedback from users of an application in something like B2B settings. 

I'm less sure about consumer research, but for the right project, maybe it would work. It has some of the features of a web-meeting interface, but it's asynchronous, so everyone doesn't need to show up at the same time.

What happens with this type of tool is that clever people figure out innovative methods once they get a look at it and start fooling around with it. As researcher Betsy Leichliter once said to me, and I'm paraphrasing: "A focus group room is just a room with some chairs. You can do a lot of things in a room with some chairs. Online tools are no different."

Breeze was started as an educational tool that could leverage Macromedia Flash for easy e-learning presentations. Now, the underlying Breeze platform is used for a variety of different types of applications, including some online qualitative marketing research tools.

You can still participate in the free beta test for ReviewBasics, by going to their site and clicking on the "RB – Click to Review Button” near the bottom of their homepage.

Getting Your Users to Tell You What to Do

Can your users tell you what to build / fix / enhance?
Sometimes.

We're seeing a lot more instances of some kind of contest or voting process where users submit ideas, and other users vote on their favorites. To see a current example of what I'm talking about, check out this contest being run by SNAP.

SNAP is a visual search engine that allows you to preview the search results without actually having to browse to the page. Pretty cool, actually. They are asking users to tell them what to build / fix / enhance in their next round of developments. And offering prizes for the best ideas, and for the best comments on ideas others have submitted.

Will we see more of this... i.e. Should I be worried?

Yes, I think we will see more of this. It's a neat way to really open up your development or innovation process to your user community. For a relatively small cost, you can enlist a lot of good ideas. 

I'm not sure what platform people are building this type of application on [comments anyone?], so I don't really know how hard the interface is to set up, but I'm guessing it's not worse than launching a bulletin board or some similar interactive application.

Why I'm not worried

This kind of approach makes a lot of sense when you have users that are fans, and that have enough knowledge to make suggestions. If you browse the contest, you will see that many of the individuals seem to have programming knowledge far beyond the average surfer. Personally, I didn't actually understand some of the suggestions.

There is a sampling issue here -- this kind of input only reaches the raving fan, or those that enter contests as a hobby. If these are your core users, then you are in great shape. If not, you will need to find other ways to tap into the customer experience.

Finally, this approach actually asks the users to suggest new features i.e. solve the problem. A more effective strategy for sustaining innovation is actually asking your customers to tell you about their problems, or tell you about their experiences, not ask them to solve the problem.  When your own development team has a deep understanding of customer experience -- including all the joys and the pain points -- they should be in a better position to articulate possible solutions.

The bottom line
I love triangulation -- where you look at a situation from as many angles as you can. And this kind of user-input is a great adjunct to anything else you might be doing to reach your users. As a nice side-benefit, it will likely also create some buzz, and engage the hearts and minds of at least some of your community. And that's all good.

Enquiring minds want to know
Like the Joker said of Batman, "Where does he get those toys!?", enquiring minds would like to know how to launch one of these contests with a vote. Is this kind of application available off the shelf, or is it a custom programming job?

Customer Driven Innovation

At a qualitative research conference a while back, I attended a session on using games to support innovation. The people you see above are other researchers, holding up their "product box". [Sorry the pics are so small, all I had was the camera phone. And it's taken me this long to get the darn pics out of the phone, but that's another story!]

This method was laid out in Luke Hohmann's book, Innovation Games, and this was a nice chance to see it in action. Tables were given lots of materials to work with, and a plain white box. The task was to create a box for an alarm clock that they would like to buy.

Participants came up with lots of interesting ideas. For example, at my table we wanted an alarm clock that knows what time it is, just like a cell-phone does. A smart clock.

Many people mentioned Apple inspired design -- features and controls so easy to use you don't actually need to read the manual, because it's intuitive. Can you tell that this group spends a lot of time in hotel rooms, trying to set yet another clock radio, or figure out what the current local time is?

What I found especially interesting about the exercise was that the room seemed to divide into two larger groups: those who were concerned about clock functionality, and those concerned about managing the morning mood and ambiance. Could these be segments, I wonder?

During the debrief, the facilitators asked multiple questions of each spokesperson, to uncover the needs and ideas around the benefits sought.

You'll find lots of people who will tell you that "focus groups can't lead to innovation", and other blah, blah of that nature. I say it depends a lot on who is running the event. If you ask customers literally "what should we do?", you may not get that much you can use. 

But the joy of this exercise was that it focused people on unmet needs, and benefits they would derive from the features they proposed. A development team could definitely learn a lot from the output of that.  And the act of creating the box helped all of us get past "design issues" and focus on what we'd really love to have.

References:
Summary of keynote speaker Paco Underhill from the same conference, talking about global trends in retail.

This method first came to my attention through Luke Hohmann's book, Innovation Games.

 

Qualitative Research Conference: Toronto October 2006

Why you need rubber soled shoes

It's cheaper and faster than ever to collect reams of information: it's wisdom in using this information that is in short supply. And you really should have some rubber soled shoes.

These were some of the things Paco Underhill talked about in his keynote last week at the joint QRCA/MRIA-QRD conference. Paco is the hugely successful analyst of shopping behaviors and author of very successful books on the subject, Why We Buy and Call of the Mall. 

Inner Context of the Shopper

One of the most interesting observations he made is that we all shop more often, even though we live in a state of accelerated time. We go into a store with a specific mission, and if we don't fulfil it, may leave empty handed.  That mission, even for the same person in the same store in the space of the same week, could have very different characteristics, based on the value system and emotion at play due to the inner context.

For example, shopping for a child's birthday cake, shopping for tonight's dinner, and shopping for the weekend when the relatives you dislike will be visiting. And in some respects, we need to treat each of those scenarios as if we were dealing with a different customer.

The Key Question

The key question, for Underhill, is always: "how do I respond to local market conditions and build same store sales?"  Depending where in the world you are, these challenges are vastly different.

When speaking of the global middle class, 2/3 of our expenditures are discretionary. We don't need to spend money on much -- someone has to entice us to spend.

In other parts of the world, money is a new and exciting toy, and that's where all the innovation is happening in retail.  Underhill shared some excellent examples of this.

Roppongi Hills, Tokyo

Underhill described Roppongi Hills as the "All Mall". It has hotels, an art gallery, residential towers, and a multitude of recreation facilities. One of the unusual elements is the large amount of public art that has been included throughout the facility, both inside and outside.

Creative use of space in Johannesburg

What you are looking at here is a drive-in movie theatre on the roof of a shopping mall (the Menlyn). A great use of space, and a good way to bring more traffic into the mall.

Another example cited by Underhill was a mall owner who built a stadium for student athletic activities on adjacent land, solely to drive traffic into the food court of the mall.

Forget about Parking in Sao Paolo

In order to bring traffic into the mall from the nearby office towers, this mall offers a free shuttle bus service that will pick people up and then run them back in complete safety so they can shop over their lunch hour.

Finding Happiness in Tokyo

Three Minutes of Happiness is the name of a chain of stores in Tokyo that sell inexpensive items for everyday use. Here's the description from Frommer's guide:

Bargain yen shops have opened all over Japan the past few years, but this is one of the best I've seen. It carries tableware, household goods, office supplies, cosmetics, watches, sunglasses, some basic clothing, and many other simple items, mostly in bright and happy colors like lime green and sky blue.

Although these shops seem very well designed, they are reminiscent of one of the fastest growing categories of shopping at home: the dollar store, and other deep discount formats, that are the antithesis of good design.

Dubai, Dubai

Underhill compared Dubai to the cantina scene in the original Star Wars movie, and called it the first place on the globe that is ready to receive the intergalactic traveller. You can get anything you want, and you can pay in any currency you choose.  Dubai apparently has fifty major shopping malls, in addition to the more traditional souks.

The Challenge

Underhill believes that landlords of shopping malls must go from being landlords to being placemakers. And the best ones are reaching for this. But there is still a lot of room for improvement.

His main recommendation, which we heartily endorse: get some good shoes, and get out the office to see what's going on.

Coming Events: Neuromarketing Webcast

Neuromarketing is the latest hot trend in research circles. Stuffing people into MRI machines and then showing them advertising, and that kind of thing.  If you want to find out what's up with this stuff, you can attend a free web-cast on this topic Nov. 2, sponsored by the Qualitative Research Consultants Association.

Neuroscience seems to be the latest buzz in the research world. There’s as much fear as there is wonderment about it. The purpose of this presentation is to define what neuroscience is and means in the research world, exploring the latest developments and their bearing on qualitative research. It will debunk the myths and set out a set of principles which can be used to inform and substantiate qualitative insight. One of the topics covered will be the neuro principle of ‘hardwiring’, which tells us why people so annoyingly always seem to remember old campaign messages despite millions of dollars being spent on later, more up-to-date campaigns – and what can be done about this.

Details:

"Brains and Brands: How Neuroscience is Impacting Marketing and Market Research"
November 2, 2006
12:00 – 1:30 p.m. ET (GMT -5)
Register HERE
(Note: registration is free, and will also give you access to the archive of other great web-casts, all sponsored by the association to educate our clients and the public. Don't you wish everyone did this?)

Paco's coming to town

If you are in the Toronto area next week, you can take in a good one-day conference on qualitative research, and also hear Paco Underhill (author of Why We Buy and Call of the Mall) speak on Friday, Oct. 20.  It's a joint event sponsored by MRIA and QRCA, about 200 people, and a nice location to boot. Details here.

I am volunteering on the conference evaluation committee, and we've done some pretty radical overhauling of the evaluation forms. (Some of it is even printed in color! What will they think of next!)  I'll show you what we did and tell you how it worked in a couple of weeks.

Survey Research: Are all the same people answering all the surveys?

Advertising Age has an interesting piece about survey research: Consumers Rebel Against Marketer's Endless Surveys: 30 Top Industry Execs Gather to Discuss 'Opinion Fatigue' Crisis

At a roundtable held among execs of large consumer products firms and their researchers in Chicago, there was serious handwringing about survey fatigue and non-responders.  Perhaps survey research is not as valid as it used to be, too small a sliver of the population is making money as professional respondents, etc. 

If you are hearing this kind of thing and wondering what to do in your own organization, my suggestions are these:

Treat your respondents with respect.

Ask smart questions. Ask only what you can act on, not what would be nice-to-know. Pre-test all surveys with a small sample of the target group and interview them. Then fix the problems with confusing or irritating questions.  This does cost a bit extra, but you will get much better results.

Out of a sense of obligation to the industry, I try to participate in all research I am not barred from. And I am frankly shocked at how much of it is frustrating to participate in. Maybe you've had this experience yourself, and it's disrespectful of you as a consumer or a customer or a voter or a taxpayer.

Be quick about it.
Challenge yourself to make your next survey half the length of the last one.

Use multiple methods.
Multi-method projects come together on the problem from several angles. This can add tremendously to your understanding. You'll also avoid being blindsided by a sampling or response bias issue from a single survey instrument. 
You could combine an internet survey with a site intercept questionnaire, with some depth interviews. If something doesn't seem to fit, test again.

Consider paying people for their opinions.
I know this is unpopular in research circles. And indeed, paying people to be on large research panels is part of the problem. But if you want people to spend some of their free time answering your questions, maybe you should reward them appropriately. After all, how do you yourself feel about this?

Spend the money for real research
Your research supplier should be a member of a professional research association in your country. These organizations have standards and ethics and auditing bodies. They conduct training and education programs, publish useful articles and journals, and generally try to maintain high standards. 

If your supplier isn't part of one of these organizations, you need to ask yourself why. Would you go to a doctor that didn't participate in continuing education? Even your car mechanic and hair stylist is likely to have recent educational certificates on display.

This problem is really the tragedy of the commons all over again. Because it is getting cheaper to do survey research, there's lots more of it, and the natural tendency of most people to co-operate and be helpful has been abused to the point where a lot of people just don't want to play anymore.

What's the story?

Reshma Anand, a qualitative researcher in Bangalore, has posted on a very innovative idea about how to enrich storytelling. She thinks that if we put ourselves in the present tense when we tell a story, we can add to the richness of the detail and enhance our recall. 

This makes complete sense to me, although I bet it takes some skill to get people to do this. If you try it, you'll see what I mean.

Instead of saying, "I stood at the reception desk for a while. No one was around, so I went to sit down." you are now saying "I am standing at the reception desk, waiting for someone to notice me. They don't seem to be around. I decide to sit down and wait." 

Interesting to remember that Einstein used just such a method to imagine what would happen to his streetcar if it accelerated to the speed of light. He tried to put himself in the moment on the streetcar, and then to put himself in the moment of someone observing the streetcar. [And people think qualitative isn't scientific...d'oh]

The nuances and recall help to enrich what is being said.

Kim Bercovitz told me recently about another innovative approach -- using actors to tell a story as a research method. It sounds a lot like the kind of improv used to entertain conference participants, except applied to get clarity around experience and emotions. The participants keep correcting the actors' performance of the thing until they get it right. And then you know you have all the ingredients, don't you? [I suspect there's a lot more to this method than I'm capturing here, sorry Kim.]

Even objects have stories to tell, as this post from ethnographer Grant McCracken shows. Grant's own stories give us insight into the mind of the researcher, the professional observer, the person who can never stop watching and thinking.

Reshma's post illustrates something else brilliantly: there is no big book out there of the right way to observe. A lot of people seem to think there is. There is trying, and testing and learning and improving. There is no one right way.

The rules are all nonsense, really, because your method is only as good as the insights you get.

You want to break out, you want to be better than everyone else? You do it by ignoring the conventional wisdom, or tweaking it, or trying something different.

Kisses and hugs to all the clients out there who help you make these moments of magic happen -- by leaving enough space to try something a little bit different. They are the ones who win the biggest prizes in insight. We bust our butts for those clients, to try to make the risk pay off. 

Choosing a Qualitative Method: handy-dandy decision tree you can download

We talk about research quite a bit on this blog, because you can't manage the customer experience or even really think about it meaningfully if you don't start diving deep into what is actually going on.  To do that, you need research.

Now that there are so many options in using new communications technology to support research, I get a lot of questions about which method is better, and that type of thing.

You can't really do it that way, though.  You need to think about the objectives of the research before you know which approach makes the most sense. 

Handy-dandy research decision tree to help you (Click image to enlarge.)

I've put together this handy-dandy decision tree with some diagnostic questions that should help you think this through.  You can even download the PDF and stick it up on your wall, and share it with friends and colleagues who will marvel at your amazing ability to dig out useful resources.

Download qualitative_choices_decision_tree.pdf

Although the graphic looks slick, the real value is in working through the diagnostic questions.  After you have done that, you'll have a clear idea of your options. Then it's all down to budget.

How much to spend?

You can get into calculating the ROI of your research, and there are formulas. 

I have one that's really simple: I think clients should get about 10 times the payback from a project, or we're wasting their money.  So if it's a $30K project, you'd better think that great insight and great strategy should be worth about $300K to the business, either in risk avoided, effectiveness of effort, or raw revenue.

I once got into a lengthy round of negotiations with a human resource department that wanted to research some employee problems, after already having surveyed the employees. I suggested that perhaps the real problem was that the executive -- quite rightly in my view -- wanted to spend the money FIXING the issue, instead of putting more $ into research. HR didn't agree, and kept pitching the research and asking me to find a way to do it cheaper, smaller, etc. Of course the unhappy end of this story is that they kept having the management team reject their request for money, and I wasted a bunch of time scoping a project going nowhere.  All of which could have been avoided by getting clear on what was needed and why. And sometimes, more research is not actually what you need.

So use the diagnostic questions, get clear, and be a star. What could be simpler?

Addendum Sept 22:

Roger Cauvin pointed out that I missed ethnography. What the heck was I thinking?

Innovative Methods in Online Qualitative Research

If you could use a break from the heat, stay in for lunch on Thursday and join me for a free webcast: When Online Qualitative Shines: Solving Business Problems.

I will be the guest speaker for a web-cast on the topic of innovative ways to use online qualitative research technologies to solve business problems.

I'll be joined by Betsy Leichliter of New York, who partnered with me in developing the background research for this presentation. Betsy and I interviewed more than thirty researchers worldwide to gather case studies.

We won't be talking about researcher methods -- this presentation is for clients who would like to know more about what other organizations are doing with the new tools.

As always, it's not about the technology, it's about how to use it to get results!

This interactive presentation format will give you access to a chat window, as well as the option to phone in with questions.  It should be lively and informative.

It's free.  You have to register for access, here.

These Q-Casts are non-commercial educational events sponsored by the Qualitative Research Consultant's Association.  Once you register, you will have access to the full archive of great presentations. 

Join us for a Web Cast on Online Qualitative Research

If you are interested in Qualitative Research, you may want to join me and colleague Betsy Leichliter for a webcast sponsored by QRCA on Thursday, August 3 12:00 Eastern Time.

When Online Qualitative Shines: Solving Business Problems

About a year ago, Betsy and I interviewed about thirty experts in the area of online qualitative research, and documented some illustrative case studies, which we presented at a conference in Beverly Hills last fall. Since then, we have had a lot of interest in the material. 

Unlike the conference, the web-cast is not only FREE, but also open to anyone.  We've restructured the material to focus on the themes most of interest to business people  - namely solving business problems. We won't be spending much time on logistics that only other researchers care about.

Q-Casts, as these are called, provide for interactive chat as well as phone questions, so I encourage you to join us and make it a lively event. 

More info about the topic of the session is here.

You do need to sign up in advance, but it's not a big deal, and you do it here.

Technorati Tags: Q-Cast, QRCA, online qualitative research, Leichliter

Qualitative Research: Storytelling

Once you have found the right people to talk to, covered in an earlier post, and you have determined what you actually need to know about, then you must determine how you will get that information. In short, what are you going to ask them?

Amateurs say things like: what do you think about X?  Do you think you would buy X?  How would you improve X?  Attempts to get directly at the information you seek are often of limited usefulness.  You may get some data, but not likely anything that is not already reasonably well known. And the conventional wisdom is seldom helpful to clients who want to differentiate themselves in the marketplace with leading edge products and services. 

It's crucial to get beyond the literal if you are going to find out anything meaningful. And good researchers will have hundreds, if not thousands, of alternative ways to do this.  Storytelling is one powerful method.

Storytelling
Stories are as old as humanity, and we can all relate something we have experienced. That experience could be using a product, shopping in a store, or managing an IT outsourcing project.  When you ask people to tell a story, they engage completely different parts of their brain. 

We don't tell stories that are boring, we add the details -- the human details -- that make it interesting to others. And in so doing, we relate much more of our full range of perceptions about the event, and even the whole category under discussion.

If invited to tell a story using a familiar template, most of us could do that too.  The template might be telling the story as a fairy tale, telling it as a news report, telling it as a country music song. Or just plain old, "tell us the story of..."

The skill of the facilitator is setting up the environment that makes it comfortable for people to tell a story in detail. 

Storytelling is not just good for research, by the way.  It's a very useful approach in change management and visioning exercises as well. 

All of the great religious traditions relied on stories like parables to convey deeply meaningful information in a simple way.  Listen carefully to the stories people tell and you can learn a great deal about what is in their hearts and minds.

Technorati Tags: qualitative research, storytelling, asking good questions

Qualitative Research: Asking the Right People

A lot of the art of good research is figuring out who to talk to.  It's never as straightforward as it looks. Let me give you an example from a recent project to illustrate.

One project I am now completing relates to a global company's brand as an employer. Initially, it seemed like we needed to talk to some soon-to-graduate students. And maybe a few people who had recently been hired.
After talking at length to the inside project team, we determined that we needed a much broader and also more specific base.  Here's approximately how we defined the group we ultimately included:

• Senior executives in the firm who are involved in talent pool recruitment and have clear ideas about the future needs of the organization and the culture.  Almost as important, in my view, is that they will need to support the outcome and recommendations of the project, and they are more likely to do this if they have been involved.
• Some recent hires into entry level positions. These are people who still remember the recruitment process, the other employment opportunities they had, and picked my client.
• Co-op and summer students who are interacting closely with the organization but are not yet permanent employees.
• Recent hires into senior positions.  These people have been working for several years somewhere else, and recently joined the organization, but not in entry level roles.
• Students. Ultimately, we decided we were looking for people from specific universities, in specific programs (some undergraduate, some masters level), who expressed an interest in specific career directions, and who were also getting good marks.

It would have made my life a lot simpler if we could just talk to anyone at all.  And finding these students has been a real needle-in-a-haystack exercise.  We got them, but it wasn't easy. But really, there would have been no point at all talking to students who were heading in a very different career direction, would there?

Every project has this problem

There are almost no projects where this kind of sampling exercise doesn't become an issue. A project currently in discussion involves a home health care product. Just imagine the possibilities... should we include doctors? Nurses? Chiropracters? Just people who used the product in the past? People who need the product but have never heard of it?

A huge part of good research is figuring out what you need to know and who you need to know it from.  If you just gather a bunch of people and stuff them in a room, give them cookies and ask them questions -- you'll definitely get data.  But you won't learn much that will help you, no matter how good your questions are.

Guiding questions to ask

Here are a few of the diagnostic questions I use to think these things through.  Please add your comments with your own useful guiding questions.

1. Who buys the product or service?
2. Who pays for the product or service?
3. Who uses the product or service?
4. Who influences the sale of the product or service?
5. Are there different segments of this market based on needs as we currently understand them?  What appears to distinguish these groups?
6. Could we distinguish repeat buyers from former buyers, from first-time buyers? 
7. Are there differences in product usage (e.g.light, heavy) that we should explore?
8. Do males and females approach this the same way, or perhaps not?
9. Are there important cultural influences that could be at work here? Ethnic groups? Language groups?
10. Is there an urban/rural dimension? A geographic dimension like west cost vs east coast, warm weather vs cold weather?
11. Are there different distribution channels involved?  On-line vs in-store vs. catalog vs. home-party vs. reseller etc?
12. What about after-sales service?  Should we talk to people who needed it? What about people who have complained?
13. What about the competitor's product or service? Should we talk to some people who prefer it?
14. Is there any way we can talk to people in the act of making the decision?  Or people who just made the decision?

How to distinguish Custom Research from a Poll
Figuring this stuff out is often what distinguishes custom research from a poll.  With a poll, you get a random sample of people in a given region or geography, like 1000 adults between 18 and 64. 

With custom research, the first part of the project often consists of developing a research strategy that will answer the business issues.

Now there's nothing wrong with polling.  But be sure that's what you need.

Technorati Tags: qualitative research, custom research, guiding questions to ask, deciding who to research, sampling

How Marketing Research is Changing: Angus Reid at MRIA 2006

Our last speaker today was Angus Reid, a leading figure, who founded the company now known as Ipsos-Reid, and who is now chair of Vision Critical, a panel management software solution.

Sorry about the blurry picture, but he was an energetic speaker!

Looking back 100 years, he pointed to three periods of major change and social upheaval, each of which was accompanied by significant shifts in marketing research.

The 1930's

• the new medium was radio
• people were uncoupling themselves from the great institutions of the day, and the depression was seen as a failure of society in many ways
• George Gallup invented the public opinion poll as we know it today.  They were doing face-to-face interviews at the door, and a poll took about 6 weeks to complete.

The 1960's

• the new media were telephone and television
• gender roles and the role of youth were redefined
• there was a renaissance in marketing research, as the shift to telephone polling made results available in as little as 24 hours, instead of 6 weeks.

Today

• the new media are cellphone, wireless and internet
• globalization is causing major shifts in society and the global economy, creating new winners and losers
• a new dynamic is taking place in the relationship between researchers, their clients, and consumers.
• it may not be the end of the era of the telephone survey, but it is definitely the beginning of the end

These shifts are always more subtle than we at first realize.  Radio was better suited to the big brassy sound of a big band than the subtle variations of the violin, often lost to static, resulting in a shift in musical tastes.

Online research has resulted in much faster, cheaper data -- but not always higher quality. Comscore networks has reported that 30% of internet surveys are completed by only .25% of the internet population. In short, there are massive measurement and sampling issues, which the industry is only just starting to grapple with.

Consumers want 'CPC' - community, participation, control

The desire for community -- something we have commented on often in this space -- is a big driver behind new trends in research.  Customers want more control, more input, more say, more access. They do not want to complete a 45 minute questionnaire on a product that looks like something that was produced in the 1930's -- but now it's online.

'Bleeding borders' between marketing and research

Reid says that some larger retailers are starting to see strong linkages between their panels and sales activity -- panelists tell their friends, take them into stores to see new product.  It is raising the question of how many of your customers you want on the panel -- maybe potentially all of them??

Like the early days of film

In the early days of film, a stage play was mounted and filmed, because people didn't yet know how to make a movie. Online research is a little bit past this stage, but you wouldn't know it to look at most online surveys.

He challenged the methodology people to start utilizing the new capabilities more creatively, instead of just porting old methods onto new platforms.

A fantastic talk.  It's so nice when an icon lives up to their reputation, as Mr. Reid did today.

[PS. Funny joke he told, on the difference between BC and Alberta: "In Alberta, the people hate the government; in BC, the government hates the people".]

Update from the MRIA Conference in Calgary:Shafeen Charania of Microsoft

This morning at the MRIA conference in Calgary I heard Shafeen Charania speak.  Here in the E-Tabs sponsored internet cafe, I thought I would share the highlights of his excellent presentation with you. 

Shafeen spoke of how Microsoft blundered past it's peak of brand coolness because of its culture in the 90's -- at the time of Windows 95 release. They used marketing research to understand market size, instead of understand customers. 

They are trying to change this now, by being much more outcome focused, much more self-critical.  He was frank in his admission of the many problems we all experience with technology. 

As evidence of their efforts to reform themselves, he showed a hilarious video (that Microsoft produced themselves) of how Microsoft would approach package design for the iPod.  If you haven't seen it, it's wonderful. 

They are also trying to find more ways to surface conversation that is "not on the party line, but still inside the party", and cited Scobleizer as one example. The other, which I hadn't heard of before, is Channel 9. When I checked it out just now, there were 40,000 on line there -- apparently all built virally. This is the insider-developer view of Longhorn development. 

They are starting to use more ethnography globally.  And learning some pretty cool stuff: 

• families share multiple cell phones in Kuala Lumpur
• males surf for porn in internet cafes, not on the family computer in Malaysia -- he called this "public is private and private is public"

His last observation really had me thinking.  That only 9% of the souls in the world use a computer somehow.  And they are looking for ways to engage with the other 91%.  Not a bad takeaway from a breakfast session.

Grant McCracken on Clotaire Rapaille: There's No Code to Crack, Just Good Research

When I saw the piece in Fast Company about Clotaire Rapaille, showman anthropologist, I wondered if my acquaintance Grant McCracken would comment on the article.  He has, and brilliantly. Both articles worth reading. Take particular note of Grant's definition of good research, which is bang on.

When I was a bank employee some time ago, I recall hearing about this Rapaille guy from my boss, who was enthralled. He made people lie on the floor and remember their first experience of a bank branch. His big insight was that people remembered pillars as part of bank branches, and that this solidity was an important part of bank branch design. Therefore, don't give up on solidity. At the time, this sounded so facile I thought I must be missing something. I was: this guy is a genius at marketing the Rapaille brand.

There is no single "code" for a culture. Do you think someone unfamiliar with Canada could talk to a bunch of us for a while and understand what makes us tick?  What would they make of the 300,000 people we import from the rest of the world every year?  And translate that code to marketing a product? Maybe, maybe not. We love Kraft Dinner, but think Red Lobster is boring. Frankly, we struggle to understand ourselves.

If you've been reading my blog recently, you know I was working in Jamaica last week. Jamaica is itself an amalgam of many cultures, including Chinese, Portugese, Lebanese, and many African nations, plastered over by the British, and now being re-engineered by Jamaicans themselves. "One Love, One Heart". It is not an easy culture to understand, even though they speak English better than I do, for the most part, and share many British traditions. I have definitely had some insights that have helped my client. But I've also been run into the boards quite painfully a few times for reasons I still don't fully understand, and continue to obsess about. [Run into the boards is a hockey analogy, BTW]

Let me give you one simple example.  My hotel put little cards on the pillow with Jamaican phrases in Patois and their meanings. Here's an example: Dawg mawgre but him head big. Translation on the card: no matter how much the dog's body shrinks, his head cannot shrink: some things cannot be changed.  My hosts at dinner one night said this was all wrong.  One thought the saying referred to someone you would underestimate who was smarter than they looked. Others thought it meant that even a mangy dog thinks well of himself. These were smart, well-educated, Jamaican-born executives who speak Patois as well as they speak English and THEY disagreed about the meaning of the phrase. Is this part of "the code"?

I often ask people what jokes they tell about themselves to try to understand what I don't understand. [What's a joke accountants tell about themselves, for example?  What's a joke Canadians tell about themselves? What jokes do the French tell about the Belgians?] What is most remarkable to me is that the jokes are almost never funny to an outsider.

In any event, I don't think there's one code. And this is the beautiful complexity of humanity. I know lots of elegant models, lots of psychology, lots of social science and I speak business reasonably well. And none of this stuff explains it all. And thank God for that.

Technorati Tags: Jamaican culture, cracking the code, Clotaire Rapaille, Grant McCracken, Anthropology