Customer Experience Crossroads

Common research mistakes

I logged in to Constant Contact a few minutes ago to start writing a newsletter. There was a button for completing a survey, so being a naturally curious [some might say snoopy!] person, I clicked.
Constant Contact now provides a survey platform. Sadly, they made several common mistakes. So egregious and common are these errors, it seemed essential to post about it and save others from the same mistakes.

This is a screen shot of the first page of the survey. I see three mistakes here:

First mistake -- ask for my name right up front. If you are going to make this non-confidential, you might want to ask for this information in the last question. Because then if I don't answer, you still get the rest of my data.

Second mistake -- non-exclusive categories. If I am in business 6 years, I have two choices. Same problem with 8 of the response categories. Oddly, the sequence changes from 8-9, and 10+.

Third mistake -- the person who wrote this survey has insufficient knowledge for the task. Two easy solutions. Either take a survey writing course -- there are lots of them around, and some are available online. Or, spend a small amount of money and hire a researcher to write the questionnaire for you. Seriously, this is not a complex questionnaire. It would be likely a half-day of work for an experienced person.

Or, you can just get poor data and look like a goof to the world.

You know, the thing is, the fact that I can use a power saw does not make me a carpenter.

Free e-book: Painless Insight Planning helps you set objectives, avoid wasting money on qualitative research

 

I've refined a number of tools to use in my work over the years. A few of them are tools designed to help clients in one way or another. This free e-book, Painless Insight Planning, is one of these tools with some narrative wrapped around it.

[Download Painless Insight Planning now]

One thing I've found that is often a challenge at the start of a project is getting clear objectives. People may have a problem or concern, but it is often not clearly articulated.

And I can tell you one thing for certain: a good way to waste money on research is not having clear objectives at the beginning.  I know this sounds perfectly obvious, but it is an issue more often than you would suspect.

Sometimes the problem is clear, but other elements are not. Such as -- who could we get insight from, otherwise known as "the research target."

I can't sit down with everyone individually and work through this, as I do with clients. So I thought I would put the whole thing into an e-book that you can download and use as much as you want. And then you'll have clear objectives for your research projects. And the world will be in a better place.

Because we really need insights these days, more than ever. And people inside companies can't afford to waste money or appear to waste money, get fired and be unemployed. Suppliers like me also can't afford to waste client's money. We all need to ramp UP on the insight quality and the innovation / ideation, but keep a lid on costs. 

What's it worth to get this answer?

An even harder thing for many clients is figuring out how much to spend. They ask me, what will it cost? 

WRONG QUESTION!  WRONG QUESTION!  WRONG QUESTION! 

The right question is "what is it worth to get good information on this situation?"

I have created a nice little rule to help figure this out. Not a complicated ROI on research thing. Just a back-of-the-envelope, cocktail-napkin kind of formula that I call the 10 Times Value Rule. It's a great place to start figuring out what you should consider spending to get some insight into your problem / challenge / situation.

[Download Painless Insight Planning now]

Easy worksheets

I hate filling in forms. Most people do. So there aren't forms in this book. There are easy worksheets designed to spur creative thinking to get you started. Here's an example.

Bonus content

If you download the book and actually read it, you'll find a link to some bonus content that will save you even more time. 

A small request - no strings attached

I'm simply putting this handy e-book out there for free with no strings attached. You can even add this e-book (with no changes) to your own site or blog for your own customers and prospects to use.  But I do have a small request.

If you think this e-book is of any value, please add this blog to your RSS feed reader. I also send out a newsletter about six times a year. You can add yourself to that list if you want to hear about other e-books I plan to release, or just get the newsletter. You can check out the previous issues here.

Thanks for reading. And good luck with your insights research projects!

[Download Painless Insight Planning now]

Listening right vs. listening wrong: Google Latitude and sampling

You've likely heard about a new service Google has launched called Latitude, that can enable you and your pals to track each other geographically via your mobile phones. There's a bit of controversy about the potential abuse of the service for stalking, but that's not what I want to talk about. At least not directly. I tested this concept a couple of years ago in research. Let me tell you about it.

About two years ago, I was planning a session with some grad students in a research program. You know the drill -- guest instructors who are practitioners come in and show their stuff. So I did an advance study with the group using an online bulletin board method. Then we talked about it in the class.

To try to find a topic of interest to all the students of various ages, I picked cell phones. Among other things, I cooked up a number of possible features and enhanced services that we concept tested. One of them I called "Friend Finder", which was essentially a product like Latitude.

I thought this Friend Finder concept was pretty clever, actually, and that if I tested such a product concept with a broad base of young adults, that it would likely be well received. I was really surprised when this group of research students reacted pretty badly to the idea, and were REALLY worried about privacy. Freaked-out and creeped-out about privacy, actually.

They also didn't like "Taxi Finder", something I personally would love. And they didn't like "Bus Finder". I don't think there was one concept that tested well.

So I pondered this finding ... and here's what I think.

This was not an average slice of humanity in any respect. Most of these people were into quantitative i.e. they enjoy statistics. A number of them had just finished a four-year degree and were running out of money. Several were mid-life career changers who were also pretty cautious about their spending. And they were in a small city on the edge of the Canadian Shield (lots of big rocks) and they told me cell coverage was often spotty. [Who knew? When I go up there to canoe, I'm pretty much ignoring my cell phone.]

And it was not a well targeted slice either.  A number of the students turned out not to have a cell phone at all, or used it very sparingly. In a real study, the client most likely would have been looking to gain insights into only those people who were already pretty keen on their cell phones and were already using some advanced features.

My colleague that day, Tania Heintzman, was personally VERY keen on Taxi Finder. But she uses taxis a lot and students don't. Off target.

I personally was very keen on Bus Finder, but the bus service in small cities is often terrible and so irrelevant to this group who all drove to campus. Off target again.

Basically, researching any number of the wrong people does not help. A bigger sample of the wrong people does not help. Statistical significance would not get you out of this issue either. The wrong sample is the wrong sample.

Sometimes only a few of the exactly right people can help you dive very very deep into powerful insight. So you really need to figure out who those right people are. Especially when you are trying to see into the future with new concepts for products.

If something like this happened in a real project, it would be a mistake to conclude that the concepts had no potential market. Because clearly, Latitude will find users. And all sorts of people are developing open-source products like my "Bus Finder" idea, even here in the frozen wastelands of Toronto. Soon it will be available via smart phone apps. 

On the other hand, some of the learning would have been valid. For example, the concern about privacy. That's an attitudinal and behavioral segment that you would want to understand. Even knowing about this issue would help you design a better service, and avoid the criticisms Google is getting now.

Innovation breakthroughs: now really IS the time or why recycling does not work for insights.

 

People are cutting back. [Oh, you heard that, too?]

I've had a couple of projects cancelled or postponed indefinitely in the last short while. One of them was to help an organization identify new markets so they could escape the commodity price-cycle trap of their current markets.

This is exactly the kind of project you definitely should not cancel -- projects that will help you crack new revenue sources. So what will this company do instead? I'm guessing, they will just push harder at existing markets, and find themselves in the same place a year or two down the road.

While having these thoughts, I bumped into a nice article in Marketing Profs. [Now that the economy is tanking, maybe we can catch up on our reading.] The quote from Jeffrey Immelt leapt off the page.

General Electric CEO Jeffrey R. Immelt has established a class of projects now well-known as "imagination breakthroughs," consisting of ideas that are "really hard or really important" and might generate significant revenues over a three-year period, the time that GE usually takes to implement a new idea. "Imagination breakthroughs are a protected class of ideas — safe from the budget slashers because I've blessed each one."

If it takes you a couple of years or more to really get an idea off the ground, then the fuzzy front end you cut now will leave you in even more serious glue in two to three years time. And if you kill the stuff already in the pipeline, you're not going to have anything new to launch in 2009 either.

I really wish clients would say instead: "Can you get me something substantive with less money?" That's a better challenge than dropping the idea creation process.

Caveat

The article this came from is filled with good ideas from the Zaltmans. But like many of their ilk, they firmly believe that no approach other than their technique (individual metaphorical interviews) can get to the good stuff. Not true, in my experience.

"These deeper ideas are not available from survey responses to statements such as "financial planning is difficult" or in focus groups where each participant has on average about 10 minutes of air time." 

OK, you are definitely not going to get these deep meanings from a survey. I agree. But individual interviews are not the gold standard either.

There may still be people running focus groups with 12 bodies in them. However, most of the kids I hang with have shifted to small groups when they are using a group discussion method. Small being five or six. And researchers frequently add advance activities such as blogging / electronic diaries / collage assignments / videos / you name it. And there's no rule about 90 minutes or two hours either -- you can keep people humming along for three hours quite easily. And yes, you can uncover metaphorical meanings in focus groups, in online research methods, and in many other ways.

In truth, the generation of insight has a lot more to do with the skill, knowledge and art inside the researcher's head than the method employed. There's no method going that guarantees insight.

As anthropologist Grant McCracken has said, the proof is in the pudding. If you got insight, you got insight. If you didn't, you didn't.

I have friends in the marketing world who have used Gerry's Zmet technique to great effect. And I believe all of methodological infighting and territory claiming must defer to this. If the method works, the method is good. The proof of the pudding has nothing to do with the theory of the pudding or the method of the pudding. The proof of the pudding is a client who says: "This was illuminating. I understand my consumer and my market in ways that I did not, could not before."

Back to the main point

This is no time to be cutting your future revenue streams by shutting down the insight machine. In fact, if you plan to be standing when things turn around, you should be cranking up the insight machine. Certainly, your customers and consumers brains are grinding away on many things right now. Not a good time to recycle some old insights.

Resources:

How to Foster "Workable Wondering", by Gerald Zaltman and Linday Zaltman. In marketing profs, reprinted from HBS.

The Zaltman Method, by Grant McCracken in his blog, 2006.

Cool new tools that enable feedback

I went looking for one thing online, and found a bunch of other stuff. Don't you love doing that on a Friday morning? So of course I wanted to share...

There are just such a variety of interesting online applications for assisting collaboration now. If you can add some others to the list, please do. If you have used any of these tools and have comments about them, please add your thoughts as well.

Mapanno Markup

This markup tool lets you invite comments from others on an image or a project. It feels a lot like Google Maps, where you can add an icon and comments, and stick it to a specific place on the document. Here's a screen shot that gives an idea...

Review Basics

I have actually tried out this tool on a demo basis at least a year ago, and it looks even better now. like Mapanno, it is currently free. But it has a broader range of functionality, and was designed more specifically with marketing folk in mind.

Here's a screen shot of what the markup environment looks like. There are bubbles that you can use to enter text and a variety of marking tools as well.

 

Review Basics also supports common video formats, as well as Word, PowerPoint, and PDF files.
I'm itching to try this out with a research project, but I think it could be really useful on a team internal basis as well. Right now, it's free, just like Mapanno.

Vyew
Vyew is one I just learned about, thanks to GeekBlog. I haven't had time to demo it at all, but it positions itself as a competitor to WebEx and GoToMeeting. In addition to those services, however, there is the potential for asynchronous involvement by multiple reviewers who can mark up documents, and view each other's comments.

Here's a comparison chart that may help.

There are some interesting case studies on the site as well, that make it clearer how one might use such a multi-featured application for things other than web conferencing. Like the other tools mentioned, it is currently FREE, and does not require software installation. There's a review by Guy Kawasaki

Concept Share

This service is specifically targeted to graphic designers. I haven't tried it out, but it looks pretty appealing. There is a free-premium model in place which affects the number of projects you can use, and the amount of online storage you get. Huge amounts of storage are available, which is needed for graphic files.

 

Basecamp

I've been using Basecamp for a few months now. Although it's not perfect, it's a very useful and low cost collaboration tool. I haven't figured out a way to use it for research projects yet. But you can store files, there are whiteboards, a really good messaging function that beats running things on e-mail, and some project management tools.

This stuff all just continues the flattening process where little tiny organizations have access to tools and technology as good as the stuff the big organization's have. Sometimes it's even better.

If you want to read about even more of these tools, here's a reference article from GeekBlog. And their handy comparison table.

Global Village Study on Teens and Technology

About this time last year I was helping to plan a global study on teens and technology use, called Global Village. This idea was cooked up by my colleagues Ilka Kuhagen (Germany) and Corette Haf (South Africa) as a way to demonstrate the power of using bulletin board research to gather global insights.

We sponsored this research as a group, and conducted the research with 12 groups. This was an impressive team of researchers, it was my honor to work with them. 

During the planning process, we had many logistical issues to work through. For example, our platform sponsor, Focus Forums, had to add Mandarin character capabilities to the bulletin board.

A decision was reached to focus our efforts on teens aged 14 - 16 years old. A challenging target in some ways, as ethical guidelines require parental permission to conduct research with this age group, which added to the logistics of the project. But we saw that this group is entering young adulthood with access to a level of technology that they have enjoyed since childhood. We wanted to see where things were heading with this group. So we found teens that had a cell-phone and used the internet relatively heavily.

The bulletin board was a fantastic tool for the project, and the teens loved the format. Age and gender barriers that would have been a major burden in a face-to-face setting evaporated, and we had high engagement and considerable openness and honesty.

In addition to this, the research team was also easily able to watch the research unfold, and share our own discussion about questioning strategies over the project schedule. This meant we could change and shift the later stages of the project based on earlier stages.

It was fascinating to see the style differences among the research team. As findings started to emerge, we were all impressed by how similar, in many ways, the attitudes of the teens were in each country. In picture sort exercises, for example, the same concepts -- and even the same images -- came up over and over again. We weren't really expecting this, it was a fascinating finding.

One thing is for certain -- these teens are joined at the hip to their cell-phones, and this is their life-line. If their house was burning down, it's one of the things they'd grab. The research team found text messaging a much more effective way to send out reminders to participate. (With an adult group, we would use e-mail). Almost none of the teens in the whole study had heard of Second Life. A few had been on Habbo Hotel, but these sites no longer held  -- they are way too busy forming real-world relationships to care too much about the virtual world. Yet.

Overall, a fascinating group of teens. The Canadian group were very much "the New Phoenicians" -- ethnically diverse, open to new ideas, enterprising and optimistic. I wanted to give them all a virtual hug, they made me feel proud to be Canadian. I have no doubt the other researchers felt the same about their own youth.

I'll be presenting on this project with my colleague Betsy Leichliter from New York at the ARF conference next week, and also in Winnipeg at the MRIA conference in May. Other team members are presenting at various conferences around the world, including the upcoming AQR/QRCA conference in Barcelona. The slide at the top came from a presentation done by Diva Oliveira in Brazil.
 

You can also read the write-up that Ilka and Corette did for the QRCA publication VIEWS, which is available free online. How Global is Your Village?  (the link spawns a PDF. Article is on pages 18 - 27)

You know advertising is hot when it's displacing IT among IT research houses

I always thought of Forrester as an IT-oriented research house. And then they were on stuff like supply chains and outsourcing. However, they are now getting talked about for their thought pieces on where advertising agencies are going.

This suggests two things to me:

1. Forrester is good at figuring out what's hot
2. Marketing, advertising, and brand building are hotter than they have ever been, but also under more scrutiny than ever.

I have other evidence of this trend.

The Globe and Mail, my daily must-read newspaper, never deemed marketing topics worthy of business coverage. [Well, any kind of coverage, actually.]  Not long ago -- within the last 18 months for sure -- they have added regular material on this topic. It's thin, but it's there.

Friday, I heard Terry O'Reilly speak at a conference on qualitative research. Mr.O' Reilly is a regular feature on CBC radio, and known for his expertise in advertising, as both a creator and a commentator. Here's one of the notable things he said on the topic of insight and advertising:

"Creative ideas born of intuition are the last legal means of gaining a competitive advantage over your competition."

I thought this was a) brilliant, and b) true.   

Forrester seems to think that advertising agencies are not helping marketing people connect with consumers and customers in a meaningful way. This is the summary of their recent research report, The Connected Agency: Partner With An Agency That Listens Instead of Shouts

"Today's agencies fail to help marketers engage with consumers, who, as a result, are becoming less brand-loyal and more trusting of each other. To turn the tide, marketers will move to the Connected Agency — one that shifts: from making messages to nurturing consumer connections; from delivering push to creating pull interactions; and from orchestrating campaigns to facilitating conversations. Over the next five years, traditional agencies will make this shift; they will start by connecting with consumer communities and will eventually become an integral part of them."

I have not read the whole report, but I am having trouble seeing this as news. And it's far too easy to just blame the agencies, who are under tremendous pressure to deliver results these days. But they can only do what they get a mandate and a budget to do from the marketing people. Surely marketers themselves must take responsibility for connecting with consumers.

And so must executives.

Being in the insight and intuition business myself, I can only agree with Mr. O' Reilly: we need better strategies. We need insights. We need innovation in marketing and advertising.

At the same conference where Mr. O' Reilly was a headliner, I presented with a client a case study on some innovative research we did together. When we got to the Q&A, the biggest question that came up -- which we brainstormed as a group -- was HOW to convince executives to do something less conventional than a focus group or a survey.

These researchers and marketers -- they want to deliver innovation, they want to deliver category-changing insight, but feel challenged to sell it into the organization.

We came up with a flip-chart full of ideas of how to do this. Things like: start small, start with intractable problems where there is motivation to try something different, and so on.

But as one of the research buyers said, "We want innovation, we want to do something new, and we want amazing insights, but we only want it once it's been done before. How do you sell this into the organization?"

Of course, we on the practitioner side have another challenge. Having promised these category-changing insights, we need to stand and deliver.

I only wish these insights that can change the landscape could be produced on demand. [I'll be hunting for a good one this week, wish me luck.]

Facebook Polling: a quick tour and a surprising finding

After holding forth about the future business applications of Facebook, I decided I should put my money where my mouth is, and do a quick little research study.
Facebook makes that easy for you with their polling feature.

Here's what I wanted to know -- what kinds of applications for business would interest people who are already on Facebook? So here's the question I asked:

What application for business would you be most likely to use on Facebook?
[a] A way to post RFPs / find suppliers?
[b] A way to bid on RFPs / find projects?
[c] A project team collaboration function / group?
[d] A way to sell content like e-books / e-learning?
[e] I would not use any of these

Facebook polling is so inexpensive, I thought I could afford to get 100 responses, which would cost about $25.00. [I do want to be clear that this is not a scientific poll, it's just a way to take a quick read on something.]

I couldn't select exactly the group I wanted -- freelancers and SMB people. So I went for age as a proxy, and asked for age 35 - 49. I was surprised that the oldest selection was age 49, actually, since surely there are many people using Facebook over the age of 49. I know several well, in fact.

I sat excitedly at my computer waiting for the colorful dashboard to show results. Which it did, looking like the chart above, with several other colorful charts breaking down my poll by age and gender.

Surprising Finding

When I had 49 responses, Facebook shut down the poll.

Before they did that, here's what I learned:

1. About half would be interested in one of the proposed functions
2. About half indicated no interest in any of them
3. The most popular of the proposed functions was [c], the project team collaboration function. This is the one I want too.

I received an e-mail from Facebook explaining that they had shut down my poll.

Hi, Susan,

Facebook Polls are intended for market research purposes. Facebook Polls created to promote products or services or that are otherwise deemed not appropriate are removed accordingly. Along these lines, Facebook has automatically removed your Facebook Polls (order #Facebook Poll).

To see the final results, go to: link 

Responses Desired: 100

Actual Responses Received: 40

Price per Response: $0.25

Your credit card has been charged an additional $10.00 This completes your order.

The Facebook team

I'm surprised that they thought this was promotion. So maybe it was deemed otherwise inappropriate. Perhaps the only people allowed to do research on Facebook itself are officially authorized. Perhaps using the word Facebook in the poll flagged my little project. As far as I know, they can't stop me from blogging about it.

Anyone have some insight into this?

I was thinking it would be cool to hire some savvy Elance developer to just make me the application I want, but now I'm reassessing.

Holiday notice

I'll be skiing in Banff. If you're there, say hi. I'll be the person doing the face-plant right in front of the lift line, wearing a fuzzy hat with ear flaps [that looks really cute, actually]. So I likely won't have the strength to post after a hard day of skiing. See you in the new year! And thanks for all the reading and the comments, which really make it fun to keep at it.

Jason Oke talks about what's wrong with qualitative research

Jason Oke is a planner for Leo Burnett -- a guy who spends a lot of time thinking about finding leverage points for the creation of great advertising. And is, incidentally, an excellent speaker. He thinks there's a lot of bad research being done, which has created a credibility problem.

Chief among his complaints is that there is far too much direct, literal Q&A going on, which we know -- I mean we really know this -- is not very useful. This kind of technique lacks so much subtlety, it is akin to "strip mining their heads," according to Mr.Oke.

By literal Q&A I mean approaches like asking people to tell us "what's important to you in a vodka", which will generate nothing that can help a vodka marketer. If the target group even knows, which is unlikely, they are hardly going to say things like, "I want to feel like part of an exclusive club" or "I want a wee boost to my social status."

This is not just true for products like vodka, by the way. I just completed a business study in which one group of people were clearly motivated by reducing their fear level. Now just how likely do you think they would be to [a] know that and [b] say it out loud? In fact at least one of them did know that, but she didn't say it, she wrote it down as part of an exercise. [Of course this creates an equally large challenge for the creative team, but that's another article.]

Jason Oke also believes that people are bored by the idea of sitting in a dimly lit room for two hours being poked at with inane questions and limp exercises. He thinks this is a major reason why it is getting difficult, and expensive, to recruit participants for focus groups. This was a new idea for me, and I think he's on to something.

Mr. Oke's recipe to turn the situation around:

[1] Take brain science into account.

People can tell us many things, but they don't have direct access to unconscious motivators. When we ask them questions depending on this access, they will tell us something, but it's unlikely to relate to the problem at hand. It may not even be true. But most people will generate an answer to a question like "what's important to you in a vodka" just because people are very good at generating reasons.

[2] Plan research activities that meet the needs of #1, and are also fun and engaging.

Mr. Oke's solution to the Vodka problem, for example, is to ask small groups of people to design a new vodka and how it will be marketed to people. Exactly. Generates great data, and is a lot of fun for the participants. He showed us clips from the tape, it was brilliant.

[3] Stand up to clients when they ask for bad research. Evangelize for good research.

The thing is, once clients have experienced really good research, they are left unsatisfied by the other kind. At least I think that's the case. I hope and pray that is the case. But it feels risky the first time you get on.

[4] Think of research as a brand touch-point.

Absolutely correct, and great advice.

Resources:

Jason Okes' blog is here

A related Malcolm Gladwell blog post is here

Clips from the Prelinger Archive film about marketing research, here

Book he recommends [I have not read, but it sounds great]: Herd by Mark Earls.

Mr. Oke presented to the qualitative division of MRIA at "Qual in the Fall" on Nov. 9, 2007 in Toronto.

Rethinking research

I'm off to the QRCA conference for the week in Vancouver this year. My mission, beyond catching up with friends and colleagues, is to try to really push the boundaries of how I think about qualitative research.
The reason is: fieldwork costs keep going up, and are going up faster than client budgets are. The whole infrastructure that surrounds qualitative research -- lists, panels, recruiters, facilities, catering, incentives -- is a large part of what is costing a fortune. It ain't me.

One of the worries about rising incentive costs is that this encourages people to become professional respondents, supplementing their income by participating in focus groups and interviews. Surely we have not tapped out the adult population of North America. There must be people who would like to share their opinions without being paid a huge sum to do it. Or are people so exhausted from constant requests to complete poor/badly written/excessively long surveys that qualitative is just one more intrusion?

I think there must be other ways of finding people -- perhaps Facebook? Is anyone finding respondents on Facebook?

I'm hoping to get inspired for a few days. Pick up a few new ideas. And challenge my own assumptions about how things should be done.

And, according to my own personal tradition, I'll keep you posted about anything interesting that I see or hear there.

Designing Experiences: prototype, pilot, tweak and adjust

James Rait posted some interesting commentary in Design-at-the-Edge in response to my Curves pieces from last week. Mr. Rait's wife is a Curves member, at a location where they sometimes give away doughnuts. [I know, it's hilarious, except it illustrates just how difficult it is to manage services with consistency to standards.]

I had outlined some possible backfires to the adoption of RFID tags to plan and track individual goals and progress. These were: [1] staff having to take their own initiative to interact with clients, instead of being forced to do it by the operational routine; [2] two classes of member emerging; [3] clients losing the happy ritual of taking their pulse rate together.

Mr. Rait, a guy with serious industrial design credentials, proposes some interesting alternative solutions to these issues in his post, here.

He also suggests that running some live market tests would have uncovered the potential issues I mentioned. I would be amazed if they had not done live market tests, actually. The execs at Curves have strong marketing backgrounds, and with 10,000-plus locations, I assume their budget supports quite a bit of research.

Sometimes it takes a while to understand what is going on. Plus the hype surrounding new technology can generate enough excitement to mask the longer-term impact. These phenomenon would have been unlikely to come to light in a short test. And if the test markets were conducted with the very best franchises -- which is usually the case -- these individuals may well have responded differently than the average.

I suspect the social evolution is not yet done. [There's lots of observing still to do. Yippee!]

Mr. Rait's post made some other interesting points about design in general, which I'll pick up tomorrow.

If you go back to read the Curves posts, note the passionate comments from one fan of the company. This kind of evangelism is a huge asset to a brand.

The photo came from a film in the Prelinger Archive, Exercise and Health, 1949.

New media measurement tools: Xuuk

Eye-tracking technology is cool, but it's always been a kind of headset-and-wires operation, not something you could use in a real advertising setting.

Xuuk has launched a gizmo that can track eyeballs for posters, billboards and displays. It's a low-end disruptor -- and although it is unlikely to have all the functionality of laboratory units -- it's extremely cheap at $999. And it potentially brings a whole new group of customers into the market.

Xuuk claims to be able to detect eye-balls looking at it from up to 10 meters away. It plugs into your USB port, so you can look at statistics.

Like many technology crossovers born in academia, Xuuk could use a little marketing support to make their message clearer. But don't let that stop you. This baby could be great tool to refine your merchandising and visuals for those times when you don't have a big budget available.

What we can learn from squirrels, and other research tips

I planted my garden over the last few weekends. A few wildflowers native to this part of the world (trillium, jack-in-the-pulpit), some herbs, and the usual impatiens, fuchsia, and such. [Yes, it's true, I do have a few domestic skills.]

For days after a new plant goes in, it's a certainty that the squirrels will dig it up. I replant every morning. This morning it was the hanging planter of morning glories.
Why do squirrels do this?

It came to me this morning. They do this because they can smell something in the freshly dug dirt. They know it has recently been disturbed. And this is a good place to look for snacks -- perhaps nuts that other squirrels have buried. It must be, otherwise why would they bother?

I've been annoyed with this behavior for years, but have never stopped to de-construct it. From a squirrel's point of view, it makes perfect sense. 

And so often this is what we need to  do when we study customers. Especially when their behavior seems irrational. Because behavior is not random. It is purposeful. If it is random, the randomness itself is purposeful. It may not be rational in the conventional sense, but it achieves a result, it tips the odds, or it has been rewarded in the past. There are reasons, there are patterns, even if the patterns are unconscious, or seemingly irrelevant.

We cannot change our customers. But we can understand them if we work at it.

More collaboration tools - maybe for research?

One of the joys of having a moderately successful blog is that I now get requests from people to check out their a) book b) new research application c) creative marketing or promotional strategy.
[One could easily do this full time, if said one were independently wealthy....]
In any event, some worthy candidates have passed by recently. Here's one I want to tell you about.

ReviewBasics

ReviewBasics is a feedback gathering platform which allows professionals across many industries to exchange ideas, gather thoughts, and review various types of creative, interactive, motion, and written content. We take a familiar, paper-based content review process and bring it online." [From a news release sent to me.]

I spoke to Tim Shih today, who is VP and sales manager for this new company. Mr. Shih says ReviewBasics is, "The next generation of tools for the online marketing research industry."

The company was started by a technology consultant and a product developer, both with experience in the e-learning space. They saw an opportunity for a rich feedback platform to support collaboration and research, and this is the result. The development team is in Russia, but Mr. Shih and team are based in New York.

Going for a test drive

I spent some time on their beta site, which is very nicely set up. I particularly liked their easy-to-follow graphic walk-through (see the screen shot below). I'd love to see more applications that do this.

This next shot shows the basic interface. You have a page where you put up an image, a graphic or whatever. This could be a screen shot of your web page, or creative concept, or a new packaging design.

You then invite some people to review what you have put up. Reviewers can mark items with arrows, text boxes, call-outs, emoticons and similar tools. There is also a space at the bottom right where they can provide generalized feedback in an open input box.

All this input is gathered together for the researcher to review, but the reviewers cannot see each others' comments.

Application in marketing research

For creative testing, we usually want to have a fair bit of control over the exposure time and conditions for the stimuli. So I don't see this being the starting point for that. But it could be a nice add-on at the end of an online questionnaire, for example.

I could also see using something like this to gather ongoing feedback from users of an application in something like B2B settings. 

I'm less sure about consumer research, but for the right project, maybe it would work. It has some of the features of a web-meeting interface, but it's asynchronous, so everyone doesn't need to show up at the same time.

What happens with this type of tool is that clever people figure out innovative methods once they get a look at it and start fooling around with it. As researcher Betsy Leichliter once said to me, and I'm paraphrasing: "A focus group room is just a room with some chairs. You can do a lot of things in a room with some chairs. Online tools are no different."

Breeze was started as an educational tool that could leverage Macromedia Flash for easy e-learning presentations. Now, the underlying Breeze platform is used for a variety of different types of applications, including some online qualitative marketing research tools.

You can still participate in the free beta test for ReviewBasics, by going to their site and clicking on the "RB – Click to Review Button” near the bottom of their homepage.

Getting Your Users to Tell You What to Do

Can your users tell you what to build / fix / enhance?
Sometimes.

We're seeing a lot more instances of some kind of contest or voting process where users submit ideas, and other users vote on their favorites. To see a current example of what I'm talking about, check out this contest being run by SNAP.

SNAP is a visual search engine that allows you to preview the search results without actually having to browse to the page. Pretty cool, actually. They are asking users to tell them what to build / fix / enhance in their next round of developments. And offering prizes for the best ideas, and for the best comments on ideas others have submitted.

Will we see more of this... i.e. Should I be worried?

Yes, I think we will see more of this. It's a neat way to really open up your development or innovation process to your user community. For a relatively small cost, you can enlist a lot of good ideas. 

I'm not sure what platform people are building this type of application on [comments anyone?], so I don't really know how hard the interface is to set up, but I'm guessing it's not worse than launching a bulletin board or some similar interactive application.

Why I'm not worried

This kind of approach makes a lot of sense when you have users that are fans, and that have enough knowledge to make suggestions. If you browse the contest, you will see that many of the individuals seem to have programming knowledge far beyond the average surfer. Personally, I didn't actually understand some of the suggestions.

There is a sampling issue here -- this kind of input only reaches the raving fan, or those that enter contests as a hobby. If these are your core users, then you are in great shape. If not, you will need to find other ways to tap into the customer experience.

Finally, this approach actually asks the users to suggest new features i.e. solve the problem. A more effective strategy for sustaining innovation is actually asking your customers to tell you about their problems, or tell you about their experiences, not ask them to solve the problem.  When your own development team has a deep understanding of customer experience -- including all the joys and the pain points -- they should be in a better position to articulate possible solutions.

The bottom line
I love triangulation -- where you look at a situation from as many angles as you can. And this kind of user-input is a great adjunct to anything else you might be doing to reach your users. As a nice side-benefit, it will likely also create some buzz, and engage the hearts and minds of at least some of your community. And that's all good.

Enquiring minds want to know
Like the Joker said of Batman, "Where does he get those toys!?", enquiring minds would like to know how to launch one of these contests with a vote. Is this kind of application available off the shelf, or is it a custom programming job?

Customer Driven Innovation

At a qualitative research conference a while back, I attended a session on using games to support innovation. The people you see above are other researchers, holding up their "product box". [Sorry the pics are so small, all I had was the camera phone. And it's taken me this long to get the darn pics out of the phone, but that's another story!]

This method was laid out in Luke Hohmann's book, Innovation Games, and this was a nice chance to see it in action. Tables were given lots of materials to work with, and a plain white box. The task was to create a box for an alarm clock that they would like to buy.

Participants came up with lots of interesting ideas. For example, at my table we wanted an alarm clock that knows what time it is, just like a cell-phone does. A smart clock.

Many people mentioned Apple inspired design -- features and controls so easy to use you don't actually need to read the manual, because it's intuitive. Can you tell that this group spends a lot of time in hotel rooms, trying to set yet another clock radio, or figure out what the current local time is?

What I found especially interesting about the exercise was that the room seemed to divide into two larger groups: those who were concerned about clock functionality, and those concerned about managing the morning mood and ambiance. Could these be segments, I wonder?

During the debrief, the facilitators asked multiple questions of each spokesperson, to uncover the needs and ideas around the benefits sought.

You'll find lots of people who will tell you that "focus groups can't lead to innovation", and other blah, blah of that nature. I say it depends a lot on who is running the event. If you ask customers literally "what should we do?", you may not get that much you can use. 

But the joy of this exercise was that it focused people on unmet needs, and benefits they would derive from the features they proposed. A development team could definitely learn a lot from the output of that.  And the act of creating the box helped all of us get past "design issues" and focus on what we'd really love to have.

References:
Summary of keynote speaker Paco Underhill from the same conference, talking about global trends in retail.

This method first came to my attention through Luke Hohmann's book, Innovation Games.

 

Qualitative Research Conference: Toronto October 2006

Why you need rubber soled shoes

It's cheaper and faster than ever to collect reams of information: it's wisdom in using this information that is in short supply. And you really should have some rubber soled shoes.

These were some of the things Paco Underhill talked about in his keynote last week at the joint QRCA/MRIA-QRD conference. Paco is the hugely successful analyst of shopping behaviors and author of very successful books on the subject, Why We Buy and Call of the Mall. 

Inner Context of the Shopper

One of the most interesting observations he made is that we all shop more often, even though we live in a state of accelerated time. We go into a store with a specific mission, and if we don't fulfil it, may leave empty handed.  That mission, even for the same person in the same store in the space of the same week, could have very different characteristics, based on the value system and emotion at play due to the inner context.

For example, shopping for a child's birthday cake, shopping for tonight's dinner, and shopping for the weekend when the relatives you dislike will be visiting. And in some respects, we need to treat each of those scenarios as if we were dealing with a different customer.

The Key Question

The key question, for Underhill, is always: "how do I respond to local market conditions and build same store sales?"  Depending where in the world you are, these challenges are vastly different.

When speaking of the global middle class, 2/3 of our expenditures are discretionary. We don't need to spend money on much -- someone has to entice us to spend.

In other parts of the world, money is a new and exciting toy, and that's where all the innovation is happening in retail.  Underhill shared some excellent examples of this.

Roppongi Hills, Tokyo

Underhill described Roppongi Hills as the "All Mall". It has hotels, an art gallery, residential towers, and a multitude of recreation facilities. One of the unusual elements is the large amount of public art that has been included throughout the facility, both inside and outside.

Creative use of space in Johannesburg

What you are looking at here is a drive-in movie theatre on the roof of a shopping mall (the Menlyn). A great use of space, and a good way to bring more traffic into the mall.

Another example cited by Underhill was a mall owner who built a stadium for student athletic activities on adjacent land, solely to drive traffic into the food court of the mall.

Forget about Parking in Sao Paolo

In order to bring traffic into the mall from the nearby office towers, this mall offers a free shuttle bus service that will pick people up and then run them back in complete safety so they can shop over their lunch hour.

Finding Happiness in Tokyo

Three Minutes of Happiness is the name of a chain of stores in Tokyo that sell inexpensive items for everyday use. Here's the description from Frommer's guide:

Bargain yen shops have opened all over Japan the past few years, but this is one of the best I've seen. It carries tableware, household goods, office supplies, cosmetics, watches, sunglasses, some basic clothing, and many other simple items, mostly in bright and happy colors like lime green and sky blue.

Although these shops seem very well designed, they are reminiscent of one of the fastest growing categories of shopping at home: the dollar store, and other deep discount formats, that are the antithesis of good design.

Dubai, Dubai

Underhill compared Dubai to the cantina scene in the original Star Wars movie, and called it the first place on the globe that is ready to receive the intergalactic traveller. You can get anything you want, and you can pay in any currency you choose.  Dubai apparently has fifty major shopping malls, in addition to the more traditional souks.

The Challenge

Underhill believes that landlords of shopping malls must go from being landlords to being placemakers. And the best ones are reaching for this. But there is still a lot of room for improvement.

His main recommendation, which we heartily endorse: get some good shoes, and get out the office to see what's going on.

Coming Events: Neuromarketing Webcast

Neuromarketing is the latest hot trend in research circles. Stuffing people into MRI machines and then showing them advertising, and that kind of thing.  If you want to find out what's up with this stuff, you can attend a free web-cast on this topic Nov. 2, sponsored by the Qualitative Research Consultants Association.

Neuroscience seems to be the latest buzz in the research world. There’s as much fear as there is wonderment about it. The purpose of this presentation is to define what neuroscience is and means in the research world, exploring the latest developments and their bearing on qualitative research. It will debunk the myths and set out a set of principles which can be used to inform and substantiate qualitative insight. One of the topics covered will be the neuro principle of ‘hardwiring’, which tells us why people so annoyingly always seem to remember old campaign messages despite millions of dollars being spent on later, more up-to-date campaigns – and what can be done about this.

Details:

"Brains and Brands: How Neuroscience is Impacting Marketing and Market Research"
November 2, 2006
12:00 – 1:30 p.m. ET (GMT -5)
Register HERE
(Note: registration is free, and will also give you access to the archive of other great web-casts, all sponsored by the association to educate our clients and the public. Don't you wish everyone did this?)

Paco's coming to town

If you are in the Toronto area next week, you can take in a good one-day conference on qualitative research, and also hear Paco Underhill (author of Why We Buy and Call of the Mall) speak on Friday, Oct. 20.  It's a joint event sponsored by MRIA and QRCA, about 200 people, and a nice location to boot. Details here.

I am volunteering on the conference evaluation committee, and we've done some pretty radical overhauling of the evaluation forms. (Some of it is even printed in color! What will they think of next!)  I'll show you what we did and tell you how it worked in a couple of weeks.

Survey Research: Are all the same people answering all the surveys?

Advertising Age has an interesting piece about survey research: Consumers Rebel Against Marketer's Endless Surveys: 30 Top Industry Execs Gather to Discuss 'Opinion Fatigue' Crisis

At a roundtable held among execs of large consumer products firms and their researchers in Chicago, there was serious handwringing about survey fatigue and non-responders.  Perhaps survey research is not as valid as it used to be, too small a sliver of the population is making money as professional respondents, etc. 

If you are hearing this kind of thing and wondering what to do in your own organization, my suggestions are these:

Treat your respondents with respect.

Ask smart questions. Ask only what you can act on, not what would be nice-to-know. Pre-test all surveys with a small sample of the target group and interview them. Then fix the problems with confusing or irritating questions.  This does cost a bit extra, but you will get much better results.

Out of a sense of obligation to the industry, I try to participate in all research I am not barred from. And I am frankly shocked at how much of it is frustrating to participate in. Maybe you've had this experience yourself, and it's disrespectful of you as a consumer or a customer or a voter or a taxpayer.

Be quick about it.
Challenge yourself to make your next survey half the length of the last one.

Use multiple methods.
Multi-method projects come together on the problem from several angles. This can add tremendously to your understanding. You'll also avoid being blindsided by a sampling or response bias issue from a single survey instrument. 
You could combine an internet survey with a site intercept questionnaire, with some depth interviews. If something doesn't seem to fit, test again.

Consider paying people for their opinions.
I know this is unpopular in research circles. And indeed, paying people to be on large research panels is part of the problem. But if you want people to spend some of their free time answering your questions, maybe you should reward them appropriately. After all, how do you yourself feel about this?

Spend the money for real research
Your research supplier should be a member of a professional research association in your country. These organizations have standards and ethics and auditing bodies. They conduct training and education programs, publish useful articles and journals, and generally try to maintain high standards. 

If your supplier isn't part of one of these organizations, you need to ask yourself why. Would you go to a doctor that didn't participate in continuing education? Even your car mechanic and hair stylist is likely to have recent educational certificates on display.

This problem is really the tragedy of the commons all over again. Because it is getting cheaper to do survey research, there's lots more of it, and the natural tendency of most people to co-operate and be helpful has been abused to the point where a lot of people just don't want to play anymore.

What's the story?

Reshma Anand, a qualitative researcher in Bangalore, has posted on a very innovative idea about how to enrich storytelling. She thinks that if we put ourselves in the present tense when we tell a story, we can add to the richness of the detail and enhance our recall. 

This makes complete sense to me, although I bet it takes some skill to get people to do this. If you try it, you'll see what I mean.

Instead of saying, "I stood at the reception desk for a while. No one was around, so I went to sit down." you are now saying "I am standing at the reception desk, waiting for someone to notice me. They don't seem to be around. I decide to sit down and wait." 

Interesting to remember that Einstein used just such a method to imagine what would happen to his streetcar if it accelerated to the speed of light. He tried to put himself in the moment on the streetcar, and then to put himself in the moment of someone observing the streetcar. [And people think qualitative isn't scientific...d'oh]

The nuances and recall help to enrich what is being said.

Kim Bercovitz told me recently about another innovative approach -- using actors to tell a story as a research method. It sounds a lot like the kind of improv used to entertain conference participants, except applied to get clarity around experience and emotions. The participants keep correcting the actors' performance of the thing until they get it right. And then you know you have all the ingredients, don't you? [I suspect there's a lot more to this method than I'm capturing here, sorry Kim.]

Even objects have stories to tell, as this post from ethnographer Grant McCracken shows. Grant's own stories give us insight into the mind of the researcher, the professional observer, the person who can never stop watching and thinking.

Reshma's post illustrates something else brilliantly: there is no big book out there of the right way to observe. A lot of people seem to think there is. There is trying, and testing and learning and improving. There is no one right way.

The rules are all nonsense, really, because your method is only as good as the insights you get.

You want to break out, you want to be better than everyone else? You do it by ignoring the conventional wisdom, or tweaking it, or trying something different.

Kisses and hugs to all the clients out there who help you make these moments of magic happen -- by leaving enough space to try something a little bit different. They are the ones who win the biggest prizes in insight. We bust our butts for those clients, to try to make the risk pay off.