InnovationInspiration

The Problem with Focus Groups

If Facebook had concept tested their newsfeed with users 4 years ago when it first came out - odds have it, they would have immediately shot it down. Judging from the backlash it generated, if Facebook trusted in focus groups or in the prescience of their customers, they would never have built what is arguably their core distinguishing feature.

Similarly, Apple, who is known for it’s “No focus groups” policy, may have never built the IPad if they listened to the initial reactions of customer concept testing: 

“Its completely useless, it doesn’t do anything. Why would I want another computer?”

Even my friend Kareem, who is an avid technology user, admitted that when wireless first came out, he didn’t understand why he would ever want to use his computer away from his desk.

It’s somewhat scary to think about the whole process. Months of hard thinking and hard work by a product designer, and then a concept can be scrapped because a room of potentially disinterested people are not able to see the full impact of the idea. 

Beyond that, just because you don’t have a design “right” on the first shot, doesn’t mean you don’t have the inkling for what can potentially be an amazing product. Sometimes, you need to wade through the initial doubt to realize great accomplishments.

The problem with focus groups has many aspects to it:

1) What is the appropriate time frame for evaluating if a product resonates with users?

If you had shown the newsfeed to users for only 5 minutes - they may have been terrified by how much content is being thrown at them. The initial reaction is being overwhelmed by content. Generally, users prefer the familiar. But, with time, users can adapt to a product, and learn how to use it.

Especially in the case of user experience, I think that there are important considerations and trade-offs to be made between initial discoverability/enjoyment, versus eventual ease of use and enjoyment. 

A great example is the Iphone’s delete/move application feature. Why would it ever occur to me that to delete an application I need to hold down an application until they all start to shake? Yet, when the time came that I wanted to delete an application, I figured out how to do it. And, I will continue to remember how to do it. In the rare or eventual occasion that I actually needed to perform some more difficult functionality, I put in the effort to figure out what needs to be done. I think this type of “burrowed” functionality, that is exposed and discovered only when the user eventually needs it, is crucial to enabling a more elegant and human design.

We can be kind to users by walking them through this process -cater for both the initial and eventual experience by gradating the UX as time goes by. Facebook is a good example of this. Where initially they had text + icons for some menu items, they eventually transitioned to having just the icons. The association had been enforced with users over time, and so it was acceptable to transition from the initially beneficial design to the eventually beneficial one.

Inquiring what the optimal length of time for AB testing is an excellent question from a methodology point of view. For each product it varies, but I would argue that months are needed before fully understanding whether a product has achieved its functional goals.

2) The users have not been thinking about the problem incessantly, and realizing that it can bring value to their lives.

We need to appreciate that a lot of the time, users are not constantly contemplating what the world of the future may look like. They may not understand why a technology they are being shown today, is relevant for the context of tomorrow. 

Also, they may not appreciate that something is a good solution to a problem, because they have not actually tried solving the problem themselves. When engaging in the act of trying to solve the specific problem, you start to see the nuance, and the nature of a good solution starts to jump out. They’re not usually experts in the tech field. Even though some would say that because they are the users, they are the experts… we need to take that notion with a grain of salt.

Democracy vs. Dictactorship

An apt analogy is do we design our products by democracy, or by autocracy? Apple and Facebook seem to have gone down this route to some extent (I don’t work there, but you hear stories about their CEOs having an involved role in most product decisions).We can wonder whether this is a sustainable approach, and whether the employees who engage with this type of culture are ultimately going to be satisfied.

But, expanding on this analogy, we can think about the difference between countries and companies. One common economic argument in the past is that dictatorships are a good form of government to carry a country through a period of industrialization (not from a humanitarian point of view, but from an economic growth point of view). But, in the case of a country, the main aim ought to be to ensure the happiness of its citizens. For a company, the main aim tends to be different: the maximizing of profits, not necessarily maximizing the happiness of employees (Zappos claiming to be the exception). In order to do that, progress is necessary, and in order to progress, vision is necessary. Is groupthink avoidable in more democratically inclined companies? The question is, what form of company governance/culture is more likely to have vision, and to act on that vision - a democratic one, or an autocracy. I don’t know what the answer is.

But, I do have my hesitations about focus groups, and suggest this:

next time we think about scrapping a product because a certain set of users doesn’t like it, we should rethink whether our experimental methods are valid. Do we need to run longer focus groups? Do we need to talk to more users, to explain the context better (without pressuring them for validation)? Perhaps, when we hear dissatisfaction, we need to deafen our ears, take a leap of faith, and see where that less-traveled road takes us.

P.S. so much for me not writing long posts any more.