Steve Jobs famously said that asking customers what they wanted in the way of a new product or service was a waste of time. He said it was Apple's job to anticipate their needs because "customers don't know what they want until we've shown them." There's been a lot of discussion about whether this means that market research is basically useless or whether it means that the way market research has been used by many businesses isn't helpful, instructive or productive. I'd argue that it's the latter.
I don't think Steve was condemning an entire industry, although he didn't have any great love for gathering or relying upon numbers for numbers sake. His basic disinterest stemmed from the issue that-- way back then and right up to today--too many companies use market research like a drunk uses a lamppost, for support rather than illumination. Or worse yet, they approach it like the guy looking for his lost keys beneath the street lamp because that's where the light is best. Convenient, but not constructive. Comforting, but not convincing or compelling.
And, basically, that's what you're doing if you're designing and deploying your market research to support foregone conclusions and to justify customer assumptions that you've already built into your offerings. If you know what the answers are going to be because you "fixed" the game, then it's not research-- it's redundant window dressing not worth the time, money or effort. It's not an experiment if you already know the results. And if you undertake a mainly CYA exercise to satisfy someone else's fears or anxieties, there are quicker, cheaper and easier ways to deal with these needs. Even medication and mixed drinks are better cures for corporate "concerns" and bureaucratic BS than throwing money away on silly surveys.
On the other hand, if you want to make a sincere effort to use the available tools to get better and smarter about understanding the needs and desires of your existing and potential customers (and to even build on those foundational beliefs to create new needs and drivers), there are some effective ways to go about it. Well-designed field research is absolutely essential when you're focused on incremental and iterative improvements because committed customers are relatively adept at identifying the margins, and in describing and detailing adjacent adjustments and tweaks. Please don't leave these things up to your engineers or you'll be adding useless features and functionality that no one needs. Leave the development of bloatware to the big guys.
When you're looking into "new" as opposed to "next" and when you're trying to cross the chasm and change the game, defining the scope and objectives of your market research efforts is harder. Many experts would tell you that it can't be done by asking your customers. I'd say that it depends on what you ask them and what you're looking for. If your inquiries are oblique rather than obvious, and contextual rather than confrontational (not in the aggressive sense of the word, but simply meaning too direct), I think you can learn a great deal.
I like research that focuses on the 4 Cs: conditions, circumstances, concerns, and community. You're looking laterally and you're trying to learn more about the customers' lives than about their expressed likes and longings. These are different, longer range lenses and they're likely to return data relating to longer term, but much more valuable and substantial opportunities as well.
The innovative products of the future, the ones creating sustainable and valuable new offerings driven by behavioral changes, aren't going to be predicated on new flavors or even fabulous new features. They're going to be ones that address fundamental needs that are often apparent from the outside, but which consumers and customers are often reluctant to address or admit. I saw an interesting example of this process recently in Japan.
Japanese companies have been eliminating office coffee makers. I don't know whether that's about cutbacks or that no one wants to make and wait for an entire pot of coffee, or the new abundance of coffee shops. But this has basically killed the coffee break and, much more importantly, the conversations, collisions, and even some of the social interactions and community that used to be a part of these places. Maybe people still gather at the water cooler for chats, but I doubt that as well.
This is real problem if, like Nestlé, you sell happen to sell coffee makers and coffee. It was clear from Nestlé's research that the employees missed the breaks and the chatter far more than the coffee.
So, the CEO of Nestlé Japan decided to build a single-serve coffee maker for offices that lacked their own machines. Importantly, his rollout plan also took into account the real pain point for the target consumers, which was actually the loss of connection and community. The new machine has been a huge success, but not for the reasons that you might expect. In fact, if Nestlé had simply tried to sell machines and cartridges into the many offices that had just dumped their machines, the project would have been a bust. And simply giving away the machines while planning to sell the "razor blades" to the companies wouldn't have worked either because the aggregate weekly costs of the cartridges are not insignificant.
Instead, Nestlé's plan enlisted individual advocates and influencers in each office who "work" for Nestlé, for free, by collecting payments from co-workers each weekly for their coffees and remitting the sums to Nestlé to replenish the cartridge supply. Not only has the connection point been restored, and the community reinvigorated, but the very acts of the collection process have increased the inter-personal interactions and engagement among the participating employees in the program.
Needless to say, there's no way that any traditional front-facing research would have elicited the underlying drivers and real desires of the customers. Even more importantly, it turns out that the real "customers" aren't the businesses themselves, but the individual workers who are drinking the coffee. A standard survey of employers wouldn't have uncovered any part of the problem or helped to determine an innovative and immediate solution to the issue of funding the program.
Customers can't tell you what they don't know--especially if they're not actually the customers who matter.