I get asked a lot about how much to rely on data versus gut instinct when innovating the next big thing. I wrote about this extensively in my book Leapfrogging. The answer may surprise you.

Disruptive Innovators Use Their Gut

Apple never did formal market research to tell them what to do. Steve Jobs once said, "We do no market research. We don't hire consultants. The only consultants I've ever hired in my 10 years is one firm to analyze Gateway's retail strategy so I would not make some of the same mistakes they made [when launching Apple's retail stores]."

Target has had a similar philosophy. Robyn Waters, the former vice president of trend, design and product development, once told me the same thing. Target didn't do direct consumer research, especially focus groups. They used their own instincts to infuse leading-edge design into everything they did, from transforming the stores' layout to creating partnerships with world-renowned designers--which ultimately led to transforming the retailer from "Target" into "Tar-jay."

Use "Unconscious Thinking" to Innovate

Many people debate the merits of data analysis versus intuition. It's not an either-or issue. There's much more to it.

Imagine you're asked to compare four different apartment listings. Each has a set of 12 bullet points describing their respective features; in all, there are 48 different pieces of information you need to consider. Apartments differ in size, have different amenities, and some are in slightly better neighborhoods. One apartment has a landlord that isn't very friendly, but some of its other features are quite positive. You're asked to select the "best" apartment of the four.

What you don't know is that two researchers from the University of Amsterdam intentionally skewed the apartment listings so that one apartment has the most desirable features, one has predominantly negative features, and two are somewhere in between. In addition, there are three groups of participants. Participants in the first group are asked to select the best apartment immediately after reading the comparisons--they don't have time to really think about their choice. The other two groups are allowed a three-minute break in between viewing the apartment listings and making their selection. Of these two groups, one is instructed to review the listings and ponder their answer during the three-minute interlude. The other group, however, is distracted with another activity for the three minutes prior to naming the most desirable apartment. Which group do you think picks the best apartment most often?

The findings are always the same. The third group--the one that's distracted with the irrelevant activity consistently showed a preference for the most desirable apartment. The reason for this is simple. The seemingly intrusive distraction actually gives them valuable "unconscious thinking time" to evaluate their options without even being aware that they are doing so. In other words, they're processing the glut of data subconsciously.

Intuit the Big Picture

The researchers also found that these unconscious thinkers were better able to cluster and categorize the large amounts of information given to them, such as the laundry lists of bullet points describing the apartments. When asked about their thought processes, they reported that they used holistic judgment (a feeling for the big picture) in making their decisions, versus the other participants who tended to base their choices on only weighing trade-offs between a few of the data points.

With unconscious thinking, our brains essentially work behind the scenes to synthesize data and draw conclusions without us even knowing it. This is why we'll often hear someone tell us to "sleep on it," which basically means give your brain a rest and let the answer come naturally. When we're under stress or trying to solve a pressing problem, we tend to rev our brains into overdrive. The thought of setting the work aside for a period seems counterproductive. But it might just be the best thing to do.

This research also tells us something about finding patterns to uncover opportunities. When we try to hold large amounts of information in our heads--like all the trends, customer needs, competitive activity, etc. going on in an industry, or even multiple industries--we simply can't process such complexity with what we've got between our ears. The patterns and themes that represent opportunities can more easily be discovered using a broad brush that reveals the overall challenges or latent needs of the market, stakeholders, or business situation. When we find ourselves staring at a lot of data, statistics, or mind-numbing charts and graphs, we've taken a step too far into the analysis zone. The goal is to seek big conclusions that provide overall direction, versus details that will leave you drowning in data.

Great innovators don't get bogged down in the data. They look for patterns and move fast. They let their unconscious thinking do the heavy lifting.