DJ Patil has an interesting job: He's the data scientist in residence at Greylock Partners, the legendary venture capital firm that helped launch LinkedIn, Facebook, and Pandora, among many others. Why does a VC firm need a data scientist? For three reasons, Patil says. "First, data helps us assess a particular company as an investment. Next, we can identify opportunities that haven't gotten as much attention as we think they should. And third, we can help companies in our current portfolio use data to find new ways of doing things, or who are struggling with scalability problems."

As Patil is well aware, "big data" is one of today's most over-hyped phrases, particularly after the New York Times Magazine reported that Target could use data to identify pregnant customers. "We've covered big data every possible way," is how one tech publication editor puts it.

"One of the questions we should put on the table is: 'Is data a bubble?'" he asks. "Why would someone like myself still be bullish on data? It's powerful when there's a way to create efficiency. But things shouldn't be purely data-driven."

Asked which industries will likely be transformed by big data, Patil points to healthcare. "But we all get too focused on using data to replace humans." Instead, Patil says the future of healthcare should be like Star Trek, with doctors running an instrument over a patient's body to get detailed information. "We don't take the doctor out of the loop, we make the doctor smarter," he says. "For instance, you and I could both come in with 103-degree fever and the doctor might assume it's the flu that's going around. But better data would show that one of us just came back from Nicaragua, which is having  one of the worst outbreaks of malaria in recent history."

What can data do for you?

1. It can improve your product.

"How do you provide a better product? By using data. Tumblr and Dropbox fall into this category," Patil says. For instance, he says, Dropbox uses data to create the most efficient use of its storage, and to iterate new versions and features that customers value. "Data can become the weapon of choice by which you make your stuff really work."

Patil points to Zynga as one example of a data-driven company. "In FishVille, one fish was selling at six the times the rate of everything else. So they made a whole bunch of fish with similar characteristics. Eventually, they learned that translucent fish sell much better than all other fish."

2. It can help you spot change.

"Data is one of the lead indicators that things are changing," Patil says. "With data insight we can ask, 'Are we comfortable with the way the system is going?'" But rather than simply making decisions based on data, Patil recommends using data findings as a departure point. "The right approach is to have data start a conversation," he says. "Now everyone's getting smarter together."

And companies ignore data signals at their peril. Patil claims, for instance, that not paying attention to data was behind the downfall of Friendster, the first significant social network which was eclipsed early in its life first by Myspace and then by Facebook. "One of the key things Friendster didn't do correctly is that they didn't focus on how long it took for their pages to load," he says. "It's painful to wait six or seven seconds to get a result. Imagine if you entered a search term into Google and had to wait six or seven seconds. They weren't measuring that and if you can't measure it, you can't fix it. It allowed other competitors to gain on them."

3. It can help you make human connections.

Huh? It may sound counterintuitive, but data can help companies connect people with each other, Patil says. For instance, a social network is most useful when you have a critical mass of friends there, he notes. "So you can ask, how long does it take to get to, say, 120 friends? What can we build for the product to get that done in half the time? And then half again? Now you can construct a product road map based on what the data is telling you." Calculations like these lead to features such as LinkedIn suggesting people you may know. "That's a data product," Patil says.

Google, he adds, is taking it one step further by trying to write algorithms that analyze how people get along. "They're asking: 'How can we scale in human ways?'" he says. "'What kind of people work well together?' We can use analytics to make our corporate culture better."

Follow Patil on Twitter @dpatil.

Published on: Jun 14, 2012
Like this column? Sign up to subscribe to email alerts and you'll never miss a post.
The opinions expressed here by columnists are their own, not those of