In his Wired article in 2004, Chris Anderson pioneered the use of the phrase “The Long Tail” as a proper noun. He observed that the reduced costs of distribution over the Internet are making it easier for businesses to serve consumer demand for niche items, and that collectively, the niches added up to quite a significant market for companies like Rhapsody, Netflix, or Amazon. This collection of all niches, “the long tail,” he argued, generates substantial value for a variety of businesses.

The long tail effect liberates consumers from having to buy what everyone else is buying, and enables businesses to serve specialized needs, rather than just serving the lowest common denominator. This idea flies against the traditional distribution networks, which only stocked those items that are most likely to sell a lot of units. Traditional supply chains needed to play this “blockbuster” strategy because their fixed costs of carrying any item and making it accessible to customers were very high. And the only products that could justify that cost were the ones that were likely to sell a lot of units.

For example, in the movie industry, this “supply chain” consisted of theaters, and DVD sales through large retailers, both of which have high fixed costs and limited shelf space. The Internet reduces many of the fixed costs, removes the restriction on shelf space, and makes every item easily accessible through searchable interfaces. No wonder, then, that Internet companies have chosen to carry larger inventories than their offline counterparts, enabling them to cater to niche interests.

Backlash against the long tail

Despite the widespread use of this idea, there has been recent backlash against it. Criticism of the idea recently started with an analysis by Anita Elberse, and was picked up by many others in the media. The critique centers around the idea that long tail companies make most of their profits from a small percentage of items sold; the classic 80-20 rule still applies. Based on this, the critics recommend that entrepreneurs and managers should continue to focus on the blockbuster strategy.

This analysis, however, overlooks the fact that it’s impossible to predict what will be a hit. Consider the list of top 100 rentals on Netflix a good indicator of consumer demand. At least three of these movies are independent films, each with a budget of $6-8 million. Juno, with a budget of $6.5 million, went on to gross 35 times that in earnings, and Little Miss Sunshine, which cost $7.7 million to make, earned more than $100 million in revenues. The Last King of Scotland, with a budget of $8 million, completes the trio.

Compare these 10x returns with a mainstream movie like The Dark Knight, which made less than six times its budget of $185 million -- and that’s among the more successful hits. Before they made it big, movies like Juno, Little Miss Sunshine and The Last King of Scotland seemed like films that would end up in the long tail. And the entertainment executive who focused purely on the blockbuster strategy would have missed out on the financial returns of these movies.

When the critics of the long tail theory account for revenues and profits, they look at data compiled after the masses have picked the winners and the losers. Some of the “winners,” the items that made it to the top 20 percent of revenues, might have come from the long tail of investments (e.g., the low budget movies). So when the critics recommend not investing in the long tail, they’re confusing the tail of revenues with the tail of investments.

The real value of the long tail is that it helps pick the winners like Juno. You might think the cost of picking winners in this way is prohibitive, but that’s no longer true in the new digital world. In the traditional supply chain, an average long tail product would not make any money at all, because it wouldn’t be stocked anywhere. However, in the digital world, even the product that starts life in the long tail and stays there makes some money because it reaches a niche audience. And, who knows, there’s always the chance that some of these long tail products could become popular someday, and make tons of money.

The author Nissim Taleb deals with phenomena like these where the outlier (like Juno) has a significant effect on the average performance. He calls this effect the “Black Swan.” In the blockbuster strategy, which requires heavy investments, the returns on all movies would perhaps cluster around one, and an outlier like The Dark Knight returns six times its investment. The exposure to the outlier, or the Black Swan, is much greater in the tail. As we’ve seen, an outlier in the tail can have a much more significant return.

These effects only get amplified with digital distribution, where shelf space is unlimited and the investment in distribution costs is negligible. As The Arctic Monkeys, Clap Your Hands Say Yeah, and many others have shown, it is possible for small indie bands to rise to stardom by the buzz they create on the Internet.

While the long tail is exposed to the positive Black Swan (unexpectedly good returns), the blockbuster strategy is only exposed to a negative Black Swan, because the strategy requires heavy investments.  A prime example is the movie Waterworld, made with a budget of $175 million, which grossed only $88 million worldwide.

Does this mean that the blockbuster strategy is dead? Not at all. After all, the Netflix 100 has a lot of room for high budget movies. Amazon and Netflix understand this very well, and put as much focus into the likely hit as they do into carrying all the indies. Their infinite shelf space allows them to stock the Harry Potter book or The Dark Knight DVD as well as lesser known title, and their search and review mechanisms help the community vote up an occasional obscure title to become a hit. The lesson?  Success is about finding the right balance of long tail strategies and more traditional approaches.

Vijay Chittoor is the director of product management at Kosmix, an exploration engine that offers a 360 degree view of any topic on the Web.  A former McKinsey consultant, Vijay is a graduate of Harvard Business School and the Indian Institute of Technology, Bombay.  He shares his thoughts on technology at his blog..