The Intelligence Community Crowdsources Ideas. Maybe You Should Too
It's not easy to be an analyst for the Central Intelligence Agency. As the CIA says, these positions are filled by "some of the brightest people in the country" who bring "integrity and objectivity" to the work of evaluating information of all types from around the world. The agency lists 13 different types of analysts whose combined work with top secret information is supposed to illuminate what will happen so the U.S. can more intelligently plan strategy.
But it turns out that 3,000 regular people have been predicting world events as well as part of the Good Judgment Project (GJP), run by three psychologists with the backing of the U.S. Intelligence Advanced Research Projects Activity program.
Their predictions are often better than those oh so bright people with access to the latest in secret intelligence, even though the regular folk just use what they find on the Internet. According to NPR, one of the top is Elaine Rich, a 60-year-old pharmacist with no background in international affairs and little formal math training. Rich is one of the projects "superforecasters," whose work is 30 percent better than the professional analysts.
You're probably not looking to join the intelligence community, but there are some great lessons for entrepreneurs in this story about crowdsourcing.
Want a good answer? Ask a group.
One is the theory of the wisdom of crowds. Forget all the popularized stuff you may have heard as yet another technology fad that came and went. The idea is that if there is something true that a group of people respond to and you get enough diversity in a large-enough crowd, chances are their biases will cancel out and you'll get something close to reality.
Entrepreneurs will want to twist this a bit. You're not going to put each of your strategic decisions up for a plebiscite. However, consider widening your list of advisers.
Find the experts out there.
One of the problems with popular interpretations of the wisdom of crowds is to assume that a random group of people will outperform experts every time. That's not really the case. For example, GJP works with experts who are particularly good at forecasting. The difference is that they aren't all in one specialty.
Get people with different backgrounds, interests, and biases on your advisory board and not necessarily the typical list of experts and consultants. You still have to make the decision, but you might get some insights that you wouldn't otherwise hear.
Measure and take a reality check.
One of the keys to making crowdsourced predictions work is to record accuracy and provide feedback to the participants. Otherwise, the participants may not even know when they are right or wrong. It becomes impossible, then, to learn lessons and improve the process. It would be like asking someone to learn how to cook but then never letting them see, taste, or smell the results.
So when you're working with advisers, prepare reports after the fact and let them know how close they were, as well as what you ultimately decided. That way they can get better and might take even more vested interest in your success.
ERIK SHERMAN | Columnist
Erik Sherman's work has appeared in such publications as The Wall Street Journal, The New York Times Magazine, and Fortune. He also blogs for CBS MoneyWatch.