Leaders Say They Love Experiments. So Why Don't They Actually Do Them?
Duke University-based behavioral economist Dan Ariely tells a story about what he calls a "typical case" of experiment phobia. A consumer products company asked him to design an experiment that would help it choose the best pricing model and subsequently agreed to his proposal to create multiple websites with varied offers to discover what customers responded to best.
But then, predictably, the company got cold feet. "I've often tried to help companies do experiments, and usually I fail spectacularly," Ariely says. In this case, the company's leaders got squeamish because the experiment would require treating some customers unfairly (by charging them more than others).
The company's team asked Ariely if, instead of levying any mistreatment, he might just tell them which pricing approach would be smartest. "I told them I was willing to share my intuition, but that intuition is a remarkably bad thing to rely on," Ariely says. "Only an experiment gives you the evidence you need." The company called the experiment off.
Ariely finds this behavior amusingly irrational. "Companies pay amazing amounts of money to get answers from consultants with overdeveloped confidence in their own intuition. Managers rely on focus groups--a dozen people riffing on something they know little about--to set strategies. And yet companies won't experiment to find evidence of the right way forward."
He also says this behavior is entirely predictable. Companies shy away from experiments for two big reasons:
1. "As the people at the consumer goods firm pointed out, experiments require short-term losses for long-term gains. Companies (and people) are notoriously bad at making those trade-offs."
2. "There's the false sense of security that heeding experts provides. When we pay consultants, we get an answer from them and not a list of experiments to conduct. We tend to value answers over questions, because answers allow us to take action while questions mean we need to keep thinking. Never mind that asking good questions and gathering evidence usually guides us to better answers."