The Case for User Reviews
It was late August 2007, and Michael Ortner and Rakesh Chilakapati, co-founders of Capterra, had gathered a handful of key managers for a meeting in the company's small conference room. As the group assembled around a blue Ping-Pong table that sat in the center of the tiny, windowless room, Ortner made his case for a big step forward. He wanted Capterra, which operates an online directory of business software vendors, to start including product reviews from users on the site. Surveys showed that the site's users were craving the feature, and Ortner figured that providing it would help build community and loyalty.
Chilakapati, Capterra's chief technology officer, with whom Ortner had teamed to launch the company in 1999, listened politely but wasn't convinced. He was worried that introducing the feature would overtax the company's 18 employees, wasting time and money, and that opening the site to nasty reviews would only rankle the very software sellers that are the sole source of Capterra's revenue. Moving forward could invite only headaches and lost momentum. "What if a big company wants to sign on, but they first want a bad review taken down?" he asked. Ortner's response: "So, let them walk."
The two co-founders are friends but don't always see eye to eye. Chilakapati, the introvert techie, is the skeptic -- "the last line," says Ortner, "to make sure my ideas are vetted." Ortner, the extrovert salesman, on the other hand, often embraces risk. After four years of working in the IT departments of Price Waterhouse and J.P. Morgan, he joined a Washington, D.C., Web hosting company called Digex. There, while looking for software partners, he hit on the idea for Capterra. "I saw how fragmented the business software market was," he says. "And it was very expensive for those software companies to reach their target audience. It made a lot of sense to create a website that would connect the buyers and software vendors."
The problem was that when Ortner and Chilakapati launched Capterra, there was no ready-made audience of online software shoppers, and few sellers were willing to sign on. Capterra's model hinges on vendors receiving preferential placement on Capterra's online directory in exchange for paying for sales leads and click-throughs to their websites from Capterra's. It took a year and a half to land the company's first customer and another excruciating 13 months to land the second. By that time, Ortner and Chilakapati had run through the money they raised from friends and family, and Ortner had racked up about $250,000 in credit card bills. Finally, in 2002, Capterra started landing more vendors. Capterra's customers now include IBM, but most are small and midsize companies that sell customized software to help businesses with, say, accounting or logistics.
In early 2007, Ortner began toying with the notion of adding customer reviews. In surveys of Capterra's buyers, reviews were the most frequently requested feature people wanted added to the site. Ortner's first thought was to start gingerly by posting testimonials. That way, vendors wouldn't feel threatened that they may wind up paying for the privilege of getting slammed by some unhappy customer. The vendors would also control the content, taking down any testimonials they didn't like.
Sharing his idea with his core management team and sales and marketing staff, Ortner got some positive feedback, including from the company's product director, Cristina Stensvaag. But soon, Ortner started to have second thoughts. "I realized this was dodging the real issue," he says. "Buyers want reviews and ratings -- and negative reviews are part of that."
Ortner and Chilakapati spent hours poring over review-heavy websites like Amazon, eBay, and online car buying guide Edmunds.com. Both liked the requirement on Edmunds.com that reviewers give both pros and cons, something they agreed would eliminate overly puffy and overly negative reviews. But Chilakapati remained wary that vendors would ever agree to cede so much control of their message.
The pair also agreed that a review feature, should they go ahead, would not permit anonymous reviews -- to protect against competitors posting phony, damaging reviews. Vendors, moreover, would verify that reviewers were, in fact, customers and could respond to any negative reviews. But Chilakapati and several members of his staff were still not convinced that building the review feature would, in the end, be worth the trouble.
The discussion came to a head in the August meeting around the Ping-Pong table. To win over his doubters, Ortner proposed sending a survey to Capterra's 1,000 paying vendors to get their reaction. But the survey simply sowed more doubt. The results showed that although most vendors would encourage customers to submit reviews, about half also worried about negative reviews.
Ortner, increasingly convinced that unfiltered reviews were the best path toward credibility, began working the phones. He started his lobbying campaign by calling vendors who had expressed major reservations about the reviews. Among them was Glenn Martin, president of Promantek, a seller of performance-appraisal software aimed at small and midsize companies. "I recoiled," says Martin about his initial response to the plan. Among his concerns were the possibility that bigger companies would prompt their army of customers to submit glowing reviews, while Promantek would always run the risk of getting hit with negative postings among a smattering of reviews.
Ortner says the feedback from Martin and some other critics convinced him they were all missing the opportunity to get great reviews posted for their companies. "Those conversations cemented in my mind that we should do this," Ortner says. "I wanted to prove they were wrong."
Chilakapati, though, saw it differently. Discontent from a big customer over a bad review "was one of the heavy cons," he says. "So, how do we handle it? My point was, we needed to take a stand: Either we allow all reviews, or we don't." If they allowed all reviews, a lack of buy-in from vendors was a major risk. And, without a sizable pool of reviews, the system would have little utility.
The Decision For three months, Ortner and Chilakapati debated the issue, often ending in a charged game of Ping-Pong to relieve stress. Finally, on the first Monday in October, Ortner set out on his twice-weekly five-mile run from office to home. The stretch of quiet solitude cleared his thoughts. "We had beaten this issue to death," he says. "Now I was listening to my gut, and it said we have to do this."
The next morning, Ortner instructed his team to move ahead, though he was careful to respectfully acknowledge Chilakapati's concerns. "While we were not in perfect agreement, he was satisfied with our evaluation process," says Ortner. "So we decided to proceed."
Over the next 10 months, Chilakapati's team assembled the software to run the program. (He estimates the total time spent on the job was 2,000 man-hours, valued at $150,000.) And in August 2008, Capterra asked vendors to contact customers and encourage them to go to Capterra's site and submit a review. The company collected about 500 reviews over the next year. "Once we started gathering reviews, I realized this was worth our investment," says Chilakapati. "That was the turning point."
The review system went live last summer. Since then, the company has continued to collect reviews solicited by vendors. But visitors also submit reviews directly, and those unsolicited posts account for about 40 percent of the 2,000 reviews now on the site. Most are positive, generating four or five out of five stars, but some 10 percent to 20 percent are three stars or fewer. An occasional less-than-glowing review is a small price to pay, says Promantek's Martin. He now sees the reviews as a boon for his company, increasing leads through Capterra more than 30 percent. "We are visible and highly ranked," he says. "And it gives potential customers the confidence that we are a worth a look."
Ortner's hope all along was that such satisfied vendors would spread the word and bring increased business listings to the directory. That seems to be happening. Last year, revenue was up 29 percent, to $4.9 million. Today, both Ortner and Chilakapati agree that their worries about negative reviews alienating vendors were overblown. Says Ortner: "Not a single one walked."
The Experts Weigh In
Good Start, But Needs More Reviews
Capterra has done lots of the right things to provide customers with the reviews they crave. Where Capterra appears to have fallen short is in the number of reviews it has attracted. In most categories, only one or two products have reviews. Capterra might arrange with vendors to pull in an RSS feed from their customer support sites, where customers strut their stuff and ask each other questions about how to do things. This would give prospective buyers a view into what actual customers are doing and talking about.
CEO, Patricia Seybold Group
Don't Overdo the Controls
The reviews provide differentiation for Capterra. Customers are looking to make informed decisions, and if they don't get the information from Capterra, they will just Google their way to other sites to find it. But where this gets challenging is vetting the customers. You do need an ongoing effort to prevent people from gaming the system. But by having vendors review the reviews and verify that the writer is a customer, the software user will think twice about writing a review. If the site is overvetted, people will stop submitting reviews. This gives too much power to the vendor. My gut tells me nobody will submit a negative review.
Vice President, Gartner
Look Beyond the Purchase
Discouraging anonymity was a smart move. There are places where it is appropriate, but I don't think this is one of them. The more someone can identify with the person who submits a review, the more likely the reader is to believe it and find it valuable. But customer service is keeping customers engaged before, during, and after their purchase. The next step is to think about the postpurchase experience. One option is to open up a space on Capterra for individual products where customers talk about using the product. This keeps customers engaged with the product and with each other.
President, Get Satisfaction
PRINT THIS ARTICLE