When you buy products online, either for yourself or your business, reviews probably weigh heavily in your decision making. We check scroll down to see other buyers' opinions on Amazon, opt for the five-star option rather than the one with only four and a half, or book the Airbnb with the most enthusiastic former guests.
Of course we all also know these reviews can be bogus - either paid for by the seller to or maliciously placed by the competition. But most of us think we're pretty good at separating phony reviews from the genuine article.
New research suggests we should think again.
Why you should be extremely cautious about online reviews
When a team of Cornell University researchers decided that building a computer program that could spot bogus recommendations sounded like a useful thing to do recently, they first had to test how good humans are at the task. To this end they rounded up a panel of three student volunteers and presented them with both known fake reviews and ones that had been verified as indisputably real.
How did the humans do at spotting the fakes? In a word: terrible.
The students frequently could not agree on whether a particular review was phony and eventually came to the right decision less than 50 percent of the time. They would have done better if they had just flipped a coin.
The results suggest why so many of us end up booking terrible hotels or buying subpar products -- we're taken in by bogus 'opinion spam' way more easily than we like to admit.
How to spot a fake
Can a computer do better? To find out the team used machine learning to train a computer program to judge the trustworthiness of reviews by feeding it a sampling of fake and genuine reviews of 20 Chicago area hotels. Unlike the confused undergrads, the algorithm soon got the hang of spotting fakes, blowing away the human competition by correctly flagging fraudulent reviews 90 percent of the time.
One day that algorithm may develop into a hugely useful product for online vendors and the companies that host them (the researchers stress this is preliminary work and that more research is needed on other types of reviews beyond hotels, but the New York Times reports they've already been approached by a host of companies, including TripAdvisor and Google). In the meantime, the research is already providing useful lessons in how to spot fake reviews.
So what are the tells that a "five-star" hotel room might end up being moldy and cramped, or that highly rated toaster might die before you get through a single loaf? According to the Cornell research, you should beware if a review,
Lacks detail. It's hard to describe what you haven't actually experienced, which is why fake reviews often offer general praise rather than digging into specifics. "Truthful hotel reviews, for example, are more likely to use concrete words relating to the hotel, like 'bathroom,' 'check-in' or 'price.' Deceivers write more about things that set the scene, like 'vacation,' 'business trip' or 'my husband.'," explains the research release.
Includes more first person pronouns. If you're anxious about coming across as sincere, apparently you talk about yourself more. That's probably why words like 'I' and 'me' appear more often in fake reviews.
Has more verbs than nouns. Language analysis shows the fakes tend to include more verbs as their writers often substitute pleasant (or alarming) sounding stories for actual insight. Genuine reviews are heavier on nouns.
Of course these subtle tells alone probably won't make you a master of spotting fakes, but combined with other methods of checking a review's trustworthiness, like watching out for various types of verified buyers and suspicious timestamps (Wise Bread has a good rundown of these techniques), you should be able to do a lot better than random chance and maybe even give the Cornell review robot a run for its money.
(Or, if you're booking hotel rooms specifically, you can just ask the robot itself to weigh in. The researchers have made it available to the public in the form of a tool called Review Skeptic.)