If you take Uber rides or hire helpers through TaskRabbit, you can thank the use of a five-star scale of user-generated ratings for the high quality of service. If you are an Uber driver or you task for TaskRabbit, you can thank those stars for putting your livelihood in a precarious position.
While they're one of the key innovations that allow on-demand app companies to function as well as they do, user-generated ratings have plenty of shortcomings. A recent deep-dive story by the Verge argues that the evaluations startups use to evaluate contracted worker often don't correlate with actual job performance. Yet ratings have become a sort of currency. "We're working for ratings," one Uber driver told the Verge, "but ratings have no value. Ratings serve only to prevent you from getting fired. Only bad things can happen to you. We're scurrying like rats after these things with no value."
With an increasing share of the economy converting to freelance work, more people are going to find themselves on the receiving end of customers' evaluations without the buffer of a manager to separate signs of legitimate problems in worker behavior from poor ratings based on bias or a rater having a crappy day.
Here are three ways reviews routinely backfire or go amiss.
Bad reviews can reflect bias.
Racism and other biases matter when it comes to how on-demand workers are treated and rated. One study showed that black Airbnb hosts were paid less than white hosts for similar listings. Another found that black taxi drivers received lower tips than white ones. Companies would have to release ratings data for someone to determine whether the trend carries over to the five-stars on the screen of your smart phone, but you see where this is going.
Bad reviews can be the result of following the rules.
Most states ban open containers of alcohol in moving vehicles. Some Uber and Lyft riders don't care. If someone climbs into a driver's car with a red Solo cup, the driver may be inclined to kick that person out. One Uber driver said she thinks that when she was temporarily deactivated, it stemmed from low ratings for refusing to let riders consume alcohol in her car. Drunk passengers create so many problems for drivers -- demanding that they be allowed to smoke or cram in extra passengers -- that some drivers told the Verge they avoid doing pickups in nightlife districts -- even though Uber has been touting its ability to be everyone's designated driver.
The review system goes two ways - but it isn't balanced.
For many on-demand companies, workers rate customers as well, but that doesn't mean they can always choose when to avoid customers with low ratings. Uber drivers can get deactivated for skipping too many customers with lower ratings.
Moreover, the ratings system sometimes incentivizes drivers to provide worse service than they would otherwise. Drivers know they're likely to get lower ratings if they're slow in arriving, for instance, so they will sometimes cancel rides preemptively -- even though it means that passenger will have to wait even longer for the next car.
On-demand companies themselves face similar pressures. Home-cleaning startup Handy reportedly hired someone from TaskRabbit to work on matching prolific Yelp reviewers with the startup's highest rated workers so that the company could get its own ratings game on track. What goes around, comes around.
"Customers have always had power, there's always been feedback, and workers could always be disciplined because a customer complained about you," Harvard Law professor Benjamin Sachs pointed out to the Verge. "What's new is the ease with which customers can give reviews and the almost complete substitution of customer reviews for management."