We've long known that university students favor male professors and rate them higher than female professors. While you can argue that overall men tend to be better professors than women, there was a strong suggestion that some unconscious bias was at play. (Additionally, I don't think students are actually the best judge of a professor's performance, but that's a topic for a different day.)
A large American university changed their ratings from a 10 point scale to a 6 point scale for reasons unrelated to gender bias. Researchers Lauren Rivera is an Associate Professor of Management & Organizations at Kellogg School of Management and András Tilcsik is an associate professor at the Rotman School of Management thought this was the perfect data set to look at bias.
Here you had 105,034 student ratings of 369 instructors in 235 courses to look at. And they could look at the ratings for the same professor in the same course, so they could compare apples to apples. They found that while there was a significant gender gap on the 10 point scale, it almost completely disappeared when students used the six-point scale. They write, in the Harvard Business Review:
When analyzing the words that students used to describe the instructor's performance, we found that the top score on the 10-point scale evoked images of brilliant, extraordinary performance. We also found that raters tended to associate that kind of performance with John rather than Julie. This result is consistent with the longstanding cultural association of the number 10 with perfection, as well as earlier research showing that evaluators more heavily scrutinize women's performance for errors and reserve labels like "star," "superstar," and "genius" for men. Given the stereotype of male brilliance, the perfection that a 10/10 connotes is an elusive performance bar for women.
Because we have no such perfection connotation with 6/6, professors who were "good" were deemed worthy of the higher score.
To check if, they conducted a test by having students evaluate identical lecture transcripts on either a 6 point or 10 point scale. The only way the lecture notes differed was by the name at the top--John Anderson or Julie Anderson. They found gender differences in the 10 point scale evaluations but not in the 6 point scale.
Take these results into consideration when designing your own performance reviews. It turns out that maybe the scale can influence the results.