INNOVATE

OKCupid's Experiment Is Lazy, Unethical-and a Creative Failure

By letting their engineers experiment on users, OKCupid and Facebook showed an astounding lack of imagination
Advertisement

First, Facebook manipulated thousands of news feeds to see if being exposed to more positive or more negative posts would make people happier or more upset.

Then OKCupid told couples they were good matches, when its own algorithm had determined just the opposite.

Both of these experiments show that both Facebook and OKCupid have a dismayingly cavalier attitude toward the well-being of their users. But they also show an astounding lack of creativity on the part of both companies. Was there really no other way for OKCupid or Facebook to get the information they were after? Or were they simply too lazy or impatient to go after it in a more ethical way?

Both OK Cupid and Facebook, because of their scale, have an almost unprecedented opportunity to conduct natural experiments of the type that most social scientists can only dream of. In a natural experiment, the "experiment" is conducted by nature, politics, or some event outside of the researchers' control. Economists, after all, can't engineer a banking crisis followed by a nationwide austerity program just because they're curious about how these events affect an economy. But you can bet they're watching Greece pretty darn closely.

Similarly, the research conducted by OKCupid and Facebook didn't necessarily have to manipulate or lie to users. Surely, OKCupid's customers have dated in defiance of its algorithm before. Why didn't OKCupid, rather than misleading people, simply follow the fortunes of those who dated with a low predicted probability of success? In some of those cases, the algorithm would be right, and in others it would be wrong. With enough interactions between its users, it seems that OKCupid could have put together a data set similar to the one it got by subterfuge. 

Likewise, it's hard to see why Facebook had to manipulate its users' news feeds to figure out the effect of emotionally-charged messages. With enough users and computing power--both of which Facebook has in spades--the researchers could have isolated news feeds in which the emotional tone had changed over time. Then they could have analyzed the emotional content of those users’ posts before and after their news feeds got unexpectedly cheery or somber. That would have yielded the same unimpressive data as the more intrusive experiment, without angering Facebook users or setting off accusations of mind control.

 

Neither Facebook nor OKCupid immediately replied to a request for comment about why they didn't try to use a natural experiment instead of a more manipulative one.

This is the kind of thing that researchers do all the time. They don't have to pretend that A/B testing and lying to users are one and the same, as OKCupid's co-founder has. They don't have to get embroiled in arguments over what constitutes informed consent, and whether the editors of academic journals were told how the underlying data in a study had been vetted, as happened in the case of Facebook. They just have to get a bit creative, and ask the right questions.

If they don't think they can manage that, here's a hint: Social science grad students are a lot cheaper to hire than engineers.

IMAGE: AP
Last updated: Jul 30, 2014

KIMBERLY WEISUL | Staff Writer | Inc.com Editor-at-Large

Kimberly Weisul is editor-at-large at Inc. and co-founder of One Thing New, the digital media startup that is rebooting women's content. She was previously a senior editor at BusinessWeek.




Register on Inc.com today to get full access to:
All articles  |  Magazine archives | Livestream events | Comments
EMAIL
PASSWORD
EMAIL
FIRST NAME
LAST NAME
EMAIL
PASSWORD

Or sign up using: