One of those standards is the demand websites not place Google-served ads on the same page as "inappropriate" content. It's reasonable, as advertisers often insist that their promotions don't appear in such contexts. What isn't reasonable, though, is how often Google will label something as inappropriate when the call seems wildly overreaching. Such popular and long-standing sites as Fark.com, BoingBoing, and Skepchick have found themselves suddenly without ad revenue -- or a clear way to rectify what ultimately turned out to be a mistake on the part of Google.
How Fark got waylaid
Fark made its problem public this week. For a period of about five weeks in the last quarter of 2016, the site had no revenue from Google AdSense, and the money already in the account was confiscated.
Google's "algorithm incorrectly flagged an image put on our site 6 years ago [without warning]," founder and CEO Drew Curtis told me. He describes Fark as what would happen "if the Daily Show ran the Drudge Report." He was told that the site had an image on a 6-year-old post that was "child pornography."
Here's the quick version of events that Curtis posted as part of a longer entry.
The thread in question, which contained the image flagged by Google Policy, was originally posted back in 2010. The thread linked to an ABC.co.au article about a guy who was acquitted of charges of possession of underage material in Puerto Rico. He'd bought a pirated DVD in Venezuela and was busted by U.S. Customs because there was a picture on the cover that looked like she was underage (I'm certain this picture isn't the same as the image Google flagged, by the way). At the trial a child psychologist even testified that there was no way she could be an adult. Someone managed to locate the actress, however, who flew down and testified in person that she had been 19 years old at the time. He was cleared of all charges.
The image in the thread was of the actress, who in addition to being an adult at the time, was fully clothed in the photo.
Why did a fully-clothed picture of an actress posted in a comment on the story set off Google's system? Curtis wasn't explicitly told, but the image also had an icon with the words "pedo bear" and a picture of a cartoon bear. (Check the link to his post, scroll down, and you can see for yourself.) The icon is a widespread internet meme about pedophilia that was often edited into existing pictures, no matter what their nature.
"I contacted everyone I knew at Google," when the ads went down and money in the AdSense account was unavailable, Curtis said. He eventually contacted "some high up guys" in Google "and none of them could figure out what it was." The fourth quarter, when this happened, is a critical revenue time for Fark. It ultimately took five weeks to resolve the issue, except he didn't receive a refund of the missing money. The financial hit was so hard that he's asking regular visitors to consider a paid membership. "We're done with advertising," Curtis said. "We need to get subscribers."
BoingBoing gets the bounce
Fark isn't the only site that has found itself on the wrong side of Google's due diligence. Curtis said that he's heard from many other sites with the same experience, although most don't want to speak publicly for fear of being blacklisted by Google.
One major site that was willing to talk to me about its experience was BoingBoing, which bills itself as the "award-winning zine, blog and directory of wonderful things" and started in 1988. Publisher Jason Weisberger discussed a similar experience that happened between May and June in 2016. "It was a 10 year old news story ... about an animated character in Japan called Pedobear," which was the start of the meme. "There may have been a picture of a dude in a bear outfit." The article was a news piece that mentioned the character in the headline.
As happened with Fark, Weisberger found out by noticing that the ads were cut and, like Fark, he's heard from other publishers with the same experience.
"It was completely done by an automated system," Weisberger said. "When we talked to humans [at Google], they said, 'Oh yeah, it shouldn't be happening, but it will fix itself.' We kept hearing the automated system caught it and would fix it, don't worry. Every day would go by and [nothing was fixed]."
"We've relied on Google as a source of revenue for 8 to 10 years, and this was the first time something went terribly astray" Weisberger said. "Suddenly something that is double digit percentages of your income disappears. Google's largest source of revenue was being run by robots. A human looking at it would immediately understand." To add to the injury, when Google dropped ad placement, the site suddenly had much more inventory. Automated ad bidding and placement systems used by advertisers noticed the change and so suddenly started offering far less for their ads.
A third side is Skepchick, a collection of blogs about science and critical thinking with a feminist twist. Founder and publisher Rebecca Watson has received regular challenges to her content since 2011, when her AdSense account was closed for suspected fraudulent traffic that was actually a post three major sites linked to. She had 5 warnings in 2015 and 11 last year.
"They're [almost] always completely ridiculous," Watson said. "I think the most recent one was 'sexually explicit material.'" The link she was provided through AdSense, which she's learned to check frequently, was for a tag called "spanking."
"I have dozens of writers on the site," Watson said. "Sometimes they're a little snarky and a little salty. I clicked the link and literally every single post was about the science of spanking your children [and intended for parents]. Not a single post was anything sexual." However, the notifications don't include an option for inaccurately tagged material. "The only options are things like, 'I've edited the content to remove any sexually explicit stuff' or 'I deleted the content' or 'I deleted the ads.'"
Watson says that some of the notifications are reasonable, because the site will deal with women's health issues or sexually explicit subject matter. In those cases, she'll edit the post in question or link out to someone else's. But every notification brings worry.
"If I can't get through to someone at AdSense [about an alleged problem], I'm screwed for any money that month," Watson said. "Even with AdSense I barely keep up with server costs." Any income comes from other projects and reader donations.
What does Google say?
I contacted Google and spoke with a representative who said very little. The company will not discuss specific situations with publishers, although the spokesperson did say, "We have a combination of human reviews and automated technologies that help us to scale." She also claimed to have "outreach programs" and ways for publishers to get in touch, such as asking questions on social media or using an online partnership group.
"Many of the publishers we work with don't understand how [the system] works," the spokesperson said. To be fair, according to the highly experienced and long-established publishers I spoke with, many of the people they know at Google don't seem to understand how the system works either.
I could have received more details, but only "off the record," which meant I couldn't even mention them in this article, so I declined the secret information.
[UPDATE (12-Jan-2017 7:45PM): The spokesperson emailed later to stress that all flagged accounts are reviewed by humans and that "many violations for inappropriate content are flagged by users and advertisers." She provided a link to the written policy on "illegal content image violation," but I lacked the permissions to see the link. However another link to AdSense policies yielded the following:
AdSense is a family-safe network. Our policy regarding adult or mature content may include any material that is not appropriate for all audiences. While this obviously includes full nudity or sexual activity, it may also include textually explicit sexual content, image or video content containing lewd or provocative poses, strategically covered nudity, see-through or sheer clothing, and close-ups of anatomy that would be inappropriate if shown nude. Additionally, topics such as sexual health and sex tips may be held to a higher standard of professionalism than content that isn't bordering on mature. Even if there's no adult content hosted on your page, links to adult pages or displaying adult ads from a third party will also be considered as adult content. When in doubt about whether an image or text might be construed as adult content, our general guideline is this: if you wouldn't want a child to see the content, or if you would be embarrassed to view the page in front of colleagues, then it's probably not family-safe and you shouldn't place AdSense ad code on it.
So, the pedobear phrase at Fark or BoingBoing could have been seen in theory as enough to trigger a violation. That said, if the actions were reversed, it does call into question whether the level of human review is enough. There's an interesting Harvard Business Review article, which I mention below in the body of the post, by two researchers at Microsoft Research about the human review that takes place with AI systems. The article raises the questions of whether the people who perform the review are necessary have the training, cultural references, or even English skills to perform the work. One reviewer that the authors met in India had to ask her teenage sons whether some word was "bad" in English. She allegedly does temp work for such companies as Microsoft, Facebook, Twitter, and Google to "train the algorithms that will curate online content."]
The coming scary future
Clearly sites that depend on Google for advertising face a problem. Highly automated systems can make decisions that can seem arbitrary, and getting human review may be harder than vendors claim. Another complication, according to senior researchers at Microsoft Research Mary Gray and Siddharth Suri in a Harvard Business Review article, is that often, human review when "AI falls short" happens in other countries with other cultures and may be undertaken by people whose English skills are poor.
Just because a business doesn't use Google AdSense doesn't mean it is free from the bigger trend and issues. Companies are increasingly pushing decisions onto AI-powered systems. Bridgewater Associates, the world's largest hedge fund, is looking to "automate decision making to save time and eliminate human emotional volatility."
[UPDATE (12-Jan-2017 7:53PM: Another example is a Japanese insurance firm, Fukoku Mutual Life Insurance, which is replacing 34 employees who used to calculate insurance payouts with an AI system.]
What happens when these decisions are used by vendors to make credit decisions, manage product availability for clients, and otherwise handle processes that are critical to smaller customers? Such efficiencies might prove to be great ways for large companies to save money but poor for the entrepreneurs who have to live with sometimes painful and damaging consequences.