Facebook has been feeling the pain of revenue pressure from investors who expect more from a company worth nearly $60 billion (a drop from the $104 billion market capitalization of its IPO opening day). And it doesn't help that increasing criticism has focused on the idea that Facebook advertising isn't effective when compared to other choices.

But now Facebook is fighting back with a new report from comScore that purports to show the value of marketing on the site. The numbers look good: For example, tracking Facebook fans of Starbucks against a control group of Starbucks customers not exposed to Facebook marketing showed a 38% difference in the level of purchasing. The social network company is using the report to claim that it really can get the job done for marketers, as the Wall Street Journal reported:

In an interview, Brad Smallwood, head of measurement and insight at Facebook, said the results proved "it's a myth that Facebook advertising doesn't work."

But... not so fast. First, Facebook is a comScore client and helped put the report together, so this isn't an objective study. More importantly, you have to dig into the report, because it doesn't actually seem to say what Facebook would like it to say.

For example, the report wasn't about Facebook advertising. It was about so-called earned media: "favorable publicity gained through promotional efforts other than advertising," according to a Wikipedia definition that Facebook quotes.

Earned vs. Paid Impressions

Most of this study focuses on earned media impact on fans, which would mean that which comes from interacting with a brand, not from being exposed to traditional brand advertising. In this case, earned media on Facebook does not include such paid advertising choices as promoted page posts or sponsored stories.

The study's findings mostly depend on using earned media, not paid. When it mentions the amplification ratio, and how companies were able to reach more people than they otherwise might have, the study specifically defined the term as "a metric that divides the number of Friends of Fans reached by the number of Fans reached with earned media, or the number of impressions reaching each of these segments." To put it differently, the earned media contacts with fans can get a boost when they're passed on to fans' friends.

Now it's time to stop again. Earned media means that someone is reading, liking, or sharing something on the brand's Facebook page. What the friends of fans see is that someone... read, liked, or shared something on a brand page. That could mean a notice in the small news feed in the upper right-hand corner of the friend's Facebook page. But how often do you pay real attention to a brand page a friend has liked or commented on? Of course, anecdotes aren't the same as statistical data, and it may be that there is a significant effect, but touting amplification without a tie to other adequate study is not useful.

Trying to Make Ends Meet

There was more data from comScore, trying to bridge the gap. But comparing a typical week in October to the promotional week in November that included Black Friday and Cyber Monday simply isn't realistic. Interest in marketing increases everywhere during the holiday season because people are looking for bargains. And then comScore compared "the online and offline purchase patterns of the exposed Fan and Friend of Fan segments compared to the Total Internet baseline for the full holiday season period of November and December 2011." Lo and behold, fans and friends of fans spend as much or more.

But assuming for a moment that friends of fans are more likely to share some of the same interests and predilections, is it really surprising that both groups, when exposed to a given intensive marketing campaign, might buy as much or more than a general populace, which might not have been exposed to marketing at all, or to a limited degree? And could it be that someone who would become a fan of a brand page might be more likely to use that brand than someone who didn't?

So, Starbucks may have seen a 38% lift in purchasing--from 1.54% of a control group to 2.12% of the test group of fans by the end of four weeks, as the study graph below shows:

The study claims to have matched the control and test groups, which would clearly be necessary. However, something sticks out of this story. Starbucks clearly went to some trouble to get people to be fans of its page. It then ran a Facebook marketing campaign with earned media exposure. After four weeks, only 2.12% of the people had made a purchase. Doesn't that seem low for a product like coffee that people tend to purchase on a regular basis?

Other test studies for Target also showed lifts in purchase behavior, but after a social marketing campaign, not an ad campaign. A nameless retailer used a paid ad campaign for a 16% lift from 1.27% purchasing in the control group, to 1.47% in the test group after four weeks.

It may be that companies can make Facebook marketing pay. But it isn't necessarily as simple as placing ads, and you would have to run your own study to know whether it really works for you. Don't blindly trust data touting someone's product or service when they are the ones delivering the information to you.