Last week, Facebook said it found flaws in the metrics it reported to advertisers—the measurements by which those advertisers judge the success of their ad campaigns on the platform. The company said it overstated the reach of Facebook Pages and Instant Articles, as well as its count of referrals to apps from ads. This admission of miscounting came just a few months after Facebook said it had inflated how much time on average viewers spent watching video ads for two years.
Facebook has promised more transparency. But in media and advertising circles, some critics are starting to ask whether they’ve been spending their money wisely on Facebook. Were they duped into making costly business decisions based on wrong information? Yes, Facebook is still an enormously powerful platform. Close to a fifth of the world’s population checks the social network every day, and all those eyeballs are incredibly valuable to advertisers and publishers. But the revelation that it exaggerated its own reach points to real flaws in the narrative about what Facebook—and indeed, digital advertising on the whole—can accomplish.
Miscounting Metrics
‘Facebook is supposed to be the best of the best. There’s an expectation that you don’t get this thing wrong.’
And yet Facebook did. It overstated app referrals by 6 percent on average by counting not only posts that directed traffic back to the app maker’s website or app but also clicks to view photos or video that kept a user inside of Facebook. Facebook Pages—the pages brands maintain as their home bases on the site—double-counted repeat visitors, leading to greatly exaggerated estimates of the size of their audiences. (The company said page owners should be prepared to see their “28-day reach” fall by 55 percent.) Facebook also said that, due to a math error, it estimated audiences were spending 7 to 8 percent more time reading fast-loading Instant Articles than they really were.
The company doesn’t say exactly how it came to discover these new discrepancies, pointing to “bugs” in the system. But Wieser says it’s likely that Facebook looked into the issue more deeply after finding the mistake in counting that video metric back in October. “Pick your analogy or theory,” Wieser says. “The cockroach theory, where you see one somewhere, then you see more, and say, geez, this must be something systemic. Or the Comey analogy: the problem was brought up then seemed to be settled, but then it comes up again.” Whatever the case, he says, the mistakes definitely amount to a “sackable offense.” (Facebook says it has established what it’s calling a measurement council to respond to advertisers’ concerns.)