On Nielsen, the MRC, the VAB, the Panel, and Big Data
I wanted to offer some commentary– call it an op-ed– on the recent MRC announcement regarding Nielsen, and the overall controversy about Nielsen data as they migrate from Panel-only to Panel+Big Data as currency of record.
First, some caveats and disclaimers. (1) From March 2020 through January 2024 I was Chief Measurability Officer at VideoAmp . As a result, I own VideoAmp stock options, which I continue to hope will be worth something. (2) For much of my career, I worked at companies competing with Nielsen, including Arbitron, Comscore and VideoAmp. (3) I have received, and taken to heart, feedback that my public commentary regarding Nielsen might betray some bias. That’s not something I want in the ol’ brand equity, so I’m working on it. (4) Manish Bhatia and I recently co-authored a paper for Coalition for Innovative Media Measurement (CIMM) on the economics of US national video currency (“Funding the Fiesta”). I heartily recommend it, and give it two thumbs up!
OK then.
If we read the trades and the White Papers, we know that there is much agita over the variation between Nielsen’s panel-only, versus panel-plus-big-data, audience estimates. Notably, the Video Advertising Bureau published a report highlighting what they characterized as Nielsen’s “instability” and “volatility”; Nielsen countered that the analysis was flawed.
But here’s the thing.
The primary finding in the VAB report was about how often, and by how much, the P+BD results differed from the panel-only data. That’s not volatility though; in statistics, “volatility” refers to variance over time, or “bounce”; not differences between two data sets. I’m reasonably certain that the introduction of the big data removed volatility, as in, made the data more trendable over time. The beef, then, is really that the introduction of big data introduces change versus the previous methodology.
But… SHOULDN’T IT? Aren’t we wasting a ridiculous amount of time and money to reinvent audience measurement if our collective expectation is that once we’re done, nothing changes?
One of the things I learned at Comscore and VideoAmp is that Big Data Doesn’t Lie. (I capitalized that because I’m trying to make it catch on.) 50 million Elvis fans can’t be wrong; neither can 50 million devices worth of data. If you start with a panel-only approach, and then introduce data from tens of millions of devices, I think we can agree that, if you know what you’re doing, all that big data is going to help.
Remember back in 2021, when Nielsen lost MRC accreditation, and everyone said that this would herald the age of multi-currency? Well, I’m here to tell you, that was never the case. The opportunity was not in the accreditation loss; it was that in revamping their methodology, Nielsen was essentially becoming an “alternative currency” themselves. Ratings users would incur switching costs whether they changed vendors or not, putting the incumbent on more even footing with the challengers than ever before.
And I’m sure Nielsen knew this. So I believe that in incorporating big data, they deliberately did so in a “panel-first” way, using the big data to address zero cells and volatility, while relying on the panel (an expensive, differentiated asset) to have as much bearing as possible on reported data.
Based on what I’ve heard over the last 18 months, I suspect that all the issues Nielsen has faced with migrating from panel-only to P+BD, accrue from the tension between wanting to maintain trends, on the one hand, while wanting to reap the (seismic) benefits of big data on the other.
When I was at Comscore, every year when there was a somewhat discontinuous change in reported data (rolling in new universe estimates, new data partners, and methodology changes), we had to devote a ton of time and resources to helping clients navigate the impact, and it was painful for all. Painful, but necessary.
I grant that hindsight is 20/20. But in hindsight, I think Nielsen would have been better off if they’d bit the bullet and told clients, “P+BD will be different. It SHOULD be different. And this will suck. But it will only suck the one time, and once we’re on the other side, we can all move on.”
I think that incorporation of big data naturally tugged the results away from historical panel trends, and frankly, toward iSpot, VideoAmp, and Comscore results. It pretty much would have had to– remember those 50 million Elvis fans.
Based on what I’m hearing about new Nielsen methodological enhancements– e.g. changing the demographic model, adopting DASH for technology Universe Estimates (remind me some time to tell you how I made the DASH study a TV Universe study)-- I think they’re eventually going to end up in the right place (big data for signal and volumetric estimates; a panel to provide calibration and training on demography). This will make the Nielsen view of video consumption different from the one we grew up with, and more similar to the “alternative currencies.” But that is progress. Sometimes you have to disintermediate yourselves.
https://s3.amazonaws.com/media.mediapost.com/uploads/MRCrelease.pdf