It’s no great secret that the landscape of television has changed in our lifetimes. And it’s no secret that the Neilsen group has struggled to keep up with the media we consume. As we get closer and closer to the end of year two of the Pandemic World, the relevance of Nielsen data becomes more and more questionable.
A Brief History of Data Collection
Founded in 1923, the original plan was measuring the results of competitive sales, which allowed people to understand what a ‘market share’ was. For those of us who didn’t study that part of economics, it means if we compare what everyone does, we know who’s best and who’s not. Pretty straight forward, except in the part that collecting that data has always been kind of messy.
I actually was a Nielsen Buy family for a while, we had to scan in our grocery receipts and use a weird hand-held device to basically fax them in every week. After 6 months, we tapped out because it’s simply too much hassle and the device constantly froze and was otherwise annoying.
But what people are more familiar with is the concept of Nielsen Watch, which is what measures TV, Radio, and streaming media. Kind of. See the way you track that is, similar to that grocery story, you get a special device that hooks into your cable box and reports to momma how much of what shows you watched. They also had paper diaries (starting in 1953) which I’m sure everyone fudged on, since that’s just a lot of messy work, and while they introduced the People Meter in the 80s, that diary wasn’t officially retired until 2017.
Semi-related to this is that the Nielsen’s only started tracking demographics of same-sex couples in 2018. Please note that is couples. And as I’ve mentioned before, queers watch the same stuff everyone else does, with different priorities, which changes at scale.
That comes with a big ‘but…’
How We Watch has Changed
I don’t have ‘traditional’ cable. I haven’t had a ‘cable box’ since something like 1995. Even when I had ‘traditional’ cable, I didn’t have a box set, I had a little device that used my TV’s software instead. Today, I do have a digital antennae to pick up local channels, but really I use it for sportsball and the CW (which is not generally allowed live on streamers — and that’s another drama).
I’m not abnormal. Few of my coworkers have cable. I remember the day my dad got it for our 1 bedroom shack and we thought it was the height of luxury. But now most of us use digital systems that allow us to watch whatever we want, whenever we want. I personally use YouTube TV which has an ‘unlimited’ DVR. It doesn’t have a DVR at all, it just gives you access to something it saved on it’s servers (commercials and all), and I can watch it forever if it’s in my favourites’ list.
We also don’t have water cooler TV as much anymore. Back when ER was king, I remember Friday mornings at the office when we’d get coffee or water and someone would ask “Hey, did you see that episode last night?” It was appointment TV! You watched or you were left out. Now, with Twitter and me on the West Coast, I usually know the Big Twist of any episode 2 hours before I can watch it. This means I don’t watch the late night shows live anymore! I watch them the next morning while I’m having coffee and clearing out my email.
Add into this binges. Netflix dumps a series on us, wholesale, and we watch. And all of this has led to serious concerns about data collection.
Data Fails the Sniff Test
Back in May, some news hit the fan. The Media Rating Council, who maintains standards in research for the media industry as a whole, issued a statement regarding the Nielsen analytics, and it wasn’t good.
Based on Nielsen’s analyses of the potential impacts of the changes to its panel procedures, MRC believes that Total Usage of Television (TUT) by Persons 18-49 was understated by approximately 2 to 6 percent for the February 2021 measurement period (the period covered by the analyses). In addition, MRC believes that Persons Using Television (PUT) estimates among the Persons 18-49 group was understated by a range of 1 to 5 percent in February 2021.MRC Statement on Nielsen National Television Viewing Estimates Analysis
The MRC did acknowledge that, due to COVID, Nielsen was no longer able to perform in-person visits (to check on existing members) and recruitment (to vet new members). This resulted in some households that previously would have been removed from the system to hang on. One year into the pandemic, and the critical mass was hit. There is now a significant area of concern that the Nielsen numbers just aren’t up to snuff.
To summarize, Nielsen identified approximately 9,400 of its total of about 40,000 TV panel homes in the U.S. as of February 2021 that had triggered alerts within Nielsen’s systems that, in pre-pandemic times, would have led the home to be withheld from contributing to Nielsen’s estimates until the alerts had been addressed by Nielsen personnel. Of these 9,400, approximately 2,400 of the homes had triggered alerts that were considered “high priority,” in that the condition that led to the alert was deemed to be highly likely related to an issue that would affect Nielsen’s ability to collect complete and accurate viewing data from the home. Another 3,000 of the 9,400 homes were in a medium priority category, while the remaining 4,000 homes were assigned a lower priority. As Nielsen began its return to in-person field processes in March 2021, it used these priority classifications as guides to determine the order in which homes that were known to be in need of maintenance issues were to be addressed.MRC Statement on Nielsen National Television Viewing Estimates Analysis
Nielsen Takes a Hiatus … Not Like That
The subsequent audit and next-steps that stemmed from the MRC caused the Nielsen to issue their own statement in September. And it was a doozy.
I’m writing to inform you that members of the Media Rating Council (MRC) have responded to our request for a hiatus with a vote to suspend accreditation for our National TV Ratings service, as well as our Local TV Ratings service.MRC Update – Nielsen’s letter to clients
Have a second read. Nielsen, after being flagged as having unreliable data and working for just about half a year with the MRC on fixing it, decided to step back and no longer have their data accredited.
In other words, they pulled a Netflix: Just believe our numbers, why would we lie?
This does not mean the data won’t be out there anymore, it means it’s no longer got that seal of “This is good data and we can prove it because of an independent validator!” That scares me a little. I don’t trust numbers when they’re not vetted, and if that makes you think “Mika, how can we trust your numbers when you post stats?” you’re right to question! Our data’s public, but the collection is rife with human error. The point, though, is that the Nielsens have one job, and that would be data.
No More “Traditional” Viewing
Both Nielsen and the MRC have the same problem at the end of the day. How the heck do you measure ‘viewing’ when the old way was thrown out the window, baby, bathtub, bath water and all? Linear TV is, at best, half of how people consume TV content. Media has moved from relying on this stalwarts like Nielsen to their own, home grown measurements.
To think of this another way, when we look at our own websites, we have our own tracking systems (Google Analytics, for example) to measure who visits us, what devices they use, and so on. We don’t have access to anyone else’s unless they give us that data, so we really don’t know ‘how good’ we’re doing compared to other people unless we use a third party. We’re doing pretty well for a super niche site, by the way, especially considering we don’t advertise.
But it’s one thing to claim “The most popular LGBTQIA+ TV database!” and another to claim “The country’s most watched network!” The former is a very small pie slice. The latter is how you make or break pay checks. And now that TV networks are moving into our world of self-declared, isolated, data collection and study, the concept of that Nielsen Unified Structure is crumbling.
That brings us to the announcement that the Nielsens are taking an “Impressions First Initiative.”
The move to impressions will occur in conjunction with the integration of broadband only (BBO) homes into Nielsen’s local measurement metrics in January 2022. The move to an impressions-based currency will deliver a more complete, precise and representative audience measurement, along with the added benefit of enabling cross-platform audience measurement. In today’s fragmented media landscape, the move to impressions lays the groundwork for implementing Nielsen ONE across local, national and digital measurement.Nielsen Announces “Impressions First Initiative” and the Integration of Broadband Only Homes into Local Measurement in January 2022
The idea is pretty cool, actually. Nielsen has finally figured out a way forward. They’re gonna track everyone! Broadband, streaming, you name it! Except … the idea of “Impressions-based” buying and selling of advertising sounds a heck of a lot like internet advertising, doesn’t it?
Ultimately, it remains to be seen how well this is going to work out. If they’re still not well-vetting the households, we’re not going to get a good picture of who’s watching what. And if they don’t stop and expand the concept of ‘households’ more, they’re going to be left out in the cold.
We’re All Losers
The losers in all of this, however, are you and me.
If the gamble by Nielsen fails, then the media moguls will continue their branch into self-managed data. This means no more verifiable and validated metrics. This means no more overnight ratings that are reliable and directly relatable to each other. Everyone will have their own stats, and while that’s good for each network to decide what shows to keep and which to ditch, it means we will no longer be able to fully trust that data.
Consider this real example of Netflix. They don’t share the nitty gritty details of who watches what and when, like other places. Their numbers are shared but lack the context we need to determine actual success. Not even show-runners really know how things are going until Netflix tells them “You’re done” (RIP One Day at a Time), which means a lot of us find that data sus. On top of that, Netflix says you’ve watched a show if you’ve watched at least two minutes of it.
Yeah. I know.
Now imagine if everyone does that.
The landscape’s about to get really ugly, folks.