Media Quality and Measurement: The Gap Between Seeing, and Knowing | Behind the Numbers

In today’s podcast episode, we discuss what’s broken in how we evaluate media, the problem with reach, and what “quality inventory” actually means. Join Senior Director of Podcasts and host Marcus Johnson, along with Analyst Arielle Feger and David Simon, Chief Revenue Officer of Verve and President of Verve Marketplace. Listen anywhere, or watch on YouTube or Spotify.

Subscribe to the “Behind the Numbers” podcast on Apple Podcasts, Spotify, Pandora, Stitcher, YouTube, Podbean or wherever you listen to podcasts. Follow us on Instagram.

Made possible by

Illuminate. Connect. Activate.​

Verve’s global omnichannel advertising platform redefines what’s possible beyond walled gardens. Verve illuminates, connects, and activates high-fidelity signals that drive outcomes for brands, agencies, and publishers at scale.

Learn more

Episode Transcript:

Marcus Johnson (00:00):

Verve's global omnichannel advertising platform redefines what's possible beyond walled gardens. Verve illuminates, connects, and activates high fidelity signals that drive outcomes for brands, agencies, and publishers at scale. Learn more at Verve.com. That's verve.com.

(00:24):

Hey gang, it's Friday, April 17th. Arielle, Dave, and listeners, welcome to Behind the Numbers, the EMARKETERs podcast made possible by Verve. I'm Marcus, and joining me for the conversation today we have two folks. One of them lives in New York City. She's one of our analysts. We call her Arielle Feger.

Arielle Feger (00:41):

That's me. Happy to be here.

Marcus Johnson (00:43):

Happy to have you. We're also joined by a special guest currently in Florida, the chief revenue officer of Verve and president of Verve Marketplace. Dave Simon joins the show. Welcome.

David Simon (00:55):

Great to be here. Thanks for having me.

Marcus Johnson (00:57):

Yes, of course, sir. We start any time we have a special guest from outside the company with a speed intro. Dave, what do you do in a sentence?

David Simon (01:09):

Yeah, we at Verve on the Marketplace side, we're connecting publishers with advertisers to try and maximize yield for those publishers and make them money so that they can continue to do all the great journalism and content creation that we all love.

Marcus Johnson (01:24):

Very nice. And second question to both of you, Dave first, if you could have one arcade machine in your house, what would it be?

David Simon (01:31):

Oh, Golden Tee. Golden Tee. It's the only one I've ever held a high score on. Golden Tee-

Arielle Feger (01:37):

I don't even know what that is.

David Simon (01:38):

Oh, it's the golf one where you roll the ball. It's like one of the best party games.

Marcus Johnson (01:40):

Oh, you're good at that?

David Simon (01:42):

Yes.

Marcus Johnson (01:42):

My goodness.

David Simon (01:44):

It's the one video game that I'm good at.

Marcus Johnson (01:46):

High score? Wow. That looks the most difficult. Maximum difficulty level with the ball rolling. Arielle?

Arielle Feger (01:54):

I would have to go old school Tetris machine. I love playing Tetris. I've always enjoyed it, and I think that would be super fun.

Marcus Johnson (02:03):

Okay. The answers we were looking for was NBA Jam.

Arielle Feger (02:09):

No, thank you.

Marcus Johnson (02:10):

Yep. And/or Time Crisis. Did you guys ... With the gun?

David Simon (02:14):

Yep.

Marcus Johnson (02:14):

The one with the gun?

David Simon (02:14):

The big old gun. Yep.

Arielle Feger (02:15):

No. Yeah.

Marcus Johnson (02:17):

Okay. Rough start. That's all right.

(02:20):

Anyway, today's real topic. Media quality and measurement, the gap between seeing and knowing. All right, folks. Today's episode is centered around some research that was very recently done by the two folks we have on the episode. And we're going to talk through some of the findings, the really brilliant findings from that piece. Arielle, do you want to set the table with what you and Verve got together and put together?

Arielle Feger (02:52):

Yeah, absolutely. So many people in the industry know that evaluating media quality can be quite difficult, especially because there's a lot of ways to think about it, and there's a lot of technical barriers and challenges. So we really wanted to get to the bottom of how marketers, how ad buyers are evaluating inventory, how confident they are in their measurement frameworks, and where the gaps remain. So we did a survey of 119 US-based media buyers in March 2026, and the results were really, really interesting. And I can't wait to dig into that a little bit more.

Marcus Johnson (03:35):

Yeah. Dave, anything to add on putting together this research?

David Simon (03:39):

Yeah, just to be clear, I mean, I've been using EMARKETER and I think every deck I've presented for the last 15 years has had at least one slide from EMARKETER. So the chance to do this kind of research was really exciting for us. And I think in particular, when we look at the market, we see that there's just such a variety of definitions of quality. And so to get some primary research from a firm that gets really good response rates and really good data was an exciting thing. So we jumped at the chance to do this with you guys.

Marcus Johnson (04:11):

Yeah. Let's start there then. You mentioned quality. What does quality mean to folks? And why does it not mean one thing or the same thing?

David Simon (04:25):

Sure. I have teenage daughters, and if you have kids, they all know six-seven. I think quality is like our industry's six-seven. It means nothing. It genuinely is so subjective that I think doing the research was interesting because there were directly contradictory data points in the research that came back.

(04:51):

Some people say quality is super important and a lot of people say, [inaudible 00:04:55] people said, "I can't buy quality." And it's like, "Well, if it's so important and you know what it looks like, then why aren't you buying it?" So I think the thing that I'm taking away from the research is that quality is in the eye of the beholder and that really what the challenge is, the hard part is that if you're a brand, you have to define what that means for you.

(05:15):

And you can look at, take Chick-fil-A versus Taco Bell, very different corporate ethos between the two companies. What's quality for one is going to be very different from the other, even though they're looking for the same consumer or similar consumers at that same moment in time. And I think that's the hard part, is that you have to really own this as a brand.

(05:35):

On the other hand, the thing that we remind customers all the time is that this is a bidded marketplace. It's a competition. And so if you're able to identify what is quality to you that may not necessarily match to the industry's standard of quality, it gives you an advantage to secure inventory and find consumers at prices that are not as competitive. And so I think there's this really interesting takeaway that we have from the research that's you really have to understand what matters to your brand with your consumer, with that mindset and the contextual moment that they're in. And I think the research backed that up, which was really interesting.

Arielle Feger (06:10):

Yeah, absolutely.

Marcus Johnson (06:11):

Please, Ari-

Arielle Feger (06:12):

Go ahead, Marcus.

Marcus Johnson (06:12):

No, no, no, no, please.

Arielle Feger (06:14):

I was going to say, yeah, the data was really interesting because there were definitely a few things that people had consensus around, reach being kind of one major definition of quality and brand safe environments also being a big definition of quality. But there was also everything from high dwell time, high quality audience signals, exclusive or major platforms.

(06:46):

So I think it all goes back to what Dave said, is really narrowing down what is it that your brand is looking for? What are the KPIs, the objectives you are trying to achieve, and then going from there and saying, "Okay, this is what we want. This is the type of media we're looking for. These are the partnerships we need to have to have that in place." So I think it really does speak to how complex it currently is and the need for maybe a more individual definition of quality rather than across the industry.

Marcus Johnson (07:22):

So Dave, does it need to be both then? Does it need to be there are standards of what quality means, but also quality at the same time, a part of it is going to be unique to the individual, to the company, to the entity?

David Simon (07:38):

Absolutely. A couple of weeks ago, we had this whole TSA line issue, and I think the airlines were doing a lot of anti-keyword targeting to avoid news stories around delays and that kind of thing. Well, what if you're at an airport that doesn't suffer from that problem? What if you're in a geo where that's not an issue? That's a unique buying opportunity. You're able to find users for whom this topic is contextually relevant.

(08:03):

And if you take this blanket sweeping approach that airlines have taken, like avoid the word delays, avoid the word plane crash, there are moments where that stuff actually can present an opportunity. And if you can figure out how to bring your brand to bear in that moment and show up in an interesting way with creative and messaging that's appropriate, you can massively differentiate. And so that's an example of where having your own very specific criteria is important.

(08:30):

Now at the same time, there's a baseline. We want to see real people. We want to see a page without 28 ads on it. And there's some sort of basic stuff we can clear the bar on, but I think what happens is that we're taking a bulldozer trying to thread a needle here. And I think that what winds up happening is that a lot of valuable inventory is presented in the programmatic space as having zero value, not lesser, zero value. And so there's this massive buying opportunity that brands could tap into when they get comfortable with exactly what their brand should represent and how it shows up in the right environment.

Arielle Feger (09:05):

There's also, interestingly, kind of the inverse that we found in the studies. Marketers, buyers are finding that things labeled premium aren't delivering what they promise. And so I think that's a really interesting juxtaposition, where there may be more environments that aren't necessarily considered premium, but will deliver on what a brand needs versus something that would be considered premium that maybe isn't delivering. So I think there's, again, a lot of wiggle room there to think about what are the things that I actually need and what am I trying to do?

Marcus Johnson (09:53):

Yeah. A shocking number, 43% of marketers said they bought inventory labeled as premium or brand safe that didn't meet expectations. But Dave, what's going wrong here?

David Simon (10:05):

Yeah. It's an interesting thing. Look, I remember early on in the days of AdTech, it was easy to accidentally serve brand ads on porn sites before all these tools were available. And I think by and large, we've cleaned up that problem. Fraud is an always ongoing battle. You've got to work with the right vendors to make sure they're helping you tackle that problem because it's constant innovation. I mean, if you think the AdTech innovation curve is high, you should see some of the tactics the fraud players are deploying. And fortunately, we partner with a bunch of companies that help us tackle that.

(10:40):

But when you clear that baseline, I think what's really interesting is the industry needs to start thinking about quality in the context of, did I get a profitable outcome from a consumer that fit my criteria? And I think what ends up happening now is that we target, how many times have we all been watching a streaming show with ads where you see the same ad five or six times? And it's because not enough brands are competing-

Marcus Johnson (11:07):

Five or six. I wish.

David Simon (11:11):

I know. It is quite redundant. But this is my point, right? You've got somebody defining somewhere this metric of quality. And I think what's important is that the brands take the onus on themselves to define what that really means, and especially through the lens of where am I generating revenue or ROAS positive outcomes that equate to success? And I think what's interesting is that through my career, I've seen dozens of examples. I'll share my favorite one because this was years ago.

Marcus Johnson (11:38):

Please.

David Simon (11:39):

We were running a big campaign for World of Warcraft. And let's be honest, World of Warcraft, the paid service, it was a fairly nerdy game. It was mostly played on computers. It was a subscription. It was not like a real action game. It was sort of a nerdy game.

(11:54):

We found, we had a bunch of eBay data at the time, and we found that there was a huge overlap between the number of people who signed up and paid for the game and people who had shopped for used motorcycle parts in the prior 90 days. So what we did is we went out and bought 20 cent CPM motorcycle forum ads and it performed phenomenally. Well, if I were to look at that forum, it's the long tail of the internet, it's a forum based format. It's not right for a lot of brands. You don't know what kind of content's on there. But for World of Warcraft, it wound up being an incredibly high performing asset. Those are the kinds of insights that we're going to see more and more and more as we use machine learning driven decisioning to decide where we're going to serve ads.

(12:35):

And so I think the key thing is that brands have to figure out how to both open their aperture while establishing guardrails rather than relying on the standards that we've all defined in the industry, but might not really match the brand. And so I think a lot of the answer behind the 43% of people not buying quality stuff is that they weren't doing the diligence to figure out what quality meant as defined by their buy-side platform [inaudible 00:12:59].

Marcus Johnson (12:59):

Yeah. I want to circle back to something you said, Arielle, which was the word reach, because it seems as though if you ask what does quality mean, reach seems to be one of the first words that people reach for. See what I did? We'll cut that out. We'll probably keep it in. Why is reach such an important signal for folks, even though it doesn't always necessarily mean that it's going to be effective?

Arielle Feger (13:30):

I'll take it and then I'm sure Dave has some thoughts too, but I mean, I think the bottom line is reach is, it's the number of people. The truth is that brands need to continually reach new eyeballs, need to continually engage new consumers. And I think that's just probably the easiest way to know that you are reaching a certain number of people.

(13:59):

Now, of course, reach doesn't necessarily mean you're reaching the right people and you've got to account for that as well. It's like, is it quality? Is it quantity? And that makes it very, very difficult. But I think reach, it's a legacy way of looking at things for sure, but I think it's kind of the lowest common denominator type thing that ad buyers can reach for to say, "Okay, this is working."

Marcus Johnson (14:25):

Yeah. I mean, David, it doesn't mean you're reaching the right people. Well, it doesn't mean you're reaching the most engaged people. It does seem like there's an element of diminishing returns here. More doesn't always mean better.

David Simon (14:40):

Yeah. So here's what's interesting. I was at a conference, I don't know, a couple of years ago and the CMO of General Motors was talking about the core KPIs that she has tracked. And the number one KPI was defined as unique reach. And I was like, "That doesn't make much sense. How are you connecting that to cars? I don't understand that." So after I asked her, I was like, "Help me understand this."

(15:02):

And she explained something that stuck with me. She said that, "Look, at best, we're going to sell you a car every three years. So what do we do between the day you bought the car and the next three years when your lease is up?" And what's interesting is that the KPI wasn't, did I walk on the lot, did I configure a car on the site? Did I locate the dealer? Did I request the brochure? Did I explore financing? Those are things that happen in that 23-day period when I'm in my consideration set. But if she doesn't reach me as a consumer for the two years and 11 months between, then she's not in my consideration set.

(15:39):

And so what's interesting is that they want to make sure they're reaching the unique reach. Now, what's also interesting is that they're segmenting that into very specific audiences. So I forget the exact term for it, and I feel embarrassed not to know this, but she nailed me to a T. She's like, "It's essentially like guys who drive trucks, but don't work construction and don't camp." And I was like, "Oh, that's me." So I forget the term they give it. And she had me pegged. And so her reach is not just like, I want to reach everybody, but I want to reach this audience to make sure that that truck is in the consideration set so when he hits that 30-day consideration period, it's there.

(16:14):

And so what's interesting is that that's for General Motors, that's how it works. If you're trying to sell subscriptions to Headspace or to Apple Music, it's not the same thing. It's a very different problem you're solving because you are looking very bottom of funnel. And then when you exhaust the available customer base of who was in market for these products, then you start moving up funnel. How many D2C brands have we seen that have just plastered Instagram with ads and then the next year they run a Super Bowl ad? That's not like a vanity exercise. That is a very scientific approach to expanding the size of the footprint. And so unique reach is important because you have to expand your customer base at the same time that you're capturing the people that fall through the bottom.

Arielle Feger (17:00):

Yeah. And I thought it was interesting in the survey we asked about, we really kind of honed in on CTV because it's just such an interesting but really fragmented channel. And one of the, the top way that buyers evaluate CTV inventory is audience data quality. So it's not just, again, that's showing us, it's not just the number of people. It's really, are these the type of people that we want to reach? So I think that's a really interesting insight of we are seeing a little bit more of a focusing in on the right type of audience.

Marcus Johnson (17:43):

There was a quote in the research that I wanted to pull out. It says, "Without consistent transparency, it's difficult to assess quality before, during, and after a campaign." I thought it was really interesting in the research what you had to say about transparency, but Dave, what to you would be one or two of the main takeaways on how transparency can help out here?

David Simon (18:03):

Yeah. So look, this is an interesting dynamic and I don't mean to be necessarily ... I guess I do mean to be a little controversial here, but here's my perspective on the transparency piece. If I sell you a chicken sandwich for eight bucks, do you ask me how much the chicken cost? You don't ask.

Arielle Feger (18:25):

No.

David Simon (18:26):

You get the chicken sandwich and you're willing to pay a price for it, right? Eight dollars is a good price for a chicken sandwich.

(18:31):

If I'm selling you a revenue positive outcome, how much do you care about the transparency? Assuming it's a real user who put their credit card in and made a purchase, do you really care at an intimate level like how exactly I made the decision to do that? And look, transparency in terms of where the user was found or what we might know about the ... Sure, fine. Okay. That's helpful for a whole bunch of reasons behind the ad buy, like what color should the packaging be? What geos should you focus on? That's all informative stuff.

(19:03):

But at the end of the day, transparency often becomes a code for, "I want to understand exactly how much money you're making because there's a finite amount of dollars and I want to make sure I'm getting my share." Well, at the end of the day, if I'm selling you something that is revenue positive, do you really care how much I'm making off of that? You do, but only in the [inaudible 00:19:24] based world where P&G looks at buying ads the same way they look at buying wheat futures to secure their ability to make bread. But at the end of the day, the way we talk a lot about to our customers about enforcing transparency is by generating digital data points through your conversion process that give you the indicator that something good is happening in exchange for the money that you're spending.

(19:51):

And so insurance is my favorite example. It's basically five pages from landing page to quote completion page. They know every single step of the way and they have really good data about what happens when somebody makes it from the third step to the fourth step and from the fourth step to the fifth step and how long that process takes relative to the LTV of that customer. If I'm getting you quotes that convert to profitable policies that get written well, do you really care exactly where I ran that stuff, assuming it meets your bar for quality, which you should have defined before the campaign?

(20:23):

And so I think the transparency thing is important, don't get me wrong. We share a ton of stuff with our advertisers. But complaining about transparency is also a bit of a shadow game about complaining about how much money you're making versus me, and it's a proxy for not having done the hard work to understand exactly what you're buying through these systems.

Arielle Feger (20:45):

Yeah. What a really interesting take. And I definitely have to agree with you on a lot of those points because I mean, I'm not an ad buyer obviously, but when I think about what I would want and what I would expect, I think about transparency in the way that I think about partnership. You want whoever you're working with to be on the same page as you. We are both working towards a goal that is going to benefit us both. And I think, like you said, breaking down every little cost, every little fee doesn't necessarily matter if I know and I trust that you're going to get me to the outcome that I want. And so I think that's how I think about transparency, is just being open with partners and being able to really level set and be on the same page.

(21:43):

So I completely agree that the nuts and bolts may not matter as much if you are getting to that goal. But if you're not, if you are struggling with a partner who maybe isn't helping you deliver on your outcomes, you may want to dig a little bit deeper and say, "Okay, well, where's my money going?"

David Simon (22:03):

Yeah. Look, what's interesting is I've spent time on both the traditional DSP side where you deal with the big Fortune 1000 brands and I spent a lot of time on the performance DSP side where you deal primarily directly with advertisers, a lot of app install, a lot of gaming, that kind of stuff. I'm never asked about transparency on the gaming performance site ever. In fact, what you'll see is that most games ... Take Instacart, great advertiser. They have a six-step process from the install to the second purchase you've made to define success. And as long as the second purchase is made with a spend of media being below a certain dollar value, you're profitable for them. They pass those signals to their buying partners, every single one of those hops.

(22:52):

Compare that to working with some of the more traditional big brands. Some big brands won't even share sales events, like sales data, with their agency because of legal requirements or whatever. That's fine. I get it. Everyone's got their own parameters. I understand that Amazon has a certain level of risk about opening up their data and everything else. But the key thing to understand is that the more that media is bought through machine learning systems, the harder it's going to be for us to define exactly why an ad is bought.

(23:21):

If you look at some of the most sophisticated models that predict the likelihood of an outcome and are in charge of buying media profitably, companies like Liftoff, [inaudible 00:23:30], Performance Max with Google, Meta, they have models that are millions of features long. To identify why that model made an exact decision is virtually impossible,

(23:42):

But it doesn't matter because you got the decision that you wanted and the outcome was generated. And so I think that if there was a call to brands here, the first thing I would say is you've got to find ways to share more data with your agency, and then that gives the buy-side platforms, the sell-side platforms, the publishers, the incentive to meet you in the middle and bring more data together to make it more transparent. But if someone's not going to give us anything past the quote completed page, what am I supposed to do in understanding how much value I'm actually creating and what's the incentive for me to try and provide more transparency? It's a one-way street. I think the brands need to figure that piece out as we head more and more into this machine learning driven world.

Marcus Johnson (24:18):

Well, if you guys want access to the full survey on media quality and measurement, keep an eye out for it. It will be coming sometime early May. We're going to be promoting it in our newsletter, and also check out Verve's social media for more on it there as well. And that's what we have time for for this episode. Thank you so, so much to my guests for today. Thank you first to Arielle.

Arielle Feger (24:39):

Thank you. I probably could have talked for at least another hour about this.

Marcus Johnson (24:43):

I know. I had to cut you guys off. Well, I could have, too. And thank you so much to Dave as well.

David Simon (24:49):

Yeah, thank you, guys. It was great.

Marcus Johnson (24:50):

Yeah, we left about a thousand questions at the door, but we got through some really great stuff, so thank you both for that. Thank you, of course, to the production crew. We've got Danny and John in the background helping us out. Thanks to everyone for listening to Behind the Numbers EMARKETER podcast, made possible by Verve. Watch upcoming episodes of our video podcast on YouTube, Spotify, and coming this spring on Apple Podcasts. We'll be back on Monday, of course. Until then, happiest of weekends.



 

Get more articles - create your free account today!