Jon Gibs, Nielsen Online
Jon Gibs
Vice President-Media Analytics
Nielsen Online
Specializing in research methodology design and development, Jon Gibs previously managed Nielsen Online’s US-based survey group employing its MegaPanel, BuzzMetrics and consumer-generated media analysis tools. His expertise involves helping clients develop surveys and measurement methodologies to meet their strategic and tactical needs.
Prior to joining Nielsen Online, Mr. Gibs was an analyst and analytics director at JupiterResearch, where he developed client reports based on primary survey analysis and specialized in developing analyses merging behavioral and survey analysis.
In this interview, Mr. Gibs discusses Nielsen’s research panels and methodology and the need for time-based versus impression-based measurement tools.
eMarketer: What are the most effective methods of online brand measurement?
Jon Gibs: I think that there are fairly significant problems with most of the methodologies in the marketplace. What’s happened is that it’s caused, in some cases, branding research to be seen as sort of an add-in on a deal, but not particularly constructive or educational to a marketer.
I like panel-based methodologies, and obviously this will be viewed as self-serving since I work for Nielsen. The response rate for most panels are within the 10% to 20% response range, which is probably not as high as some market researchers might like it, but it’s certainly higher than the 0.01% response rate that you typically get with an online pop-up survey. And the challenge with having that low of a response rate is that the results you get tend to be much more of a remnant of who happens to answer the survey, rather than being a representative sample of people that view the advertisement.
eMarketer: But there are complaints about the panel-based methodology as well.
Mr. Gibs: The first concern is that with any form of panel-based measurement we typically require a much higher impression load to do analysis—somewhere in the 50- to 60-million-impression range. Dynamic Logic [a provider of brand measurement services that does not use panels] can survey a campaign of any size.
The second concern is that a panel, because it’s not a pure census kind of a population, doesn’t fully represent every specific aspect of a population. We believe very strongly in the quality of our panel, and we’re going through an MRC [Media Rating Council] audit process to basically prove it.
We think that while it’s fair to have concerns about panel quality, we believe that we have invested the personnel, financial and technological resources to develop the best measurement structure we can.
eMarketer: Is all audience measurement inherently flawed? Do the existing methods actually reflect that someone is engaged with an online ad?
Mr. Gibs: I think flawed is the wrong word. Not to sound too much like a geeky scientist type, but every type of measurement method, be it online media research, TV media research or the way in which you measure the rice that you put in a measuring cup, has some level of error associated with it.
The problem with the word “flaw” is it suggests that there’s some sort of magic bullet and that all of a sudden you’re going to find a measurement method that has no error rate in it, and that’s not going to happen. I mean, the only way to fundamentally do that is to go out to a pure census kind of audience which, one, isn’t possible because of privacy reasons and two, isn’t financially feasible. That said, sampling is a pretty good method.
There are some areas, particularly when we’re looking at things like unique audience and time-spent metrics, where a panel-based methodology has a fairly low error rate. But the capture might not be as robust on things like page use or actual stream counts, and that’s the reason why Nielsen has moved to a census-based measurement for those things.
eMarketer: How does a marketer pinpoint online advertising as the driver of offline sales?
Mr. Gibs: There are two core methods. One is more of an old-school method where you take a media or market mix model where you place all of your media inputs in a giant pot, stir them and then figure out the degree to which there are sales left in any given market based on your media allocation in that marketplace.
And the online method that we’re using currently using, which I think is a lot more straightforward, is we’re matching online ad exposure to the Nielsen Homescan Panel [a consumer shopping survey panel]. So if a person saw an ad and they buy a product, that means that the ad made them buy the product. We’re taking into account all online ad exposure with the brand.
eMarketer: Which consumer categories can precisely tie online ad exposure back to the sale?
Mr. Gibs: Packaged goods and retail can precisely tie online ad exposure back to the sale. We’re doing a lot of online/offline effectiveness research in those categories.
The other area that we’re looking at from a similar type of effectiveness measure is TV ratings, and understanding how online advertising drives TV ratings. [Editor’s Note: Nielsen in April announced it would begin measuring TV and Internet usage in 375 of the homes in its national TV ratings sample before making a decision this fall to expand it.]
eMarketer: What is the single biggest problem with online brand measurement today?
Mr. Gibs: The single biggest problem is that the current methodologies that are used are insufficient to deal with the complexities of online advertising. And more specifically, they are built around an advertising model itself that values frequency over engagement or time of exposure.
The solution is to use time-based measures rather than impression-based measures. Rather than buying, for example, 100 million impressions on a Website, it would be buying X% of a person’s time. That creates a very different value proposition for publishers and encourages them to create a more engaging environment for the consumer to interact with advertising. We think marketers should plan and buy digital media around time-based metrics.
eMarketer: Looking at the entire consumer purchasing funnel, from awareness to final purchase, where is the measurement model most broken?
Mr. Gibs: I think we’ve all pretty much agreed that the click-through is a bad metric. We’re working on a media allocation modeling, which is focused on assigning weights to each element of a campaign and not all elements are equally weighted. The last click doesn’t get all the credit.
We’re testing this model, which enables marketers to understand the role of things like search, video, Flash, standard display ads and microsites play in an online campaign.
The problem is there are no two campaigns that are like so it’s really hard to create rules. All research companies like to have things that follow rules because it’s easier to scale them. So one of the challenges that we’ve had is to try to find structures within those models that we can use from a plug-and-play standpoint.
eMarketer: With respect to measuring online campaign performance, are there conflicts of interest among publishers, marketers and agencies in terms of who owns the data?
Mr. Gibs: It’s tricky. If it’s data that we are paying for, the collection of with the purpose of selling, then it’s our data. If the client is paying for the data collection, then it’s the client’s data. I run a custom analytics group, so 99% of the work I do is owned by the clients when it’s finished. The intellectual property isn’t necessarily owned by them unless they’re paying for a specific model—just the results.
eMarketer: What are the measurement challenges with respect to online video? How can they be solved?
Mr. Gibs: I think there are two challenges with video—one [is] brand marketers are definitely looking for some form of a GRP metric that can be applied both for online and TV. There are challenges here that go beyond simply multiplying together reach and frequency. There are issues around how you define start time, how you define time at all and dayparting. And there are issues around the size of the audience watching or engaging with video.
Right now, we’re not doing a very good job of measuring what happens inside the video itself.
eMarketer: What do you mean by that?
Mr. Gibs: It means we need to look at the actions consumers are taking when they engage with a video. Are they pausing? Are they rewinding? Are they stopping altogether?
The other issue revolves around video content. We’re increasingly moving to a place where we’re going to watermark video content so we’ll understand what happens in it. And so from a syndicated basis, it’ll be a lot easier to measure content, rather than knowing that a certain video ran on ESPN.com.
From a brand marketer’s point of view, if you have a product integration where your brand is in a TV show and the show is being re-run on Hulu, you need to know if that video is watched.
We’ll be able to look at what’s in the content of the video, where the person goes relative to the video and survey people to understand if they made it to minute 35 of the video versus minute 10. It will enable marketers to make smarter choices about how well the video is working for them.
eMarketer: GRPs—some marketers want an online equivalent to help them measure the effectiveness of their online campaigns. What do you think?
Mr. Gibs: The GRP is more of an advertising volume metric than a brand effectiveness metric. The thing that bothers me about the GRP is it’s an incredibly blunt-force measure.
I mean, if you think about reach and frequency as impressions, we provide impressions to the marketplace now. But we have an impressions measure for online [media]. You can buy a certain number of impressions. Great. But that doesn’t tell you what’s happening.
I hope we’re moving to a place for TV and online advertising where we’ll see more integrated metrics, next-generation metrics. The GRP is a 50-year-old metric that isn’t designed for a world where there are hundreds and hundreds and hundreds of TV networks and millions and millions of Websites. The GRP makes TV buying fairly straightforward and you don’t have multiple sets of numbers in the marketplace.
But on the online side, we’re obviously in a different place where there are multiple vendors and multiple methodologies that are looking at an online currency and trying to develop their own online currency.
eMarketer: Well, should the industry be creating a standardized online brand measurement system?
Mr. Gibs: I feel somewhat mixed about it. I think innovation is good and as a person who likes doing research for a living, I like being pushed. I like having an environment that’s dynamic.
We have to create a currency model that’s [built] on transparency and data quality. Transparency means that the clients need to know exactly what goes into it so they can trust the numbers. The advertisers need to know exactly what goes into it so they trust that these numbers that are being used for the media buying. We’re trying to find the right cocktail before we try to serve it to the industry.
