Drill Down: What Are the Problems?

What are the fundamental problems with online brand measurement today? Ask a few dozen experts—from agency executives and marketers to representatives from the various trade associations and research firms—and you will likely get a few dozen answers.

eMarketer posed the question directly to experts in the field of online measurement, and the responses were, predictably, all over the map. And that fact alone—that the players can’t even agree on where to begin—is the overarching problem. Fortunately, though, a closer analysis reveals several common themes that indicate the problems are so highly interrelated that solving one can solve others at the same time.

Q&A: What is the single biggest problem with online brand measurement today?

“In our marketing accountability survey, only about 33% of marketers were satisfied with their marketing accountability and measurement programs. We don’t have enough standardization or enough useful marketing media mix intelligence and understanding. Studies and research offer guidance, but marketers complain that by the time they get the answers back, the media world has shifted and the mix or model is no longer relevant.” Full Interview

“I think it’s really hard to create a panel of size and scale and a technique to do surveys that aren’t interruptive. You can do custom one-offs with one of the research companies that don’t scale or are very focused on individual campaigns, but they don’t tell the whole story.” Full Interview

“There are two big problems: I’m hearing from our publisher members that there is a kind of panel fatigue among people who are asked to take online surveys that are quantifying brand impact. Panel recruitment and response rates are issues. The second is an assumption that the Internet is not good for branding—there is a continued lack of awareness on the part of marketers and agencies.” Full Interview

“Marketers monitor the front-end and the back-end so they see clickstreams and commerce. The difficult part is the qualitative part in between, which is the level of engagement. The challenge is to begin to build technological capabilities that allow them to see the complete digital footprint that a consumer leaves when they engage with the brand and then be able to address that consumer in a relevant way—behaviorally, contextually or both.” Full Interview

“I think it’s not having those foundational reach, frequency and GRP metrics. You will never see P&G and Unilever spend more than single digits (millions) unless we give them reach, frequency and GRPs. Their entire business model is based on media mix models where those are the inputs and the outputs.” Full Interview

A General Apathy Toward Branding

In this harsh economic environment, does branding even matter to marketers?

“Branding is the hardest thing to justify, and everybody looks to that budget first,” as Bob Thacker, the senior vice president for marketing at OfficeMax, told Advertising Age in May 2009.

Any progress or advancements made in the field of online brand measurement will be heavily dependent on the overall demand for brand-oriented advertising.

Due to the economy and resultant short-term sales pressures today, it appears branding will be taking a bit of a backseat to other business priorities, at least according to most of the studies and surveys eMarketer evaluated for this report.

“However strongly you believe in your brand, you have to do your bit to reduce spending at tough times like these.”
—Marty Ordman, vice president, marketing and communications, Dole Food Co., speaking at the ANA Brand Conference, as cited in Advertising Age, May 13, 2009

In the one clear exception to the rule, the ANA released results from a “Brand Building in Tough Times and Beyond” study at its May 2009 conference of the same name. In close alignment with the theme of the conference, 74% of senior marketers responded that “brand equity” is very important to their company’s success.

Yet a wealth of other survey data suggests otherwise—namely, that brand-building will remain a low priority during the recession. The ANA survey provides the first case in point. In its sampling of 129 marketers, two-thirds admitted that the recession had shifted the focus of their companies to short-term results, as opposed to the longer-term results associated with branding.

According to a recent MarketingProfs survey conducted among 670 marketers in October 2008, while more than one-quarter of respondents identified customer acquisition (29.9%) and customer retention (26.6%) as top goals, only 15.4% cited “creating awareness for long-term brand-building” as a top priority.

Similar results were found by Heidrick & Struggles in a December 2008 survey of US senior executives. Branding took sixth place among their top priorities over the next 12 to 18 months.

Another survey, from Datran Media, shows that the marketers’ current focus on driving customer acquisition and retention at the expense of brand-building goals (such as increasing brand awareness or favorability) is not just a US trend. The survey, conducted among 3,000 marketers worldwide, found that while increased customer acquisition and retention were rated most important by 63.2% and 43.7% of respondents, respectively, only 14.1% rated branding measures as highly.

Similarly, in the context of interactive campaigns, Forbes conducted a survey of 112 senior marketers primarily in the US in early 2009 and found that direct response measurements were significantly more popular than branding metrics as a gauge for success. While 70% of all respondents identified conversions as the leading measure of success and 49% mentioned click-through rates, only 25% cited brand-building metrics.

The low ranking for branding is partly explained by a survey from the ANA. In its October 2008 study, just when the financial crisis began in earnest, marketers were asked to cite methods they were using for measuring brand growth. Fully 70% said the answer was “sales and net income,” which is about as blunt and bottom-line a measure as you can get.

Again, even though marketers appear to be temporarily retreating from branding initiatives, including deploying and measuring online brand campaigns, it does not mean that is the right thing to do. Mr. Liodice, the CEO of the ANA, summed it up best: “It has been demonstrated empirically, time and again, that stronger, higher-valued brands lead to stronger, better business results.”

A Preoccupation with Search, at the Expense of Branding

In the InsightExpress poll conducted for this report, 83.7% of marketing professionals from a variety of disciplines and companies agreed or strongly agreed with the following statement:

“Online search—because it’s so easily measured and is often the last click before a purchase—is getting too much credit. We therefore undervalue the branding effects of online advertising formats such as banners, interactive rich media and video.”

Only 16.2% of respondents were neutral, and not a single one disagreed with the statement.

Too many marketers are not even measuring their online branding efforts. According to a May 2008 PROMO magazine survey of 148 US marketers (who subscribe to the magazine, which skews toward direct response marketers), only 41.5% said they measured metrics for online brand awareness. A much higher percentage (58.5%) used the far less meaningful click-through metric.

A January 2008 survey by Sapient found similar results, with only 48% of marketers saying they measured online branding campaigns; in contrast, 71% measured search and 82% measured Website analytics.

Not surprisingly, brand marketers are frustrated. Having found themselves in hot pursuit of direct response metrics, they are not getting the answers to the most fundamental questions they have about their online branding efforts.

Wenda Harris Millard, president of consultancy Media Link and immediate past chair of the IAB, explained it this way in an interview with eMarketer: “Brand marketers really want to know who they’re reaching. We keep talking about impressions. They [brand marketers] want classic demographics. Impressions are not a substitute for knowing who you will reach. They also want to know how long someone looked at an ad and what happened. Did it leave a favorable impression of the brand? Will someone try this product? And did somebody take an action at any point? They want to create awareness and then know whether they changed a consumer’s perception or induced trial.”

Ms. Millard, who previously held senior sales and media positions at Yahoo! and Martha Stewart Living Omnimedia, continued, “Brand marketers are frustrated about not getting answers to those basic questions, so they end up looking at online as more of a performance-oriented medium. The agencies, in many cases, are using classic direct marketing metrics and trying to measure brand impact using those metrics—but that doesn’t answer the marketers’ real questions.”

Killer Stat for the Purchase Funnel: 94%

When a Microsoft Atlas Institute study examined the 90-day timeline for a typical purchase funnel, it found that companies often disregard 94% of the data available to them when assessing online campaigns. The study also revealed that marketers attribute far too much weight to activity occurring at the very bottom of the sales funnel, concentrating heavily on the last ad clicked, which often appears on a search engine.

Commenting on the study, Esco Strong, senior group manager of the Atlas Institute, said, “It’s a myopic view that disregards the points at the top of the funnel that brought the consumers down to the bottom.” Mr. Strong added, “You make someone aware at the top of the funnel, target them with specific information in the middle, and drive them to buy with search at the end.”

Not everyone in the industry is in agreement about where in the funnel the most work needs to be done. Some, such as Curt Hecht, president of VivaKi Nerve Center, believe the top of the sales funnel is trickiest since it deals with attitudes. “Once you get into real behavior around intent and helping clients come up with proxies to show levels of interest, it’s the attitudinal metrics at the top of the funnel that are harder to come up with,” said Mr. Hecht, in an interview with eMarketer.

Others, such as Ken Mallon, senior vice president-custom solutions and ad effectiveness consulting at Dynamic Logic, a firm that measures brand metrics for online campaigns, think the ad industry has got the top part of the funnel licked, but the bottom part—tying intentions to end results—presents the biggest challenge.

“The hardest part is the last step, [measuring] what happens after someone makes an intent to purchase,” said Mr. Mallon, in an interview with eMarketer. “What happens after someone raises their hand and says, ‘Yes, I do intend to go to that movie. I do intend to buy that car in the next 90 days.’ How does intent translate into actual purchase?”

Agreeing with Mr. Mallon is Jeff Lanctot, the chief strategy officer of interactive agency Razorfish, who told eMarketer in an interview: “[Our clients] see the opportunity to tie attitudinal measures to behavioral measures, and that’s where the complexity comes in. There’s value in measuring those tried-and-true attitudinal metrics but also in recognizing that there’s another piece to the puzzle—actual behavior. Did consumers purchase? Did they register? Did they take that end action? Digital yields valuable insights and also creates incredible complexity.”

To make things even more complicated, some stakeholders even question whether the standard sales funnel model is adequate to represent the complexity in today’s digital world.

“We’re asking ourselves whether the funnel construct is actually the right one. The research we’ve seen shows that the purchase decision process is far more complex and nonlinear than a simple funnel. [For example,] I don’t think we have a good idea of how peer influence works in a funnel construct. There’s been very little progress to evaluate the impact of those [social] conversations on brands.”
—Jeff Lanctot, chief strategy officer, Razorfish, in an interview with eMarketer, April 28, 2009

“Some marketers are starting to use the technology that manages online ad campaigns (the ad-serving platform) to assess the impact of all online touchpoints, instead of basing the optimization of media on the last click before a conversion.”
—Jacques Bughin, Amy Guggenheim Shenkan and Marc Singer, The McKinsey Quarterly, October 2008

Clearly, if marketers are going to spend precious dollars executing online branding campaigns, they ought to be measuring the results—even if it costs a little extra in terms of time and effort.

Today, the tendency among online marketers is to measure only search and give it most or all of the credit for online conversions. They do this because search results, which often represent the last ad clicked, are relatively easy to measure. But a wealth of data suggests marketers should be applying much more rigorous measurement analytics and integrating search and display results to get the complete story on their campaigns.

Another factor is frequency. We all know that multiple messages do a better a job convincing consumers to buy a product than a single message. By exposing a given consumer to multiple display messages, in addition to the search text ad they get after a search inquiry, marketers are likely to improve conversion rates.

Numerous quantitative studies prove that when display ads are combined with search, marketers can expect a significant increase in sales conversions, whether those take place online or offline. (See Data Spotlight section of report.)

“The challenge is that people are applying direct metrics in many cases to what is inherently a branding campaign. We need to continue to evolve the brand metrics in order to make sure we are using the right metrics to measure what the campaign objectives are. But to simply use direct metrics to measure brand campaign performance is never going to be successful.”
—Pam Horan, president, Online Publishers Association, in an interview with eMarketer, April 27, 2009

An Addiction to Clicks

Clicks represent the low-hanging fruit of measurement—and that is why marketers tend to seek them out with a vengeance. But clicks do not even begin to capture the full value of online display ads.

According to comScore, two-thirds of Internet users never click on display ads over the course of a month; moreover, only 16% of Internet users account for 80% of all clicks. That helps explains why the average click-through rate for display ads has plummeted over the years to a mere 0.1%. (Note that online video ads can still elicit click-through rates in the 1% to 4% range, but that is likely to taper off over time.)

“The majority of marketers are not actually using the Web for brand advertising. They’re using it for direct response and promotion.”
—Jim Spanfeller, president and CEO, Forbes.com, treasurer of the OPA and chairman emeritus of the IAB, in an interview with eMarketer, April 2009

A January 2009 study commissioned by iProspect and conducted by Forrester Consulting shows that when Internet users were exposed to a promotional ad, less than one-third (31%) of them clicked on the ad itself; however, a plurality did take some other form of action:

27% searched for the product, brand or company using a search engine.

21% typed the Web address directly into their browser and navigated to the advertiser’s site.

9% investigated the product, brand or company through social media.

Another 37% of respondents in the iProspect study claimed to be completely nonresponsive to “any such ads.”

“If I don’t have a display campaign to support my paid search campaign, I’m basically giving the traffic away to my competitors.”
—Robert Murray, CEO, iProspect, “Search Engine Marketing and Online Display Advertising Integration Study,” May 2009

Overall, the iProspect study concluded that Internet users were more likely to engage with or make a purchase from brands with which they were already familiar, and online brand ads are one way to get there.

Additional comScore research, conducted with Starcom USA, reveals no correlation between display ad clicks and basic brand metrics. In other words, if you look only at clicks, you’re considering only a tiny fraction of the economic value of an online advertising campaign. Study after study has shown that online display ads generate awareness and interest—even if those ads never get clicked.

Why have clicks become the default standard for online brand measurement? Very simply, because they are so easy to measure. Jim Meskauskas, vice present/director of online media at Omnicom-owned ICON International, put it this way in an e-mail to eMarketer: “Every client is a branding client, until they get their first tracking report. Then they turn into direct marketers.”

Or, as Kathryn Koegel, former DoubleClick researcher and a principal at media research consulting company Primary Impact, wrote in her May 2009 “The State of Digital Display” white paper: “We have built a parallel universe that does online little service to its power for marketing and placed too much responsibility on a click on a small graphic image, rather than the fundamentals of media planning and creative.”

If there was one point on which all the professionals eMarketer interviewed for this report could unanimously agree, it was that if your goal is to build brands, clicks are the wrong way to go.

“Because of our direct response heritage [in the online advertising industry], we’ve toiled under the tyranny of the click for too long.”
—Randall Rothenberg, CEO, Interactive Advertising Bureau, as quoted in Advertising Age, March 30, 2009

The Scourge of Clicks

If there are any lingering doubts about counting clicks, talk to Gian Fulgoni, chairman of comScore.

In an interview with eMarketer, Mr. Fulgoni gave a clear and impassioned argument for moving beyond the click-through as a metric for online branding success. The problem arose, he said, when the Internet was first evolving as an advertising medium. The technology community at the time had “a very short-term view of how advertising works that was direct-response-oriented, not branding-oriented,” said Mr. Fulgoni.

Because the click-through rate was so easy to measure, it got the industry into a direct response groove, which has in turn led to a serious undervaluing of branding online.

“Let’s just accept that the click is not telling the whole [branding] story,” said Mr. Fulgoni. “We don’t hold traditional media to that same level of accountability. We don’t say you need to immediately pick up a phone and call somebody if you see a TV ad or if you’re listening to a radio ad or read something in a magazine. Why should the Internet be measured by this immediate response metric called the ‘click’?”

Mr. Fulgoni recently wrote a white paper entitled “How Online Advertising Works: Whither the Click?” which culled data from more than 200 studies and was published in the June 15, 2009, issue of the Journal of Advertising Research. The white paper concluded that an ad impression on the Internet works just like an ad impression in traditional media.

According to Mr. Fulgoni, people who are exposed to a display ad—whether or not they click on it—have “an increased likelihood of visiting the Website of the brand in the ad, an increased likelihood of conducting a trademark search query, an increased likelihood of buying online, an increased likelihood of buying offline.”

“[The display ad] has a lasting effect,” concluded Mr. Fulgoni. “Cumulatively, impressions build up over time and create an image of a brand. Then when you go off and do a search, you’ve got the brand better established in your mind.”

While direct response is attractive to marketers because of the immediate gratification of almost-instant results, they must take a longer view when measuring branding effects, which are cumulative in nature.

Click-obsession leaves marketers with a very crude and inadequate accounting measure that ignores most of the potential value of an ad. They do not know if the ad created awareness in the mind of the consumer, if it served as a reminder to buy the product at a later time, if it created a more favorable impression of the brand or reinforced loyalty to the brand. These attitudinal shifts simply do not happen in the instant of a click.

Do Traditional Measurement Techniques Work for New Media?

Having accepted the fact that clicks are not the answer, what about adopting the measurement techniques of traditional media?

For years, those inside and outside the Internet ad industry have debated whether standard offline metrics, most notably GRPs (gross rating points), page impressions, reach and frequency, should be applied to the digital space.

On one hand, many have said that the typical forms of measurement for traditional media do not translate well online. Nor do these metrics take into account the Internet’s unique interactive qualities that allow for two-way communications with consumers. In a word, they feel these forms of measurement are inadequate, or even irrelevant.

Scott Knoll of Datran Media put it well in a recent iMedia Connection article: “The models that have been in use up to this point most resemble the proxy- and panel-based models of broadcast [television] and fail to represent the very thing that makes [the digital] industry different—its interactivity.”

“It’s like going to a 3-D movie without the glasses. The Internet is more dimensional, but [for the most part] measurement criteria are the same as for a one-way medium. You don’t have the glasses so you’re not appreciating the dimensions.”
—Matt Freeman, CEO, Betawave, as quoted in AdWeek, March 23, 2009

Others argue that the power and possibilities of social media programs that allow for deep consumer engagement are completely ignored by traditional reach and frequency measurements.

But don’t write off traditional media measurements and metrics just yet. There is a growing movement to support the application of tried-and-true GRPs to Internet ad planning and measurement, and this may well be a key to opening up the floodgates for more media dollars and advertiser interest.

The key here is making online comparable to other media. “On the one hand, you’ve got incredible [measurement] granularity on the Web and a lot more insight into your brand metrics,” said Jim Spanfeller, president and CEO of Forbes.com, in an interview with eMarketer. “On the other hand, you really can’t compare that in any way, shape or form to your offline spend.”

Integration Is Hard When Data Is Locked in Silos

In eMarketer’s interviews with professionals who are experts in the field of building models and processes for online brand measurement, it was clear that unlocking data silos is a critical step toward connecting the dots of media measurement. As Razorfish’s Jeff Lanctot told eMarketer, “For all the talk about the integration of traditional and digital media, they’re still bought in isolation much more than they’re bought in integrated packages.”

“The single biggest issue advertisers are dealing with is how to use online properly in combination with other marketing activities. To get there, we need data integration and that’s the hard part.”
—Bob Barocci, president, Advertising Research Foundation, in an interview with eMarketer, April 2009

In an interview with eMarketer, Cary Tilds of Mindshare-Team Detroit said, “The current approach to planning media is what I would call the buckets-of-money approach. You have a TV bucket, a digital bucket, a search bucket, you might have an emerging [media] bucket. You have buckets of money. The unspoken controversy is about switching money from one bucket to another.”

Ms. Tilds added, “But the buckets of money approach has nothing to do with how people consume media. A good media planner needs to figure out how to allocate money against the consumer journey—and that is a very complex problem. You have to identify all the knowns and unknowns in the long string of mathematical equations that we need in order to move from buckets of money to consumer-centric planning.”

Further proof that data silos are a problem comes from quantitative surveys. According to a November 2008 study by JupiterResearch and Verse Group, 78% of US marketers agreed that internal silos act as the biggest barrier to integrating marketing. A related concern was “managing our brand across multiple platforms.”

A more recent study, conducted in March 2009 by TNS Media Intelligence and sponsored by rich media provider Eyeblaster, found that while 67% of senior marketers are now running cross-channel campaigns, a scant 12% are integrating performance data across channels.

When it comes to integrating digital and traditional media, too many marketers feel as though they are operating in the dark. Two separate surveys bear this out.

In the study by Heidrick & Struggles of 111 US senior marketers, a paltry 15% of respondents claimed satisfaction with their optimization of the marketing mix.

Another study, jointly conducted by the 4A’s and the ANA, found that only 7% of US marketers said they were “very satisfied” with their progress toward integrating online and traditional media.

Among the obstacles to integrating traditional and online media, according to the joint 4A’s/ANA study, were the lack of metrics and an insufficient understanding of digital media among senior management.

The top frustration among agency executives, on the other hand, was that their clients do not fully understand how consumers use digital media.

The study also identified challenges in measuring marketing ROI. On a scale from 1 to 5 (where 5 represents “a major problem”), the top challenges were data scattered across the organization (3.2), difficulty allocating enough resources to do the work (3.2) and inconsistent data formats (3.1).

In an interview with eMarketer, Michael Mendenhall, senior vice president and chief marketing officer of Hewlett-Packard, had much to say about the problems and solutions for data silos. “Many marketers have disparate databases that don’t speak to each other, so they don’t have a 360-degree view of a customer…. As you think about Fortune 500 companies that are selling across a portfolio and selling brand experiences, it becomes very hard to sell based on a portfolio approach without appropriate analytics.”

He added, “The challenge for marketers is to begin to build technological capabilities that allow you to see the complete digital footprint a given customer leaves when they engage with your brand. You need to be able to understand that behaviorally and/or contextually and address that consumer in a relevant way…. Where most marketers struggle is they do not have the information technology capability and/or management to do that. You need a sophisticated CRM platform that allows you to identify somebody coming through search or display, and gets them to opt in and [allows you to] start to manage them…. Marketers need the ability to pull strategic information out of the digital footprints that consumers are leaving.”

In an interview with eMarketer, Martin Nisenholtz, senior vice president for digital operations of The New York Times Co., provided a publisher’s perspective on the online/offline data integration problem. “The thing that I continue to hear over and over again from people is, ‘Gee, I wish I had a way to tie my online brand measures to my offline brand measures so that I could standardize,’” he said. “If I’m spending X amount of money on television and Y amount on the Web, wouldn’t it be great if I could look at one set of metrics and determine whether the offline plus the online were more powerful together, whether the online on its own has efficacy and whether the offline without the online is much weaker or not.”

Online Silos

Of course, data silos also exist within the interactive platform. According to an iMedia Connection article by Scott Knoll, a vice president at Datran Media, “The problem is not the existence of the data. The problem resides in separate measurements for search, display, email, video, and mobile, creating…‘data smog.’”

Kathryn Koegel, in her May 2009 white paper about the state of online display ads, identifies intra-Internet silos as a major obstacle to online advertising efficiency and success. “The silo-ing of media into online vs. offline, direct vs. branding, search vs. display, behavioral vs. contextual, undercuts the creation of media plans that address actual consumer purchase processes in the most efficient manner,” she wrote.

Too Much Information, Too Much Complexity

Another major obstacle to online brand measurement is the sheer tonnage of data to be extracted, filtered and integrated—as well as the mind-numbing complexity that data creates.

“The problem with online is we have too much data, actually.”
—Jon Gibs, vice president-media analytics, Nielsen Online, in an interview with eMarketer, April 17, 2009

While the Internet offers an abundance of possible metrics, that very same plenty creates complexity and confusion. It is relatively simple and straightforward when the objective is direct response. The desired behavior is not only easy to measure, it is often near-instant. The consumer sees the direct response ad, follows through with the call to action, and the marketer can quickly and easily measure the results.

“Every day that I read about a new technology…I get worried…. The scary thing is how complex [digital advertising] can be.”
—Mark Addicks, senior vice president and chief marketing officer, General Mills, in an interview with eMarketer, May 4, 2009

When branding is the objective, though, the whole measurement process is much more complex. First, the intended result—a change in feeling, perception or attitude—is harder to measure. Second, the impact of brand-building occurs over a period of time, often weeks, months or even years. Third, there are far more dots that need to be connected.

“Branding is a longer-term proposition. It’s not about recalling a specific ad you saw 5 seconds ago. It’s about whether people feel better about a brand or not.”
—Young-Bean Song, senior director of analytics and Atlas Institute, Microsoft Advertising, in an interview with eMarketer, April 16, 2009

This brand measurement complexity problem is not unique to the online space, though the Internet adds another layer or two of complexity with its interactive properties.

But it is essential to integrate online data with data from other media, because branding campaigns are clearly most successful when message exposures come from multiple media—a mixture of television spots, a magazine ad, targeted banner and online video ads, an outdoor billboard, an e-mail promotion and so forth.

“How can you possibly isolate the effect of any particular piece of communication? All of the problems that exist in measuring advertising and marketing efficacy exist in every environment—offline and online.”
—Amy Fuller, group executive, worldwide consumer marketing/global products & solutions, MasterCard Worldwide, in an interview with eMarketer, April 28, 2009

Additional complexity comes from the fact that research and measurement firms have widely differing approaches and methodologies, resulting in wildly varied results. As Mr. Lanctot of Razorfish noted, “For a brand marketer, when there’s such a disparity in what would appear to be the most basic of metrics, it plants a seed of doubt about the reliability of the data sets or the measurement program as a whole.”

Current Measurement Models Have Limitations

None of the prevailing techniques for measuring online branding impact today are without flaws.

“Every type of measurement method, be it online media research, TV media research…has some level of error associated with it. There’s [no] magic bullet out there.”
—Jon Gibs, vice president, media analytics, Nielsen Online, in an interview with eMarketer, April 17, 2009

Site Intercept/Brand Impact Studies

Although widely popular with advertisers, the use of site intercept/brand impact ad effectiveness studies, such as those conducted by Dynamic Logic or InsightExpress, is expensive when multiple studies are run and do not tell the whole branding story. Relying on the control versus test group approach, they work well for measuring the brand impact of a specific ad or campaign, but lack scale and do not take into account the longer-term effects of brand advertising unless they are run repeatedly over multiple campaigns.

In addition, the heavy use of such surveys has led to consumer survey fatigue, where the same individuals are polled over and over. This can result in respondents who are not representative of the population being measured. Some, too, question the intercept method itself, since interrupting people with surveys while they are surfing can lead to somewhat artificial results.

Mr. Hecht of VivaKi, in an interview with eMarketer, pointed out some of the pitfalls of these intercept measurement techniques. When asked what the single biggest problem is with online brand measurement today, Mr. Hecht replied: “It’s the problem of doing either custom one-off research which doesn’t scale, or doing studies focused on individual campaigns which don’t give you the whole story. It’s really hard to go out there and create a panel of size, or create a technique to do surveys that are not interruptive and that are useful. I’m not seeing a lot of innovation in terms of a company stepping into that void to provide the kind of research and tools needed.”

Mr. Hecht added, “The market’s at the pressure point, in a good way, and it’s going to force innovation and solutions to come forward.”

Another problem with the intercept survey method is that it is not very good at measuring multiple forms of online ad messages. Joe Laszlo, director of research at the IAB, said, “One thing…that’s lacking is a way to assess the effectiveness of your campaign if it runs across a broad swath of the Internet. For example, if it runs across 15 different sites and in conjunction with a search engine campaign, plus some social media.”

Panel Measurement Systems

The alternative to brand intercept studies is panel measurement systems, such as those deployed by comScore and Nielsen. While this technique has the ability to measure actual surfing behavior, it has its own limitations. Panels are best for measuring Website traffic and are less effective for measuring ad campaigns that may run across dozens or hundreds of sites. They also tend to miss or undercount smaller, long-tail sites.

In addition, there are the long-standing concerns about conflicting data:

The individual panel measurement firms often show widely differing numbers, such as for site traffic reach or audience size.

The numbers obtained from site-server logs are often contradicted by any given panel measurement firm, causing frustration and confusion.

Measurement of Social and Video Are Even Further Behind

While the widespread use of social and video sites by consumers evokes excitement (and sometimes a little fear) among advertisers, the measurement of advertising in these environments is vastly underdeveloped. Most advertisers are still trying to understand the nature of this consumer activity and how it might translate into appropriate advertising messages, let alone how to measure campaigns in these environments.

Social Media

In a study by the Marketing Executives Networking Group (MENG) of 126 US marketing executives, more than one-third of respondents admitted they did not even measure social media activities. One-half claimed to measure raw traffic and one-third measured clicks, both of which represent crude metrics, even by Internet standards.

A full 59% of respondents said they never or hardly ever measured the ROI of social media marketing efforts. Only a little over 12% did so all or most of the time.

The primary reason so few measure social campaigns is that it is difficult—59% of marketers in a separate Aberdeen Group study said social media was either “somewhat difficult” (39%) or “very difficult” (20%) to measure. Note that the respondents here were self-identified “best-in-class” companies. The survey also found that only 18% of marketers link activities on social media sites back to revenues or other financial metrics.

Online Video Measurement: Promising, but Still Early Days

Like social media, the online video advertising market is booming with promise, but somewhat lagging in dollars. That is partly due to a laundry list of measurement issues. A central issue, though, is the tendency to focus on short-term, direct-response-type metrics that ignore the primary branding-oriented benefits of video.

“The instinctual reaction to try and show the sales impact of video campaigns is misguided,” said Mr. Song of Microsoft Advertising. “We have studies that show that video, photo-sharing, weather and social media placements look horrible in the online ROI equation because they’re upper-funnel. People don’t see ads on Facebook and then go buy something. Reach, frequency and GRPs…need to be there upfront.”

However, the lack of transparency for video advertising metrics is only one of several problems confronting the format, according to 2008 research from MarketingSherpa.