Academic versus reality – a tale of 2 studies (Part I)

Nico Neumann, assistant professor and fellow at the Centre for Business Analytics at Melbourne Business School, looks at the academic approach to ad effectiveness and the limitations from using aggregate data in two recent studies.

Two recent academic studies investigating advertising effectiveness have sparked vivid online debates.

Both are high quality, independent studies (not those commissioned by industry groups or your ad vendor claiming to be independent). Both studies deserved the attention they got, but maybe not always in a way they should have.

The first of these two thought-provoking studies is the “Freakonomics” centrepiece investigating TV advertising effectiveness of 288 CPG brands in the US, conducted by three professors from Chicago. In a nutshell, the ‘Chicago work’ presents two main messages.

To begin, many previous studies have measured advertising effectiveness incorrectly, often even hiding negative results. Second, many brands seem to over invest in TV advertising as the authors find a negative return on investment (ROI) when comparing ad effects and costs.

What happened? Adland was not amused. Whole blogposts, tweets and LinkedIn sections have been dedicated to attack and ridicule the study, including by (supposedly neutral?) consultants and agencies. Critical comments largely revolved around the following three themes, many of which I have heard in Australia too:

  1. “These guys don’t work in advertising and therefore don’t understand media.”
  2. “The research design is flawed.”
  3. “ROI is not the right metric to assess advertising effectiveness.”

Let’s address these points one after another. First, all authors actually have worked on ad effectiveness studies before. Even more importantly, the rules of mathematics are not different for media: 1+1 is still 2. Once you have the data, your results really come down to a proper statistical analysis. And the three Chicago professors are very experienced here.

Second, no research is perfect. One can certainly find potential weaknesses, such as the limitations from using aggregate Nielsen data. However, it was sad to observe that many online commentators have not read or understood the actual work, but still felt the need to shout out to the world that the study must be bad.

Third, advertising can provide a range of benefits. But let’s be honest – all brands expect some measurable return from their investments, including advertising. The analytical question we all try to answer is: Which revenue would not have occurred if I had not run my ads?

With regards to ROI conclusions, there often seemed to be a misperception that the presented analyses would only consider weekly numbers and short-term ROI. However, the Chicago team also looked at an ROI analysis across a period of five years.

When taking a closer look at these long-term results, the ROI estimates were not as bad as many readers seemed to have believed. The Freakonomics study suggests that between 24-40% of brands have a definitive positive ROI in the long run. Moreover, for the remaining 60% of brands the case is less clear. Several of these brands may also have a positive ROI when taking into account a long-term view. So about half of all brands are likely to have positive returns for their advertising spending overall.

Not an unrealistic observation, isn’t it? And rarely new. We need to keep in mind the old Wanamaker wisdom: “I know half the money I spend on advertising is wasted; the trouble is I don’t know which half.”

What could be the reasons for brands having no positive return for their ad dollars today? Well, there could be underperforming creatives or poorly executed campaigns. Not every ad has the effect that advertisers would like to achieve. Let’s recall the diverse opinions about the Australian TV ad for Advangen.

Moreover, many brands may simply spend too much overall on advertising because they run too many ads. This can happen because numerous media executives – including in Australia – believe in the share-of-voice winning formula. The guiding principle is that media dollars are never a bad investment and that we can always spend more on advertising to beat our competitors and be the ‘loudest’ in the market.

This is a dangerous game though. If all companies in a category keep increasing advertising to get more share-of-voice than the others, everyone keeps the same market share eventually but at higher costs. As a result, profits and ROI shrink every year. This is not a sustainable business strategy. And the only players benefiting from the share-of-voice strategy are the ones making money from selling ads. Apart from this zero-sum game fallacy, there is also sufficient evidence showing that too much advertising can actually backfire and lead to lower conversions and negative consumer sentiments about a brand.

What does this all mean? While it’s easy to fall for confirmation bias, from a sober perspective, the two main messages of the Chicago study should be taken on board by prudent management. Let’s measure ad effectiveness properly – using scientific best-in-class methods – and let’s be careful how much we spend on our advertising investments. It should not be about who has the most advertising, but who can leave a memorable impression using all available marketing levers effectively.

Nico Neumann is an assistant professor and fellow at the Centre for Business Analytics at Melbourne Business School.


Get the latest media and marketing industry news (and views) direct to your inbox.

Sign up to the free Mumbrella newsletter now.



Sign up to our free daily update to get the latest in media and marketing.