Mumbrella to launch major research project examining advertising agencies performance
Mumbrella is set to undertake one of the most ambitious studies of advertising agency capabilities ever undertaken in the Australian market.
The major project is being led by Mumbrella deputy editor Robin Hicks with the results revealed at this year’s Mumbrella360 conference.
Criteria to be examined include creativity, effectiveness, integration, planning, talent, commercial success, account management, impact on the industry, momentum and client stability.
In its first year, the study will examine Australia’s top 25 advertising agencies.
Hope “Talent” covers HR issues such as turnover, retention, training, performance reviews and remuneration.
I cant believe the 3rd element is focussed on awards. I thought the industry had grown up and gotten over the so called importance of awards?
Hi Dragon,
I agree with you that there is an over emphasis on awards – and particularly on rankings of awards where a single good piece of work can get hugely over-recognised in league tables if it wins in more than one place.
But equally, creativity does matter. And when the world’s best creatives sit down and arbitrate against a world standard – as they do at Cannes – their views mean something. So we’d be foolish not to include it in the mix at all.
Similarly, when agencies go to a great deal of trouble to demonstrate effectiveness – again, an essential – it adds an additional usueful signal.
But to emphasise, awards will be one small part fo the mix because it seems foolish to ignore them altogether – but they are not the main game.
We’re lookign for a far broader story than that.
Cheers,
Tim – Mumbrella
It would be neat to take ‘how much a client spent with an agency’ and compare that to ‘increase in client sales’.
Hope it’s not toooooooo long this time!!
Quality of work life / opinions of employees would be a good measurement to add in the mixer
brlliant idea. clients will love it for the quality of info. major agencies may sulk if they dont score.
Tim, what is the source of the Top 25 agencies?
Hi Sam,
That’s a good question – and one we’ll loo to our panel on. Having done a calculation on the back of an envelope, my sense is that the top 20 or so will be resonably obvious. Where it may be slightly more controversial will be in the marginal calls. I suspect there’ll be another 10 or so vying for places.
Cheers,
Tim – Mumbrella
Tim,
The moment you included awards, you lost me. Creativity comes in many forms. Awards only concern themselves with one school of creativity. Under your criteria an agency that wins a Cannes Bronze Lion for a one-off piece of print work is considered more creative than an agency that creates something like the Bunnings campaign, or JB Hi-Fi’s work. These campaigns exhibit great creativity because their creators understand their categories, their target audiences and what motivates them perfectly. They mightn’t be fashionable, but sometimes the most creative thing you can do is NOT be anything like a Cannes winner.
Anonymous contribution of pitch consultants, clients, former clients and – headhunters? How many dinners at Bilsons do I need to shout to fix these results?
On a serious note, I love the fact Tim is taking the initiative to find a more holistic approach to ranking agencies rather than who’s scratching who’s back. If I were to add a few pointers:
1. Client wins 1 point, client losses -1 point unless it was a result of a global realignment. This is a good measure of relationships with clients.
2. Effectiveness / ROI – +1 point for every 10% measured, -1 point for every 10% lost. This should come from the client, not agency case studies.
3. Staff retention – indicator of how happy the cows the agencies milk are. +2 for every employee that’s stayed another year, +1 for every employee hired, -1 for every employee that’s left within three years, -2 for every employee that’s left under 3 years.
4. Innovation / bravery – +1 point for every campaign that is shifting advertising forward, no matter if you ‘like’ the creative or not, -1 point for every ad that’s been done before, 0 points for maintaining the status quo. This is important for the industries health as a whole, and could also encompass media platforms.
5. Awards – +1 point for any campaign that has won in the major local or international shows, this doesn’t include a single campaign that’s won at every show, e.g: Best job in the world = 1 point, not 20.
6. Relationships – +1 point for every positive relationship from a client AND supplier (including directors, photographers and headhunters), -1 point for every bad relationship.
7. Averages – overall points X number of clients ÷ number of staff should average things out a little better.
8. Billings divided by staff numbers to scale the agencies health.
It would be great to see the scores broken down as well, an agency may be great at making a profit and treating their clients well, but treat their staff and supplies poorly and produce bad work. This would help a marketer in making an overall decision.
Great initiative.
All sounding very much like the TV Week Logie Awards for Agencies!
Further, how does this panel inject objectivity into evaluation? Example: a pitch consultant (!) rejected by agency x, decides to mark down agency x.
A fatuous PR stunt.
Tim,
Great initiative. Just promise us all that you’ll come up with some sort of neat data visualisation that displays how all the agencies rank against each other. That way the ad industry can finally be rid of Mr Lynch’s (edited by Mumbrella) Hot or Not Chart. It would be so refreshing to have something with some sort of robustness behind it.
Hi Tim,
You almost lost me as well when you said you’ll also include performance in industry awards.
I have never chosen an agency based upon their success with awards and none of the companies I’ve worked for have either. It has simply never been a criterion for assessing the capabilities of an agency.
I’m also not sure how truly valuable this will be without the input of the agencies own clients. From my point of view there is really only one organisation that can genuinely assess the ‘success’ and ‘performance’ of an agency and that is the client.
I’ve always been involved in measuring the performance of an agency through processes like ‘Project Level Management’ which has the marketers and agencies rate each other’s performance on a particular campaign or program immediately after it has completed. These scores are than aggregated for a year round score and issues surfacing through this process are actually addressed going into the next year.
To be fair to agencies, I’ve seen as many bad briefs coming from marketers as I do bad response to briefs and I’ve always said performance is a two way street between the client and agency.
The Project Level Management process would evaluate both the agency and client’s performance on a scale of 1 to 10. I’ve put up a screen grab of just one version of this type of performance evaluation spreadsheet, this one for agencies – http://twitpic.com/4hjeae There are others out there and Forrester has a pretty good evaluation spreadsheet as well but you’ll also find no mention of awards.
I applaud the initiative but without significant performance evaluation from each agencies clients, I can’t see how it can be considered useful, accurate and truly representative of each agencies current capabilities?
Like it! Time to arrest the traditional approaches and create more wholesome benchmarks. One thing I’m a little confused about though is that this process will examine ‘creativity, effectiveness, integration, planning, talent, commercial success, account management, impact on the industry, momentum and client stability’
Seems a bit of an overkill, given that a firms creativity should really only be evaluated on how it CREATES client value, which from my experience comes from effectiveness, integration, planning, talent, commercial success, account management, impact on the industry, momentum and client stability…
I suggest to keep it simple and just judge creativity – in its true form, not in terms of being flamboyant.
And maybe include all firms that offer this value proposition to those charged with marketing. Or does 360 stand for ‘thousand’ of minimum budget?
A long overdue initiative. But the argument over methodology and criteria will rage for longer than the newspaper readership debate I’m sure. Those who won’t do well will already be lining up to shoot it down. Strap on your bullet proof vest Tim and go for it. Research should only ever be an aid to judgement and agency evaluation sure needs it.
How about losing 1000 points for wrongly entering an award category?
I agree it’s a fantastic initiative. The ‘talent’ section must be a real focus of this report. Staff turnover is a huge black mark on this industry and needs to be addressed. Agencies have to be more accountable for staff development and retention. I’d love to see an update on % of staff leaving their agency role in the first 18 months of employment.