Are undergrads really more influential than Gruen panelists? Klout thinks so
In this guest post, Sarah Ballard and Sean Rintel warn that marketers who rely too heavily on online influence measurement system Klout may be foolish to do so
Klout is increasingly becoming a hot new marketing metric. But basic tests suggest that it offers highly questionable results.
We asked: Could undergraduate students really have more Klout than Gruen Transfer panellists Todd Sampson and Russel Howcroft?
While The Gruen Transfer’s panelists may be among the top Australians in the business of influence, the Klout online social influence measurement service shows that typical undergraduates appear to have as much online influence as Gruen panelists.
Online social influence is becoming one of the hottest new marketing metrics.
Companies such as PeerIndex and Klout claim to measure influence by weighing up combinations of platforms, followers, and sharing. Klout is currently aggressively promoting itself by pairing with brands; for example, Cathay Pacific is currently offering free entry to its SFO lounge to passengers with Klout scores above 40, and Red Bull has worked with Klout in Australia. It’s clear that brands are looking to partner with the most influential advocates available, but is Klout a reliable way to measure influence?
Klout scores start at 0 and Klout claims that the average user gets a score of 20 out of 100, but we decided to put that to some simple tests – in particular examining whether people who link it to their Facebook profile get a disproportionate lift.
Twitter-linked accounts: You’re a 10 even if you do nothing
To determine the baseline Klout score, we conducted the “Snow White and the Seven Dwarves” test. Seven Klout accounts were created named after the dwarves and appropriate content was tweeted daily (e.g. jokes from Happy). The dwarf Klout scores stayed at 10 for the entire month–even Sleepy who posted nothing and had no followers. This result suggests that baseline score is 10 and that the claimed average Klout score of 20 is very low.
To determine the baseline Klout score, we conducted the “Snow White and the Seven Dwarves” test. Seven Klout accounts were created named after the dwarves and appropriate content was tweeted daily (e.g. jokes from Happy). The dwarf Klout scores stayed at 10 for the entire month–even Sleepy who posted nothing and had no followers. This result suggests that baseline score is 10 and that the claimed average Klout score of 20 is very low.
Facebook-linked accounts: Is 40 the new 20?
Next we tracked the Klout of three male and four female undergraduates, aged between 18 and 28, who used Facebook, averaging around 400 friends and reporting only light Facebook activity. Astonishingly, all of their scores rose dramatically above the claimed average of 20. The average peak score was in the mid-40s. The scores dropped slowly and all but one maintained scores above 30.
Birthday bump versus consistent posts
Several of the undergraduates had birthdays just after signing up for the study, and their Klout scores reached the high 40s and dropped slowly thereafter. This suggests that the increased activity surrounding major life events considerably affects the Klout score.
However, two subjects did record scores in the 40s without the occurrence of any major life event. Instead, these subjects recorded consistent posts and Likes on Facebook pages between themselves and around three friends. Just one or two posts reduced the level of decay or even increased Klout scores, indicating that Klout does value consistency over one-off events. Still, the birthday bump is a source of bias.
Undergraduates on Facebook versus Influencers on Twitter
Klout accounts with only Twitter logins required thousands of followers and high levels of activity to achieve comparable scores to Facebook only accounts with far fewer friends and much less activity. During the tracking period we followed a highly active Australian social media expert with 1800 Twitter followers and hundreds of retweets and mentions. As shown below, with some variations one undergraduate (red) had a higher score but eerily similar decay to the social media expert (blue), and they ended at around the same score of 45.
Similarly, as the graph shows below, the average peak Klout scores of the undergraduates were also remarkably similar to the average of Gruen Transfer panel members. In fact, were it not for Wil Anderson, the undergraduates would trounce the Gruen Transfer group.
Todd Sampson is just below the average of these undergraduates (42), and Russel Howcroft is, apparently, far less influential than all the undergraduates (35). 
Nice work if you can get it
Clearly, Klout scores should not be taken at face value. NYU Professor of Business Sinan Aral’s recent Science article argues that online social influence is more about measurable action taken on posts, rather than the number of followers or posts themselves.
We found that Klout favours Facebook over Twitter. Klout may argue that this is because Facebook connections are deeper and more personal than Twitter. However, what it really means is that Klout rewards users who join and connect their Facebook account. This is not a measure of influence, then, it is a measure of willingness to join Klout. 
This still leaves the problem of purporting that an undergraduate has as much influence as Todd Sampson or far more than Russel Howcroft. The biggest problem with the Klout score is that it has no idea who people are or what they are saying.
The Gruen members are all are prominent in their field and have significant media exposure. The Klout score apparently does not attempt to include measures of social capital such as employment or media coverage and has no concept of the meaning of posts.
Klout is not really measuring influence, rather, it is measuring activity on social networks.
Brands that want to use Klout should consider the scores only as a rough measure of activity. Premiums should target those with scores of 60. Scores above 10 will capture all Klout users. Scores above 20 will capture very sporadic Internet users. Scores up to around 45 or even 50 will capture users with average levels of visible Internet activity. These are potentially useful guides to likely activity, but they are a long way from a measure of influence.
- Research and analysis by Sarah Ballard and Dr Sean Rintel, School of Journalism and Communication, The University of Queensland
Thank you! I’ve been screaming that Klout has no clothes for a while, and am still amazed at how much stock some experts place in it.
It’s easily gameable. Because ll scores are normalised out of 100 it doesn’t give you a true measure but a normalised measure against everyone else on Klout – most of whome you shouldn’t be benchmarked against. Klout is useless as an analytic tool on your own activity. It is marginally useful in identifying potential influencers you should attempt to reach.
But, strategically, it is no more useful than common sense.
User ID not verified.
Super interesting… great post “Klout is not really measuring influence, rather, it is measuring activity on social networks.”
I de-activate my Facebook every two weeks when I’m not promoting A Rational Fear… I’ve noticed a correction to my Klout correlating with my facebook use… often the difference is up to 8 points….
User ID not verified.
Fuck. My klout score is 62… Should I take this off my CV?
User ID not verified.
Exactly. All it measures is quantity of activity. Not quality. Or other ‘social equity’ such as media exposure etc.
A useless tool designed to collect and sell your personal information.
User ID not verified.
Excellent piece.
User ID not verified.
Great post / analysis. I signed up to Klout to see what it was all about, and after a couple of weeks deleted the account as it seemed so irrational.
User ID not verified.
To be fair – only an undergrad would take Klout seriously as anything but a bit of fun.
User ID not verified.
another data scam.
someone will overpay for it – they always do!
User ID not verified.
My favourite thing about Klout is the list of topics it thinks you’re influential on. Pretty sure they partially employ the “close your eyes and point” technique
User ID not verified.
Anon #7 – sadly, I have encountered a lot of people who don’t know much about the social space and are putting huge emphasis on klout because someone or other told them it was important/significant/the next big thing.
Great article guys! Really interesting
User ID not verified.
Blind Freddy has more Klout than Tod Sampson and Russel Howcroft and rightly so. These two guys are worse than 2nd hand car salesmen (apologies to 2nd hand car sales guys).
I’m just so thankful the Gruen Transfer gives these legends in their own lunch times a platform to demonstrate how brilliant they are to us lesser mortals.
User ID not verified.
I don’t think Klout is a serious measure of online influence – nor do I think it makes it self out to be.
Don’t forget sole revenue stream is brands paying to be a Klout Perk.So Klout wants the ‘massses’ to sign up, believe they have huge influence and share their perks with their friends as they’re now social gurus
User ID not verified.
If anyone in their right mind recommends social activity/measurement using Klout then I would be incredibly suspicious of their work.
User ID not verified.
Mo Klout, Mo Problems
User ID not verified.
Interesting article. If I had a dollar for every time I saw someone try and use quant variables to construct an engagement score, influence score, amplification score or appreciation score I would probably have a few hundred on the hip by now. They are looking to use a combination of quant data to create a surrogate for something qualitative.
As a quant researcher I have learnt over time that the pursuit of this holy grail of turning quant into qual is the road to hell. Millions of hours have been lost and we have little to show for it.
At a respondent level these scores often look ridiculous as is demonstrated above. I don’t understand why these metrics aren’t labelled as what they are instead of what they clearly are not. It just gives the measurement a bad name and perpetuates the ‘snake oil’ reputation that online and social media metrics already have. The fact is there will be a strong relationship between activity and ‘klout’ (influence and amplification) but this needs to be demonstrated not assumed.
User ID not verified.
Klout – that’s just a game to get the highest score… not real influence
Why not take a look at Kred http://kred.com – transparent, real-time and as we look at communities much more accurate.
We also have the full twitter firehose so we can look at influence across 120 million profiles.
Andrew Grill
CEO, Kred (and Sydney expat)
@andrewgrill
User ID not verified.
True influence measurement is still 3-5 years off and it won’t be a startup that cracks it.
User ID not verified.
For those looking for further reading, this Wired article is the most unbiased and realistic summary of Klout I’ve seen: http://www.wired.com/business/2012/04/ff_klout
note: I have no affiliation with this article, so you’ll have to forgive the complete lack of self-promotion
User ID not verified.
What’s clear in the social influence measurement space is that nothing is clear, with all the different “measurement” tools varying in their analysis of the same social properties!! Klout is useful sometimes as a benchmarking tool, but noting else … unless your property has a really high score!!
User ID not verified.
Thanks for the tip on Cathay Pacific. I might try them out.
My Klout sits in the high 40s, which I find amusing as I find it, PeerIndex and Kred extremely flawed one-dimensional systems.
Apparently offline influence is meaningless 🙂
User ID not verified.
I’ve got a score of 62 also… what does that get me?!
User ID not verified.
Thanks for the study! I am getting so tired of people advertising their Klout-scores, as if it means anything…
Since the beginning of Klout I knew it doesn’t work. Seeing me listed under “Fundraising” although I have not posted on fundraising for the past two years, was quite enough. Add to the neverchanging score over the past years, because I have not added Facebook – to make it clear, that Klout is more toy than tool.
User ID not verified.