Testing: audience knows best

Predicting the future of a film or television program and avoiding a financial disaster doesn’t necessarily require clairvoyant powers; testing your project with an audience at different stages of the production process can make a huge difference. Aravind Balasubramaniam reports.

“Nobody wants others to tell them their baby is ugly,” says the founder and CEO of Audience Development Australia (ADA) David Castran. “But it’s very important that producers listen with two ears and try to challenge their strongly-held views. Testing empowers producers by giving them information about the audience’s relationship with the pilot or program. I know at times that can be controversial.”
Drama has the highest production cost per hour of all TV programming, so well-executed testing can inform producers and broadcasters about the potential response to their shows.
Castran’s company was founded in the early 90s to provide comprehensive audience study services to the industry. Traditionally, groups of approximately 40 people in Melbourne and Sydney were shown the program and then handed a questionnaire to complete, followed by a moderated session, but ADA’s concept and pilot testing, as well as program evaluation, have mostly moved online – using an interface similar to catch up TV services – allowing the company to work with a bigger sample. “The laggers were the 50+ audience who were slower to get online, but they came around and now you can’t keep them out of the surveys; they absolutely love doing research online,” said Castran.

“Instead of having a sample size of 80, we are now doing 300 and the costs are the same. Plus, our clients are no longer subjected to the plight of watching judgment being passed on their projects.”
Gathering information is far easier than analysing it; things have to be put into context for the client, using normative data. Castran believes that, in the end, it’s all about the ‘actionability’ of the data – in other words, what can you do with the research material?

“We ask open-ended questions, resulting in people typing in quite a bit, a whole battery of things, which in turn calls for a filtering process to highlight general patterns. It’s really about guiding the client; when a
viewer or respondent says something, it’s often code for something else, and we can identify that because we’ve heard similar replies in the past.
“The normative database contains results of up to 400-500 tests conducted in the last 15 years, so we can make comparisons on, for example, how a typical comedy compares to others. Of course, we don’t show clients each other’s scores, instead we aggregate them into benchmarks, categories and genres. We try to answer questions such as ‘Are these results bad or good, high or low?’ or ‘Is that something I should be crying or be happy about?’”