The Notes You Don’t Play

In this guest column Antony Giorgione looks at how good copywriting can unravel the issues behind even the most complex and delicate situations – such as ethnic diversity imbalances – to reveal solutions.

The recent release of the 2016 WARC 100 presented some fantastic case studies in effectiveness, and it brought to mind another ingenious and effective piece of creative from last year – perhaps not eligible for WARC but for me just as inspiring by the Behavioural Insights Team (BIT) at the Avon and Somerset Constabulary, in the UK.

It revolves around some copywriting used to address an ethnic diversity imbalance, and was the work of the Behavioural Insights Team (BIT) out of the UK. BIT was established in 2013 by the Cameron Government as an entity to address behavioural change in areas of public policy and services through the application of statistical insight and an understanding of the behavioural sciences.

The Problem

The Avon and Somerset Constabulary was finding Black and Ethnic Minority (BEM) applicants were unaccountably underperforming at a specific stage of their recruitment process.

The problem appeared within the second stage of a seven stage recruitment process. The first component of the second stage is a Behavioural Styles Assessment to ascertain the behavioural suitability of the applicant, and the second component is a Situational Judgement Test (SJT) to test (amongst other things) the applicant’s empathy and decision-making abilities.

As is illustrated, the SJT presents a number of scenarios to the applicant in both video and transcription form, with applicants grading a list of response options from ‘counter-productive’ to ‘effective’.

Pass levels for BEM applicants on the SJT were not commensurate with those of non-BEM applicants.

The Strategy

Was this underperformance the result of a racial bias embedded within this recruitment process? Sort of.

BIT cited two papers in their online case study; Steel and Aronson (1995) and Harackiewicz et al. (2013). Steel and Aronson discuss ‘stereotype threat’ which they describe as thus: ‘It focuses on a social-psychological predicament that can arise from widely-known negative stereotypes about one’s group.

It is this, the existence of such a stereotype, that means that anything one does or any of one’s features that conform to it make the stereotype more plausible as a self-characterisation in the eyes of others, or even on one’s own eyes.’ (1995, p. 797).

For example, in one of the studies conducted for their paper, Steel and Aronson found that African American students who were asked to identify their race immediately prior to an intellectual test performed significantly worse relative to white students than African Americans who were not asked to identify their race.

Even this ostensibly innocuous question appeared to have a detrimental effect on performance (1995).

In other words, when considering the task at hand it may be possible that the underperformance of BME applicants was driven by a self-handicapping response when confronted with stereotypes within the SJT scenarios, or even perhaps prior to commencement.

‘Values affirmation’ is a theory predicated upon the idea that self-affirmation via an intervention can combat stereotype threat-induced self-handicapping. Harackiewicz et al. applied values affirmation interventions to first generation undergraduate students who, as a group, were found to be underperforming academically relative to their continuing generation peers (2013).

These interventions were presented as non-assessable 15 minute writing exercises timed early in semester then again just prior to exams.

Two groups of students were given a list of non-academic values such as ‘sense of humour’, ‘independence’ and ‘being good at art’. The values affirmation group was asked to circle the values most important to them and the control group was asked to circle the least important values.

The values affirmation group was then asked to write from their thoughts and feelings as to why these values may be important to themselves and the control group was asked why their chosen values may be important to someone else.

Overall, it was found values affirmation intervention narrowed the academic gap substantially between first generation students and continuing generation students (Harackiewicz et al., 2013).

In prompting the students to ruminate upon their self-integrity and self-worth, the intervention helped them recalibrate or reassess their self-esteem allowing them to apply themselves to the academic process with vastly reduced self-handicapping.

The strategy from BIT – more specifically Elizabeth Linos, senior advisor; Joanne Reinhard, advisor; and Simon Ruda, Principal Advisor: Head of Home Affairs, Security and International Development – to overcome the underperformance of BEM police applicants lay in creating a values affirmation intervention prior to testing, rather than in trying to modify the test itself.

The Solution

In the email that follows the Behavioural Styles Assessment inviting the applicant to sit the Situational Judgement Test, two very simple sentences were added. The first sentence congratulated the applicant for successfully completing the behavioural assessment component.

The second asked the applicant to: “take some time to think about why they want to become a police constable’ before commencing the SJT.

The first sentence is a demonstration of the type of positive priming which BIT have successfully used in other case studies. Though it doesn’t appear to be significant, in its absence the email (which I have not viewed) would have come across with less warmth and encouragment.

Most of the efficacy in this intervention, however, appears to come from the second sentence. It lies in an assumption of the applicant’s mindset; that the applicant has altruistic motives for wanting to join the police force and that accessing this altruism via a values affirmation prompt can override or negate any stereotype threat.

As the BIT case study states; ‘It is possible therefore that our intervention is helping BME applicants go with their gut instinct when answering questions, rather than what they expect the ‘correct’ answer to be.’

The Results

BME applicants who were subjected to the intervention were 20% more likely to pass the SJT than the BME applicants in the control group who did not receive the intervention.

The probability of passing the test with the intervention became effectively the same for BME applicants as it was for non-BME applicants. And significantly, while this intervention appeared to have a positive effect on BME applicants, there was no effect on non-BME applicants. A jigsaw-fit result.

The Intuitive Leap and the Burden of Proof

By necessity, the work of the BIT is founded upon theoretical and statistical proofs. As they state on their blog: “Often the evidence base consists of experiments that are conducted within a different context, aimed at different populations or use different outcome measures”.

The theories around stereotype threat and values affirmation are clearly far more complex than what I have been able to present to you today, as are the findings of the BIT in this specific case study.

But in coming to this solution a great intuitive leap also appears evident. This leap takes prior research and reduces the methodologies to the absolute barest minimum. Instead of multiple 15-minute writing exercises, the applicant is asked for a moment’s consideration.

In its sophisticated underpinnings and crystalline brevity, this extraordinary solution reminds me of a quote by Miles Davis: “It’s not the notes you play, it’s the notes you don’t play.”

References:

Steele, C. M., & Aronson, J., (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69(5). 797-811.

Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Giffen, C. J., Blair, S. S., Rouse, D. I. & Hyde, J. S., (2013). Closing the social class achievement gap for first-generation students in undergraduate biology. Journal of Educational Psychology, 106(2). 375-389.

Antony Giorgione is a freelance creative strategist