PUBLIC RELATIONS
Tuesday 20th November 2018
In public relations correlation is not causation
Bill runs a small family owned toy shop that’s doing okay. He runs it himself and all his staff are family who all pride themselves on their friendly service. Then his daughter Ruth needs to go on maternity leave so he hires Lucy to cover for her. It’s a phenomenal success as sales the following month shoot up by more than 200%. Lucy is clearly a fantastic salesperson.
Except I haven’t given you all the data. Ruth started her maternity leave in November so the massive increase in sales probably wasn’t because of Lucy, it’s because Christmas is always Bill’s best month. In fact this December sales were lower than last December when Ruth was working in the shop. But that doesn’t necessarily mean Ruth was a better salesperson than Lucy as it could be that online competition had increased. It’s logical fallacy.
When I work with PR professionals to help them to improve how they measure and evaluate communications one of the most common mistakes I see is not taking sufficient account of the difference between correlation and causation.
The problem is when people look at some data that is accurate, but then interpret it wrongly. Sometimes it’s because people haven’t looked at enough data so interpret it based on what they can see, where additional data might lead to a totally different interpretation.
It’s human nature to jump to the easiest, most obvious, common sense conclusion. Unfortunately, because it’s the lazy approach it can often be wrong.
Tell ‘em about the honey, Mummy
John Webster was a founder and creative director of advertising agency Boase Massimi Pollitt (BMP) and is “one of advertising creativity’s towering figures”. He was responsible for some of UK television’s most memorable adverts including Cadbury’s Smash Martians and Jack Dee’s campaigns for John Smith’s bitter.
At a time when many advertising industry creatives eschewed research, fearing it stifled their creativity, he was renowned for embracing research data and using it to improve creativity. If John Webster hadn’t understood the importance of really understanding data we’d never have seen arguably his most famous creation – the Sugar Puffs Honey Monster.
The Honey Monster loved the honey in Sugar Puffs and broke things if it didn’t get any. Unfortunately, it was a massive failure in research. The data was crystal clear. Mums and children hated it.
But rather than accept the results at face value and shelve the concept Webster went back to delve into why they hated it.
Mums hated it because it deliberately destroyed things. Conversely, children hated it because it wasn’t much of a monster.
Understanding this Webster made the monster much bigger to please the children, while for the mums instead of its wilful destruction and naughtiness he made it clumsy.
The tag line “Tell ‘em about the honey, Mummy” has entered our national consciousness as one of the all time great advertising lines.
AMEC Measurement Month
AMEC (the International Association for the Measurement and Evaluation of Communications) Measurement Month appears to be a good time to clarify for communications and PR professionals what the difference is between causation and correlation.
Causation is action or occurrence that can cause another action or occurrence. The result of the action is always predictable and the relationship between the two is certain.
Correlation is an action or occurrence that can be linked to another.It doesn’t necessarily mean that the action is what caused the other action or occurrence. The possibility that the action caused it might be great, but it doesn’t have the certainty needed to show causation.
Confusing causation and correlation can have serious consequences and the mistake is often made even when huge amounts of money are involved.
In the 1990s New York city mayor Rudi Guiliani famously implemented the ‘broken windows’ approach to reducing crime. It says that ignoring one broken window will lead to more broken windows and a spiral of decline. Guiliani and his police chief Bill Bratton embarked on a crack down on petty crime such as graffiti believing it would create an atmosphere of lawfulness and therefore reduce serious and violent crime. It appeared to work as crime rates did indeed fall.
Other cities tried to emulate New York’s policies and success in reducing crime. It didn’t work. There were lots of factors other than Giuiliani’s ‘broken windows’ policy at play in New York. Crime had already started to fall under Giuiliani’s predecessor David Dinkins who’d also been responsible for expanding New York police department by 7,000 extra officers many of whom only came in when Giuliani took over. Other factors included falling crime rates nationally, unemployment fell and more controversially NYPD changed the way it recorded crime.
Spurious correlations
One of my favourite ways of illustrating spurious correlations is the wonderful Spurious Correlations website (and book) by Tyler Viglen.
Who knew that if you eat less margarine you’re less likely to get divorced?
Or that eating cheese increases the likelihood you’ll die by becoming tangled in your bedsheets (perhaps as a result of the nightmares of eating cheese just before bedtime).
The British Cheese Board has even done research to find out if eating cheese before bedtime really does give you nightmares. Unfortunately the British Cheese Board research doesn’t appear to take account of causation and correlation as despite 85% of female participants reporting that after eating Stilton they had “super-crazy, vivid dreams including talking soft toys” it doesn’t mean Stilton definitely gives you crazy dreams. In fact that’s an experiment you could try yourself at home.
Causation and correlations implications for communications measurement and evaluation
For communications and public relations professionals the challenge of causation and correlation makes it even harder to demonstrate the impact of their activity on the business or organisation they are working for.
Just being able to show a huge increase in the quantity and quality of earned media coverage and a corresponding increase in sales doesn’t actually mean the media coverage even contributed to the increase in sales. It could be like Ruth and Lucy and another far bigger, unrelated factor is at play.
There is an important ethical consideration to make as it is unethical to makes claims of success based merely on correlation. It’s also unprofessional as making the claim without knowing it is dubious calls into question ability to actually do the job.
How PR professionals can avoid spurious correlations
One of the many benefits of the AMEC Integrated Evaluation Framework tool is it provides a process which forces you to think hard about how what you’re doing relates to what happens next. Because the tool makes you list your activities you can then identify the outputs from each activity and if there are any corresponding outtakes (are people aware and thinking or acting differently) from the original activity. Following a clear process doesn’t stop you from incorrectly attributing an activity to a result, but it does make it far less likely.
If you or your team are interested in exploring how you can improve your public relations or communications measurement and evaluation then please get in touch to discuss how my consultancy or training services might help.
Except I haven’t given you all the data. Ruth started her maternity leave in November so the massive increase in sales probably wasn’t because of Lucy, it’s because Christmas is always Bill’s best month. In fact this December sales were lower than last December when Ruth was working in the shop. But that doesn’t necessarily mean Ruth was a better salesperson than Lucy as it could be that online competition had increased. It’s logical fallacy.
Post hoc ergo propter hoc | After this, therefore because of this
When I work with PR professionals to help them to improve how they measure and evaluate communications one of the most common mistakes I see is not taking sufficient account of the difference between correlation and causation.
The problem is when people look at some data that is accurate, but then interpret it wrongly. Sometimes it’s because people haven’t looked at enough data so interpret it based on what they can see, where additional data might lead to a totally different interpretation.
It’s human nature to jump to the easiest, most obvious, common sense conclusion. Unfortunately, because it’s the lazy approach it can often be wrong.
Tell ‘em about the honey, Mummy
John Webster was a founder and creative director of advertising agency Boase Massimi Pollitt (BMP) and is “one of advertising creativity’s towering figures”. He was responsible for some of UK television’s most memorable adverts including Cadbury’s Smash Martians and Jack Dee’s campaigns for John Smith’s bitter.
At a time when many advertising industry creatives eschewed research, fearing it stifled their creativity, he was renowned for embracing research data and using it to improve creativity. If John Webster hadn’t understood the importance of really understanding data we’d never have seen arguably his most famous creation – the Sugar Puffs Honey Monster.
The Honey Monster loved the honey in Sugar Puffs and broke things if it didn’t get any. Unfortunately, it was a massive failure in research. The data was crystal clear. Mums and children hated it.
But rather than accept the results at face value and shelve the concept Webster went back to delve into why they hated it.
Mums hated it because it deliberately destroyed things. Conversely, children hated it because it wasn’t much of a monster.
Understanding this Webster made the monster much bigger to please the children, while for the mums instead of its wilful destruction and naughtiness he made it clumsy.
The tag line “Tell ‘em about the honey, Mummy” has entered our national consciousness as one of the all time great advertising lines.
AMEC Measurement Month
AMEC (the International Association for the Measurement and Evaluation of Communications) Measurement Month appears to be a good time to clarify for communications and PR professionals what the difference is between causation and correlation.
Causation is action or occurrence that can cause another action or occurrence. The result of the action is always predictable and the relationship between the two is certain.
Correlation is an action or occurrence that can be linked to another.It doesn’t necessarily mean that the action is what caused the other action or occurrence. The possibility that the action caused it might be great, but it doesn’t have the certainty needed to show causation.
Confusing causation and correlation can have serious consequences and the mistake is often made even when huge amounts of money are involved.
In the 1990s New York city mayor Rudi Guiliani famously implemented the ‘broken windows’ approach to reducing crime. It says that ignoring one broken window will lead to more broken windows and a spiral of decline. Guiliani and his police chief Bill Bratton embarked on a crack down on petty crime such as graffiti believing it would create an atmosphere of lawfulness and therefore reduce serious and violent crime. It appeared to work as crime rates did indeed fall.
Other cities tried to emulate New York’s policies and success in reducing crime. It didn’t work. There were lots of factors other than Giuiliani’s ‘broken windows’ policy at play in New York. Crime had already started to fall under Giuiliani’s predecessor David Dinkins who’d also been responsible for expanding New York police department by 7,000 extra officers many of whom only came in when Giuliani took over. Other factors included falling crime rates nationally, unemployment fell and more controversially NYPD changed the way it recorded crime.
Spurious correlations
One of my favourite ways of illustrating spurious correlations is the wonderful Spurious Correlations website (and book) by Tyler Viglen.
Who knew that if you eat less margarine you’re less likely to get divorced?
Or that eating cheese increases the likelihood you’ll die by becoming tangled in your bedsheets (perhaps as a result of the nightmares of eating cheese just before bedtime).
The British Cheese Board has even done research to find out if eating cheese before bedtime really does give you nightmares. Unfortunately the British Cheese Board research doesn’t appear to take account of causation and correlation as despite 85% of female participants reporting that after eating Stilton they had “super-crazy, vivid dreams including talking soft toys” it doesn’t mean Stilton definitely gives you crazy dreams. In fact that’s an experiment you could try yourself at home.
Causation and correlations implications for communications measurement and evaluation
For communications and public relations professionals the challenge of causation and correlation makes it even harder to demonstrate the impact of their activity on the business or organisation they are working for.
Just being able to show a huge increase in the quantity and quality of earned media coverage and a corresponding increase in sales doesn’t actually mean the media coverage even contributed to the increase in sales. It could be like Ruth and Lucy and another far bigger, unrelated factor is at play.
There is an important ethical consideration to make as it is unethical to makes claims of success based merely on correlation. It’s also unprofessional as making the claim without knowing it is dubious calls into question ability to actually do the job.
How PR professionals can avoid spurious correlations
One of the many benefits of the AMEC Integrated Evaluation Framework tool is it provides a process which forces you to think hard about how what you’re doing relates to what happens next. Because the tool makes you list your activities you can then identify the outputs from each activity and if there are any corresponding outtakes (are people aware and thinking or acting differently) from the original activity. Following a clear process doesn’t stop you from incorrectly attributing an activity to a result, but it does make it far less likely.
If you or your team are interested in exploring how you can improve your public relations or communications measurement and evaluation then please get in touch to discuss how my consultancy or training services might help.