Evidence is power?

This blog was written by Polly Neate, CEO at Shelter. Polly delivered the opening keynote address at the Gathering 2018, which explored the role of power, impact and change in the UK youth sector.

I used to be convinced that the first question you should ask of any service for people who need care or support should be whether it is evidence based. I was a journalist specialising in social policy in the 1990s, and I thought people ignored the importance of evidence. In fact I thought evidence could save some ideas that I saw disappearing – particularly in young people’s services. I still thought this when I went to a large children’s charity in 2005. It’s interesting to me that when I started to question it was when I moved to Women’s Aid, and the organisation and sector I was in had less power: less money, less influence, less credibility. Because then I faced those questions myself and didn’t have the resources to answer.
 
I’ve now come to believe that the search for the perfect evaluation can blind us to what we know, or stop us understanding what’s important. For example, during the time I was at Women’s Aid, a highly regarded review of domestic abuse services omitted anything that didn't meet the most stringent evaluation standards, and then concluded that there’s no evidence to support women-specific services. This was used as justification for decommissioning a large number of services for women – it was not only wrong but dangerous.
 
We need to recognise that evidence is a security blanket for powerful actors. For example, commissioners often have no expert knowledge, and they are weighing evidence of impact and price. A larger, wealthier organisation that has both an evaluation (whatever its quality) and economies of scale will win. Many commissioners don’t ask what influence the people most affected had over the evaluation – often they were merely asked the questions that those with power thought should be measured. Which is nowadays often the so-called risk factors.
 
In any case, though it lends power to some, evidence itself has much less power than we like to think. People stick with their beliefs and preconceptions despite credible evidence that refutes them. To shock someone out of a belief and into a new one is incredibly difficult and usually facts won’t do it. People with power can choose to stick to their beliefs despite the evidence (even the evidence they themselves demand), and this affects everyone they have power over – that's how public policy works. Changing policy is often not about persuading policy-makers that their belief is incorrect, but that enough people disagree with them for it to threaten their power.
 
Which brings me to the other type of evidence journalists, politicians, funders and commissioners love, which is stories. Stories trigger emotions that make us want to believe the statistics – or not to.  If you have an illness and you’re told that 25 per cent of people recover, you will feel terrible. If, instead, you’re told the story of someone who overcame that illness, you’re inspired and more optimistic.
 
Individual stories matter. Successful people have stories about how they got to where they are in life. The stories disguise the truth that in reality it’s impossible to tell whether they got where they are by their own merits. Usually it was privilege. It’s easy to prove, with evidence, that our society is run by people who have no particular qualities to qualify them other than privilege, but this is routinely ignored by tacit agreement. And by the same token, we don’t want to think that people who have no power, whose lives are spectacularly unsuccessful by any popular measure, are no different from us.
 
Those of us with power make people who are relatively powerless tell us their story all the time. We can choose when and how to tell our story, they can't. They have to do it again and again. You need a positive story of developing resilience, gaining self-esteem, improving behaviour, the story the powerless have to tell the powerful about themselves in order to be valued and to be given the things we all need, like money, a home.
 
But many people can’t own a simple narrative – instead it’s complex or episodic. How do we measure the success of an “intervention” with people who don’t have a story? Much more importantly, what are our moral obligations to them? Do we only have an obligation to help if we can prove that it works? Even worse, in the age of social return on investment, do we only help in ways that save the state money?
 
That’s why I have a big problem with the power given to “what works”.  I think “what’s right” is more important. I know that sounds frightening and a bit evangelical – but to me it’s baffling that we’re more scared of the notion of objective morality than the subjective selection of so-called facts. Very often, it’s society that needs to change, not individuals who have a set of so-called “needs” or – even worse – are creating a risk that needs managing.
 
We need to be very careful we’re not sleepwalking into a dystopia in which we meet needs, manage risk, and save money – while leaving power structures unchallenged and letting social injustice continue to flourish.