In this piece for our November 2016 newsletter, Kenton Hall, Communications Officer for the Centre, offers his reflections on "proving" and "improving" and the questions this discussion have raised within the youth sector.
Imagine for a moment that you know nothing about working with young people.
For those who have dedicated their lives to the study and practice of supporting better outcomes for young people, this may seem an impossible leap to make. It is too much part of the fabric of who they are; they are confronted, day-to-day, by challenges both personal and procedural, by the dichotomy between the complexity of working with individuals and the minutiae of enabling the provision of this work.
And, most importantly, there are actual young people involved, whose lives and futures can be affected positively or negatively as a result of that work.
It is, therefore, easy to understand, in what must often feel like an all-consuming role, the temptation to take sides, to seek allies, to push back against anything that feels as though it conflicts with your core goals or appears to further politicise an already demanding mission.
Within the youth sector, this often flourishes in a sense of conflicting agendas, or in some cases, the feeling that such an agenda has been imposed: to satisfy funders, to conform to a “new” way of doing things or, worst of all, to undermine or discredit instinctive and professional approaches.
The idea that seems to cause the most consternation, in some circles, is that of “proving impact”.
In what can be highly pressurised conditions, does this feel like criticism? A suggestion that the work has not made a difference?
To which, it may seem a natural response to ask:
“Why am I being asked to prove it, when I could just be getting on with doing it?”
“Of course I can prove it. I already know the work makes a difference.”
Sometimes, the concern is more about whether anyone is listening.
It is difficult to imagine, at first glance, why anyone who works with young people would be averse to the idea of improving the offer they make to them, the support they provide.
And yet, under challenge, has it become the case that the need to prove, whether willingly or otherwise, has displaced the desire to improve?
And if the key problem is that terminology, and the ideology it reflects, are obscuring shared goals, preventing us from acknowledging and embracing them, then perhaps it would be worthwhile to look at both ideas with fresh eyes.
How would it look if we aligned “proving” and “improving” along an axis that, broadly, equated to the “head” versus the “heart”?
Let’s take the position that proving is often seen as an exercise, a way of satisfying, or convincing, those on whom practitioners, often grudgingly, rely for funding and resources to do the work about which they are passionate. They put their “heart” into the day-to-day work of relating, of listening, of supporting real life ‘outcomes’: an improved quality of life for the young people with whom they work.
Being asked to prove the benefits of what they’re doing, to measure and demonstrate the impact of their work, may feel like a distraction and an imposition: further time subtracted from a clock that already seems to be running too fast. For others, it is antithetical to their approach; it fundamentally undermines and interferes with their relationships with young people. It is a manifestation of a political climate that emphasises competition and austerity.
But the work is important; the proof should be obvious: what they do matters. They can see the things – feel the things – they are being asked to prove.
Those who work in impact measurement, on the other hand, might feel a similar frustration with the suspicion, in some quarters, of their work. They see, just as clearly, the necessity, with such important work, to be evaluate, to ask systematic questions, to understand what makes a difference and to whom. By being able to ‘prove’ the impact the work is having and, perhaps most importantly, by going through the process of considering what ‘proof’ might mean, then not only can organisations explore, question and reflect upon their inner certainty that a difference is being made but also upon where the work can be improved, even evolved, to better support young people.
And so, it may feel that the head is concerned that the heart will miss something vital.
What would happen, however, if we reversed these positions? If we considered “proving” as a function of the heart and “improving” as a function of the head?
Can impact truly be measured or outcomes demonstrated without an emotive component? Without the drive, the passion, to do the best for the young people with whom you work, can sufficient honesty being generated to identify flaws, identify approaches that do not achieve what was intended, to thoroughly investigate processes to ensure that they are fit for purpose?
Or could it be that this is the source of the resistance? Is there an element of fear in play when asked to prove something, a fear that admitting a need for improvement and the associated uncertainty would be, in essence, an admission of failure?
Anyone who has ever shared knowledge with someone else, whether in a group setting or one-to-one, knows that one of the subsidiary benefits is that in the act of communicating knowledge, we refresh our own, we sharpen our understanding of our own practice.
So, consider the process of ‘proving’ impact, of measuring and communicating that impact to others, as an act of sharing learning and insight. You may have to demonstrate the impact of your work for what appear to be procedural reasons – to funders, for instance – but is this not an opportunity, a means of helping them to understand what you do and why, and, in turn, reflecting yourself on what you do that ‘works’ and where you can improve?
Conversely, those focused on impact measurement, on processes, tools and methods by which this can be refined, made more reliable, made more robust – must never forget that the result for which they’re working is not better measurement, but better outcomes. The eventual target of improvement is the quality of the young people’s lives.
With that in mind, a different sort of rigour may be applied to the scientific and academic approaches by which tools are developed and processes honed. “Improving” with all its potentially ephemeral qualities becomes an essential factor in the work. “Proving” can no more be a means unto itself within the academic community than it can be for those working with directly with young people.
These are, however, simply thought experiments, designed to make us think about two broad and, seemingly, contradictory ideas from a different angle.
Is there really that marked a divide between the head and he heart, between “proving” and “improving”? Surely, as in life, to achieve all of which both are capable, they must work together in concert?
The head and the heart must be joined.
We prove because it offers the opportunity to improve in ways we may not have otherwise identified. We improve so that the act of proving becomes less of an onerous addition to our work and more an inherent component of it.
Obviously, in practice, it would be short-sighted to imagine that any division in practice is easily bridged. The head and heart are, too often, prone to conflict. But they are part of the same organism. They inform and feed each other. They must admit and understand the limitations of the other, in order for the body to function.
It is much the same with “proving” and “improving”. If we start from the assumption that we are all working towards the same goal albeit, at times, using different methods and following different paths, then the act of proving and the art of improving can serve most effectively in tandem, for the benefit of the young people with whom we work.
In this guest blog for the Centre, Dan Barton, Senior Area Youth Worker for Devon Youth Service discusses his experience of measuring evidence in youth work and the questions and thoughts it has inspired.
If you are reading this, the value of youth work is a subject about which you either already have a strong opinion, or genuine questions. It is certainly discussed regularly across the sector, with passionate arguments being made for a variety of positions.
Personally, I think that, for years, we have been guilty of searching for a “holy grail” of outcomes, a “silver bullet” that would, finally, solve the eternal mystery of why the value of this work is often undervalued outside of the sector.
Conversations about teenage pregnancy, anti-social behavior, employability and, increasingly, radicalisation and child sexual exploitation are commonplace; people want to believe that there is a place for youth work but are often unsure as to where that place lies, or what form it should take.
The following questions, therefore, are ones we all need to ask ourselves: What part do we have to play in these strategic objectives? How do we contribute to these outcomes?
How can we stop young people being a problem?
The at times controversial “P” word has been less explicit in recent years, but it remains an important part of the puzzle.
Simply put, I would argue that it is impossible to practice good youth work and not meet these outcomes, or contribute in our own small way (and it is sometimes a small way, something we all have to learn to accept as part of this journey.)
I was reflecting to my wife last week that I never have to challenge my colleagues about their behaviour. I never feel uncomfortable. I never feel threatened, for instance, bullied or undervalued. Their values are intrinsically linked to youth work – inclusion, diversity, honesty, respect and a real sense of optimism for young people. As a group, we seem to know instinctively that quality youth work has value. We are lucky enough to see evidence of it every day and luckier still to have confirmation from the young people themselves.
For example, I can’t get through a day at work without seeing at least two or three of the following things happening:
Young people being spoken to in a way that is absolutely without a power or status divide, aiding a sense of self-respect and the development of trusting relationships.
Young people being given their third or fourth chance, when they tell us stories of how they have been isolated from other areas of their lives (school, home, work), aiding the development of resilience, belief and self-worth.
Young people being challenged to be more than they ever thought they could be: in the language they use, how they treat others, and how they expected to be treated by others, nurturing respect and tolerance.
Young people being offered responsibility – real responsibility, such as allocating funding, making decisions that affect their communities, taking an active role in social action. This fosters a sense of belonging to society to as a whole and to their communities.
Young people turning to us daily when others have let them down in their hour of need and receiving support that helps to increase their sense of worth.
There are many more examples, of course, but the demands of a word count will stop me from rambling.
Yet, despite this seemingly definitive, real-world evidence, we must contend with words and phrases that seem to defy the demonstrated value: ‘austerity’, ‘tough choices’, ‘we can’t provide the gold standard anymore’ and ‘doing more with less’.
Of course, when money is tight, who is going to simply take our word for it when everyone else is talking about their value too, and when judgments are being made on a sweeping scale about services and the contributions they provide? Again, perhaps we have been caught reaching for the gold when a sufficient amount of silver would accomplish more: the thousands and thousands of small steps that young people make every week.
Can we prove that we have stopped a young couple become pregnant (or even judge conclusively that this would be a positive outcome)?
Can we prove that our intervention was responsible for young person X gaining employment?
Can we state empirically that we were responsible for young person Y being less at risk of being sexual exploited?
This would be particularly difficult.
That's where the Centre for Youth Impact and Project Oracle come into our story. They understand youth work; they understand the small steps we take in working with young people. Crucially, they have the ear of funders, commissioners and service directors (who are willing to learn) and others that so desperately need to understand the value of our work.
We designed our Theory of Change (no mean feat), we drew up our evaluation plan, we designed or adopted recording and measuring tools and we started to tell our story in a different way – our way. Our impact measurement practice now fits our work. Developing the voice of young people and ensuring that it is heard remains a core expectation of our youth work.
We just record it differently now, in a way that allows us to gather the statistics and qualitative data that are relevant to us. Pride amongst our staff team has, in turn, improved. We know that our work does change lives, a little bit at a time, and we are learning more about how that happens.
We especially enjoyed developing our Theory of Change (‘what we do and how we do it’ as the staff called it). We liked discovering how we make assumptions in our work and owning them – allowing ourselves to unpick what has previously been regarded as ‘given’ or ‘implicit’ in our practice: talking respectfully to a young person for example – something young people tell us is unusual, believe it or not!
Working through the Standards of Evidence validation process with Project Oracle was quick and painless. We heard great ideas about how to tweak our frameworks and were given some tools to get the desired effects. We also had some help with our evaluation plan, which was great – I got to role-play, challenging the staff with things like “so what?”, “who cares?”, and my personal favorite, “we can’t say we do it if we can’t prove it”. Fine tuning our outcomes framework, looking at what we can measure against each area, figuring out creative ways to evidence each statement was very enlightening and great to share with others. I believe we have a collective understanding now of the above and its helps staff to focus their efforts on specific tools rather than use a scattergun approach.
I would certainly recommend embarking on this journey to any organisation that works with young people. You owe it to our staff and to your young people to do all you can to make quality youth work sustainable.
With potential changes to how relationship education could be delivered in schools and Scotland taking the lead in the UK for putting youth work back on the map, there has never been a more opportune time to combat the frustration that many are feeling, by everyone within our sector grabbing a song sheet and singing to the same tune.
Our monthly newsletter collects news, events, research and blogs from the Centre, our networks and practitioners and organisations around the world. Sign-up here