Just one question
Taking inspiration from the success of ‘TeacherTapp’, in response to COVID-19 we are now using an adapted version of our existing ‘Asking Good Questions’ survey tool to ask youth practitioners to respond to one key multiple-choice question each week.
What we do
How do I get involved?
Does your organisation have a theory of change for its work with young people?
[15/01/21 - 21/01/21]
This week we heard from 60 survey respondents, who could each select one answer.
Before we get started with this week’s analysis, we thought it might be helpful to clarify exactly what we’re talking about when we talk about a ‘theory of change.’ A theory of change is just that - a theory or set of ideas, developed by all those involved, about how provision creates change with and for young people. It’s both a process - of consultation, reflection and engagement - and an output - usually a diagram or image that captures:
- Aim: the long-term impact that young people and their communities achieve for themselves. This impact is usually experienced in the future, rather than ‘in the now’.
- Outcomes: the skills, knowledge, and assets that your provision supports young people to develop and that create the best conditions for young people and communities to experience the longer-term impact.
- Mechanisms of change: these are the experiences that young people have in your provision that affect a change in outcomes. They are often referred to as the ‘active ingredients’ in your work and are markers of high-quality delivery.
- Activities: what you do, how often, with whom, and in what setting.
As theory of change is not static! As you develop and refine yours, you may find that more questions arise, and gaps in your thinking and rationale may crop up. This is a very healthy sign that the process is working!
We’ve shared more thoughts on why we think a theory of change is important (and indeed essential) below.
How many organisations currently have a theory of change for their work with young people?
Just over half (52%) of respondents reported that they have some sort of theory of change - either an overarching one for their organisation (22%), or several different ones for specific areas of their work (30%), examples of which included a youth crime prevention project, and open access youth provision.
It is worth noting a potential bias: those engaging with the survey, and/or the Centre’s work more broadly, may be more engaged with evidence and evaluation, and therefore more likely to have a theory of change. However, the responses we received this week are similar to those we’ve seen on other projects where organisations might not be expected to be similarly engaged with evaluation, such as the Youth Endowment Fund Covid-19 Fund and the #iwill Fund Impact Accelerator.
A further quarter (27%) of respondents do not currently have a theory of change at their organisation, although a fair few comments noted that development work was in progress:
“We've had a couple of different iterations over the years, and the last set we developed two years ago weren't really adopted. However we're going through a ToC process at the moment, as part of our overall strategic review, and it has more senior buy-in, so I'm hoping the new ToCs (one organisation-wide ToC and some more specific ones for our offers) will be used more across the organisation.”
“We are planning to create an overarching Theory of Change this year as we've recognised the need for it. We want to bring our programmes under a unified approach, and define key concepts at an organisation level.”
One person also noted a particularly positive process of developing a theory of change with external support, involving young people, staff, trustees, and other key stakeholders.
A theory of change can help you to feel confident in knowing what is core to your work, and then adapting around those components as and when required - something that is particularly relevant right now. However, as highlighted by the first comment and noted above, writing a theory of change is only one step in this process. 15% of respondents this week do not know whether or not their organisation/provision has a theory of change, an indication that perhaps organisations aren’t embedding theories of change within their practice and processes, or sharing them widely with their staff and volunteers.
Of course, the coronavirus pandemic means that most youth organisations are working hard and fast to adapt their work. If you’re thinking about how your organisation’s theory of change relates to your practice in the current context, start by having a read of Research and Methods Lead Mary McKaskill’s reflections on how to engage with a theory of change in a way that supports intentional decision making and adaptations to delivery, specifically during the COVID-19 pandemic, originally shared back in June during the first lockdown. To support your thinking about how this extends to your evaluation planning and implementation, we’d also recommend BetterEvaluation’s three-part series on adapting evaluation in the time of COVID-19. NPC and Evaluation Support Scotland have also shared some really helpful reflections and recommendations on this topic.
Finally, we also received a few responses this week that indicated some people were both unfamiliar with the theory of change concept and sceptical about its potential value. Theories of change in youth work have never been uncontroversial, and - whilst they are becoming more widespread - there remain voices that challenge and question:
“What the f**k is that, we just do youth work”
“What's a "theory of change"? Sounds like nonsense to me”
In response to this, and building on the outline included at the start of this week’s reflection, we would add that a good theory of change should:
- Create time and space for you to reflect on your intentions and practice, alone and with young people and partners
- Help your team consolidate what you already know and uncover new questions to explore
- Explain why you do what you do, and what you do, clearly and succinctly
- Set out the ‘core’ and ‘flexible’ components of your work
- Help you design new projects and programmes with and for young people
- Provide a framework for learning and evaluation
Why is all of this important? There are lots of reasons, but it might include: building consensus on the ‘how’ and the ‘why’ of your work; communicating clearly to young people, partners and funders about your work; feeling confident defining what quality looks like - especially at times when activities need to be adapted; and providing opportunities for a wide range of stakeholders to be involved in shaping provision. There are plenty more, and we’d love to hear from you if you have any more reflections on how a theory of change has supported your work with young people.
If you are starting to think about a theory of change for your work and are looking for some guidance to support with this, check out this resource hub on our website, which includes a ‘theory of change toolbox’ that you can use for theory of change workshops within your organisation, as well as some background on how a theory of change will support you in asking key questions about your provision.
How prepared do you feel to continue offering provision for young people during the third lockdown?
[08/01/20 - 14/01/20]
We received 64 responses to this week’s survey. Each respondent could select one option.
52% of those who took part in the survey reported that they feel very prepared to continue offering provision for young people over the third lockdown, with delivery that they have already adapted. One respondent commented:
“Very prepared this time, good advice from Public Health and the NYA, good PPE, robust and reviewed risk assessments. Youth workers ready and willing. Shame about the weather.”
Another respondent also noted how they found the NYA alertness level guidance extremely helpful.
41% are feeling somewhat prepared, with plans in place but changes that still need to be made. A small number (3%) are feeling not at all prepared, and 5% told us something else.
Three main delivery challenges were flagged in this week’s comments. The first is a broad concern about limited resources, across a wide range of areas that include (a lack of) access to safe community building spaces, funding, and staff capacity. One participant commented that they were not currently receiving grants from major funders, and that they did not have the budget to qualify them for other grant streams, and another observed that staff in decision making roles within the youth organisation were ‘incredibly behind, focusing on other aspects of their work’ - perhaps emphasising the many demands on practitioners and youth organisations at this time.
Another practitioner highlighted a lack of capacity to meet young people’s individual needs:
“Every student we work with has such varied circumstances, it's so individualised and we don't have the capacity to be bespoke. I think our generic or blanket provision just won't work for everyone.”
Young people’s engagement and virtual fatigue also continue to make planning and delivering youth provision a challenge, particularly now that many young people are also now experiencing the school day from their homes and behind screens:
“We were really looking to meeting in person, as the groups were all zoomed out! This has affected some of our planned activities and some of our members are now not wanting to log on again, after a full day of on-line schooling!”
“We have the plans in place, however young people are increasingly not wanting to engage as much. They feed back they just want somewhere open access to relax with friends.”
“We can carry on, but it feels harder to keep the offer fresh and be innovative. Young people feel more demotivated and less enthusiastic about online delivery.”
Young people are not the only ones feeling this fatigue; the need to continually shift and adapt continues to take its toll on those who are offering the provision:
“Feel prepared for something and then the goal post gets shifted, so prepared for that and they move again. It's an ongoing battle that is feeling exhausting.”
On top of this, there is the emotional and moral weight of having to make decisions about safe delivery:
“We have a very dedicated team who are committed to delivering for young people but access to community buildings has become a challenge and also the responsibility to keep staff safe as well as deliver the services is a challenge.”
Some are still unsure about whether it is safe to be delivering permitted in-person activities:
“I think there has been good learning from Lockdown One - there remains a worry around the infection rate of the new strain so whilst we can continue to deliver we need to ask if we should?”
“Moral is the dilemma and proving to [the] community we are offering essential and much needed service to young people.”
This week’s announcement that youth workers currently holding or actively training for National Youth Agency (JNC)-recognised qualifications are classified as key workers is a positive step, and will hopefully provide support on this last point. However, there is no doubt that conditions for ongoing delivery are still very challenging.
A huge thank you to everyone who contributed a response this week. We’re shifting a gear with the next question and thinking about theories of change - does your organisation have one for its work with young people, either across the organisation as a whole or for individual activities? Let us know in the usual place, before 17.00 next Thursday 21 January.
Before Christmas, we asked ‘when do you think your provision for young people will return to how it was pre-pandemic?’
This survey was open between 11/12/20 and 07/01/21.
The situation across the UK has changed, furthermore, since we posed this question, as we now begin a third lockdown with restrictions much like those we saw back in March and April last year, and as vaccinations are rolled out at a larger scale. Whilst stricter measures for January were anticipated by many, would those who responded before Christmas respond in the same way today? As one participant commented, responding to this question “very much depends on government restrictions changing and vaccine roll-out being quick and effective. What are the chances?”
How did people respond?
A small percentage of respondents anticipated a return to their pre-pandemic delivery between January and March this year. This could be due to their unique context and type of provision (current National Youth Agency COVID-19 guidance does allow for small group or 1:1 sessions with vulnerable young people indoors.)
“We are piloting small youth group sessions within the building for the first time. In January we are hoping to extend that provision.”
In contrast, 13% do not expect to see a return to pre-pandemic provision happening in 2021 at all.
22% were hopeful for a return between April and June this year, the largest proportion of respondents anticipated this happening between July and September, and then 18% believed this would be most realistic between October and December 2021.
12% said something else. As ever, the additional comments shared in response to the survey highlighted the nuance and complexity that surrounds this topic.
Provision won't return to how it was pre-pandemic
Some practitioners anticipated resuming operations from spring or summer 2021, but expected that things will not be ‘back to normal’ until 2022, especially with some social distancing measures expected to stay in place for a while yet.
Others felt that provision will never look the same again. For some, there is the harsh reality of provision having to close permanently, or needing to invest significantly in rebuilding provision:
“Many of the voluntary clubs we support have lost volunteers and revenue which will force them to close permanently.”
“Hopefully some aspects should return sooner, but there will be much to rebuild across some locations/services that are suspended and some areas of work (e.g. youth volunteering).”
There is also a concern that the economic impact of the pandemic will lead to a further reduction in resources:
“I think even before the pandemic we knew that young people were disproportionately impacted by austerity, cuts, etc and my worry is that with Government talking about having to "pay it back" that young people, again, will be the losers and my big worry is that there will be more cuts on already devastated services.”
...and nor should it
Others noted that blended delivery is likely to be a more realistic way forward. For some, this is felt to be a good thing, especially where adapting delivery has led to new learning and positive changes to provision:
“I believe we will retain elements of online provision into the future, as it has been helpful to some of our young people and also offers an alternative method of contact?”
“We are aiming to return to pre-pandemic as soon as possible, maybe incorporating some changes from the things we've learned during the pandemic.”
“Never and nor should it. It’s changed how things are done and shouldn't go back.”
“We will continue to deliver remote and virtual programmes (something we had never done before March 2020).’
How long is a piece of string?
For some people, the situation still feels too unclear to estimate when provision might return to how it was before the pandemic, although the vaccine roll-out was felt to be a clearer indicator, especially for increasing the number of young people attending face-to-face group work.
“This very much depends on government restrictions changing and vaccine roll-out being quick and effective. What are the chances?”
“No clue, how long is a piece of string?”
“[...] I think face to face work will return to similar to pre-pandemic levels once the vaccine roll-out has been widened as most of our work is one to one or small groups.”
Another respondent detailed their hopeful plan for a phased return, continuing remote and virtual programme delivery in the immediate future, re-starting groups and clubs by March (at a smaller level and outdoors where possible), beginning camps and residentials by June, and resuming international work by August 2021.
They concluded by saying “we are hopeful, resilient and will do what we can.”
It is really difficult to plan and predict under the current circumstances, especially as contingency planning, increased risk mitigation, and responding to local and national contexts draws further on already stretched resources. The sector has responded and adapted, continuously, since this all began.
As well as demonstrating how tricky this is, this survey has also highlighted how practitioners are making the best of an incredibly challenging situation - drawing on learning gained from adapting provision - and maintaining hope that 2021 will see the return of some more provision for young people.
As we move onto the next survey, we are asking how prepared do you feel to continue offering provision for young people during the third lockdown? We recognise that this is about more than having a plan for delivery, with many other variables such as availability of staff and volunteers, local contexts, resourcing, and more.
We look forward to your responses and thank you, as always, for contributing to Just One Question.
Are you currently reaching the young people who need your organisation's services the most?
[13/11/20 - 19/11/20]
This week we heard from 63 survey respondents, who could each select one answer.
Before we dig into this week’s results, here’s a reminder of some of the headlines from previous surveys that have focused on reaching and engaging young people with provision:
- In late May/early June, 58% of Just One Question respondents flagged ‘reaching young people’ as their biggest challenge in moving delivery online. From the most recent Data Standard in June and July, 80% of sampled organisations also reported that they are either already providing an online and digital offer, or that they were expecting to within the next six months.
- In July, 19% of respondents were seeking deeper insight into the young people in their local area who are facing the most risks, and 21% wanted a better understanding of how to reach them.
- In September, 43% of Just One Question respondents had found that moving provision online has not enabled them to reach or engage with new young people (or if it had, it was just a few).
- In October, 24% of respondents felt engagement from young people would be one of their biggest barriers to delivery in the weeks that followed. 27% also felt that evaluation should focus on whether we’re reaching the young people who need us most.
We are not reaching the young people who need our organisation's services the most
Now, in mid-November, 46% of survey respondents say that they are not currently reaching the young people who are most in need of their organisation’s services. Many additional comments described how current lockdown restrictions are influencing this - for example, one practitioner reports how not being able to run open access sessions means that there is no opportunity ‘for some young people to self-identify that they need additional support’. Another person identified the value of young people being able to ‘congregate’ as a group and provide support to one another - something else that cannot take place right now.
For those working in partnership with others, referrals and outreach work is also being limited:
'We aren't receiving as many referrals as usual because other services don't have the capacity to complete our referral form (they are focusing on crisis work) so there may be young people who aren't being identified / referred to us support due to the impact of COVID-19 on other services, e.g. schools and social care.'
A previous survey in early October highlighted some of these challenges, particularly when collaborating with schools (‘Are you/is your organisation collaborating more since the start of the pandemic?’ 02/10/20 - 09/10/20).
For those operating some kind of online provision, there are continued concerns about young people who are ‘digitally excluded’ being left behind, or about those who have found that digital engagement does not work for them, and are therefore ‘hiding away.’ One practitioner shared specific concerns about young people who are staying at home, despite it being a place where they do not feel safe.
'We are completely digital and I am confident that we are meeting the needs of young peope that we have established relations with but I have concerns that we are not meeting the needs of young people that we are trying to establish relations with.'
One person highlighted their frustration at the situation.
We are reaching them, but...
One practitioner noted that whilst they are reaching the young people who need their organisation’s services, they ‘don't feel that [they are] able to fully meet their needs due to the restrictions that [they] are currently working within.’ For example, existing members might only be engaging through social media or in different ways to how they might otherwise. Another person spoke about how whilst they are ‘seeing more young people than ever’ they ‘cannot be sure [they] are not missing people.’ Someone else added that whilst they are reaching these young people, there are still many more who need their organisation’s service.
Increased targeting means that we are reaching these young people - but not everyone
19% of this week’s survey respondents do feel that they are currently reaching the young people who need their organisation’s services the most. Several people described how their work has taken on a more refined focus, such as detached work, 1:1 provision, specifically supporting young carers, or focusing on ‘vulnerable group work.’ This does however mean that not all young people who might ‘normally’ be supported are able to access provision:
'But only in the sense that we are hitting the absolute minimum targets. There is no capacity for extras of any kind.'
One practitioner also noted that their provision is currently very targeted as they can only operate in smaller groups - and that this meant that they are reaching those who need it most. However, they also added ‘at least those that we know who need our support the most!”
How do we know?
Related to this point, over a quarter of this week’s respondents (27%) are not sure whether or not they are currently reaching those young people most in need of their provision. The question this week was deliberately totally subjective. Another question we might ask is ‘how are you currently determining which young people need your organisation’s services the most?'
The situation right now is frustrating, as one respondent shared this week, uncertain, and highly changeable - back in August, 87% of surveyed practitioners expected to adapt their offer in some ways over the following months. Whilst ‘knowing’ and objective knowledge is particularly difficult right now, it’s still an important place to aspire to - particularly where it helps us to respond, adapt and reach out.
As well as trying to meet need and demand over the coming months, it will be a challenge to determine quite what this looks like: understanding the need for your provision, checking whether you’re reaching the young people you are most focused on, and/or measuring overall impact.
If you’re currently focusing on these questions, we’d love to hear more from you about how you are approaching this.
We'd also recommend taking a look at questions one and two of the Asking Good Questions framework.