Just one question
What we do
This platform allows users to register and receive immediate feedback on how others have responded, as well as a follow-up ‘final score’ via email once the survey has closed, and reminders to complete each weekly survey.
How do I get involved?
What is the number one thing that would make evaluation a really useful experience for you in your work right now/moving forward?
(09/04/21 - 15/04/21)
This week’s survey looked at how youth practitioners are currently experiencing evaluation in their work. We received responses from 47 practitioners, who could each select one response.
‘More time to think about it’ was the top response (30%), followed by ‘less pressure to do things for different funders’ (23%) - although additional comments indicated that others who did not select this response would perhaps have selected this jointly with ‘more time’, and also that there is overlap between the two responses:
“Only let me chose one (otherwise i would have chosen funders as well)”
“A consistent approach would simplify the burden we face to report in so many different ways to different funders. Time invested in improving our ability to record and provide that data would then be effective.”
“Connection between evaluation results, impact, inclusion and how it influences change - often feels disjointed and there is limited control to be responsive to evaluation.”
There are clear recommendations here: more consistency in approaches and expectations from funders, more time to invest in these consistent approaches and to enable ‘responsive’ evaluation, and a more collaborative system whereby practitioners feel that they can take ownership of and use evaluation efforts to influence change. “A well thought out system that is adaptable to both the individuals and organisation”, is how one participant described it. Another contributor called for greater guidance at a national level:
“Greater national direction on tools to be used to help develop the picture of what works in youth work, to enable greater conversations across the sector and a shared understanding & common language around evaluation.”
‘Training in tools and techniques so I feel more confident’ was the third most popular response, selected by 17% of people.
Several participants also touched on some of the more foundational issues around how evaluation is perceived and experienced. One person needs evaluation ‘to be reflective’ and wants to see ‘ongoing reflection at all times.’ Another participant also wanted to jointly select ‘less fear of failure’, which was the ‘number one most useful thing’ selected by a small percentage of this week’s contributors (4%). One more person shared an experience of how evaluation had taken on a less helpful role at their organisation:
“Evaluations have turned into a competition within our service who gets the most engagements, safeguarding issues, trauma, drama etc winner keeps their job.”
This week’s survey was a repeat of a question that we asked back in February 2020, as part of the Asking Good Questions survey (before we adapted the platform for Just One Question). To see how this week’s responses compare to that, and for a deeper look at some of the issues and topics highlighted above, take a look at this month’s Our Thoughts.
What training would be most beneficial to you right now?
(25/03/21 - 01/04/21)
For this week’s survey, we repeated a question from back in July 2020. At the time, multiple studies and surveys had highlighted a need for increased training for practitioners who were adapting, rapidly, to the new context of youth work delivery.
Over a year since lockdown began, practitioners have now been delivering in this ‘new normal’ for some time. New methods have been developed, tested, and shared, and in many places we’ve seen a proliferation of opportunities for online training and networking; in part due to need, as well as increased accessibility and the removal of some barriers with virtual training provision. See surveys from January and February this year for more thoughts on these topics.
At the end of January this year, 46% of survey respondents told us that they’d been able to join more useful training and support before the pandemic - but as many said that they only want to join if it’s really important or relevant. 13% wanted to join in with all of the training and support opportunities that were available. 12% reported that they had less time training and support than before the pandemic hit, and 13% had been flooded with opportunities that they were not interested in.
With this in mind - what does relevant and useful training look like for today’s context?
52 respondents shared 104 responses to this week’s question.
The most popular training topics flagged this week were on meeting the increased needs of young people as a result of the pandemic (29% of respondents), digital youth work (23%), more advanced safeguarding (23%), and data collection and evaluation (21%).
17% indicated that they don’t need any more training right now.
15% would like support in shifting to long-term blended delivery. Other topics included managing teams remotely (13%), digital facilitation (12%), responding to feedback from young people (12%), and improving the quality of our youth work (10%).
We saw the smallest number of responses for training in detached and street-based youth work, specific practices for how to work with young people (e.g. creating safe spaces), and how to follow social distancing rules and regulations. This is a notable shift from July’s survey (see more below).
10% shared ‘something else.’ Specific training topics that were highlighted include:
- Working with volunteers
- Practical examples and ideas for (indoor) socially distanced youth work
- HR (human resources) management
- Outdoor learning and engagement
Two additional comments were also shared in response to this week’s question:
“Ensuring we have the skills to support the whole team in going from COVID-19 restrictions to the new normal.”
“Leadership that allows BAME youth workers to get jobs with youth sector SMT positions etc - the % of BAME staff in position is so poor and no one seems to doing anything about it just talks.”
How does this compare with last July?
As the number of respondents varied between July 2020 and April 2021, we have used the ‘% of respondents’ metric for the comparison graph below. We also added a few new responses to this week’s survey (‘improving the quality of our work’, ‘shifting to long-term blended delivery’, and ‘data collection and evaluation’) which are not included.
Here are some key reflections from what’s changed over the past 10 months:
- Key priorities have not changed - the two most popular responses at both time points were meeting young people’s increased needs as a result of the pandemic and digital youth work. A greater proportion of all respondents selected these answers in the first survey, however.
- Confidence in other areas has shifted. In April 2021, fewer practitioners expressed a need for training about how to follow social distancing rules and regulations (14% to 4%), detached or street-based youth work (21% to 6%), and specific practices for working with young people, e.g. creating safe spaces (28% to 6%). There’s also notably less demand for training in digital facilitation (27% to 12%)
- This time around, a greater proportion of respondents also indicated that they don’t need any more training right now (7% to 17%).
For both surveys, respondents could select up to three responses, although the average number of answers per respondent was two.
Does this resonate with your experience? Let us know - firstname.lastname@example.org.
For our next survey, we’re going to be digging into the response about training in data collection and evaluation by asking what is the number one thing that would make evaluation a really useful experience for you in your work right now/moving forward?
Choose two words to describe how you’re feeling about your work with young people right now.
[12/03/21 - 18/03/21]
We’ve asked this question twice before, first in July 2020 and again in September 2020. We know that as lockdown conditions have eased, tightened, and eased once more, practitioners' feelings towards their work with young people has varied significantly. We were interested in gauging the current ‘mood’, given that lockdown conditions are slowly set to be lifted over the coming months.
This week we heard from 75 practitioners, and received a total of 104 responses.
The comparison graph below shows how responses have changed since mid-July (presented as a % of responses).
Looking across the last eight months, we can see a slight decrease in overall positivity from practitioners towards their work with young people. Whilst feelings of motivation, frustration and excitement are the most prevalent feelings at all three time points, reported excitement has remained lower, with 21% and 20% of respondents reporting this in September and March respectively, vs 24% in July. Levels of frustration have also increased, from 15% in July, 19% in September, to 22% reporting feeling frustrated in our latest survey. More positively, practitioner’s motivation has increased significantly, with 22% of respondents feeling motivated, compared to just 16% in September.
Respondents related frustrations to the fact that it is ‘very difficult to plan ahead’ and there is a continuous need to be ‘responsive’:
“Young people respond really well, but our systems and processes are not as fleet of foot as they need to be to meet their needs”.
This notion of constantly needing to respond but struggling to match the pace of change and young people’s needs, was echoed in respondents’ own ‘something else’ answers to better convey what they are feeling: including words such as ‘tired’, ‘powerless’, and ‘uncertain’. Two respondent’s also alluded to impact of ongoing capacity and financial challenges on their feelings towards their work, particularly being perceived to have slipped to the ‘bottom of government spending priorities’ and youth services funding being cut.
These negative feelings were part of conflicting or ‘tumultuous’ emotions, and one respondent suggested:
“Most of the suggested words could apply at differing times and in different ways but the underlying feeling is the importance of the work.”
Despite some of the frustrations and continuing feelings of uncertainty, it is clear that as a reduction of social distancing measures appears to be in sight, many practitioners feel less anxious around their work with young people. Reported nervousness had continued to decrease, from 11% back in July, to just under 4% in this week's responses. Whilst confusion (understandably) peaked in September with the ever-changing rules around face-to-face delivery, just 2% of respondents described feeling confused this week, suggesting there is much more clarity in the guidelines around delivery. Feelings of preparedness have also gone up, with just 5% of respondents reporting feeling prepared in September, vs. 11.5% this week. It suggests that tools such as the NYA COVID-19 Guidance have been well received and utilised. It also suggests that as the road map out of lockdown progresses and organisations have consolidated some of their learning over the past year, organisations do feel more ‘ready to go’. As one respondent reflected:
“Now is the time to be excited about creating new opportunities alongside young people”
Whilst overall this week’s responses suggest that practitioners are feeling more confident around the practicalities of delivery and motivation has increased, levels of frustration are still high, and feelings of stress are hovering at around 10%. We know that the next six months or so are not going be easy, and we will continue to focus on how we can support organisations - through the information and tools we share, and conversations we facilitate - to keep motivation and excitement levels high, and reduce levels of stress or confusion.
Has digital delivery widened your geographical reach?
[05/03/21 - 11/03/21]
Comments from previous surveys indicated that some practitioners were finding that they are now reaching a wider range of young people through their provision, due to moving some or all activities online. We were interested in finding out whether this was the case more widely.
This week 52 practitioners shared a response.
From this week’s survey responses, the picture is fairly split; 42% of respondents report that they are now reaching young people from further afield, whilst 40% share that their geographical reach is the same. 12% already had a wide geographical reach, and a small percentage don’t know or provided a different answer.
The potential for increasing reach in this way will vary, of course, depending on an organisation’s type of provision, capacity, and resources. One participant commented that their organisation’s funding limits them to a particular geographical area, so they would not be able to widen their reach even if it was practically feasible to do so. Another participant does not do so much digital delivery, so this question is less applicable for them.
On the converse, one respondent shared that digital delivery has enabled them to develop links with projects from around the country, meaning that the young people they support are now joining online groups from other areas of the UK - and beyond. This is supported by an increased ability to link staff up across geographical areas for networking (an opportunity that also came through clearly in a previous survey about areas of work that are working better online). Another person added that they have been engaging groups at an international level.
Finally, a wider geographical reach might not be based solely on digital provision - one participant commented that they have managed to increase capacity of locality based staff, which could also be said to increase their geographical reach.
Are the young people who are engaging with your online provision now the same ones who were engaging at the start of lockdown?
[26/02/21 - 04/03/21]
This week we received 52 responses. Participants could select one answer each.
The question for this week builds on responses and feedback from a number of previous surveys:
- In two early surveys from late May last year, 56% of respondents reported that fewer young people were engaging with online provision compared to face-to-face, and there were mixed experiences of whether the same or different young people were engaging regularly each week.
- Then, in November, 47% of respondents told us that they did not feel that they were reaching young people most in need of their organisation’s provision.
- Finally, last month in February, 47% of respondents and many comments reported that young people were less keen to join activities online due to digital fatigue.
A major concern shared at the start of lockdown was around the limitations of building and maintaining relationships with young people, whilst pre-pandemic provision was not possible and sustained engagement felt more challenging.
Almost a year into lockdown restrictions, the picture of young people’s engagement with provision continues to be mixed. As one respondent commented, “some [young people] have been retained, some have dropped off and some young people have come on board.” 20% of respondents report that they are currently still engaging with the same group(s) of young people that they were at the start of lockdown, and 50% report that they are engaging both the same groups and new young people. Responses this week indicate some consistency and continuity, although we cannot draw conclusions from these figures alone (more might be learned through organisational user and engagement data, for example).
17% are engaging the same groups, but have found that overall, engagement has dropped off. One person commented “numbers attending have reduced substantially” and another also noted that “some levels of engagement [are] unclear in the more 'anonymous' world of social media.”
A small percentage (6%) are now engaging totally different groups. As would be expected, it is not possible to move all types of provision and engagement online - those who primarily worked with young people through detached work, for example, are not easily able to move this engagement online. For some, adapting provision has led to new groups engaging:
“Because we have changed our offer as the first offering was not working. We are attracting new groups .”
At the same time, there are large groups of young people not accessing provision:
“We are seeing [an] increasing number of young women engaging whereas take up from young people with refugee status has tapered off. This is particularly at the referral stages of our services and provision. We suspect that this has to do with the digital provision changing accessibility for certain groups but are looking into this further in the next few months. In terms of our group work provision we see smaller numbers engaging but relatively more young women participating.”
“We're actually reaching a far more diverse group of young people than before - care-experienced, different geographical regions - all as a result of the move online. It's superb to see. And we've focused on designing our programmes to be accessible, recognising some of the barriers young people face. Young people have been calling in, have been joining by text, video and all sorts of things. That has made a huge difference - accessibility is at least as much about design as it is device.”
Challenges with accessibility continue, but comments such as those above demonstrate ways in which practitioners and organisations are prioritising and responding to these through their work.
We are not reaching the young people who need our organisation's services the most
Now, in mid-November, 46% of survey respondents say that they are not currently reaching the young people who are most in need of their organisation’s services. Many additional comments described how current lockdown restrictions are influencing this - for example, one practitioner reports how not being able to run open access sessions means that there is no opportunity ‘for some young people to self-identify that they need additional support’. Another person identified the value of young people being able to ‘congregate’ as a group and provide support to one another - something else that cannot take place right now.
For those working in partnership with others, referrals and outreach work is also being limited:
'We aren't receiving as many referrals as usual because other services don't have the capacity to complete our referral form (they are focusing on crisis work) so there may be young people who aren't being identified / referred to us support due to the impact of COVID-19 on other services, e.g. schools and social care.'
A previous survey in early October highlighted some of these challenges, particularly when collaborating with schools (‘Are you/is your organisation collaborating more since the start of the pandemic?’ 02/10/20 - 09/10/20).
For those operating some kind of online provision, there are continued concerns about young people who are ‘digitally excluded’ being left behind, or about those who have found that digital engagement does not work for them, and are therefore ‘hiding away.’ One practitioner shared specific concerns about young people who are staying at home, despite it being a place where they do not feel safe.
'We are completely digital and I am confident that we are meeting the needs of young peope that we have established relations with but I have concerns that we are not meeting the needs of young people that we are trying to establish relations with.'
One person highlighted their frustration at the situation.
We are reaching them, but...
One practitioner noted that whilst they are reaching the young people who need their organisation’s services, they ‘don't feel that [they are] able to fully meet their needs due to the restrictions that [they] are currently working within.’ For example, existing members might only be engaging through social media or in different ways to how they might otherwise. Another person spoke about how whilst they are ‘seeing more young people than ever’ they ‘cannot be sure [they] are not missing people.’ Someone else added that whilst they are reaching these young people, there are still many more who need their organisation’s service.
Increased targeting means that we are reaching these young people - but not everyone
19% of this week’s survey respondents do feel that they are currently reaching the young people who need their organisation’s services the most. Several people described how their work has taken on a more refined focus, such as detached work, 1:1 provision, specifically supporting young carers, or focusing on ‘vulnerable group work.’ This does however mean that not all young people who might ‘normally’ be supported are able to access provision:
'But only in the sense that we are hitting the absolute minimum targets. There is no capacity for extras of any kind.'
One practitioner also noted that their provision is currently very targeted as they can only operate in smaller groups - and that this meant that they are reaching those who need it most. However, they also added ‘at least those that we know who need our support the most!”
How do we know?
Related to this point, over a quarter of this week’s respondents (27%) are not sure whether or not they are currently reaching those young people most in need of their provision. The question this week was deliberately totally subjective. Another question we might ask is ‘how are you currently determining which young people need your organisation’s services the most?'
The situation right now is frustrating, as one respondent shared this week, uncertain, and highly changeable - back in August, 87% of surveyed practitioners expected to adapt their offer in some ways over the following months. Whilst ‘knowing’ and objective knowledge is particularly difficult right now, it’s still an important place to aspire to - particularly where it helps us to respond, adapt and reach out.
As well as trying to meet need and demand over the coming months, it will be a challenge to determine quite what this looks like: understanding the need for your provision, checking whether you’re reaching the young people you are most focused on, and/or measuring overall impact.
If you’re currently focusing on these questions, we’d love to hear more from you about how you are approaching this.
We'd also recommend taking a look at questions one and two of the Asking Good Questions framework.