Building the foundations of a data literate youth sector: what we’ve learnt so far from the Youth Investment Fund (YIF) Learning Project

This blog was written by Sarah Williams, Research and Learning Officer at the Centre. As we enter the third and final year of the Youth Investment Fund (YIF), this blog reflects on what we have learnt about the collection and management of impact data, and how these lessons can be applied across the youth sector. The YIF is a £40 million fund from the Department for Digital, Culture, Media and Sport (DCMS) and The National Lottery Community Fund, supporting around 90 youth providers over a three-year period (2017 – 2020) to develop and expand their open access youth provision. NPC and the Centre for Youth Impact are leading the learning and impact strand of the fund.

In the early design stages of the YIF learning project, we met with the funded organisations, aiming to get a greater sense of their aspirations and areas of challenge. What emerged from these conversations was a strong sense that, whilst their data collection has advanced considerably over recent years, they often face challenges doing the data justice in terms of analysing it and really understanding what it means (which in turn will help them to take action). Data literacy – referring to the ability to derive meaningful information from data – is a fundamental yet often neglected factor in discussions about impact measurement and evaluation in the youth sector.

The existing data management landscape
Understanding and improving impact relies on appropriate and practical tools not only for collecting data, but also for storing, analysing, sharing, and ultimately using it to acquire meaningful and actionable information about a service or programme. There are a number of digital products or tools on offer that are used by youth organisations to manage their data (covering everything from collection to analysis), including customer relationship management (CRM) systems, case management systems, and specific data analysis tools (such as Microsoft Excel).

The specialisation of these digital tools towards a specific aspect of data management can result in a difficult balancing act for youth organisations, leading to frustration at the duplication of data entry tasks across multiple tools. Equally, the existing products that are on offer have been criticised for being: targeted towards the commercial sector (and therefore lacking relevance to a youth work setting); overly complicated and reliant on high levels of IT confidence (and therefore inaccessible to small or inexperienced youth organisations); and, perhaps most importantly for many, too expensive. Existing tools also have a major drawback from our perspective, which is that they are not usually designed to aggregate and share comparable data across multiple organisations.

It is as a result of all of these factors that there is no one tool or product that has been universally or even widely accepted by youth organisations as a central place to record, report, and understand their social impact. This has resulted in a fragmented data landscape, limiting opportunities to integrate, compare, and draw out wider trends, and disadvantaging many smaller organisations with limited resources.

Our approach in the YIF Learning Project
The YIF Learning Project takes a broad approach to understanding impact, focusing on five types of comparable data collected from across the cohort of around 90 grant holders (you can read more about the YIF evaluation approach here). As you can imagine, our approach generates large quantities of data, and so we have been careful not to underestimate the role and importance of digital technology. From the outset, the learning partners included an ambitious objective to provide a digital platform that enables the collection, analysis, and sharing of data for the organisations involved, all in one centralised and consistent place.
The digital platform that we are using has come a long way in its development, and it is continually being adapted and improved to optimise its value for the grant holders. There have been some challenges along the way, and we have relied on honest and constructive feedback from the users to inform the development of the tool. Based on this experience, below we highlight some core values that we believe are most important for effective data management.

1. Simplicity: we know that there is significant variation in skills and experience of data-handling across the youth sector, and this variation is reflected strongly amongst the YIF grant holders. When digital platforms are slow to operate, clunky, and require second-guessing, both the quality and the value of the data diminishes – if delivery organisations use them at all. We need to emphasise and focus on the user journey, to develop digital tools that are accessible even for small organisations operating with part-time staff and volunteers, and to provide regular opportunities for training and troubleshooting.
2. Utility: youth organisations need to be able to analyse and get reports on their data in real time, swiftly generating new insights and perspectives on their work. This is important for moving towards a youth sector where impact data produces real value as the basis for decision-making and improvement, rather than to be used merely externally as ‘proof’ of impact achieved.
3. Security: the introduction of new EU GDPR standards in 2018 has created much greater awareness and concern about data protection, and there is no doubt that this has had an influence on attitudes and behaviours in the youth sector. It is, therefore, essential that digital tools offer transparency and reassurance regarding what happens to the data, who has access and ownership, and the measures that are taken to keep data secure.
4. Interoperability: switching between data management systems poses a large yet sometimes unavoidable burden. This challenge arose in the YIF learning project, when we asked grantees who were already invested in the use of an existing data system to also use the digital platform for the YIF. To address this, we have engaged with other platform developers to create functions for the easy transfer of data-sets from one platform to other. More widely in the youth sector, we need to ensure that data-management tools are as interoperable as possible, so that we can meaningfully compare and collate data from diverse contexts, and recognise the value of different types of youth provision.

Looking ahead
We are excited to continue to support the data literacy of YIF grant holders, and to learn from their experiences to inform data-driven behaviours in the sector more widely. We believe that building the foundations of a data literate youth sector is essential for enhancing the extent to which we can meaningfully learn and improve.
We will continue to share learning from our experiences in the YIF evaluation and to openly reflect on the challenges faced along the way. If you have any questions about this blog, or the YIF evaluation approach more broadly, get in touch with yiflearning@thinknpc.org.