YPQI - Frequently Asked Questions
What Do YPQI, PQA and SEL stand for?
YPQI - Youth Programme Quality Intervention
PQA - Programme Quality Assessment
SEL - Social and Emotional Learning.
What is the YPQI?
The YPQI is a continuous quality improvement process designed to support organisations working with young people to improve the quality of what they do, and thus the impact of their work on the lives of young people. It has been designed with, tested and used successfully by thousands of youth organisations in the US.
The process involves a cycle of three stages: Assess, Plan and Improve.
Each cycle takes place over six months.
Over 40 years of research on US youth programs by the High Scope Educational Research Foundation and the David P. Weikart Center for Youth Program Quality has demonstrated the characteristics of high-quality youth provision. This recipe includes a safe environment, a supportive environment, positive interactions, and engagement with young people. These four ingredients can be measured and improved upon through reflection, observation and building the skills of staff within youth organisations.
You can find research on the development of the PQA here (Smith et al, 2016), as well research on PQA used for continuous quality improvement here (Smith et al, 2012).
What does ‘Assess, Plan and Improve’ involve in practice?
- The Assess stage involves a team within a youth organisation using the PQA tool (described in detail below) to arrive at a set of ‘scores’ for a particular service, project or activity they run.
- In the Plan stage, the team uses these scores to identify areas of their practice that they want to focus on improving.
- Finally, in the Improve stage, the team receives training and support to help them improve those areas they have identified as priorities.
The cycle then begins again, to give the team the opportunity to assess whether they have achieved their improvement goals, and identify what they want to focus on next.
What is the Programme Quality Assessment (PQA)?
The PQA is the tool that sits at the heart of the YPQI, and is used in the first stage of the process: ‘Assess’. There are several versions of the PQA, but we are using the version that focuses on social and emotional learning (SEL). The PQA is a quality framework used for observing and scoring the quality of practice. It involves a ‘self-assessment team’ of staff in youth organisations observing one another’s work against a detailed list of staff behaviours (which are called ‘items’) that promotes SEL amongst young people. Members of the self-assessment team observe and score each other for each of these items, then discuss the scores together, to see where they are doing well, and where there is room for improvement.
What does the PQA measure?
The PQA measures the presence of quality markers – referred to throughout as ‘items’. These are observable behaviours, demonstrated by staff working with young people, that influence the quality of provision. The PQA groups these items under four domains: safe space, supportive environment, interactive environment and engaging environment.
The PQA has a simple system of scoring either 1, 3 or 5 for each item in the four domains. This scale corresponds to how often a behaviour is observed (for example, whether a team member does it all the time for all young people, or just some of the time or for some young people) and/or how authentic/meaningful a behaviour is (for example, when young people are offered a choice, is it a truly open choice, or just a selection between options the staff member has already decided upon).
What do the PQA ‘items’ look like?
Below are some examples of items: for each one, a description of staff behaviour is given for a score of 1, 3 and 5.
1 Staff do not circulate to interact with young people during the session.
3 Staff circulate and interact with some (but not all) young people at some point during the session.
5 Staff circulate and interact with every young person at some point during the session
(one of eight items in the Safe Space domain)
1 Staff do not acknowledge, validate, or name young people’s emotions.
3 Staff occasionally acknowledge, validate, or name young people’s emotions.
5 Staff consistently acknowledge, validate, or name young people’s emotions (e.g., “It seems you are disappointed that you didn’t get the part.”).
(one of 11 items in the Supportive Environment domain
1 Staff frequently take over or intervene intrusively in assigned tasks or activities.
3 Staff occasionally take over or intervene intrusively in assigned tasks or activities.
5 Staff never take over or intervene intrusively in assigned tasks or activities, supporting young people to carry out roles or responsibilities as independently as possible.
(one of 12 items in the Interactive Environment domain)
1 Staff do not provide opportunities for young people to make choices.
3 Staff provide opportunities for young people to make at least one choice within the framework of the activities, but the choices are limited to discrete options presented by the staff (e.g., “pick one of the following topics,” “do it this way or that way.”)
5 Staff provide opportunities for young people to make at least one open-ended choice within the framework of the activities (e.g., “pick any topic,” “use these materials any way you want”).
(one of 13 items in the Engaging Environment domain)
What does “low stakes accountability” mean?
Low stakes accountability is an approach intended to move away from ‘high stakes’ accountability regimes, where there is fear of ‘failure’, where individuals can be penalised or publicly shamed, where targets may be unattainable and produce perverse incentives, where compliance is seen as mandatory but not helpful and where there is little or no support to improve.
By contrast, low stakes accountability means that no one will be penalised for one ‘low score’, and standards set are attainable for all. Support is available to improve and the process is continuous. Those who participate see it as a good use of their time.
In the YPQI, this means that:
- individual PQA item scores are used only by self-assessment teams, in the process of agreeing overall scores for the team as a whole.
- behaviours included in the PQA framework are achievable for all team members, and there is dedicated support to improve.
- data relating to the practice of individual staff are not used in any individual performance management processes, or shared with anyone outside of the self-assessment team.
Low stakes accountability is a large part of what makes the YPQI different to other quality assurance processes.
How do the ‘Plan’ and ‘Improve’ stages work?
Self-assessment teams receive dedicated training in using data gathered through the PQA to plan their improvement efforts. Based on this plan, the team then receives training and coaching support to develop specific areas of their practice with young people.
The YPQI process is cyclical, meaning that there is no one-off assessment or set standard to reach. Sites are encouraged to assess, plan for improvement, implement these plans and assess again, on an ongoing basis.
What’s the difference between a site, a setting and an organisation?
A site is the location that is engaging in the YPQI process. It may be one location out of many where an organisation operates, or it may be the only delivery location. The site is where the observation takes place. The observation generates site-level data through the PQA.
The setting refers to the space or place where staff teams interact with young people. The PQA measures items of quality within a specific setting.
The organisation is the charity, social enterprise or local authority (for example) that oversee and leads provision in the site, and is seeking to learn and improve its practice through the YPQI. One organisation may run a number of different sites, and there may be more than one setting at each site.
What’s to stop a team ‘gaming’ the self-assessment and entering scores that make them look good?
Nothing. However, there would be little point since the scores are for internal planning purposes only: so the team would only be gaming itself.
Is the YPQI accredited? Can you pass or fail?
The YPQI is not accredited and has no pass/fail criteria. There is no ‘award’ (such as gold, silver or bronze) and no external validation. This is a critical part of the low stakes accountability approach, as described above.
So how do know if we’re doing it right?
Organisations or sites can ask for a trained External Assessor to come and observe their practice and score the PQA. External Assessors have been trained to consistently score the PQA accurately, in line with the design and intention.
An External Assessor supports a self-assessment team, by providing a reliable set of scores to factor in as they identify the areas of practice they wish to improve. In keeping with low stakes accountability, the External Assessor’s scores are not taken as the ‘correct’ scores, but included in the overall assessment and reflection process, alongside the team’s own scores.
As part of the UK pilot, cohorts of External Assessors are being trained across England, Scotland, Wales and Northern Ireland and we are matching them with pilot organisations to support their self-assessment.
What is the incentive for going through this process?
The primary value for organisations implementing the YPQI is that it’s a process that has been shown to consistently improve the quality of practice. In the US, organisations that use the YPQI are understood to have a commitment to continuous improvement, organisational learning and high-quality experiences and settings for young people.
As a result, some US funders ask that organisations they support commit to the YPQI. Over the course of the UK Pilot, we will be building the case for funders in the UK to follow suit, and provide additional resources to support organisations to implement the YPQI on an ongoing basis. A key aspect of this process will be developing the evidence base for the connection between the quality of provision and positive outcomes for young people.
Does the PQA measure outcomes for young people?
No, it only looks at the behaviours of staff teams within youth organisations. Through many years’ research and testing, the PQA has focused in on the ‘quality markers’ that are most strongly correlated with young people’s social and emotional development and learning.
Many organisations in the US currently measure young people’s outcomes alongside using the PQA. As part of the Centre for Youth Impact’s pilot of the YPQI in the UK, we will be undertaking a research study to gather data on young people’s SEL outcomes alongside data on the quality of practice in order to explore the relationship between the two.
We hope that this means youth organisations in future will be able to focus on understanding and improving quality, rather than measuring outcomes, because our research will have already looked closely at the link.
What about the voices of young people? Can they use the PQA?
It is possible for young people to use all or part of the PQA to observe and score practice in youth organisations. The experience of the Weikart Center in the US is that this is most effective where young people are already routinely involved in sharing their feedback and views on provision, and where the organisation has well-developed routes to listen and respond to young people. It is far less effective when the PQA is used to begin a new approach of involving young people in assessment or ‘inspections’. Throughout the PQA itself, there is a focus on the extent to which young people are offered authentic choice, leadership roles and opportunities to shape provision.
Is the YPQI specifically designed for youth workers?
The YPQI was designed for youth workers in the United States, working with young people in ‘out of school time’ settings. The term ‘youth worker’ doesn’t translate exactly to the UK context, because there is no qualification route in the US. ‘Youth workers’ include mentors, sports coaches, group leaders, key workers and so on. ‘Out of school time’ provision covers a wide range of activities including sport, arts, health and wellbeing, support for literacy and numeracy, citizenship and ‘service learning’ (similar to what we call social action) and outdoor learning. The common thread is a commitment to the personal and social development of young people, and provision taking place outside of the school setting.
At the Centre we believe that the YPQI is relevant for anyone who works with young people in informal and non-formal settings. This includes volunteers, paid staff including qualified youth workers, and managers. We think that the domains of quality correspond well with the values and ethics of youth work in particular, and that the YPQI process is well aligned to youth work settings.
YPQI materials currently refer to ‘youth workers’ throughout, which is a reflection of the US context, and does not exclude other adults who work with and for young people.
As part of the UK pilot, we will be working with a range of staff within youth organisations to review and, where appropriate, adapt the language used in the YPQI materials for a UK audience.
How was the YPQI developed?
You can read about the history of the YPQI’s development here.
If the YPQI has been created in the US, how will it relate to the UK context?
At the Centre we believe that the YPQI is a valuable reflective process that will be beneficial to youth organisations across the UK. We don’t feel that the difference between the US and UK contexts and practice is so significant that it will render the YPQI and the PQA invalid or unhelpful. We feel that there are common strands to how we understand quality in provision for young people that are ‘portable’ from the US to the UK.
The YPQI pilot is an opportunity to test these beliefs and work through in detail what adaptations are needed for the process to be successfully implemented in the UK. Alongside supporting organisations to implement the YPQI in the UK, we are carrying out action research with all four nations to explore any adaptations needed for the UK context.
Aren’t there already quality frameworks for the UK youth sector?
Yes, there are a number of quality frameworks that are available in the different countries of the UK, including the Welsh Government Quality Mark, the NYA Quality Mark and Ambition Quality. We are not intending to duplicate or displace any of these frameworks, and are working with all of the stakeholders involved as part of this pilot.
We believe that the YPQI is a valuable addition and support to these existing frameworks. The YPQI has a number of important differences from existing quality assurance frameworks:
● There is no pass/fail criteria or standard that has to be reached in order to complete the process successfully
● There is no accreditation or award offered to those reaching a certain standard
● It focuses solely on the quality practice rather than health and safety, legal compliance, or training and development for example
● The scores or data gathered through the process are not intended to be shared externally
● The framework is based on research data that demonstrates the link between each of the quality markers and improvement in young people’s social and emotional learning outcomes
Our hope is that the YPQI will become a valuable component part of the quality landscape for UK youth organisations, and can be used to demonstrate that organisations are meeting key aspects of the requirements set out in other frameworks.