The OYF Network uses multiple data and evaluation approaches to drive improvement and understand outcomes and impacts. We understand that different kinds of data and information can be used in different ways, to drive change, improvement and impact. As such, we encourage OYF members to draw on, at minimum, three distinct sources as they plan to improve their collaborative and individual programs, change systems, and drive towards better outcomes for young adults in their communities.
Information Sources Used by OYF Collaboratives to Assess Impact
1. Annual OYF Self-Assessment: this survey asks questions about the characteristics of the collaborative, the collaboratives capacity in 4 domains, the collaborative’s impact on 7 kinds of systems change, and their work in 3 core values areas that are essential to this work. Collaboratives can track their progress over time, and compare themselves to peers.
2. Common Measures and Local Stats: the four Common Measures are run annually and given to sites, describing community-wide population level stats on the numbers of OY and their education and workforce attainment status. Sites set future targets for improvement against the Common Measures. Sites supplement these measures with their own other local statistics as well.
3. Site-Level Data Collection: OYF sites collect information from direct services providers who are collaborative members, to understand the numbers of OY being served by programs, and also the outcomes being accomplished – disaggregated by race and other demographics when possible.
More detail on each of these areas follows.
1. Annual Self-Assessment
The Opportunity Youth Forum is regularly assessed for progress and impact by our independent evaluators, Equal Measure.
This assessment is based on our theory of change, which is that to attain the goal of reducing the numbers of opportunity youth, localities need to have collaboratives with strong capacities that are focused on improving systems meant to support young adults, all embedded in a community that values equity and shifting power imbalances.
Within the assessment’s three main domains are 14 different sub-domains, which the assessment looks at in great detail; over three hundred questions are asked annually in the assessment survey.
An important aspect of the Annual OYF Assessment is the return of data back to the contributing OYF members. Within 2 months of completion of the survey, OYF member receive back a detailed report which includes their own responses, and summary information on how they scored on the three major domains, as well as their scores within the 14 individual sub-domains in the survey. Additionally, each individual site is show how their scores have trended across time. And in addition, each site is shown their current year’s scores against the network-wide average, and also against a peer-group of OYF members (for example, a more mature urban site would see its data compared to other mature urban sites).
Summary findings, of improvements across the entire OYF network, from the annual OYF assessment reports, can be found here:
- 2020 OYF Annual Assessment
- 2019 OYF Annual Assessment
- 2017 OYF Annual Assessment
- 2016 OYF Annual Assessment
- 2015 OYF Annual Assessment
2. Common Measures and Other Community-Wide Data
Opportunity Youth have historically been difficult to count, as they are defined as a group which does not appear in either of two data sets – those in an educational program and those not working. With the OYF Common Measures, however, we can annually measure, for each unique geographic area of a OYF collaborative, the overall opportunity youth disconnection rate; and the disconnection rates for high school, post-secondary and workforce annually for the entire community.
The four measures are:
- Overall community disconnection rate: The rate of young people disconnected from work and school (i.e., opportunity youth).
- High school disconnection rate: The rate of young people without a high school diploma/GED and not working who are disconnected from high school.
- Postsecondary disconnection rate: The rate of young people with a high school diploma/GED but without a postsecondary credential, who are disconnected from postsecondary education and not working.
- Workforce disconnection rate: The rate of young people with a postsecondary credential, but not enrolled in postsecondary, who are disconnected from the workforce.
One important attribute of the Common Measures data is that it is disaggregatable by various demographic variables, such as race, gender, income, etc. We are also able to disaggregate it by other OY-pertinent variable such as having a child in the home, nativity, and more. The Forum for Community Solutions provides the Common Measures detail to each OYF member as the underlying data becomes available, typically annually, in partnership with our assessment provider Equal Measure.
Each community receives a detailed data report, which it can use to track progress, set future goals, understand which parts of the OY population are increasing, understand OY characteristics to inform program-level design, and much more.
To realize the full potential of the OYF Common Measures, each urban area-focused OYF community is encouraged to set targets for improvements in OY outcomes, based on the Common Measures. Having targets helps collaboratives and their partners identify where to focus their efforts and to measure their progress in improving opportunity youth outcomes. Communities may also choose to set additional targets against other, more local data sets. Read more about target setting approaches and practices, which are a strategy of our Data for Impact Initiative.
3. Program-Level Data
Established OYF member collaboratives are asked annually to collect program- (or service-) level data from collaborative partners who work directly with OY. Such partners can typically include CBOs, school districts, colleges, workforce intermediaries, etc. OYF members utilize a variety of approaches to collect this data.
The data request includes:
OY Outputs: the numbers of OY being served by programs. Includes breakdown by race and gender when possible.
OY Outcomes: the numbers of outcomes being achieved, in the last year, by programs; this includes major outcomes such as:
- Receiving a High School credential
- Attaining employment
- Attaining an internship or apprenticeship
- Attaining a college credential
We also request this data to be broken down by race and by gender when possible.