Insights
One of our objectives is to bring robust evidence from all over the world to the UK. Straight Talk on Evidence is a good source of information on rigorous evaluations of programmes in the US. They have recently reported on positive findings of a large evaluation of an initiative called Career Academies, which has run for more than thirty years.
In this blog, I summarise the findings on the long-term effects of Career Academies from a large multi-site randomised controlled trial (RCT) carried out by the MDRC considering labour market outcomes, educational attainment, and family formation. The evaluation focused on nine high schools that implemented the Career Academies model in poorly performing, large, urban school districts around the US. Within the host high schools, the programme was targeted at students at a high risk of dropping out of school before graduation.
Career Academies offer an alternative learning environment for youth (students in 9th or 10th grade) to enhance school engagement and performance, and at the same time, to provide them with the skills required to make successful transitions to higher education or the labour market. They have three specific characteristics: 1) they are organised in small learning communities to create a more supportive, personalised learning environment; 2) they combine academic classes with occupational specific training; and 3) they offer workplace opportunities through partnership with local employers. Each Academy usually focuses on occupational training in a particular field, adapting the approach and intensity of the three components to the local needs. They have a recruitment period for students to submit their applications and successful applicants enter a Career Academy in the 9th or 10th grade (when they are 14 or 15 years old). They work with the same team of teachers until graduation.
From a research design perspective, the nine Academies being evaluated had an appealing feature: the number of eligible applicants was larger than the number of places, which gave the scope for running a lottery (random assignment) to choose the participants. 55% of applicants were randomly selected to participate in a Career Academy (the treated group), while the remaining 45% stayed on the high schools’ regular education track (the control group). Both groups of students, participants and non-participants, share similar pre-programme characteristics such as ethnicity, gender, performance in school, attitudes toward school, and socio-economic characteristics. This means that the difference in mean outcomes between the treated and the control group should measure the average effect of participating in a Career Academy rather than other factors.
The study started in 1993, and the evaluation used a follow-up survey eight years after high school graduation, which asked about postsecondary education, labour market experiences, and family formation. 81% of the students who participated in the randomised trial responded to the follow-up survey, with no significant difference in characteristics compared to the 19% of non-respondents.
The findings that are of most interest to our work are the following:
- Participants had an 11% increase in average annual earnings over the full eight years, earning $2,555 more per year (in 2017 dollars) than the non-Academy students. The effects persist over these eight years without showing a declining pattern along across the period. Oddly, these effects are only observed for men (who experienced a 17% increase in earnings), without finding statistically significant results for women´s earnings.
- There was no significant impact on educational attainment. No systematic differences were found regarding the completion of high school or an equivalent degree between participants and the control group. This may be due to the fact both groups showed similar levels of motivation, as evidenced by the fact that they had all applied to the programme. The cost of the programme ranged widely from $3,800 to $7,600 per student for the entire Academy participation, being partly offset by increased tax revenue associated with the gain in earnings.
Some questions remain:
- Can these results be replicated in another context? Which elements of the model are most important?
- How can we minimise dropouts? Other studies have shown that only 55% of the students assigned to an Academy end-up attending and finishing the programme.
- What is happening with women? We need to understand why they do not seem to be benefitting as much as men from participating in Career Academies.
Unsurprisingly, this educational alternative is increasingly popular, with more than 2,500 Career Academies now running across the US. In England, similar types of schools called University Technical Colleges have been running since 2010. All we need now is a similarly robust evaluation of their progress here – we’re aware of one UK evaluation working on that. Watch this space!
