Evidence review
What was the programme and what did it aim to do?
This study investigates the effects of a large subsidised training programme for disadvantaged youth in Columbia, introduced between 2002 and 2005. The analysis evaluated the last cohort who started training in 2005. Overall, the programme reached 80,000 young people. This is equivalent to 50 percent of the target population of all unemployed young people (18-25 years old) in the bottom 20 percent of the income distribution. Public spending for the programme amounted to 60 million US dollars. Participating youth were provided three months of classroom training and three months of on-the-job training. The training was thus not equivalent to, but closely resembling, European-style apprenticeships. The training covered a wide range of occupations (with a focus on administrative occupations) and classroom training was largely provided by private firms. On-the-job training was provided in the form of unpaid internships with private firms. The aim of the programme was to improve disadvantaged youths’ employment and income prospects.
What’s the evaluation challenge?
Evaluating the effectiveness of education and training is challenging because young people who choose to get training may be different to those who do not. Often, they may be more ambitious or have higher ability. Therefore when comparing their labour market outcomes to untrained people differences between the two groups may be driven by pre-existing characteristics, such as ambition and ability. Since these variables are largely unobservable, it is difficult to eliminate their impact on labour market outcomes, so as to gauge the effect of a particular training intervention.
What did the evaluation do?
The study deals with these unobservables by implementing the policy as a randomised controlled trial (RCT). The RCT made use of randomisation in the process of who was admitted to training. The training institutions received applications and were asked to select, for each class they offered, a list of 45 potential trainees. This number exceeded the number of available slots (usually 30). Applicants were then randomly assigned to participate or not participate in the training programme. As a result, all applicants were as likely to benefit from training by the training providers, but only a subset actually received training. Thus, the study compared labour market outcomes for comparable trained and untrained persons to infer the effect of training on these outcomes.
How good was the evaluation?
According to our scoring guide, randomized control trials can achieve the maximum score of 5 on the Maryland Scientific Methods Scale (SMS). This is because randomisation can ensure that individuals subject to a policy intervention (training) differ from other individuals only by whether or not they benefit from the intervention. In this evaluation, the randomization worked satisfactorily for women, but not for men. Trained and untrained men differed significantly, as measured by a set of sociodemographic variables. We therefore only consider the results for women. To further ensure the comparability of trained and untrained young people, the study only compared individuals who had applied to the same training course and at the same site. The authors demonstrate that the evaluation sample largely resembles the corresponding population (that is, all unemployed 18-25-year-olds in the bottom 20 percent of the Columbian income distribution). Finally, being offered a training place actually resulted in taking the training in 97 percent of the cases, meaning that the evaluation could address the effect of training itself and not just the effect of having been assigned to training. For all these reasons, we score the study at the maximum of 5 on the SMS.
What did the evaluation find?
The study found that women’s paid employment rate increased by seven percentage points in response to training. Their wages increased by about 20 percent. These increases were accompanied by, and probably largely due to, a significantly increased probability of working in the formal sector (as opposed to being unemployed or working in the informal sector). Balanced against the costs of the programme, the study estimates a net benefit per woman in the range of $700 – 3,000 (that is, an internal rate of return of 20 – 35 percent), depending on length of working life and depreciation of training, and abstracting from indirect and non-monetary costs and benefits.
What can we learn from this?
For women, the study demonstrates that the programme achieved its goal of increasing wages and formal sector employment. Formal-sector employment is an important policy goal not only because it yields higher wages than other employment, but also because it gives access to health care and pensions. In terms of magnitude, women’s wage gains in particular were very substantial. It cannot be entirely ruled out that the gains of trained women may have come at the expense of untrained women. That is, the gains may be more modest if all young people received this training. However, the large magnitude of the estimated effects suggests the programme was highly successful. While not equivalent to European apprenticeship it involves some elements that may also make apprenticeships successful: By involving private firms as training providers, training contents should have corresponded to labour market demand; firms had the opportunity to screen potential employees during the internships; and likewise, the trainees got access to information on job opportunities (and requirements) they may not have got without the internships. Of course, proper evaluation would be needed to know if these findings generalise to apprenticeship programmes. This study also does demonstrate how such schemes could be much better evaluated to see if these results do generalise.
Reference
Attanasio, O., Kugler, A., and Meghir, C. (2011). Subsidizing vocational training for disadvantaged youth in Colombia: Evidence from a randomized trial. American Economic Journal: Applied Economics, Vol. 3 (3), pp. 188-220.
