Insights
In my last blog, on the lessons from our full set of evidence reviews, I highlighted the importance of experimentation, piloting and evaluation at the early stages of policy development to help make decisions on scaling up (or down) and improving effectiveness.
One important area where such an approach is likely to prove more challenging is in capital investments where the size, lumpiness and longevity of investments makes experimentation tricky (to say the least).
As our transport evidence review makes clear these features don’t necessarily rule out high quality evaluation. As I’ve explained before, the key to high quality impact evaluation is to look at changes for those benefiting from a policy compared to a similar (control) group that don’t benefit. While it is fairly easy to understand how we might construct control groups and undertake evaluation for policies targeted at individuals, households or firms, it is harder to think about how we might do this for policies – such as transport investment – that target areas. Fortunately, however, there are methods currently available (and more being developed) that would allow us to improve evaluation along these lines. The section on impact evaluation for transport investment in our evidence review has more on these and other ideas. It would be good to see these approaches applied by both local authorities and DfT in the evaluation of existing schemes. Indeed, we’re currently working with DfT and a couple of Local Authorities to see whether implementing such approaches would be feasible.
But as with other policy areas, we also need to think about how changes to the policy cycle could help improve decision making. Rather than the experimentation that we highlight for other (‘revenue’) policy areas, we think that the focus for capital investment needs to be on better linking appraisal and evaluation. Considerable resources are already devoted to the ex-ante cost-benefit appraisal of transport schemes as part of the decision making process. The increased interest in the effect of transport investment on the local economy has also been accompanied by growing criticism of the appraisal approaches used to help facilitate scheme prioritisation. DfT is planning to refresh WebTAG (the set of procedures which outline how appraisals should be conducted) to respond to these criticisms and solutions. Our evidence review highlights the importance of complementing any further work on the “ex-ante” appraisal framework (i.e. analysis that predicts what might happen) with additional work to strengthen the “ex-post” evaluation of transport investment (i.e. analysis of what actually happened). Not least because, as our review makes clear, there is limited evidence that the employment (and other effects) that would underpin any changes to the guidance actually occur in practice.
Our transport evidence review contains some more detailed proposals – for both improving evaluation and better linking appraisal and evaluation – building on our work with the Department for Transport, as well as the discussions of a LA/LEP working group convened by the Centre. We’re continuing to work on developing these proposals and hope to report on some of the findings later in the year.
