Insights
An important part of our mission is to help policy makers understand evidence on what works by summarising the impact evaluation evidence related to local economic growth.
Having completed wide ranging reviews of the international evidence, we were concerned that we may have missed some evaluations that had been undertaken by local government within the UK. Often evaluations are not published, because they were intended for internal information, or their findings are negative or just unclear.
A little over a year ago, we decided to make a concerted effort to unearth such evaluations.
Between December 2019 and March 2020, we contacted local authorities, combined authorities, local enterprise partnerships and central government departments, as well as other organisations all over the country to see what evaluations they had undertaken. In total, we sent emails to 1,329 people from 416 different organisations, and also conducted in-depth interviews with some of the respondents.
We’re grateful to everyone who took part. Our efforts yielded 46 evaluations from 31 different organisations, none of which met the standards of evidence that we use for inclusion in our evidence base.
While it’s possible that this leaves a treasure trove of evaluations yet to be identified, we think that’s unlikely. We know places are busy so we made responding easy and followed up repeatedly. And our experience suggests that people are generous with their time when it comes to sharing reports that they feel might be of use to other places facing similar policy challenges.
Instead, we think the limited number of evaluations reflects more fundamental barriers to undertaking good evaluation at the local level. Based on the interviews and our experience to date, we think this is because:
- Places are focused on their core delivery obligations – and when resources are scarce, evaluation cannot be a priority; and
- Even if resources were available, incentives to undertake evaluations and capacity to deliver them are limited.
Interviewees highlighted additional evaluation challenges, including small sample sizes and evaluation guidelines that do not consider robust methodologies.
The exercise confirmed something we have long believed: places need significant help to develop their evaluation chops. With this in mind, we are committing new resources to help over the next couple of years.
First of all, our Evaluation Panel will provide more collaboration and support for local governments that want to undertake evaluation but need expert input. As well as the information online, you can contact Victoria Sutherland, our Head of Evaluation, with any requests for support or information.
Secondly, Danielle Mason, our Head of Policy, is working closely with government to improve the incentives and resources available to support evaluation under future funding programmes including the Towns Fund, Future Hight Streets, and Shared Prosperity Fund.
We hope that this work will set the stage for more impact evaluation in the coming years.
