‘How to’ resource
Since 2014, 11 English city-regions have agreed devolution deals with government. Devolution deals involve establishing and transferring a range of powers, programmes, and budgets to mayoral combined authorities (MCAs). The powers vary across MCAs (with longer established MCAs generally having more powers) and can include adult education, skills, transport, business support, and housing. MCAs also receive a funding settlement.
Both the Ministry for Housing, Communities, and Local Government (MHCLG) and individual MCAs have expressed an interest in evaluating the overall impact of devolution. An impact evaluation would assess whether devolution had an impact on (that is, caused a change in) outcomes for an area that had received devolved powers and funding compared to similar areas that had not. Alternatively, an evaluation could look at the impact of devolution across multiple MCAs.
This briefing aims to help policymakers think through the feasibility of evaluating the impact of devolution.
Overall assessment and alternatives
Overall, our assessment is that it is likely to be extremely difficult to evaluate devolution as a whole due to how the policy has been designed and implemented over time. Whilst there may be some options available, they are likely to either use less robust methods or not provide convincing answers to key policy questions.
Even though evaluation of devolution as a whole is unlikely to be possible, it should be possible to undertake impact evaluation of some policies, programmes or projects delivered by MCAs as a result of funds or powers received through a devolution deal. This could include:
- Evaluating single interventions within a programme.
- Evaluating multiple interventions with a shared outcome.
- Evaluating multiple interventions that targeted the same geographic area within a devolved area.
There is scope to use a wide range of counterfactual impact evaluation methods to evaluate interventions, especially if evaluation is built in from the design phase.
In addition, MHCLG and MCAs could learn more about how devolution is influencing practice by analysing monitoring data and undertaking process evaluation.
