At What Works Growth, we work with places to address the barriers to doing high-quality evaluation. We provide training, resources, and bespoke support, all free of charge.
Our training is aimed at people in local policymaking and designed to be practical, using real-life examples and activities from economic development.Browse our training offer
What makes a good evaluation? A robust impact evaluation is at the heart of understanding what really works in local economic growth policy.An 8-step guide to better evaluation
We support central policy makers and analysts working on local growth to use evidence effectively and deliver high-quality evaluation.Central government support
What kind of evaluation?
Different types of evaluations answer different questions. At What Works Growth, we focus mainly on impact evaluations.
Impact evaluations assess whether a given intervention – such as a business support scheme – has made a difference to a given outcome – such as business survival.
We offer training, resources, and bespoke support to help places design, deliver or commission impact evaluations. Our training and guidance also support places looking to improve monitoring and process evaluation.
Impact evaluation asks the question ‘what difference did the intervention make?’ It focuses on understanding whether the outcomes observed (for example, whether new businesses survive for two years after start-up) can be attributed to the project or programme being evaluated (for example, a training programme for new entrepreneurs). This is known as ‘causal impact’ (i.e. the project or programme is the cause of the outcomes).
Impact evaluations establish causality by using comparison. Ideally, we’d compare the same person, business or place under two different scenarios – one where they receive support (known as ‘treatment’) and another where they do not. As it isn’t possible to know what would have happened if they hadn’t received treatment, we need to establish another comparison, often referred to as a control group, comparison group, or counterfactual.
The standard approach is to create a group of people, businesses or places that are similar to those being treated, but that did not receive treatment. Changes in outcomes can then be compared between the ‘treatment group’ and the ‘control group’.
The main challenge is how to ensure that those in the control group are similar to those in the ‘treatment group’. For example, more ambitious businesses may be more likely to apply for some business advice and access to finance programmes. In this scenario, the evaluation findings may be biased upwards. Better business outcomes are attributed to the support received, when they may be due to the ‘treatment group’ being more ambitious, and thus doing better even without the programme, than the ‘control group’. Impact evaluations use a range of techniques – including randomised control trials and quasi-experimental designs – to overcome this ‘selection into treatment’ problem. Where these are not feasible, matching techniques can be used to make sure that the groups are as similar as possible on their observable characteristics.
Bespoke support for impact evaluations
We provide expert help and advice on evaluation design and delivery, including through our external Evaluation Panel.
All our support is free, and can include:
- early conversations to find the right evaluation design for the project;
- review of evaluation specification and ‘invitation to tender’ documents;
- sitting on advisory or steering group; and
- peer review of findings.
We may also be able to help deliver your impact evaluation if it is well designed and fills a gap in the evidence base.