‘How to’ resource
Our resources, training and advice are designed to help places to think through impact evaluation, to assess whether impact evaluation is right for their project and to design and commission robust impact evaluations that improve understanding of the effects of local economic growth policies.
Different types of evaluations answer different questions. At What Works Growth, we focus mainly on impact evaluation. Impact evaluation assesses whether an intervention – such as a business support scheme – has made a difference to an outcome – such as business survival.
Impact evaluation focuses on measuring the effects (impacts) of a specific intervention on specific quantifiable outcomes. This is known as ‘causal impact’– because (when done well) these evaluations establish that the intervention caused the outcome. The key question we try to understand is whether a policy affects the outcomes it was designed to influence, what the effects were, and how that compares to other policies trying to achieve the same thing.
Impact evaluation does this by trying to answer the question of “what would have happened if the policy didn’t happen?” or, in some cases “what would have happened if we tried this different policy instead?”. It does this by using counterfactuals – constructed alternative scenarios that mimic what would have happened in a world where the policy wasn’t implemented, or where a different one was.
Currently, the impact evaluation evidence on most local economic growth policies is limited, which impedes policymakers’ ability to make evidence-based decisions. Addressing this will require more local economic growth policies to be evaluated.
GUIDANCE ON HOW TO EVALUATE
How to evaluate
We offer a helping hand for places wanting to design and commission robust impact evaluations.
Understanding impact evaluation
Better evaluation in 8 steps
Evaluation case studies
Evaluation training
Evaluation support
Evaluation resources
Impact evaluation isn’t always possible, or appropriate for all projects. In these cases, many of our resources can still be useful in developing plans for monitoring and evaluation.
Impact evaluation isn’t always the ‘best evidence’.
We think it is the best approach to finding robust answers for casual questions – Did the policy have an impact? How big was the impact? Which policies had stronger impacts on certain outcomes?
However, the answers to those questions are one part of the wider evidence base.
Impact evaluations usually need to be repeated in different contexts to build up a wider picture of the type of policy they are looking at and combined with other types of evidence for a fuller picture.
This is why many of our resources, including our evidence briefings, policy briefings, and blogs, draw on both impact evaluations and a wider range of evidence, like data trends and economic theory.
