Insights
Policy making and key challenges of an evidence-based approach
Government budgets are subject to a lot of scrutiny these days. Great effort goes into thinking about how to allocate resources and how to design programmes. Nevertheless, inefficiencies in the way taxpayer’s resources are spent are always found. The magnitude of this is unclear, yet here is an example to put things in perspective about how big the issue could be: according to the Inter-American Development Bank (IADB), the amount of public funds inefficiently spent in Latin America would be enough to lift people out of extreme poverty in the region.
Many institutions and governments advocate for an evidence-based approach. The approach sounds reasonable and there is a lot of consensus around it. So, of course, we should back it. But what does this really mean? It is about not only producing and using data to inform our policy decisions. Here are three key principles we should consider when thinking or following an evidence-based approach.
Letting yourself go
First and most importantly, we need to be prepared to allow findings to guide our thinking and decisions about policy issues. We must permit our notions and priors about certain topics and issues to be challenged. This implies a slight change of paradigm: transitioning from relying on our current understanding of certain topics to acknowledging new pieces of evidence, which might contradict what we thought we knew beforehand.
If our notions are confirmed with the evidence, then that is great. However, if that is not the case this implies an opportunity for learning and adjusting both our beliefs and policies. An example is publicly funded business support programmes that provide information, advice or mentoring to firms. Despite conventional wisdom, there is little formal evidence available indicating these types of programmes actually have an impact on local economic growth, even when there are some positive effects on individual businesses.
Reading between the lines
After we have placed our focus on the evidence, we need to assess the quality of the data we have at hand and understand its caveats. The evidence should ideally provide us with a roadmap or actionable information about how to operate in the policy world. Descriptive information is less useful since it is unlikely to tell us anything about the consequence of using certain policy tool or intervention. Thus, we should be careful in how we convert data –sometimes the only type of evidence available– into actionable information or ‘intelligence’ we can use to decide if we should fund a programme and scale it or not.
For example, comparing the productivity levels of a specific region against the UK national average is not telling us how and when to intervene. What’s more, it is not even telling us there is an issue of low productivity and that we should spend public resources on this policy space. By construction, a large number of regions will always be below the UK national average. Hence, it is better to look at similar areas and to understand the evolution of productivity in that region, and if possible to grasp which specific intervention had a positive impact on productivity growth. An assessment of the evidence available should distil what the data is telling us, what it is not, and above all consider what information is missing and how it could affect our conclusions.
Sharing is caring
Tackling the most pressing issues will never be an easy task. Policies are not always successful in achieving the desired objectives. Finding that something did not work is not always appealing to the reader or to the policy debate. However, sharing an experience from an intervention that did not work is as useful as lessons from a policy that had a positive impact.
Sharing all these lessons from failure is of public interest, not only from a point of view of transparency but to improve the overall evidence base and contribute to effective use of public resources. We should not underestimate the value of experiences that might prevent us from avoiding certain policy design or anticipating certain issues for other regions across the country, or even for future governments in the same region. The purpose of an evidence-based approach is not just to conduct evaluations and get data, but to use our experiences and contribute to the debate with results, regardless of the outcome.
Conclusion
To make efficient use of public funds, local governments should evaluate the impact of their policies to verify budgets are spent wisely. Embarking on policymaking with an evidence-based approach implies being flexible to adjust our beliefs to new findings, assess the quality of the evidence and its limitations, and finally a commitment to transparency and learning from positive and not so positive experiences.
