Insights
With eleven policy reviews either completed or currently in progress, we at the What Works Centre would like to think that, by now, we are rather adept at evaluating growth policy. In our publications and outputs to date, we have often been very quick to stress the need for robust and targeted evaluation at all stages of policymaking, but are we listening to our own advice? Is the Centre evaluating its own approach to trying to answer the fundamental question of ‘What Works?’
Through its various partners, the Centre is able to draw on a wealth of academic and professional expertise and experience. Building on our practical experience over the life of the project to-date, we at Arup recently organised a ‘Charrette’, which placed the Centre’s work on show to a wider multi-disciplinary panel of Arup’s consultants.
Charrettes, oddly deriving from the French for ‘chariot’, traditionally refer to collaborative gatherings in which a group of designers meet to reach a solution to a design problem. In the context of our Charette, the session provided a unique and valuable opportunity for Arup’s project team to showcase and evaluate the Centre’s key findings, and seek the views and perspectives of a wide-ranging body of experts; this group included economists, town planners, transport planners, masterplanners, urban designers, financial advisers, engineers and management consultants, amongst others.
The discussion was both wide ranging and deeply interesting; however, two points stood out in particular, having been discussed at some length.
The first concerned the nature of the findings that the Centre reports on. A common thread that ran through the discussion was the observation that any policy intervention will inevitably have impacts beyond the economic sphere; for example, interventions can lead to innumerable social, environmental, psychological and cultural implications, particularly those which involve changes to the built environment such as estate renewal schemes, new transport infrastructure or major events. Indeed, from the perspective of an urban designer for example, a major event which is shown to have only limited economic value is still a success if it improves the aesthetics of the built environment and results in an increase in the psychological wellbeing of local residents. Similarly, a new motorway connection may bring ample aggregate economic benefits to a local economy, but at what cost for those in communities within earshot, who an environmentalist might argue are either consigned to a life of tinnitus and respiratory issues, or, if they choose to relocate, to a significant financial loss as the value of their houses have fallen by half overnight?
We have often acknowledged this fundamental issue, and appreciate that there are many indiscernible considerations that cannot be brought under the econometric umbrella. Furthermore, it would be short-sighted to, in a two-dimensional fashion, brand any policy as a failure simply on the basis of purely economic outputs. Nevertheless, it was widely appreciated that the econometric evaluations reviewed by the Centre can form a very valuable and significant component of policy formation and evaluation.
One of the most positive outcomes of the discussions was a real enthusiasm amongst the practitioners, and a sense that the work of the Centre could be usefully applied to a wide array of work. It was acknowledged that robust policy or project evaluation is not always at the forefront of the agenda. However, several stressed that, in practical and logistical terms, robust evaluation could be built into almost any project, from bridge building to place-making, and is particularly useful where the goals of such projects are grounded in the assumption that they will bring about some form of economic prosperity. After all, things tend to get built for a reason, and few people tend to construct a new highway or redevelop a failing estate if they don’t believe (or even expect) that it will bring some form of measurable benefit. Yet despite this, as we have shown from practically all of our reviews to date, widespread evaluation of such interventions is often lacking.
It is therefore extremely encouraging that the WWC and the benefits it can offer were received so enthusiastically by such a range of practitioners, and it’s safe to say that econometric evaluation is now increasingly on their radar. In practice, this indicates that the work of the Centre need not be confined to local policymakers or the public sector, rather, there is a real opportunity to incorporate robust evaluation within the work of individual private sector firms also. We hope to be able to continue to spread the message of the benefits of evaluation in our upcoming work.
