Evidence topic: Sport and culture

Evidence review (PDF) published in June 2016.

Sport and culture have intrinsic value to people and places, promoting health and well-being. However, the evaluation evidence suggests the economic impacts of major events and facilities are limited.

What are sport and culture interventions?

Sports and culture can include everything from small-scale local events, to hosting the Olympics. Our work on this topic has concentrated on major events and facilities, such as large international sporting events, arts events, or permanent facilities such as arenas.

Major events tend to operate over at least one week, or involve shorter events on a frequently recurring basis, and are targeted at a national or international audience.

Major facilities include permanent facilities of regional or national scale that are targeted at a regional, national or international audience.

For our evidence review, we excluded conferences and conference centres, trade events, and expos.

The rationale: How do sport and culture deliver growth?

Sports and culture have intrinsic value to people and places as well as promoting health and wellbeing, cultural enrichment, and prestige and branding. In more recent decades, promoters of major sport and cultural events or facilities have claimed undertaking such projects will have economic benefits as well including:

• Creating jobs
• Supporting regeneration through, for example, improvements to the built environment, new housing and improved transport links

Evidence review: What does the evidence say about sport and culture?

The review considered over 550 policy evaluations and evidence reviews from the UK and other OECD countries. It found 36 impact evaluations that met our minimum standards.

We initially focused the review on evaluations of sporting or cultural events and facilities of any size. However, we found no evaluations of small-scale events that met our minimum standards. Whilst these findings are based upon evaluations of major projects, we believe they offer useful guidance for policymakers considering projects on any scale.

What the evidence showed:

• The overall measurable effects of projects on a local economy tend not to be large and are more often zero. Any wage and income effects are usually small and limited to the immediate locality or particular types of workers.
• Facilities are likely to have a positive impact on very local property prices. Policymakers should consider the distributional effects of these property market changes (who are the likely winners and losers).
• Projects may have been associated with increased trade imports and exports, including tourism, although these effects may be short lived (and are only considered in a small number of studies).

Where there was lack of evidence:

• We found no impact evaluations that considered visitor numbers. Far more should be done to assess the extent to which projects lead to net increases in visitor numbers for the area as a whole. Visitor numbers for the project alone and surveys of attendees may not provide strong evidence on the impact of projects on net visitor numbers.
• There was a paucity of evidence regarding cultural projects overall. This is an issue for understanding the likely impact of such projects and also leaves a gap in our ability to compare the economic effects of sport projects and cultural projects.
• We found no robust evidence on the economic impacts of smaller projects (such as arts centres or small-scale festivals) – although based on what we found for large projects, we can assume that the economic impact of such projects would be even smaller.
• We found no robust evidence for the impact of recurring sport and cultural events, such as annual festivals or tournaments.

Lessons:

• Almost all of the evaluations that we found to be rigorous are focused on projects at the grand end of the scale. However, we are confident that there are lessons for everyone facing this type of spending decision from the evidence we have looked at regarding these very large projects.
• It is important to have realistic expectations of what sports and cultural projects can achieve.
• Facilities may be more likely to produce economic benefits than events, probably due to the longevity of their impact.
• Indirect employment effects are unlikely to be large, and focus should be on the direct employment effects generated by an event or facility. Reflecting this, time and expense can be saved by forgoing complex multiplier-based appraisal systems in lieu of solid ‘narrow’ evaluations.
• As the benefits of new facilities tend to be very localised and related to property prices and regeneration, they should be part of a broader strategy rather than seen as stand-alone projects. They should not be relied upon as the major component of a job creation strategy.
• Considered together the findings raise interesting questions about who should pay for sport and cultural events and facilities in any given locality. None of this should overshadow the other real if difficult-to-measure benefits.

Downloads:

Case studies: Advice on how to evaluate sport and culture

Evaluation of local economic growth effects of events and facilities in sports and culture poses a number of unique challenges. Mega-events such as the Olympics are rare, thus reducing the number of observations for analysis. They also tend to be hosted in unique place such as global cities making it difficult to find similar control cities. Conversely for smaller local events, the effect size is potentially too small to easily detect.

In addition, events and facilities are not located randomly – policymakers may choose prestige locations, or locations which they hope have strong regeneration potential, in which case underlying factors for these areas need to be disentangled from any project effect. The criteria on which decisions are made about who hosts events or where facilities are built are not always transparent making it difficult to control for selection bias.

The effects of events and facilities may exhibit complex patterns over time and space: a sports stadium may improve a neighbourhood nearby at the expense of a neighbourhood further away; a World Cup may be expected to have effects before (e.g., construction effects), during (e.g., visitor spending) or after (e.g., due to infrastructure improvements).

To help improve evaluation of sport and culture interventions, we have four case studies that give examples of how previous policies have been evaluated.

Each evaluation case study has met our minimum standard of evidence, which means it (at a minimum) compares what changed for the places, businesses or individuals that benefited from an intervention with what changed over the same time frame for otherwise comparable places, businesses or individuals that didn’t benefit, or that received a different type of intervention. All of the case studies use statistical approaches to achieve this comparison. Statistical approaches try to ‘strip out’ the impact of the other factors that could have affected outcomes in both the beneficiary group and the comparator group.

Read more about how to evaluate, and why we think it can be helpful to learn from previous approaches in our how to evaluate guide. You can also read more about different evaluation methods in our scoring guide.