Evidence review
What was the programme and what did it aim to do?
This study looks at the effect of a major cultural event, the European Capital of Culture, on urban & regional GDP and on residents’ life satisfaction. As with most sports and culture projects, the objectives were not primarily about delivering local economic benefits. But it is still interesting to understand the economic effects. Between 1990 and 2009, 29 European cities were chosen to be European Capital of Culture (ECOC) for a given year. The host cities were given a budget for both cultural projects and infrastructure improvements – with investments being mostly (77.5%) generated from the public sector. Projects vary in type and scale but the best represented sectors are theatre, visual arts, music, street parades, open-air events, heritage/history and architecture. There are an average of 500 events in the award year.
What’s the evaluation challenge?
The fact that ECOC cities are selected by the EU’s Council of Culture Ministers makes the event harder to evaluate. The selected cities are likely to have different characteristics to non-selected cities. For example, the minsters of culture may choose cities that are struggling or (alternatively) ‘on the up’, and any differences in outcome may be due to these conditions rather than the ECOC status and related investment. Furthermore, the individuals who live in the regions that are selected may have fundamentally different levels of life satisfaction or at least different tendencies in terms of stated life satisfaction.
What did the evaluation do?
In order to deal with this problem the authors use a difference-in-difference approach to estimate the effect on quality of life and GDP. Individuals in regions that have ECOC status in a particular year form the treatment group. The control group is formed up of individuals in all other European regions that are not ECOC. The authors look at how much higher quality of life is in ECOC regions in the year that they were ECOC, compared with the control group in the same year.
In order to implement this approach the authors make use of ‘The Mannheim Eurobarometer Trend File 1970-2002’, which is a longitudinal dataset of individuals in 18 Europeans nations (i.e. it follows the same people over time). The dataset includes self-reported life satisfaction, the dependent variable, which is rated on a 4-point scale from ‘not at all satisfied’ to ‘very satisfied’. Individual-specific factors are also available such as employment situation, income, gender, etc. This dataset is combined with regional GDP per capita based on data from BAK Basel.
How good was the evaluation?
According to our scoring guide, difference-in-differences receives a maximum of 3 (out of 5) on the Maryland Scientific Methods Scale (Maryland SMS). This is because it does well to control for observable differences (e.g. population) between cities that host events and cities that don’t, but is unable to control for unobservable differences (e.g. the quality of local infrastructure). Importantly, a variety of control variables are used to account for regional and individual differences that may affect outcome variables in a particular year. The control variables for satisfaction were personal characteristics such as age, income, and so on, as well as regional economic growth. The controls for regional growth were macroeconomic factors such as population density, sectoral shares and human capital, represented by education. Since this study uses a wide range of variables to control for observable differences we award it a ‘3’ on the SMS.
What did the evaluation find?
The results for GDP show no effect of ECOC status on GDP in either the run-up to the event, during the event or after the event. The results for life satisfaction indicate a negative effect for individuals in regions hosting the ECOC during the year of the event. The effect is 0.09 points lower on the 4-point scale, which the authors suggest is considerable. There is no significant effect on life satisfaction in the year prior to the event. After the event is over, life satisfaction returns to pre-ECOC levels. Unemployed people suffer the greatest drops in satisfaction during the event – being unemployed roughly doubles the negative impact on life satisfaction.
What can we learn from this?
Taken at face value, there are no GDP effects related to having ECOC status and the wellbeing impact is actually negative. Therefore, the arguments why local leaders may to bid for such an event are weakened. However, since the control group in this evaluation is made up of all other European regions (rather than only similar regions), it is likely that there remain significant unobserved differences between ECOC and non-ECOC regions that could be responsible for lower quality of life during the event year, despite the large number of control variables. Furthermore, self-reported quality of life is a complex outcome variable to truly understand – hence policy recommendations based on evidence of this type should always remain cautious. Despite these caveats, it is important to recognise that the evidence in this evaluation suggests that effects on local residents may be negative with no offsetting boost to the local economy.
Reference
CREMA European Capitals of Culture and Life Satisfaction [Study 324 from our Sports and Culture review]
