Evidence review
What was the programme and what did it aim to do?
This study examines the impact of a partial credit guarantee program which was implemented in a large Italian region beginning in 2008. The program was allocated an endowment of £17.8m per year from a regional fund. Each guaranteed loan involved an agreement between the regional authority and a private covenant bank. If the firm defaulted on its loan from the bank then the regional authority covered up to 80% of the banks’ losses. The aim of the programme was to increase credit availability and improve the financial structure and performance of small firms.
What’s the evaluation challenge?
Evaluating the effect of credit guarantees is difficult because such programmes are typically targeted at firms who would otherwise face difficulties getting credit. This means that these firms are likely to have little credit availability (from elsewhere) and may be performing badly on outcomes such as sales, employment and credit rating. As a result of this selection, if we compare differences in outcomes for firms who receive loan guarantees to those who do not, these differences may not reflect the impact of the programme. Instead, they may simply reflect differences in the types of firms who receive support.
What did the evaluation do?
The study made use of a source of randomness in the policy’s implementation, arising from the fact that the bank that was supposed to roll out the policy was acquired by another bank in the period immediately before implementation. The study identified firms who were more likely to receive a loan guarantee simply because they historically (i.e. prior to policy conception) used the bank that eventually acquired the bank implementing the policy. The study then examined firm outcomes such as bank debt, probability of default, investments and trade debts for firms who we treated for this reason against firms who didn’t participate for this reason. This technique is called ‘instrumental variables’.
How good was the evaluation?
According to our scoring guide, instrumental variables receives a maximum of 4 (out of 5) on the Maryland Scientific Methods Scale (Maryland SMS). This is because the technique works well to control for both observable differences (e.g. sales) and unobservable differences (e.g. motivation) between supported and non-supported firms; although it doesn’t give us the full confidence of a randomised control trial. For the method to be well implemented the ‘instrument’ (i.e. historical use of the acquiring bank) must identify a set of firms that are more likely to get the policy treatment but which are not different in any other way. In this case, the instrument works since these firms are more likely to be treated not because of their underlying characteristics, but simply because ‘stickiness’ in the Italian banking system means that firms that used a bank historically are likely to use the same bank again in the future. For this reasons we score the study 4 on the SMS.
What did the evaluation find?
The study finds that the policy did improve some financial conditions for supported firms. While the total amount of bank debt was unchanged, the treatment did lead to increased long-term debt (the loan guarantee targeted this type of debt). Further, treated firms were able to secure loans at a significantly lower interest rate. However, the study also found that treated firms showed a higher probability of default when considered one or two years after the treatment (but no difference when looking at three years after). Despite the improved debt structure and overall cheaper credit available for treated firms, there wasn’t much improvement in ‘real outcomes’. Investments were significantly higher only in the first year after treatment (not the second or third years). There was no significant effect on trade debts (which proxy the amount of business a firm is doing) in all years.
What can we learn from this?
Although the policy led to a more balanced debt structure (i.e. more long-term) with lower servicing costs (i.e. lower interest rates), it didn’t increase the overall amount of debt nor did it lead to significant improvements in firm performance. In terms of defaults it actually lead to worse performance. The authors suggest that the lack of improvement on real outcomes could be due to the relatively short time frame of the evaluation (three years). Improving debt structure may in itself by a policy benefit since Italian firms have been shown to have debt structures that are unbalanced towards the short-term component. However based on the findings presented the policy was only partially successful in achieving its stated objectives.
