How Did the UK’s Assessments of Multilateral Value for Money Affect Aid Allocations?

The UK Government is currently engaged in efforts to reform the multilateral development banks, and its new International Development White Paper renews its commitment to multilateralism. In past years, the UK Government has undertaken substantial assessments of each multilateral’s effectiveness to inform UK funding decisions. 

In this blog, we explore how these major assessments of Value for Money (VfM) affected the UK’s multilateral aid allocation. To do this, we’ve accessed previously un-published  data from the UK’s 2011 Multilateral Aid Review (MAR) (which was instituted by Minister Mitchell in his previous role as International Development Secretary) and the subsequent 2016 Multilateral Development Review (MDR). 

We find that whilst high-scoring multilaterals do receive the largest shares of multilateral funding, changes in these scores do not translate into changed allocations. We go on to explore potential explanations, as well as implications for reform efforts. 

The UK’s commitment to multilateral effectiveness

The UK has consistently allocated a significant share of its aid budget to the core costs of multilateral organisations. Typically, this allocation is over one-third of the total aid budget, apart from in 2022 (a year where the bilateral share temporarily spiked due to in-donor refugee costs). And Development Minister Andrew Mitchell has said that “every penny” of this must be “justified in terms of effect.” 

The recently published UK International Development White Paper shows a renewed commitment to multilateralism, superseding the 2022 International Development Strategy’s shift towards bilateral partnerships and its commitment to reducing the multilateral share of the FCDO’s aid budget to a quarter by 2025 (which we found will mostly be achieved through the planned decline in EU contributions, and so requires limited cuts elsewhere).

How did the UK assess value for money across multilaterals?

In his former role as International Development Secretary, one of Mitchell’s first acts was to commission the MAR: a systematic assessment of the VfM provided by 43 multilateral organisations, published in 2011 (ahead of the planned increase of UK aid to 0.7 percent of Gross National Income (GNI)). This exercise was repeated in the 2016 MDR for 36 organisations, following the UK’s decision to leave the European Union. 

Both assessments were intended to inform allocation decisions across multilaterals. The 2011 MAR’s stated purpose was “to ensure that decisions about UK funding through these organisations are based on evidence about the value for money they offer and the results they achieve.” And in the 2016 MDR, former International Development Secretary Priti Patel said that the strong performance of the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM) “fed directly into our recent decision to increase funding”, and that future payments to other organisations were also being linked to performance.

The 2011 MAR and 2016 MDR both created two indices: the first measuring each multilateral organisation’s “Contribution to [/”Match with”] UK development objectives”, and the second measuring its “Organisational strengths”. The method for measuring these scores evolved slightly between the reviews, but broadly, the former index assesses the areas in which multilaterals work (e.g. their focus on poor countries, delivery of results, and work across fragile contexts, gender equality, climate and environment); and the latter index assesses how well they are managed (e.g. their consciousness of cost and value, partnerships, strategic and performance management, financial resources management, and transparency and accountability).

A four-point scale was used for the each of these indices in both the 2011 MAR and 2016 MDR, and the two reviews similarly converted these scores into discrete categories, though their labels differed somewhat. Scores of 3+ were considered “strong”/“very good”, between 2.5-3 “satisfactory”/“good”, 2-2.5 “weak”/“adequate”, and 0-2 “unsatisfactory”/“weak”. The 2011 MAR also laid out how it converted performance on these two indices into an overall assessment of VfM (though the 2016 MDR was not so explicit). An average score of 3+ across both indices was considered “very good” VfM, and an average of 2.5-3 was considered “good” VfM; a score of 2.5-3 on one index and 2-2.5 on the other was considered “adequate” VfM, and scores of 0-2.5 on both indices was considered “poor” VfM.

We accessed this data underlying the MAR and MDR via a Freedom of Information request (although the MAR and MDR reports did present their results graphically, their underlying data is not publicly available).

How do these assessments relate to allocations?

Before we look at whether changes in assessments of multilateral VfM influenced  allocations, we first look at how the most recent VfM assessment relates to allocations,  and find a clear relationship.  
In the below chart, we average the scores on the two indices from the 2016 MDR; and plot them against the UK core contributions across the 2016-19 period (to reduce any lumpiness in the timing of payments, whilst avoiding later distortions to multilateral allocations driven by the UK aid cuts from 2020 onwards). It’s clear that some of the largest contributions were received by high-scoring multilaterals—such as the European Commission (EC), International Development Association(IDA), GFATM, Global Alliance for Vaccines and Immunisation (Gavi), and African Development Fund (AfDF).

Figure 1: UK allocations to multilaterals against its assessment of VfM

Source: CGD analysis, using: the 2016 MDR (the underlying data accessed via a Freedom of Information request; reference: FOI2021/26517), and the OECD’s dataset for Members’ Total Use of the Multilateral System
Note: VfM scores are calculated as the average of the two MDR indices for “Contribution to UK development objectives” and “Organisational strengths”.

However, we shouldn’t necessarily expect to see an unconditional relationship between the level of VfM scores and the level of allocations. For example, since the MAR and MDR do not account for absorptive capacity. Gavi and the Global Fund may have been assessed to provide similar VfM in the 2016 MDR, but that doesn’t mean they could each spend the same amount effectively. Indeed, whilst Gavi asked donors for $7.5bn over five years in its second replenishment (2016-20), the Global Fund’s fifth replenishment (2017-19) asked for $13bn over three years—almost triple the annual amount.

We should, however, expect a closer relationship between changes in VfM scores and changes in allocations. If the UK felt VfM had improved (or deteriorated) at a multilateral, you would expect this to have a positive (or negative) influence on its share of the multilateral budget (this controls to some extent for absorptive capacity, which likely varies less over time for a given organisation than it varies across organisations at a point in time). 

Figure 2 presents changes in VfM scores between the 2011 MAR and 2016 MDR. A similar group of multilaterals achieved the best scores in both reviews, such as Gavi, GFATM, IDA, the International Committee of the Red Cross (ICRC), the Asian Development Bank (AsDB), EC , and the United Nations Children's Fund (UNICEF). We found that 2011 VfM scores are able to explain 61 percent of the variation in 2016 scores; but despite this persistence in levels of VfM scores, there are also clear changes between the two reviews. 

Figure 2: VfM Scores in the 2011 MAR and 2016 MDR

The greatest absolute improvements in VfM  were found for the Food and Agriculture Organization (FAO), the Inter-American Development Bank (IADB), the Commonwealth Secretariat, UN Women, and GFATM. Conversely, the largest deteriorations in VfM scores were found for the Private Infrastructurer Development Group (PIDG), the Global Facility for Disaster Reduction and Recovery (GFDRR), the United Nations High Commission for Refugees (UNHCR), the Central Emergency Response Fund (CERF), and ICRC.

Change in VfM score and allocation

Figure 3 shows there is no clear relationship between changes in 2011-16 VfM scores and changes in UK multilateral aid allocation (based on the years following each review, and measured as the percentage change in share of total UK core contributions). The former accounts for less than a fifth of a percent of the variation in the latter.

Figure 3: Percentage Change in Share of UK Multilateral Aid against Change in VfM Score 

Source: CGD analysis, using the 2011 MAR and 2016 MDR assessments, the underlying data accessed via a Freedom of Information request (reference: FOI2021/26517); and the OECD’s dataset for Members’ Total Use of the Multilateral System
Note: VfM scores are calculated as the average of the two MAR/MDR indices for “Contribution to UK development objectives” and “Organisational strengths”.

The UK increased the share of its allocations to just four multilaterals with improved VfM scores—and for two of these the increase is very high relative to the improvement in VfM (UNITAID & International Organisation for Migration (IOM)). In fact, the UK reduced its focus on the majority (14) of multilaterals with improved VfM scores, including for some of the top improvers: UN Women, the Commonwealth Secretariat, and IADB.

Meanwhile, the UK significantly increased its allocations for two multilateral organisations who’s assessed VfM decreased over the period (CERF and the United Nations Peacebuilding Fund (UNPBF). (The former being related to the need to allocate humanitarian assistance quickly).  However, it did take the decision to reduce its allocations for several of its major multilateral partners following their deterioration in VfM: PIDG, AsDB, the African Development Bank (AfDB), and Gavi.

What’s behind this apparent lack of influence on aid allocations?

We recognise some important caveats and limitations to our analysis. The UK will base its multilateral allocation decisions on other valid factors than MAR/MDR VfM assessments. The contributions of other donors matters, for instance: to either fill in the funding gaps left by others, or to match their contributions and share the burden. Additionally, the relationship between MAR/MDR VfM assessments and allocations may not be linear. The levels of scores matter as well as their changes (as reflected in the discrete VfM categories of the MAR/MDR): the Commonwealth Secretariat may have improved by a lot, but it still provided low VfM. And whilst our VfM scores averaged the two sub-indices created in the MAR and MDR, this doesn’t account for diminishing returns to each: for instance, it is better to support an organisation that works fairly well towards some of your objectives, than it is to support an organisation that works very well towards objectives you don’t share. Furthermore, not all multilateral contributions are purely discretionary, notably for the EU and UN agencies, and this will affect the relation between VfM assessments and allocations. MAR/MDR VfM assessments may not neatly affect allocations in the immediate period following the review, either: they may have an effect earlier, if results are known internally before publication; and they may have a delayed effect depending on the timing of multilateral replenishments. A more sophisticated (econometric) analysis could potentially tackle some of these issues; though we felt our approach is intuitive.

One other important contextual point is that the UK substantially increasing its aid budget between the MAR and MDR, including through the increase to 0.7 percent of GNI in 2013. This might have led the UK to increase its presence in smaller multilaterals, even if they had lower VfM, perhaps reflecting a search for influence. 

Is this a problem? And what does it mean?

Our headline finding is that changes in the UK’s assessment of multilateral VfM appeared to have no systematic influence on the share of funding each multilateral received. This is concerning because it implies that—despite high-level political level commitment to VfM/effectiveness and major technical efforts to assess it—this was not a driver of changes in multilateral allocations. That is not to say that the reviews were not influential—the reviews have been used to drive reforms within multilateral organisations; and without them, the UK could have moved away from supporting the most effective organisations. However, it does imply that other factors were the driving force: whether political, convenience, or the influence of other countries. On these points, we would speculate that the cycle of replenishments at multilaterals creates a built-in momentum that is hard to resist.  

For UK and other policymakers trying to reform the multilateral system, we suggest the analysis highlights the high level of built-in inertia in the system and the need to work with other countries to drive multilateral reform (the UK’s assessments were widely shared but unilateral). Our recent policy note draws on this analysis and elaborates how to tackle both these challenges.

The 2011 MAR and 2016 MDR data provided by the FCDO in response to our Freedom of Information request and used in this analysis can be downloaded here.