New Evidence on the Health Loss but not the Health Gain from WHO's 2009 AIDS Treatment Guidelines

We here at CGD tend to be critical of international agencies like WHO or the UNDP for establishing targets or guidelines without sufficient consideration of the impacts, for good and ill, of those guidelines in the affected countries. Such guidelines often apply standards more appropriate to rich countries and then pressure poor countries to behave as if they were rich.

Take the example of WHO’s recommendations regarding eligibility for free antiretroviral therapy (ART) in affected countries. The current guideline, promulgated in 2015, recommends that all countries, no matter how poor or how severe their AIDS epidemic, immediately start all HIV infected people on AIDS treatment. And UNAIDS reinforces this guideline by advocating that all countries attain the “three 90s,” which imply that 81 percent of everyone living with HIV be on treatment by the year 2020, rising to 90 percent by 2030. Models show that such ambitious objectives are necessary to achieve zero AIDS deaths and zero new infections. These guidelines are useful for setting targets for international donors—and perhaps for middle-income countries like South Africa and Botswana—but they provide no help to a poor country attempting to prioritize treatment scale-up without the resources to treat all its HIV-infected citizens. Decision makers in such circumstances need a decision rule—such as the CD4 thresholds that WHO formerly provided.

The subtitle of WHO’s 2002 guideline was “Guidelines for a public health approach,” signaling an attempt to accommodate the needs of a poor country with limited health sector capacity and vast numbers of HIV-infected people. In that earliest guideline, countries were advised to prioritize patients with CD4 counts less than 200 cells per microliter, a criterion that focused care on the sickest people with HIV. WHO’s justification for taking a public health approach rather than a clinical approach was explained in a 2006 article here. Subsequently, in 2009, WHO revised this threshold to 350 and then in 2013, to 500 leading to the 2015 guidelines removing the CD4 criterion altogether. 

The trend toward increasingly comprehensive treatment recommendations, which ignore fiscal and health system constraints, has been propelled by findings that earlier treatment was correlated with better long-term survival and lower infection rates among the patient’s sexual partners, without reference to the fiscal and health system burden the more comprehensive guidelines imposed. The studies that established these benefits, including HPTN-052, START, and TEMPRANO on the direct benefits to patients and HPTN-052 on the prevention benefits, compared randomly sorted patients who started treatment before their CD4 count descended to 350 to patients who began treatment promptly as soon as their CD4 count dropped that low. The study organizers went to great pains to assure that any differences observed in medical outcomes between those who immediately started ART (at a CD4 > 350) and those whose treatment was deferred could not be attributed to failure of clinics or patients to follow the guidelines. So for example, if a patient learned his or her treatment was deferred and did not voluntarily continue with pre-ART testing and evaluation, the patient would be tracked down in his or her home and given the support necessary to assure they continued testing until they were eligible and subsequently remained on treatment for the follow-up period of two or more years. 

But in real-life programs in severely affected resource-limited settings, it is difficult to assure that a deferred patient will in fact return regularly for testing and evaluation and then begin and sustain treatment when their CD4 count drops below a threshold. So the three clinical trials achieved an accurate measure of the biological outcomes of treatment on the individual with CD4 greater than 350 who immediately starts treatment. But they compare these patients to an extraordinarily compliant group who maintain their connection to the clinic and repeat their lab tests despite the fact that their treatment was at first deferred. 

Instead of looking for medical outcomes, the new paper, by Jacob Bor, Till Baernighausen, and coauthors, focuses on the important mediating variable of patient retention. Patients who do not return after an initial CD4 test cannot possibly benefit from treatment. Using the “regression discontinuity method,” the authors test the hypothesis that offering treatment to individuals improves their subsequent retention in the pre-ART and ART treatment program. Figure 1 compares retention results between the three previous clinical trials, which virtually guaranteed patient retention, and the authors’ study of the “Hlabisa compliers” using these innovative methods.

Image

Instead of explicitly randomizing patients, asking for their consent to participate, and then deferring one group’s treatment, the regression discontinuity approach exploits the fact that routine patients are automatically randomized due to the naturally occurring, day-to-day random deviations of the patient’s CD4 count on either side of the treatment threshold. In contrast to the previous studies, which were designed to prevent impact of eligibility on retention, the new study finds that patients who randomly have a CD4 count of 351 rather than 349 on the day they were tested, and comply with the recommendation they begin treatment, were fully 70 percentage points less likely to be enrolled and retained in treatment 12 months later.

The results in the last two bars of Figure 1 above are for the subset of all patients called “compliers,” whose subsequent treatment accorded with the recommendation of the 350 threshold guideline. For the entire sample of 2,300 patients, binned into 32 subgroups by initial CD4 count, Figure 2 shows the dramatic discontinuity in 12-month retention on either side of the 350 threshold.  

Image

In Figure 2 the 18 percentage point difference between the retention rates to the left and right of the threshold is less than the 70 percentage point difference in the right two bars of Figure 1. This is because Figure 1 excludes the non-compliers, including both the “never-takers” who were eligible but did not start ART, and the “always-takers” who were not eligible but took ART anyway. The former depress the data on the left of Figure 2, while the latter inflate the data on the right. So the 18 percent difference in Figure 2 is closer to a real-world estimate of the disadvantage in retention to people whose treatment is deferred because their CD4 count is too high—and much higher than the negligible difference in retention that would have been misleadingly estimated from the retention results in the first three sets of bars in Figure 1.

So should we conclude, as the authors suggest, that the 350 threshold was an incorrect guideline from WHO and an incorrect policy choice for South Africa, producing more harm than good? To draw such a conclusion would require a much more complete assessment of the costs and benefits of the guideline’s implementation in a public health setting.

When resources are limited, individuals with CD4 counts above 350 can only be initiated on ART by reallocating resources from somewhere else. The most obvious possibility for reallocation would be to withhold ART from someone with a CD4 count below 350. Since the authors do not assess the capacity utilization of the ART clinics in their study, we don’t know whether such a displacement of a sicker person (with a lower CD4) by a healthier one (with the higher CD4) would have been required in Kwa Zulu Natal. But such a displacement might well be required in other settings with much more limited resources.

As I suggest above, the missing discussion in the paper is that of the “opportunity cost” of a program-wide policy of early initiation. Deferring the treatment of a sicker person in order to begin the treatment of a healthier person is only one example of the “opportunity costs” that a country might incur from abandoning the 350 guideline. In general, the term “opportunity cost” refers to all the other valuable social ends that could have been accomplished with the financial and medical resources used to initiate ART for people with CD4 counts above 350. To neglect these other possible uses for these resources implies the belief that none of the initiated patients with a high CD4 count is displacing one with a lower count, that none of them is displacing another patient who urgently needs a vaccination or an appendectomy, and that the financing of the medical resources has no alternative possible useful purpose, such as expanding access to education for girls or building roads to rural areas.

Assuming that there are, in fact, opportunity costs to expanding ART access to those above a CD4 count of 350, the analysis is incomplete without that information. The new study tells us that the benefits of initiation might be much higher than previously thought. But the costs must also be considered. The authors plausibly suggest that the patient-level benefits they estimate for Kwa Zulu Natal Province might be generalizable to the rest of sub-Saharan Africa. But not so the public health considerations, including the opportunity costs. The three types of opportunity costs I list will vary greatly between poor countries with few donors and a high HIV prevalence rate and richer ones like South Africa. This paper should not be interpreted as evidence that thresholds are bad for public health.