Accurately forecasting the demand for health products at health clinics, and ordering the correct quantity, is a significant challenge to ensuring availability of health products at the last mile of health systems. In 2006, CGD convened the Global Health Forecasting Working Group to explore ways to improve demand forecasting for essential medical technologies. While much progress has been made since then in terms of more accurate global forecasting, clinic-level demand forecasts still remain an unaddressed area.
Generating accurate demand forecasts
An accurate demand forecast predicts how much of each medicine is needed for each health clinic—which is critical to ensure high availability of essential health products while maintaining an efficient supply chain. Demand forecasting enables ordering the right amount of stock—if demand is underestimated, it results in stockouts, and if demand is overestimated, it can lead to expired/wasted product. A more accurate clinic-level forecast is also the backbone for demand forecasting at the district and national procurement levels.
Generating an accurate demand forecast is quite complex, with dozens of factors impacting health product demand at a clinic: seasonal variations, disease outbreaks, population and migratory populations, provider prescribing preferences, and introduction of new products. Simple heuristic methods for forecasting demand do not capture all the complexities for products with seasonality and user preference. Applying more sophisticated forecasting methods often reaches the cognitive limit of human capability, especially in resource-limited clinics with staff with competing needs on their time and attention.
Why is demand forecasting so important?
In environments with chronic shortages of health products, when stock becomes available at the national level, there is a tendency to over-order. Such order inflation pushes the system out of sync with actual demand. Demand forecasting also needs to safeguard against biases and other human limitations which thwart objective and accurate forecasting.
The issue of clinic-level demand forecasting is particularly important for family planning, where informed choice by end users across the full range of contraceptive methods is crucial to voluntary family planning programs, and is often challenging to predict. Public health clinics in most low- and middle-income countries typically order contraceptives and other health products based on a simple calculation of past use (or “consumption”) of those products.
Intelligent forecasting methods have the potential to greatly improve predictions of future use by employing more complex calculations that draw on a variety of data sources and learn over time. More accurate predictions will lead to better ordering, ultimately increasing product availability for clients and potentially reducing waste through more efficient supply chain management. Some recent work has shown that AI-driven models can improve vaccine forecasting at the health clinic level in Tanzania.
The opportunity now
More than a decade of investments in supply chain information systems to capture product stock and flows (Logistics Management Information Systems) by USAID, the Bill & Melinda Gates Foundation, GAVI, and the Global Fund, have led to enhanced availability of supply chain data. While it is not perfect, many countries now have two-to-three years of historical consumption data that lends itself to more sophisticated forecasting analysis. That, coupled with dramatic improvements in computing power on mobile devices and improvements in machine-learning algorithms, have created a unique opportunity to address this issue now. In addition to structured historical consumption data captured through supply chain information systems, health clinics typically have plenty of other structured and unstructured data. New technologies can bring together multiple such data sets to improve forecast accuracy.
Leveraging open-source innovation and competition
Use of advanced forecasting methods for clinic demand could be carried out by selecting a single vendor to develop a tool or algorithm. However, using open-source innovation can bring in a greater diversity in approaches and techniques, allowing comparison and learning from those various approaches. USAID recently launched a new forecasting competition that takes this approach, utilizing clinic data from Côte d’Ivoire. To the best of our knowledge, this will be the first quantitative competition of this nature in global health.
Forecasting is well suited to a prize competition approach to model development because performance can be measured over a short time period with a clear “right answer” (the test data), a relatively large number of forecasters can develop unique approaches, and there are minimal costs (beyond team effort) for model development. A prize competition inspires and incentivizes others to explore new methodologies and technologies that improve upon traditional practices.
Forecasting competitions have been widely used in the last three decades to advance the practice and theory of forecasting. Over the years, this approach has proven more effective because it crowdsources refinement and further builds critical nuance into the algorithms. As the global health community at large utilizes, troubleshoots, and finetunes these algorithms, the science of forecasting develops and opportunities for the future open up.
In addition to helping select the approach that may perform best, a forecasting competition helps mobilize participants from diverse industry backgrounds—both within the global health arena but also forecasting experts from commercial retail, healthcare, and e-commerce companies can contribute. It helps elevate the agenda of demand forecasting and become an instrument of change.
Better real-world demand forecasts are not just about the best technical approaches
The success of a forecast modeling approach is likely to vary from country to country. Incorporating the best forecasting methods into large-scale use will undoubtedly require modifications beyond what is successful in the controlled “desktop” setting of a prize competition. For real-world deployment, a model should:
- be understood by and beneficial to the users;
- work with data that is locally available when needed; and
- forecast demand unconstrained by supply.
Furthermore, local stakeholders can improve the model based on their deep understanding of the local context and conditions. In the case of this specific USAID forecasting competition, USAID plans to address this by providing a grant to one organization to socialize their model with prospective local stakeholders and users in Côte d’Ivoire; customize and strengthen the model and data sources; pilot test the model as a decision-support tool at actual sites; and measure the model’s performance compared to traditional order calculation methods.
No matter which model wins, and whatever the outcome of the implementation grant, a forecasting competition will bring together forecasters from multiple organizations, geographies, and industries/sectors. If nothing else, it would spur greater collaboration between demand-forecasters, modelers focused on global health supply chains, and those working to solve demand forecasting problems in private sector supply chains. The links and collaborations it would create will endure long after the competition ends. If successful, this challenge will create a breakthrough innovation in demand forecasting, at great speed. Modern data science and technology have the potential to transform how health clinics order health products. This competition is the start of a journey that hopefully other global health donors can take.
While implementing better demand forecasting provides a solid foundation for getting started with the use of better analytics for optimizing family planning supply chains, the journey should not stop there. Forecasts will always be inaccurate. No matter how sophisticated planning algorithms are, there will always be unknown factors. To better cope with the intrinsic uncertainty in demand, global health donors and country governments should consider applying such crowd-sourced innovation and technical competitions to also optimize the supply planning component of global health supply chains.
See here for more details and to join the forecasting competition
* The authors’ views expressed in this publication do not necessarily reflect the views of the US Agency for International Development or the US government
Disclaimer
CGD blog posts reflect the views of the authors, drawing on prior research and experience in their areas of expertise. CGD is a nonpartisan, independent organization and does not take institutional positions.
Image credit for social media/web: Vadim via Adobe Stock
