The future of pandemic modeling in support of decision making: lessons learned from COVID-19

Abstract

The devastating global impacts of the COVID-19 pandemic are a stark reminder of the need for proactive and effective pandemic response. Disease modeling and forecasting are key in this response, as they enable forward-looking assessment and strategic planning. Via 85 interviews spanning 14 countries with disease modelers and those they support, conducted amid the COVID-19 pandemic response, we offer a qualitative overview of challenges faced, lessons learned, and readiness for future pandemics. The interviewees highlighted several key challenges and considerations in forecasting, particularly emphasizing the complications introduced by human behavior and various data-related issues (including data availability, quality, and standardization). They underscored the importance of effective communication among those who create models, those who make decisions based on these models, and the general public. Additionally, they pointed out the necessity for addressing global equity, debated the merits of centralized versus decentralized responses to crises, and stressed the need for establishing measures for sustainable preparedness. Their verdicts on future pandemic readiness were mixed, with only 43% of respondents saying we are better prepared for a future pandemic. We conclude by providing our vision for how modeling can and should look in the context of a successful pandemic response, in light of the insights gleaned via the interview process. These interviews and their synthesis offer crucial perspectives to shape future responses and preparedness for global health crises.

Background and motivation

The ongoing COVID-19 pandemic has led to over 6.9 million deaths [1] globally, with some sources suggesting that as many as 18 million individuals were lost to the pandemic from 2020 through 2021 alone [2]. It pushed millions into poverty, disrupted governments, and led to socioeconomic instability and greater inequality [3]. Despite the rapid development of highly effective COVID-19 vaccines [4], several factors have worsened the pandemic’s continuing effects. These include global disparities in vaccine distribution, the variable nature of human responses to the pandemic, decreasing political and public engagement, persistent issues related to funding and staffing, and the ongoing evolution of the virus causing COVID-19 (SARS-CoV-2). Furthermore, this is not our first global pandemic, nor will it be the last. Understanding the challenges associated with combating COVID-19 will enable better responses in the future, both in the near term by facilitating improved responses to present threats, and beyond by enabling a more prepared and agile response to future pandemics.

The pandemic response could be viewed through many different eyes—those of a first responder, a leader at a drug manufacturing company, an epidemiologist at a state or local health agency, or a nonprofit director deciding where to allocate funding, to name a few examples. As epidemic modelers, we (the authors) approach the pandemic with the goal of mitigating disease impacts [5,6,7] by collaboratively serving both decision-makers and the general public. First, we build these models to assist decision-makers at the federal, state, and local levels by providing them with insights and data to inform their decisions. For example, said models can help optimize deployment of resources and proactive mandates, support hospitals in planning staffing and bed needs, and calibrate expectations about what “best” and “worst” case outcomes look like given a set of possible assumptions. Second, we aim to make our models useful and accessible to the general public, helping everyone understand the dynamics of the pandemic and the rationale behind various public health decisions.

This paper is the result of our participation in a United States Department of Energy (DOE) program, designed to identify gaps with existing capabilities to inform future development and expansion of DOE’s technologies. The mission of the DOE Office of Science is to deliver scientific discoveries and major scientific tools to transform our understanding of nature and advance the energy, economic, and national security of the USA [8]; improving pandemic response capability falls squarely within the scope of this mission. Over the course of the program, we interviewed 85 individuals around the globe. We began our selection of interviewees with individuals who were in our professional network and actively involved in data-driven COVID-19 response, expanded to second- and third-degree connections by asking interviewees who else they would recommend we speak with, and additionally reached out to specific individuals/groups whose work we wanted to gain insights on, including modelers and decision makers in low- and lower middle-income countries (LMICs). Note that although we specifically contacted the Ministries of Health at all LMICs [9] whose official languages included English or Spanish, our survey unfortunately contains relatively few LMIC voices; our participants skew Western and within one or two degrees of connection from us. The full list of names and/or affiliations of those who chose to be identified are included in Additional file 1.

This survey focuses on individuals who were collaboratively engaging with model building, model use/interpretation, or pandemic response operations at an organizational, local, national, or global scale. Our interviewees included key individuals working in roles across the boundary of COVID-19 modeling, decision making, and community response. Those in leadership roles included Chief (Clinical/Executive/Medical/Preparedness and Continuity/Public Health) Officers, Directors, and Founders, among others. Those in technical roles included analysts, scientists, professors, researchers, and epidemiologists, among others. Those in community and medical roles included physicians, nurses, community clinic organizers, and public health professionals, among others. Their selection for inclusion here was made based on their active (rather than purely academic) and collaborative involvement in the COVID-19 response. Our goal in conducting the interviews was to try to understand the challenges individuals faced in their work and what had helped them overcome these challenges in the context of the COVID-19 pandemic. As we heard common themes emerging, including the importance of collaboration and of accruing and maintaining collective knowledge, we realized that sharing these insights beyond our program could be beneficial to the broader disease modeling community.

Similar publications related to COVID-19 include reflections from individuals [10,11,12,13,14,15,16], those that have focused on the science-policy interface [17, 18], those focused on the response within a specific country [19,20,21] or within LMICs [22], those performing a review of published literature rather than conducting interviews [23, 24], and those focused on addressing challenges such as those highlighted in our interviews (e.g., communicating uncertainty [25] and risk [26,27,28,29], methodological and ethical issues [30, 31]) rather than identifying the challenges themselves. Previous work has also asked the question “were lessons learned studies based on previous epidemics/pandemics helpful in the COVID-19 response?” [32, 33]; The findings show that lessons learned studies based on previous epidemics/pandemics, such as Ebola, were helpful in combating the spread of COVID-19 as well as enhancing the response. Our work differs in its structure (interviews), scope (modelers and those they may support across many countries and organization types), and synthesis (we recommend a path forward based on our findings). Because we rely on conversations and not publications for our insights, we capture inputs that may not have otherwise appeared in print and individuals who may not publish. We can imagine it being useful for policy and decision makers to read collections of articles such as ours, focused on challenges faced by different groups in the COVID-19 response.

Limitations of our study include potential bias due to the selection of interviewees from our professional network, resulting in a skewed perspective towards Western experiences and underrepresentation of voices from LMICs. Despite efforts to contact ministries of health in LMICs, the response rate was low, limiting the generalizability of our findings. Additionally, the qualitative nature of our interviews means our findings are subjective and based on individual perceptions, which may not capture the full breadth of challenges and lessons learned globally. In spite of these limitations, our study adds to the existing body of literature by providing detailed, qualitative insights into the specific challenges and considerations in disease modeling and forecasting during the COVID-19 pandemic. These insights, particularly around data challenges, communication, and the balance between centralized and decentralized responses, are critical for shaping future pandemic preparedness and response strategies. Ensuring sustained investment, fostering global equity in modeling capacity, and improving communication channels will be essential in preparing for future global health crises.

Responses from model creators and users about major challenges, lessons learned, and preparation

Of the 85 interviewees, 15 (18%) were decision makers, 44 (52%) were housed at academic or research institutions, 14 (16%) were in industry, and 12 (14%) worked in medicine or public health. Figure 1 shows the country locations of the respondents, which span 14 countries. Decision makers are defined as individuals responsible for implementing policies, guidelines, or strategies in their respective organizations or regions. Those employed at academic hospitals would be considered in medicine if they have patient contact, and in academia if they do not. All interviewees were actively involved in pandemic response efforts using COVID-19 models. Interviewees’ roles ranged from local to national levels, operating in the government, public, and private institutions. We do not include individuals who did not have ‘‘lived experience’’ during COVID-19 (i.e., we do not include academic or research-focused individuals who solely published papers, nor decision makers who themselves were not involved in developing COVID-19 policies).

The three standard questions we asked interviewees regarding their experiences during the COVID-19 response, the answers to which we recorded with detailed notes, were the following:

• 1. What were the major challenges you faced?

• 2. What were some of the biggest lessons you learned?

• 3. Are we better prepared for the next pandemic?

We obtained consent from the interviewees to have their insights included in this publication. Responses are anonymized throughout (i.e., individuals will not be tied to particular opinions/insights; quotes are paraphrased from sentiments expressed in the interviews as needed to maintain anonymity). Note that the responses presented here represent the thoughts and opinions of individuals, and not those of their employers. Unless otherwise stated, all exposition, interpretation, and suggestions for the future, based on the included responses, are our own.

Fig. 1

Interviewee responses to the question “Are we better prepared for the next pandemic?” shown by the country in which an individual primarily works. Primary work location and institutional location across respondents spans 14 countries. The “Mixed/Maybe” category includes those who said we both are and are not more prepared, who gave an unsure answer, who expressed hope but not a committed affirmative, and who said some version of maybe

Below, we share our qualitative interview findings regarding emergent themes, highlight relevant vignettes, contextualize within the context of the literature, and then discuss what path forward we believe should be taken based on the collective wisdom of the respondents. Major themes explored include modelers and modeling, data, the public and decision makers, internationality and equity, centralization, sustainability, and specific done-well/poorly reflections. Note that certain issues show up across multiple contexts. We include details on which of the challenges/lessons (grouped into similar responses, and mentioned throughout the following sections) were raised most frequently in Additional file 1.

Modelers and modeling

Epidemic modeling played a pivotal role during the COVID-19 pandemic, offering forward-looking analyses to mitigate population and economic impacts of the disease. However, as highlighted in the literature, while models could effectively predict general trends the majority of the time they frequently failed to provide reliable predictions at critical transition periods, such as during the emergence of new variants or shifts in public behavior [34]. While traditional epidemic models often assume homogeneously mixing populations, historic consideration of pandemic scenarios demonstrates the need for models that can dynamically account for heterogeneous behaviors and real-world complexities [35,36,37]. Key modeling challenges identified in the current pandemic literature echo these historic findings; they include inadequate complexity (e.g., inability to account for demographic and regional heterogeneity) [12], underestimation of human behavioral variability [18], and limited adaptability to real-time changes [17]. Furthermore, while COVID-19 case data were often available at the administrative level 1 (e.g., county, district, municipality) by later in the pandemic, most of the epidemiological forecasts were still provided at the administrative level 2 (e.g., states, provinces, parishes) and even administrative level 1 (national or country).

Modelers we interviewed, who worked on both mechanistic and statistical disease models, largely aligned with the literature in recognizing the limitations of existing models, particularly their inability to handle heterogeneous mixing and rapid changes in behavior. Several respondents, particularly from LMICs, noted that many models were designed without sufficient consideration for localized factors, leading to outputs that lacked practical utility. However, there was notable disagreement regarding the value of incorporating additional data streams and/or real-world considerations into models: while many called for more sophisticated models, others emphasized the importance of transparency and usability for decision-makers and the issues associated with fitting models having more moving parts. Our interviewees are notable for their questioning of the utility and appropriateness of making certain forecasts at all, especially in highly heterogeneous regions. Others critiqued an overemphasis on forecasting at the expense of scenario planning and impact evaluation.

Additional challenges highlighted by both the literature and interviewees relate to the scope of effective modeling efforts. Key tasks for modelers include understanding and propagating data uncertainty, accounting for evolving viral dynamics and co-circulating variants, and even assessing the economic impacts of mitigation strategies. Yet, as one modeler notes, traditional epidemic models often focus on single pathogens and health outcomes without considering the unintended social consequences of interventions. This, according to that interviewee, underscores the need for multi-objective modeling approaches that balance health impacts with broader social considerations. Furthermore, despite the availability of more granular data at the zip code or county level in many US states later in the pandemic, our interviews elucidate the reality that these data were not always utilized effectively to support localized modeling. One policymaker mentioned lacking connections to modelers working at the appropriate resolution to address their specific needs. Policy makers responsible for local decisions questioned the lack of granular forecasts that could enable decision making given the significant heterogeneity across their states (e.g., California, Texas, and New Mexico). In our home state of New Mexico, the authors note that there are huge differences in access to medical testing and care in different parts of the state (e.g., in Santa Fe vs. the Navajo Nation); state-level models for New Mexico will be essentially meaningless in rural or indigenous counties. To address such gaps, one interviewee described a successful effort to link local decision-makers with modelers to develop granular forecasts, enabling rural areas to receive tailored insights and more effective decision-making tools. They said this involved leveraging many years of an existing relationship, a funded multi-person modeling team, and “the modelers defining and adapting the models in intense collaboration with decision makers. It takes diving into each others’ worlds!”

Modelers emphasized the distinction between projection (“what if”, based on possible scenarios) and forecasting (“what will”, based on the best guess at reality). Many interviewees felt that scenario-based approaches or impact-related studies often provided more actionable insights, helping policymakers understand potential outcomes under varying conditions. Forecasting, on the other hand, received more skepticism, even from forecasters themselves. One interviewee even likened early forecasting attempts to “crystal ball-gazing”, suggesting that such efforts were fraught with uncertainty and, at times, guesswork. Another said we should not be forecasting when we do not even have “nowcasting” (figuring out what current disease levels actually are) down; they said “It’s like going outside and being totally sensory deprived’ – you can’t tell if it’s raining because you can’t tell if you’re soaking wet!” Others mentioned the forecasters’ dilemma, where predictions themselves influenced behavior (e.g., forecasts predicting low case counts could lead to riskier public behavior, thereby invalidating the predictions).

Finally, some modelers discussed the risks associated with doing modeling. One modeler cautions that while the standard Susceptible, Infectious, Recovered (SIR) model often used for forecasting (which also underlies many more sophisticated models) tends to predict correct numbers for herd immunity or total number of infections throughout an epidemic, it significantly misses both the peak of infectious individuals (by as much as a factor of 2) and the time to that peak; they note this inaccuracy has obvious consequences for mitigation strategies (e.g., “flattening the curve”), ensuring adequate intensive care unit (ICU) resources, and making timely public health decisions. Another modeler noted that a question having been posed does not necessarily mean it deserves an answer, saying that modelers should ask themselves, “What could we make worse by providing an answer for an ill-formed question?” (e.g., they noted, answering a question about when cases would peak could lead to intervention strategies that end following a peak under the assumption that there will not be another one). Another modeler noted the issue of maximizing the impact of available resources, saying any resources devoted to modeling could also be spent elsewhere, so when doing resource allocation one should assess the potential impact of modeling vs non-modeling activities.

In reflecting on these challenges, it is clear to the authors that modeling efforts during the pandemic often grappled with competing priorities: generalization versus local relevance, accuracy versus usability, forecasting versus scenario exploration, modeling versus non-modeling funding. Enhanced interdisciplinary collaboration and pre-crisis relationship building and planning could bridge some of these gaps. For instance, developing modular models that allow stakeholders to toggle assumptions or parameters could balance the need for sophistication with the demand for clarity. Ultimately, while the pandemic highlighted significant limitations in existing models, it also underscored their value as tools for decision-making. Moving forward, investing in models that integrate behavioral dynamics, demographic heterogeneity, and real-time adaptability while ensuring accessibility to non-experts will be critical. As George Box famously noted, “All models are wrong, but some are useful” [38]. The challenge lies in ensuring that usefulness is maximized across diverse contexts and stakeholders.

Data quality, reporting, standardization, infrastructure, and sharing

Despite the unprecedented availability of epidemiology data during COVID-19, data-related issues were the most common topic in our interviews. Over one-third (35%) of respondents cited data challenges. Key obstacles mentioned in the literature include incomplete or delayed reporting, missing demographic or geographic details, and inconsistent data standards across jurisdictions, particularly in low-resource settings with insufficient surveillance infrastructure [18, 20]. These issues were compounded by the pandemic’s rapid evolution, which often outpaced data availability and complicated timely decision-making [21]. The literature emphasizes the need for standardized, high-quality, real-time data to support effective responses. Barriers such as the sheer volume and complexity of data made it difficult for researchers to maintain and communicate modeling results [16]. The dynamic nature of the pandemic further highlighted the importance of synthesizing rapidly emerging evidence [18]. Gaps in public health preparedness, including ineffective data management and communication, have underscored the importance of robust data infrastructure and enhanced surveillance systems [20, 21]. Addressing these deficiencies requires the inclusion of diverse stakeholders to promote equity and strengthen global health systems [22].

Our interviewees largely aligned with these observations from the literature, emphasizing the centrality of data challenges to the limitations of epidemic modeling. Many described difficulties stemming from incomplete or missing data, such as the lack of age or race details in testing data, which limited the ability to develop targeted, equity-focused interventions. Others noted conversely that data overload, particularly the flood of genomic sequencing data during the pandemic, overwhelmed existing analytical capacities, leading to delays in actionable insights; for example, one interviewee noted that the number of SARS-CoV-2 sequences available from the last 3 years is two orders of magnitude larger than that of human immunodeficiency virus (HIV) sequences from the last 40 years. Some interviewees voiced skepticism about whether additional data granularity would have been utilized effectively without improvements in data sharing and modeler-decision maker collaboration.

A recurring issue noted by interviewees was the lack of standardized data collection and reporting. Noted areas in which standardization was a challenge include data from hospitals, case reporting, test positivity definitions, COVID-19 infection levels, and ways of measuring SARS-CoV-2 in wastewater. This lack of standardization, interviewees noted, makes it difficult or impossible to compare findings and/or learn across locations (particularly important when one could potentially use earlier hit locations during a pandemic to prepare in other locations). A USA-based interviewee made the point, however, that simply defining standards is not necessarily enough; they gave the example that the US Centers for Disease Control and Prevention (CDC) had definitions that some states followed, and others did not. Furthermore, a modeler noted that even if standards are consistent, a disconnect in understanding the conditions under which data are collected could lead to flawed conclusions. They gave the example of the setting of vaccination clinics potentially impacting how many people show up, noting that for example maybe people are more likely to go to their local firehouse than a pharmacy, leading to different possible interpretations behind a sharp uptick in vaccination numbers in a city depending on whether a model accounted for this setting-specific effect.

Infrastructure gaps were another commonly mentioned issue, particularly in LMICs. Respondents noted these gaps sometimes ran deeper than equipment or training: for example, getting genomics data did not just involve sending over a sequencer. “You can’t run a DNA sequencer in a place you can’t get electricity”. Multiple respondents noted the lack of automated systems for data collection and reporting, which rendered real-time surveillance infeasible. One public health official remarked, “Our ability to release daily case counts was hindered by the manual nature of our reporting systems”. Even in high-resource settings, respondents noted that the absence of robust data-sharing agreements and platforms constrained the utility of existing data. One respondent working in a hospital setting noted that first responders were swamped with patient care, rendering any system that was overly complicated or time-consuming infeasible. Finally, the often mentioned desire by modelers to account for human behavior in models was often caveated by mentioning issues with the behavior data itself. One modeler said behavioral data was risky to use because of its “noisy and not representative” nature.

The authors believe addressing these challenges requires a multi-pronged thoughtful approach that engages both top-down and bottom-up investment. Improvements in global surveillance infrastructure are essential, particularly in LMICs, to ensure timely and accurate data collection. Equally important is evaluating the feasible level of data recording and reporting under immutable constraints and devising data collection methods that impose minimal overhead on an already overburdened medical establishment. Standardizing data definitions and reporting formats across jurisdictions would facilitate comparative analyses and improve the utility of cross-border collaborations, but ensuring such standards are followed (and follow-able) is a key piece of that process. Additionally, fostering stronger relationships between modelers and decision-makers could ensure that the hard-fought data collection efforts lead to data that are both actionable and aligned with policy priorities. By focusing on these areas, we can enhance the accuracy and relevance of epidemic models, ultimately improving pandemic preparedness and response capabilities.

The public and decision makers

The COVID-19 pandemic highlighted the critical role of public response in shaping the effectiveness of public health interventions. As observed in the literature, public trust is essential for implementing interventions, yet this trust was frequently undermined during the pandemic by factors such as disinformation, contradictory policies, and rapidly evolving science [12, 17, 18, 20]. The politicization of science and interventions further exacerbated this issue, making it challenging for policymakers to gain public buy-in for necessary measures [39, 40]. Effective communication and transparency of science are emphasized as key to addressing these barriers [41, 42].

Our interviews largely aligned with these findings, with over a third of respondents highlighting communication and public trust as central to the pandemic response. Our interviewees mentioned lack of public trust being attributable to influences such as disinformation, changing policies appearing contradictory, evolving science, the emergence of non-experts (e.g., social media influencers, talk show hosts, and academics who had never worked on epidemic modeling) with huge platforms, and other social/economic factors. Public health officials said this mistrust was a problem when it came to doing interventions. As one official emphasized, “You can’t have a public health intervention without the public”. The interviewees tended to also mention actionable steps they think can or should have been taken, such as providing modelers with training in communication and ensuring that public messages are simple, clear, and empathetic. For instance, one modeler shared their frustration: “We weren’t trained for this level of public interaction. Suddenly, we were in the spotlight, and we didn’t know how to handle it”. Another respondent, a policymaker, noted, “The public isn’t interested in the uncertainty bands of a model-they want a straightforward explanation of what to expect”.

Interviewees also identified gaps in communication between modelers and decision-makers. The authors note that in a perfect response, modelers should learn from, plan with, and act in concert with both decision-makers and the communities they serve. However, in practice, the modelers we interviewed still felt at times that they were going unheard while they tried to work collaboratively and purposefully with those groups. Some reported feeling excluded from key conversations. One modeler observed, “It’s hard to build models that are useful when you don’t know what policymakers are willing to do. We ended up guessing what interventions might be considered, and that’s not ideal”.

Fig. 2

A visual of the groups and communication channels commonly mentioned in our responses. Presently, there are weaknesses in connections between the three groups. For a strengthened connection between decision makers and the public, decision makers need to release clear, trustworthy, and consistent messaging; the public needs intervention-level and civic-level engagement. For that between modelers and the public, modelers should receive communications training and learn how to speak to general audiences; an awareness of public behavior will inform and improve models. Finally, for that between modelers and decision makers, modelers again need communication and interpretation training so as to be comprehensible to decision makers; the decision makers need to be willing to provide honest information on what options are on the table, and to give modelers the information and access they need to build good models

The authors emphasize their agreement with the literature and interviewees about the importance of improving communication among modelers, decision-makers, and the public. Figure 2 illustrates the interdependence of these groups and the potential pathways for strengthening communication channels. For modelers, training in public communication and interpretation is essential, enabling them to convey their findings in a manner accessible to both decision-makers and the public. Decision-makers, in turn, must provide clear objectives and constraints to modelers while ensuring that public messaging is consistent and trustworthy. Finally, public engagement strategies must account for regional and cultural differences, addressing inequities in access to information and resources. To enhance preparedness for future pandemics, fostering stronger relationships among these groups is critical. As one respondent aptly summarized, “The pandemic has shown us how much more we could achieve if we were truly working together”.

Internationality and equity

The COVID-19 pandemic underscored the necessity of international and equitable responses to global health crises. The literature consistently highlights the importance of cross-border collaboration, emphasizing that pandemics do not respect political boundaries and thus require globally coordinated efforts [10, 20, 22]. Effective responses depend not only on pooling resources but also on ensuring that these resources, including medical supplies, vaccines, and data, are distributed equitably [22, 43]. A particular focus has been placed on the need to strengthen global health infrastructure, especially in LMICs, to ensure timely and effective responses in all regions [22, 44].

Our interviews broadly reflected themes from the literature but placed a stronger emphasis on empowering local modelers in LMICs. Several interviewees from both LMICs and high-income countries (HICs) highlighted the limitations of importing models developed in HICs, which often fail to account for the unique contexts and challenges faced by LMICs. As one respondent noted, “Models built for high-resource settings didn’t always apply in places with limited infrastructure or competing public health priorities”. Another underscored the importance of local expertise, stating, “Local modelers understand the context far better”. Equitable access to resources and decision-making emerged as a major theme, with one interviewee succinctly observing, “Democratize and decolonialize modeling, and great things will happen”. Respondents emphasized that building local capacity requires more than training alone; it also demands access to computing resources, sustained funding, and strong institutional support.

Functional partnerships were also highlighted as key, with one LMIC-based respondent noting, “The most impactful work I’ve done has been in collaboration with global and local teams”. While global networks and international collaborations were largely praised, some respondents expressed frustration with the inequities inherent in these partnerships. Although there was consensus that international collaboration should involve meaningful inclusion of LMIC scientists, not just as data providers but as active participants and leaders in modeling and public health planning efforts, one LMIC-based respondent described instances where local researchers contributed significant data but were excluded from authorship in publications, saying, “We’re expected to provide data, but we’re not given a seat at the table when it comes to decision-making or recognition”.

The authors believe that fostering global equity in pandemic modeling capacity is possible. First, international funding bodies must prioritize investments in LMIC modeling capacity, including computing resources, training, and sustained institutional support. Second, global modeling collaborations should adopt guidelines to ensure equitable participation and authorship. Third, capacity-building efforts must go beyond technical skills to include training in interdisciplinary collaboration and communication. By addressing these structural inequities and supporting the integration of LMIC modelers into global networks, the international community can better prepare for future pandemics. As one respondent succinctly summarized, “If we truly work together, the possibilities are endless”.

Centralization and decentralization

Existing work emphasizes the need for both centralized and decentralized responses to effectively manage health crises like the COVID-19 pandemic. The literature highlights the importance of empowering national and state health departments through adequate funding, staffing, and the establishment of standardized data collection and dissemination processes, as seen in [14, 15, 19, 21, 22]. These works also stress the necessity of ensuring equity in resource allocation and the inclusion of diverse stakeholders. Conversely, the importance of grassroots, bottom-up responses is also highlighted in the literature, emphasizing community mobilization, local engagement, and addressing health inequities at the local level [14, 15, 21]. The need to strike the right balance between centralized and decentralized efforts is reflected in the literature, which mentions the challenges and benefits of balancing national coordination with local autonomy. Overall, the literature advocates for a hybrid approach that leverages the strengths of centralized coordination while ensuring robust local engagement and support.

The interviews aligned with existing work in suggesting that effective pandemic modeling requires a balance of both central and local efforts, with strong coordination and communication between the two levels. Interviewees had conflicting takes on what “best practice” was in dividing power, subject to limited resources. Many interviewees cited the need to further empower (i.e., fund, staff, and listen to) national and state health departments. Others found a response centered on bottom-up actors more compelling; one modeler noted that if only the central body is engaged and invested, there will not be the on-the-ground or local support to actually do targeted regional modeling or implement interventions.

Interviewees were vocal about the tension between the different layers of response, i.e., the different actors in the system. Describing the relationship between state(s) and nation, one respondent called it “tricky”, commenting on it being unclear to what extent the state actors should contribute to natioanl data collection and modeling efforts. Another said, “federalism (decentralized governance) is both a blessing and a curse”, explaining that its a blessing in that when the federal government is giving bad (or no) advice a good state can exert its own effective response, and a curse in that data are hard to get from each state and states are free to ignore good advice from the national government. The sentiment this respondent expressed, that states should have more power only when the central response is weak or flawed, explains the mixed nature of the sentiments around centralization. The valence of responses were tied to a responder’s perceptions about the efficacy and trustworthiness of the central body in question.

The authors echo the literature’s advocacy for a hybrid response framework that leverages the strengths of centralized coordination while fostering strong local engagement and agree with our interviewees that this balance requires robust investment in both national and local health systems, as well as mechanisms to facilitate coordination and feedback between these levels. The authors also believe that the effectiveness of such a framework hinges on ensuring that local voices are not only heard but integrated into decision-making processes. This discussion reflects broader cultural norms and political dynamics, particularly in federated systems like the USA, where the balance of power between state and federal authorities has long been a source of debate and complexity.

Needs for sustainability, logistics, and resources

The literature emphasizes the need for robust infrastructure, comprehensive training, and sustainable and coordinated funding to maintain pandemic preparedness [14, 15, 21, 22]. It stresses the importance of improved health sector investments and continuous learning for building resilience [14, 15], while highlighting the necessity of political advocacy, legislation, and coordinated efforts to enhance surveillance and data sharing [21, 22].

Many interviewees echoed the above, mentioning needs for adequate and stable funding, training, political advocacy and legislation, and infrastructure for emergency response. Multiple notably placed additional emphasis on the need for preserving institutional memory and preventing burnout among modelers, topics which are less explicitly covered in the literature. One modeler noted that a pandemic response being sustainable means that individuals involved in the response should not all burn out and exit the field during or after the response. When discussing the 16+ h days worked, one individual acknowledged that “There have been marriages that have been burnt away on this alter of the public good”. Another specifically mentioned the need for enough person-power to effectively distribute the workload, suggesting this was a means to mitigate burnout risk. Cynically, a public health worker said that while they hoped more people were hired next time around, they doubted it would happen.

The authors could feel, while conducting the interviews, the emotional fraying that had (or was still) happening for many of our interviewees across all locations and fields as they engaged in the pandemic response. We, having also felt those effects ourselves, agree that more direct attention should be given to institutional memory and workforce sustainability. Burnout among pandemic responders, including modelers, jeopardizes the effectiveness of future responses. To address this, we believe governments and institutions should establish support structures and mechanisms to retain expertise and ensure knowledge transfer (e.g., knowledge repositories and documentation, mentorship and training programs, simulation exercises and scenario planning).

Noted successes, failures, and human impacts

Successful relationship building, collaboration, and data sharing, particularly through platforms like GISAID, are emphasized in the literature as critical components of an effective response [12, 15, 18, 21]. Failures are also noted in the literature, which highlights the need for better preparedness, learning from past events, and improving model utility [10, 16, 20]. As we noted in the previous section, our interviewees’ emphasis on the human aspects of success and failure (specifically emotional support, connectivity, and empathy for both responders and the public) adds a dimension that is less prominently covered in the literature.

Noted successes experienced by our interviewees included relationship building and/or collaboration, model sharing/critiquing/improvement, gathering population-representative data, containment efforts, and the metagenomic database GISAID (although this database was criticized by one interviewee for “playing politics”; the interviewee suggested it should be a public non-nationally-based repository run by the World Health Organization – WHO – rather than being associated with a specific country).

Noted failures experienced by our interviewees included under-estimating the threat posed by the pandemic, not learning from the past well enough (i.e., not learning and/or using the lessons from previous emergent disease events), personnel and/or resource limitations, the presence of many models of limited value (one interviewee asked rhetorically “were there any ‘good’ models?”), and pandemic plans focusing too much on the acute phase and not on the (important) transition to long-term combating of the disease. We also are looking ahead to facing a number of new and emerging threats, including misinformation, mutation and new variants, and new viruses.

Multiple interviewees reminded us that it is humans who are doing this work, and therefore subject to things like burnout, fatigue, ego, etc. They said that with that in mind, we need to focus on emotional support, connectivity, and empathy. Along similar lines, respondents reminded us that the public are also all human - and pandemic fatigue was “understandable (though remarkably poorly understood)” but made the response more difficult. The authors only comment here is that regularly bringing the focus back to the humans at every level, heightening empathy and understanding by modelers and decision makers, is important at all times, not just in pandemic times.

Preparation for the next pandemic

The sentiments as to whether we (the “we” here was interpreted by interviewees as either referring to their field, or to the public/society more broadly) are better prepared for the next pandemic were mixed, but leaned positive. We did not require interviewees to give a “yes” or a “no” answer, so some simply expressed uncertainty, whereas others said “yes and no”. Figure 1 shows the breakdown of how often a given response occurred, with some select caveats shown alongside the results. Of those who responded, 42.6% of respondents said we are better prepared and 11.8% said we are not better prepared. 17.6% said we both are and are not. 5.9% gave an “unsure” answer, expressing hope but not a committed affirmative. Finally, 22.1% said some version of “maybe”. Note that 17 out of the 85 interviewees did not answer this question.

Those who said “yes” to improved readiness often highlighted advancements in genomic surveillance, vaccine technology, and institutional coordination, frequently citing their experiences in well-resourced national or centralized organizations. Conversely, skeptics pointed to persistent inequities, workforce burnout, and fragile political will as evidence that critical systemic gaps remain unaddressed, particularly in low-resource settings. Respondents with mixed views acknowledged progress in areas like vaccine distribution and modeling but cautioned that readiness varies by context and the nature of the next pandemic.

This mixed verdict on future pandemic readiness contrasts with literature that often leans more towards optimistic assessments of improved capabilities post-COVID-19 [13, 15, 19, 22]. However, amongst our interviewees, decision-makers tended to lean more toward optimism regarding pandemic preparedness than modelers, so the relative pessimism of our interviewees is likely related to our population including so many more modelers than other sentiment surveys in the literature. This skepticism among our interviewees highlights ongoing concerns from those in the modeling world about sustained investment, infrastructure, and the political will necessary to maintain pandemic preparedness.

The authors’ vision of the next decade for pandemic modelers

In short, the key to successful pandemic modeling in the next decade involves centralized yet dispersed modeling, sustained investment, data, and communication. The road through the challenges to achieve this vision is shown in Fig. 3.

Fig. 3

Summary of challenges and needs to achieve a sustainable, equitable, and ready response future

Designing for the future

Globalization has created a complex environment for disease spread that requires new and more highly coordinated response efforts. Specifically, the COVID-19 pandemic highlighted major gaps that require coordination and collaboration of governments and scientists around the world to reduce the potential impact of the next pandemic. During the pandemic, there were “too many cooks in the kitchen”, many without prior epidemic modeling expertise, which resulted not only in public distrust but also in significant population and economic impacts [45, 46]. As such, we need centralized modeling centers (e.g., the WHO’s Hub for Pandemic and Epidemic Intelligence, the US CDC’s Center for Forecasting and Outbreak Analytics, or the UK’s Data, Analytics and Surveillance Group within the UK Health Security Agency) around the globe, not just in high income countries, with sustained funding that can provide real-time decision support in both “peace” and pandemic times and develop proactive solutions to mitigate future threats. These centers will need to:

• 1. Develop models for operational purposes, ensuring appropriate validation, verification, and efficient run time,

• 2. Work with data owners and policymakers on developing integrated national surveillance systems with standard data collection and reporting metrics for not just public health data but other data streams (e.g., behavioral, movement, environmental) that may be critical for understanding the spread of diseases,

• 3. Develop tools to address data challenges including quality (i.e., if it is fit for its intended purposes) and integrity (e.g., accuracy, completeness),

• 4. Collaborate with decision makers and receive specific modeling direction, questions, objectives, constraints, etc. (not just model what’s feasible and tractable),

• 5. Integrate disparate modeling approaches (e.g., multi-objective and multi-scale models that capture the complex dynamics and processes contributing to disease spread and its impacts), build consensus, and provide solutions with quantified uncertainty, and

• 6. Provide a trusted, unified voice for the modeling community.

These activities are highly dependent on sustained investments for infrastructure, research, and maintenance. In the much more mature and well-funded field of weather modeling and forecasting, communication and cooperation between these channels can mitigate injuries and deaths. For example, when Hurricane Ian recently hit Florida, although the damage was extensive, early forecasts were made and timely evacuation orders were issued by decision makers to the public, and many were able to evacuate in time.

By empowering each country and locality to have full jurisdiction over their modeling activities—through capacity building, training, and resources—we are likely to see more accurate results and reduced population and economic impacts because the models will be able to address questions more relevant to their needs. Part of the problem is that while it is possible for many modeling groups in western countries to develop models and forecasts for LMICs, without understanding cultural differences and data nuances, it will be impossible to develop models that are actionable.

One of the major challenges highlighted by the pandemic and our interviews was data. The success of the proposed centralized modeling centers will be dependent on the availability of continuous, reliable, heterogeneous, and disaggregated data at all administrative divisions. Development of integrated surveillance systems that follow the One Health approach (people, animals, and the environment) will be key for understanding and mitigating future epidemics and pandemics. However, when it comes to health data, we need to figure out how to minimize the burden that data collection activities may put on healthcare providers. Modelers and decision makers should collaborate to develop data standards and reporting metrics. This will ensure that models use appropriate data in order to provide effective decision support. However, such data processes must (a) not place undue burden on an already stressed system and (b) be demonstrably worth the resources allocated away from other possible tasks; when the US Department of Health and Human Services asked for a public comment period on whether or not they should shut off the reporting of COVID-19 hospitalization data, many hospitals said it is an undue burden on the hospital staff to do this reporting.

Science communication is an essential ingredient that needs to be improved before the next pandemic. While very few people understand the underlying physics-based models used for weather forecasting, this community has had tremendous success in communicating uncertainty (e.g., cone of uncertainty for hurricane forecasting), sudden changes in where the hurricane is expected to make landfall (e.g., multiple path projections), and mitigation strategies (e.g., evacuation orders). However, the public trust was not born overnight but was developed through continuous refinement and exposure of forecasts in digestible forms. These forecasting centers will need to educate the public and decision makers about different types of models, assumptions, limitations, uncertainty, and the role of behavior on changing the outcome (in contrast to weather). While models are anything but crystal balls, the public and stakeholders need to understand that models allow us to explore different potential futures and quantify their impact

Imagined scenario

Note: The following is a short fictional scenario set in the near future that presents our vision of a highly functioning, resilient pandemic response system. It is written in a narrative rather than academic style to highlight the real-world experiences and well-being of public health professionals. The individuals and organizations described are fictional.

A novel highly pathogenic hemorrhagic fever virus, designated EBOL-32, was detected in western Senegal. The West African Epidemiologic Modeling Consortium (WAEMC), based in Abuja, Nigeria, was alerted. They pivoted to structure their approaching monthly red team exercise on modeling and intervention assessment for Ebola-like illnesses. As the number of cases continued to rise, the global modeling and response team spun up their network, circulating the WAEMC red team exercise findings as a starting point for their work. Three months later, after having spread to 106 countries and causing 6275 deaths, the WHO officially declared the disease a pandemic.

A little over 4 months after EBOL-32 was first detected, at 10:30 pm, Priya, an epidemiologist, closed her computer and rubbed her eyes. She had just finished her team’s weekly report on the pandemic status at the county-, state-, and US-national level, and sent it to all of the state health departments. She was proud of this document, because it was part of a national effort that had already saved innumerable lives, while rebuilding public trust in science.

Priya’s report included case and hospitalization forecasts at all geographical levels (county, state, and national) for the next 2 months, and described the time-varying “trust level” she was using to indicate the certainty (or lack thereof) the model had about its predictions. It also included hospital supply tracking, behavioral projections based on social media analysis, and key points for press releases. She was tired, but she knew that after running the report by her team tomorrow and sending it to the state health department she would get a break; her team took turns running point on each report, so she only ended up having late nights around one week per month. The week after they took “point” for the month, her workplace provided an extra 4 h of leave so scientists could take a mental health break.

Jamal, a sociologist, had provided the behavioral simulations. Emma, a communications specialist, had sent over the public statement outlining how our understanding of the situation had changed since the last release, with sources supporting updated understanding. To Priya, it was reassuring to know that she was not doing this alone, that her team contained experts from many fields. They could produce rapid, relevant reports each week in part because case, hospitalization, and supply data were downloadable from the state dashboard, available in a standardized format provided at the national level. State, county, and city data were also on the dashboard for innovations and needs that were unique to local communities.

How was this comprehensive and rapid response possible? In the years following the COVID-19 pandemic, the actors involved in the response knew they needed to build structures that could support equity, institutional memory, and bipartisan public trust. The USA and the EU jointly created an annual “PandemicSight” challenge, loosely modeled after the US CDC’s “FluSight” flu forecasting challenge. In this biennial, month-long program, which is held in a different country each year, modelers spend a week being briefed on the status of that year’s pandemic, are put into teams, and then spend two weeks spinning up functional pandemic models at various geographic resolutions that can be used for prediction. The final week is spent with communications experts, honing the messaging to the public. Participants from countries with less governmental funding are supported by the shared equity fund, and a goal is to develop global partnerships spanning countries of differing socioeconomic statuses. In the time between “challenge” months, participants stay in standby mode by performing shorter local exercises and interface with regular seminars in which lessons learned from local groups are shared.

This is a fictional scenario, but it is within our reach.