4.1. KRI Motivated by an interest in appropriately integrating economic, risk, decision, and dynamic disease models to demonstrate the difference between static and dynamic policy models and the importance of changes that occur over time, KRI polio modeling efforts began in 2001 [17] with retrospective characterization of the economic benefits of polio risk management in the United States [9]. Informal discussions of the preliminary work on this topic in late 2001 with the US Centers for Disease Control and Prevention (CDC) led to the establishment of a collaboration between KRI and CDC polio subject matter experts [17]. The KRI-CDC collaboration focused throughout the rest of the decade on the polio endgame (i.e. characterization of risks and risk management options for after WPV eradication). In 2003, KRI presented the decision options for post-WPV eradication policies [8] and developed a DEB dynamic transmission model for polio that included immunity states associated with WPV infection and vaccination with OPV and/or IPV, including transmission by individuals with asymptomatic infections [10]. Given the exclusive use of tOPV at that time, this transmission model used a generic poliovirus serotype and did not consider OPV evolution endogenously [10]. KRI focused on the global policy level and developed estimates of the costs for the different post-WPV-eradication decision options stratified by World Bank income levels (WBILs) to capture some important differences that exist between countries [11]. KRI also characterized the costs and value of the information from the global poliovirus laboratory network (GPLN), which supports global poliovirus surveillance [12]. KRI provided the first quantitative risk estimates for VAPP, cVDPVs, and iVDPVs [13]. The risk estimates appropriately varied by WBIL and type of poliovirus vaccine used by national immunization programs based on statistical analyses of available data at the time and as a function of different post-WPV eradication policies [13]. KRI used the transmission model [10] to explore post-WPV eradication outbreak response policies and provided key insights to the GPEI in 2005 [190] about the benefits of both pre- and post-WPV eradication outbreak response [14], which motivated investments in improvements in GPEI outbreak response activities. Many of these papers appeared in a 2006 special issue of Risk Analysis [15], which also included perspectives on risk management in a polio-free world [16] and on the history and nature of the collaborative modeling process used [17]. The retrospective economic analysis for the US showed significant (hundreds of billions of 2002 US dollars US$2002) in net benefits from US investments in polio immunization [9], which helped to strengthen US commitments to global polio eradication and risk management. Following the development of the integrated model components (i.e. dynamic disease transmission, risk, decision, and economic), KRI performed an economic analysis of post-WPV eradication immunization policies [18]. Given the time horizons considered in the economic analyses that extended beyond the characterization of outbreak events, the integrated model included consideration of potential reinfection and asymptomatic participation in transmission of individuals with waned immunity, with paralysis only occurring in a small fraction of fully susceptible individuals. High-level policy discussions related to control vs. eradication in late 2006 motivated KRI to apply the post-eradication model to estimate the economics of eradication (followed by several different post-WPV eradication immunization policies) compared to a wide range of control options [19]. This analysis demonstrated that eradication (if technically and operationally feasible in a reasonable time) represented a better health and economic option than control with OPV in OPV-using countries [19]. Some discussions at the time included significant pessimism about the ability to stop poliovirus transmission in India and the other remaining endemic countries [191]. KRI modeling suggested that elimination could occur in India with sufficient immunization intensity [19] and demonstrated that achieving eradication is a choice (i.e. the actions that countries and the GPEI take matter with respect to outcomes, and neither failure nor success could be taken as a given). KRI also demonstrated the economic inefficiency of a wavering global commitment to eradication [19]. The economic analysis of post-WPV eradication immunization policies showed that either stopping OPV altogether or switching to IPV dominated the continued OPV use (i.e. control) after successful eradication of WPVs [18]. However, using IPV after WPV eradication represented the option with the highest expected costs and the lowest expected cases, while stopping poliovirus immunization represented an option with lower expected costs and some additional expected cases, which led KRI to recommend research and investment into strategies to reduce IPV costs [18]. KRI performed extensive uncertainty and sensitivity analyses [20]. Recognizing the importance of OPV cessation as an option, KRI demonstrated the need for globally coordinated coordination of OPV cessation due to game-theoretic considerations associated with cVDPV risks that could occur with uncoordinated OPV cessation [21]. This analysis also highlighted the importance of creating a stockpile for post-WPV eradication outbreak response [21]. Due to the complexity and scale of the GPEI, KRI recognized the importance of managing the GPEI as a major project and ensuring sufficient resources for polio eradication to succeed [22]. KRI discussions about this work with GPEI partners highlighted the importance of the GPEI taking the long view and asking for the funds that it needed to succeed with a long-term budget and plan, instead of what it thought it could raise in annual budgeting cycles. Although not specific to polio, by extending a simple integrated theoretical model [192], KRI discussed uncertainty and sensitivity analyses for integrated models [193] and explored the dynamics of priority shifting for eradicable diseases [194], the latter of which also built on prior KRI analysis of a wavering commitment to eradication [19]. Recognizing the importance of a stockpile of OPV for post-WPV eradication outbreak response [21], KRI developed a framework for optimal stockpile design [23]. Although KRI primarily used DEB models, KRI developed an IB polio dynamic disease transmission model that showed the significance of different assumptions about mixing networks, which remain highly uncertain and difficult to model at the global level [24]. In 2010, KRI performed an economic analysis that estimated 40-50 billion US$2013 in net benefits for the GPEI for 1988-2035. The range of estimates depended on whether successfully coordinated OPV cessation following WPV eradication included global use of IPV or not (with the lower end of the range of net benefits (i.e. less desirable) reflecting the use of IPV) [25]. That analysis assumed successful WPV eradication in 2012 and considered the impacts of a delay out to 2015 [25]. KRI contributed to discussions about the role of economic analyses in the evaluation of global disease management efforts [195] and the development of eradication investment cases [196], in book chapters not captured by the systematic review. In 2012, KRI explored trends in the risks of poliovirus transmission in the US and recognized that imported live polioviruses could potentially circulate in a population with high IPV coverage, although the risks in the US appeared low [26]. KRI also explored the probability of undetected wild poliovirus circulation after apparent global interruption of transmission [27] (by extending a simple SC model [197] developed and applied in the mid-1990s to support certification of elimination of polioviruses in the Americas [197–199]). Still focused on post-WPV eradication and the polio endgame, as the GPEI immunization policies evolved, KRI appreciated the need to expand and update its integrated model. Specifically, as the GPEI began using mOPVs, first mOPV1 and then mOPV3, and later using bOPV (which contains both serotypes 1 and 3) for some SIAs, KRI needed to model the transmission of each serotype. KRI identified the need to model population immunity to transmission [1], and widely discussed its key role in prevention [200]. As part of its model update, KRI characterized the global immunization policy options as of 2012 and identified prerequisites for OPV cessation [28]. KRI developed a series of papers published in a 2013 special issue of Risk Analysis that described the components of its expanded and updated poliovirus transmission and OPV evolution model and discussed the role of modeling as part of the polio legacy [2]. KRI performed a comprehensive expert review of the literature on poliovirus immunity and transmission [29] and synthesized the information from the experts to (i) numerically characterize an expanded set of immunity states for its transmission model and (ii) identify significant uncertainties despite the large literature [30]. KRI reviewed the 2012 national polio immunization strategies to characterize updated prospective polio immunization policies and reviewed the seroconversion literature to characterize variability in vaccine take rates for different vaccines and numbers of doses in different settings [31]. KRI also updated its prior review of risks [13] and reviewed the literature related to understanding and modeling OPV evolution [32]. Based on this analysis [32], KRI concluded that its prior statistical model for cVDPV risks based on the historical global use of tOPV [13] offered poor predictive value of risks after the GPEI introduced mOPVs and bOPV. Specifically, the poor performance of the statistical model based on historical data [13] when compared with evidence at the time motivated KRI to include OPV evolution and the development of cVDPVs endogenously in its expanded poliovirus transmission and OPV evolution model (i.e. to use a dynamic and serotype-specific approach) [33]. KRI focused on the need to manage population immunity to transmission considering all individuals in the population, including individuals immune to disease but able to contribute asymptomatically to transmission, most notably those with only IPV-induced immunity [34]. The expanded model of poliovirus transmission and OPV evolution offered insights from modeling a diverse set of actual experiences with wild and vaccine-related polioviruses [33]. Overall, the expanded poliovirus transmission and OPV evolution model (i) uses eight recent immunity states to reflect immunity derived from maternal antibodies in infants, only IPV vaccination, only LPV infection, or both IPV vaccination and LPV infection (to more realistically capture the differences in immunity derived from IPV and LPV), (ii) includes multi-stage waning and infection processes (for more realistic characterization of these processes), (iii) characterizes OPV evolution as a 20-stage process from Sabin OPV (as administered) to fully reverted polioviruses with assumed identical properties to typical homotypic WPVs (to allow cVDPV emergence to occur within the model), (iv) characterizes each serotype separately (to analyze serotype-specific poliovirus properties, vaccination policies and risks), (v) considers explicitly both fecal-oral and oropharyngeal transmission (to account for the differential impact of IPV on fecal and oropharyngeal excretion), (vi) accounts for heterogeneous preferential mixing between mixing age groups and subpopulations, and (vii) accounts for differences between various IPV and OPV routine immunization schedules and the reality of repeatedly missed children during successive SIAs [33, 35, 36]. KRI also updated its estimates of IPV costs in the context of exploring national choices related to IPV use with various delivery options [37] and noted continued high expected costs of IPV. KRI used the updated and expanded integrated global model to identify optimal strategies from a modeling perspective (i.e. with respect to expected health and economic outcomes) to support the GPEI partners as they worked to implement the GPEI 2013–2018 Strategic Plan [201]. In 2014, KRI modeled the dynamics of coordinated cessation of serotype 2 OPV (OPV2) without [38] and with [39] IPV, which demonstrated the importance of using sufficient amounts of tOPV in the run up to OPV2 cessation to increase population immunity to transmission prior to OPV2 cessation [38]. Despite the GPEI emphasis on IPV introduction, these analyses also demonstrated the relatively small expected role of IPV in stopping or preventing transmission in areas with conditions conducive to poliovirus transmission (i.e. relatively high R0, high contribution of fecal-oral transmission, like the countries of interest to the GPEI) [39]. Given delays in achieving eradication and requests from the GPEI partners, starting in 2013 KRI began modeling pre-eradication activities and to explore options to help accelerate eradication. KRI applied its transmission model [33] to characterize the potential impact of expanding target age groups for polio SIAs [35] and to stop and prevent poliovirus transmission in two high-risk areas in northern India [36] and in the high-risk area of northwest Nigeria [40]. Considering potential US risks, KRI developed and applied an IB model to characterize the potential for transmission of polioviruses following an introduction of a LPV into the Amish communities in North America [41]. Consistent with prior recognition of the potential for circulation of imported LPVs in areas with high IPV-only coverage based on its US modeling [26], KRI modeled population immunity to transmission and management strategies for Israel following the observation of WPV serotype 1 transmission in Israel despite its high coverage with IPV only [43]. In contrast to some other areas in the US, KRI reported relatively little heterogeneity in six counties in Central Florida at high risk of importations due to international family entertainment attractions [44]. KRI discussed some lessons from the GPEI relevant to measles and rubella eradication [45]. Insights from KRI modeling showed the importance of focusing on immunization program performance (i.e. achieving high coverage with OPV) to maintain population immunity to transmission as the key to success in the polio endgame [42]. Many KRI modeling studies emphasized the failure to vaccinate with OPV as the primary cause of delay in achieving and maintaining WPV eradication, and the importance of heterogeneity in populations that leads to pockets of preferentially-mixing under-immunized individuals that can sustain transmission [35, 36, 40, 41, 43]. KRI provided a high-level review of the policy impacts of its modeling [3]. In 2015, KRI explored the information from different types of poliovirus surveillance activities and modeled the potential for undetected live poliovirus circulation after apparent interruption of transmission [46] based on earlier exploration [27]. KRI characterized global importations and cVDPVs since 2000 and showed that over 50 countries failed to maintain sufficient population immunity to transmission to prevent paralytic cases from cVDPVs and/or imported WPVs [47]. KRI also modeled three countries that use IPV-only for routine immunization (the US, the Netherlands, and Israel) and demonstrated the decline in population immunity in transmission that occurs when countries switch from using OPV to using IPV only. At the time of global introduction of IPV beginning in OPV-using countries, KRI discussed the safety of IPV and emphasized the potential benefits of using IPV as a first dose to reduce VAPP using data from the US experience [48]. Looking closely at northwest Nigeria, KRI explored the trade-offs associated with different strategies to manage population immunity to transmission that demonstrated the high importance of using more tOPV in SIAs in the run-up to OPV2 cessation and the minimal impact of IPV [49]. KRI published a series of articles in a special issue of BMC Infectious Diseases in 2015 using its updated integrated model that aimed to help national, regional, and global health leaders navigate the polio endgame from 2013 to 2052. Modeling the long-term risks requires characterization of the potential for reintroductions of iVDPVs from a small number of individuals with B-cell-related primary immunodeficiencies [50], for which KRI reviewed the evidence collected since its 2006 statistical analysis [13]. KRI recognized that static modeling of historical data offered low predictive power for future iVDPV risks. As a result, KRI developed a DES model to support the stochastic generation of iVDPV excreters for prospective risk analyses and the exploration of the potential benefits of polio antiviral drugs (PAVDs) [50]. KRI used its iVDPV model and other stochastic risks related to containment in its integrated global model to characterize the risks, costs, and benefits of different future poliovirus risk management options for 2013–2052 compared to the 2013 baseline, which included continued widespread use of OPV for control [51]. Using both the global model [51] and a model of northern Nigeria [49], KRI showed the importance of vaccine choice and preferential use of tOPV in the run-up to globally coordinated cessation of serotype 2 OPV (i.e. OPV2 cessation), which was then-planned and since implemented in late April 2016 [52]. Recognizing the importance of significant tOPV use and sensitive to the time delays and costs of vaccine production, KRI estimated potential tOPV and bOPV needs through 2020 [53]. As global health policymakers approached the final decision point for establishing the timing of OPV2 cessation, KRI explored alternative OPV cessation and IPV introduction timing options [54] that showed substantial financial benefits associated with delayed IPV introduction. KRI demonstrated the importance of using aggressive and high-quality (i.e. rapid, high coverage, sufficiently large scope) outbreak response SIAs after OPV cessation and during the polio endgame [55]. In anticipation of coordinated OPV2 cessation, KRI explored the risks of potential non-synchronous OPV2 cessation [56] and of inadvertent tOPV use after OPV2 cessation [57]. Later work showed the potential risks of non-synchronous bOPV cessation and inadvertent use of serotype 1 or 3 OPV use after bivalent OPV cessation [58]. Using the updated integrated model [51], KRI performed an uncertainty and sensitivity analysis of cost assumptions [59] that continued to demonstrate the relatively high cost of IPV. Recognizing the importance of maintaining high population immunity for serotypes 1 and 3 prior for future coordinated bOPV cessation, KRI demonstrated the benefits of high levels of continued bOPV use and sustaining OPV production through bOPV cessation [60]. Building on prior characterization of iVDPV risks [50], KRI modeled the impact of comprehensive screening to find and treat asymptomatic iVDPV excretors and explored the impact of screening on the expected benefits of PAVDs [61]. KRI explored the potential benefits of investments in a new, ideal poliovirus vaccine assuming the best attributes of OPV and IPV [62]. Emphasizing the importance of actions taken by countries and the GPEI, KRI highlighted the importance of maintaining preparedness throughout the polio endgame [63]. KRI also demonstrated the minor role of IPV in outbreak response when used in conjunction with OPV, and showed that IPV in addition to OPV for outbreak response (in the outbreak area) does not represent a cost-effective option compared to using OPV alone [64]. KRI demonstrated the need to maintain sufficient poliovirus vaccine supplies and stockpiles for outbreak response in the polio endgame [65] and assessed the economic benefits of temporary recommendations for international travel immunization requirements for countries with transmission of WPV1 [66]. Recognizing the increasing role of environmental surveillance for polioviruses, KRI systematically reviewed published poliovirus environmental surveillance studies and reported information related to the design, cost, and effectiveness of these systems [67]. KRI also explored the dynamics of die-out of serotype 2 polioviruses after homotypic OPV cessation and lessons learned from its cessation relevant to the cessation of OPV serotypes 1 and 3 [68]. Reviewing insights from prior modeling [35, 36, 40, 41, 43], KRI demonstrated how under-vaccinated subpopulations can sustain poliovirus transmission despite high coverage in the surrounding population, depending on the degree of mixing and the size of the under-vaccinated subpopulation [69]. Building on these lessons, KRI explored the potential for silent circulation of live polioviruses in small populations [70], and the role of hard-to-reach subpopulations in characterizing the confidence about the absence of transmission for purposes of certifying the eradication of WPV1 [71]. KRI revisited its earlier characterizations of containment risks [13, 51] and explored current containment risks and their management [72]. KRI also discussed the role of system dynamics in integrated polio risk management modeling [4]. With continued failure to stop transmission in Pakistan and Afghanistan as of 2016, KRI developed a model of both countries as one epidemiologically connected area [73]. Modeling poliovirus transmission in Pakistan and Afghanistan suggested that subpopulations of under-vaccinated individuals that preferentially mix with each other probably sustain transmission and that interrupting transmission requires a significant improvement in OPV SIA coverage in these under-vaccinated subpopulations [73]. Further modeling of poliovirus transmission in Pakistan and Afghanistan suggested the need for proactive strategies (as opposed to reactive ones) to stop poliovirus transmission [74], and KRI cautioned against getting distracted by the introduction of IPV from achieving high coverage with OPV SIAs. Exploration of the potential for silent poliovirus transmission in Pakistan and Afghanistan [75] showed the role of surveillance in providing confidence about the absence of transmission. Tradeoffs in key characteristics of the poliovirus surveillance system in Pakistan and Afghanistan [76] suggest some role of environmental surveillance in assuring confidence about the absence of transmission, although KRI identified the need for further characterization of the quality of the information from polio surveillance in Pakistan and Afghanistan to fully explore the benefits of investments in environmental surveillance. Looking prospectively at the polio endgame given failure to succeed in the GPEI objectives by 2018, KRI discussed the role of different poliovirus risks and risk management opportunities [72], and the potential risks of needing to restart OPV [77]. KRI also reflected on the role of integrated modeling to support the global eradication of vaccine-preventable diseases [4]. In 2019, KRI updated its cost estimates of the GPLN including both acute flaccid paralysis (AFP) and environmental surveillance [78]. KRI characterized the impact of hard-to-reach subpopulations on confidence about no undetected circulation in the context of supporting global certification of wild polioviruses [71]. Building on prior recognition of the potential role of a new vaccine [62], KRI commented on an article that reported the results of a new OPV2 vaccine strain (nOPV2) [79] and explored the logistical challenges of modeling and implementing a restart of OPV after its cessation [80]. Although outside of the time window for this review, in early 2020, KRI published an updated version of its integrated model to account for the programmatic experience, vaccination achieved, and epidemiology through 2019 [202]. This process included updating the inputs for its iVDPV risk model [81], and focused on actual and expected performance throughout the polio endgame instead of assuming optimistic and ideal risk management from 2015 on [203] as KRI assumed earlier [51].