> top > projects > LitCovid-sentences > docs > PMC:7796369 > annotations
LitCovid-sentences  

PMC:7796369 JSONTXT 9 Projects

Annnotations TAB TSV DIC JSON TextAE

Id Subject Object Predicate Lexical cue
T1 0-69 Sentence denotes Artificial Intelligence Model of Drive-Through Vaccination Simulation
T2 71-79 Sentence denotes Abstract
T3 80-243 Sentence denotes Planning for mass vaccination against SARS-Cov-2 is ongoing in many countries considering that vaccine will be available for the general public in the near future.
T4 244-365 Sentence denotes Rapid mass vaccination while a pandemic is ongoing requires the use of traditional and new temporary vaccination clinics.
T5 366-487 Sentence denotes Use of drive-through has been suggested as one of the possible effective temporary mass vaccinations among other methods.
T6 488-661 Sentence denotes In this study, we present a machine learning model that has been developed based on a big dataset derived from 125K runs of a drive-through mass vaccination simulation tool.
T7 662-768 Sentence denotes The results show that the model is able to reasonably well predict the key outputs of the simulation tool.
T8 769-970 Sentence denotes Therefore, the model has been turned to an online application that can help mass vaccination planners to assess the outcomes of different types of drive-through mass vaccination facilities much faster.
T9 972-974 Sentence denotes 1.
T10 975-987 Sentence denotes Introduction
T11 988-1106 Sentence denotes Since the COVID-19 pandemic started, efforts for vaccine development and production began in many different countries.
T12 1107-1219 Sentence denotes It is argued that an effective vaccine can be the best solution to end and minimize the impacts of the pandemic.
T13 1220-1434 Sentence denotes While some countries have already started vaccine testing and production, it is estimated that approved vaccination solutions for large-scale implementation can be ready as soon as January 2021 or even earlier [1].
T14 1435-1635 Sentence denotes Once a vaccine becomes available, the next challenge would be the vaccination of large numbers of people in a short period of time to minimize further human and economic impacts of the pandemic [2,3].
T15 1636-1699 Sentence denotes Rapid mass vaccination requires many local vaccination clinics.
T16 1700-1917 Sentence denotes Considering that a vaccine will become available for the general public use in the near future, it is important to start planning ahead of time to be able to implement mass vaccination effectively and efficiently [4].
T17 1918-2122 Sentence denotes Therefore, access to mass vaccination modeling and simulation tools has become very important for public health units that are going to plan, manage and run different types of mass vaccination facilities.
T18 2123-2325 Sentence denotes Use of drive-through vaccination has been tested during past public health emergencies such as the H1N1 and during the COVID-19 pandemic for rapid testing and it has shown some promising outcomes [5,6].
T19 2326-2547 Sentence denotes A significant advantage of the drive-through clinics is their lower virus transmission possibility compared to walk-in clinics because people are isolated in their cars and are not in direct contact with other people [7].
T20 2548-2686 Sentence denotes They are only in contact with the immunization staff that are likely vaccinated first and are equipped with personal protection equipment.
T21 2687-2896 Sentence denotes However, studies show that rapid vaccination using drive-through clinics requires proper site selections and design, human resources management, and careful attention to operational and logistical details [8].
T22 2897-3178 Sentence denotes COVID-19 mass vaccination can be implemented using a combination of traditional approaches such as health clinics, pharmacies, nursing homes, schools, workplaces, places of worship, and innovative and temporary facilities such as drive-through and large stadiums, and parking lots.
T23 3179-3261 Sentence denotes Use of large temporary facilities allows for rapid and safer mass vaccination [9].
T24 3262-3525 Sentence denotes Drive-through facilities have lower disease transmission, low virus exposure, have large throughput, provide better security, and are more accessible and comfortable particularly for individuals with mobility issues or are geographically dispersed [2,7,10,11,12].
T25 3526-3715 Sentence denotes The drive-through method has some limitations and shortcomings as its usage is influenced by weather conditions, requires suitable and available spaces, and significant logistical planning.
T26 3716-3836 Sentence denotes Moreover, drive-through can cause traffic issues in the localities and may expose staff to carbon monoxide exposure [6].
T27 3837-4000 Sentence denotes Despite these, drive-through facilities for mass vaccinations are recommended and are being seriously considered as part of the COVID-19 immunization process [13].
T28 4001-4098 Sentence denotes Drive-through vaccination was largely used in the USA in 2009 during the H1N1 influenza pandemic.
T29 4099-4366 Sentence denotes Subsequent studies found that with reasonable processing time and negligible carbon monoxide exposure, drive-through is a feasible and effective alternative to traditional walk-in clinics by providing faster vaccination under lower disease transmission risks [14,15].
T30 4367-4539 Sentence denotes Drive-through clinics have been used for testing and provision of health services such as prenatal and pharmacy services during the COVID-19 pandemic as well [16,17,18,19].
T31 4540-4683 Sentence denotes The use of drive-through during the COVID-19 pandemic demonstrated its effectiveness in the absence of enough clinical testing facilities [20].
T32 4684-4802 Sentence denotes Drive-through mass vaccination facilities can be designed and implemented in various shapes and sizes [7,19,21,22,23].
T33 4803-4948 Sentence denotes Drive-through facilities with more dispensing lanes can provide higher throughputs and prevent traffic overflow onto neighboring streets [10,24].
T34 4949-5083 Sentence denotes Drive-through facilities need to be staffed with different skills including immunization, nursing, admin, IT, logistics, and security.
T35 5084-5166 Sentence denotes Drive-through clinics can be set to work 24 h a day in two, three, or four shifts.
T36 5167-5477 Sentence denotes Although drive-throughs have lower disease transmission risks, especially when staff and visitors use personal protective equipment and are vaccinated before vaccinating others, attention must be paid to the safety and the security issues related to traffic, drivers’ behaviors, and extreme weather conditions.
T37 5478-5614 Sentence denotes Hence, large drive-through clinics may need to be supported by local emergency services such as police, fire, and paramedics [18,25,26].
T38 5615-5903 Sentence denotes Several studies have modeled and examined mass vaccination and point of dispensing clinics using discrete event, agent-based, optimization modeling, and multi-criteria decision support systems for better layout, efficiency, resource allocation, and scheduling [2,10,22,26,27,28,29,30,31].
T39 5904-5975 Sentence denotes Some of these models have been turned into software packages and tools.
T40 5976-6151 Sentence denotes Examples of such tools included Clinic Planning Model Generator developed by Aaby et al. [22], Maxi-Vac by the US Centers for Disease Control, and Real Opt by Lee et al. [10].
T41 6152-6343 Sentence denotes However, new models and tools are needed to incorporate the specific challenges posed by the COVID-19 pandemic and emerging technological solutions in simulations and artificial intelligence.
T42 6344-6586 Sentence denotes This paper aims to use the advancements in simulation tools and artificial intelligence to develop an application that can help mass vaccination planners to quickly assess potential outputs of different drive-through mass vaccination clinics.
T43 6588-6590 Sentence denotes 2.
T44 6591-6612 Sentence denotes Materials and Methods
T45 6613-6793 Sentence denotes The artificial intelligence (AI) discussed here was developed based on a big dataset derived from an existing drive-through mass vaccination simulation created by the authors [32].
T46 6794-6927 Sentence denotes In this section, we briefly explain the drive-through simulation tool and then provide some details about our machine learning model.
T47 6929-6933 Sentence denotes 2.1.
T48 6934-6962 Sentence denotes The Drive-Through Simulation
T49 6963-7085 Sentence denotes The drive-through model used in this study is a hybrid model consisting of a discrete event and an agent-based simulation.
T50 7086-7158 Sentence denotes The model contains several agents including passengers, staff, and cars.
T51 7159-7290 Sentence denotes The physical layout of the drive-through (Figure 1) can be extended up to ten lanes with a maximum area of 20,000 (150 m by 130 m).
T52 7291-7461 Sentence denotes Passengers entering the drive-through are screened at the screening station to ensure that they meet the vaccination criteria and receive the necessary handout materials.
T53 7462-7692 Sentence denotes Cars and passengers passed the screening phase move to one of the open lanes for registration, vaccination, and recovery, which can be determined in part by the availability of High Occupancy Vehicle Lane and Pre Registered Lanes.
T54 7693-7786 Sentence denotes Vaccination will be conducted by professional immunization staff at the vaccination stations.
T55 7787-7875 Sentence denotes Cars move to one of the available parking spaces located after the vaccination stations.
T56 7876-7976 Sentence denotes Individuals experiencing complications after vaccination can be taken care of by the relevant staff.
T57 7977-8024 Sentence denotes Each station is supported by up to four kiosks.
T58 8025-8167 Sentence denotes The discrete event model of the drive-through simulation generates the service process for each lane from registration to recovery (Figure 2).
T59 8168-8329 Sentence denotes Start and end time in each station is measured through time blocks (for example, timer and timeRe refer to the start and end time for registration respectively).
T60 8330-8408 Sentence denotes A sample of our drive-through simulation experiment results is presented here.
T61 8409-8516 Sentence denotes Table 1 shows the parameters setting for the experiment and Figure 3 presents the output of the experiment.
T62 8517-8602 Sentence denotes The values presented in Figure 3 are the average of 50 iterations of this experiment.
T63 8603-8772 Sentence denotes According to these results, a total of 1933 cars with 5804 passengers will be processed in the drive-through mass vaccination during a day with three shifts of 8 h each.
T64 8774-8778 Sentence denotes 2.2.
T65 8779-8791 Sentence denotes The AI Model
T66 8792-8906 Sentence denotes The high fidelity AnyLogic model suffers at inference time, due to the computation cost of running the simulation.
T67 8907-9011 Sentence denotes A single simulation run can take up to 90 s, which may not be efficient in any real time analysis tasks.
T68 9012-9214 Sentence denotes Due to the stochastic nature of the drive-through simulation, a significant number of simulations runs (using Monte Carlo method) is needed for each parameter setting that requires more simulation time.
T69 9215-9351 Sentence denotes In order to alleviate this, we attempt to train a neural network to predict the outputs of the simulation based on the model parameters.
T70 9352-9479 Sentence denotes As is often the case with training neural networks, the problem is rooted in gathering large amounts of data on which to train.
T71 9480-9614 Sentence denotes For this, we make use of the AnyLogic parallelized computation ability, to simulate large batches of simulation runs at the same time.
T72 9615-9749 Sentence denotes In doing so, we strategically sample across a large range of parameters, near the domain of interest required of real-world scenarios.
T73 9750-9802 Sentence denotes Parameter simulation ranges can be found in Table 2.
T74 9803-9935 Sentence denotes We chose to sample all variables uniformly, as we do not have any distribution knowledge of the real-world application of the model.
T75 9936-10092 Sentence denotes Furthermore, by sampling in such an unbiased fashion, we reduce the chances of the network overfitting based on some intrinsic property of our training set.
T76 10093-10185 Sentence denotes Some conditional restrictions are built regarding the binary variables, as noted in Table 1.
T77 10186-10292 Sentence denotes Altogether, we generated about 125 k simulation samples, providing us with a dense enough training domain.
T78 10294-10296 Sentence denotes 3.
T79 10297-10304 Sentence denotes Results
T80 10305-10418 Sentence denotes We utilized an 80–20 train-test split to train a 5-layer feed forward neural network, which is shown in Figure 4.
T81 10419-10552 Sentence denotes We trained the network using the Adam optimizer with a learning rate of 0.001, minimizing the mean absolute error, in batches of 256.
T82 10553-10649 Sentence denotes Training was stopped after 100 epochs, with the test and validation error displayed in Figure 5.
T83 10650-10804 Sentence denotes Our final network is 10 Mb in size, which is small enough to be implemented even on the smallest of mobile devices and is able to infer results instantly.
T84 10805-11072 Sentence denotes More precisely, by entering the parameters (within reasonable ranges) listed in Table 1, our model is able to predict information such as cars and passengers passing through the vaccination center, average wait times throughout the day and overall completion metrics.
T85 11073-11116 Sentence denotes An example of this can be seen in Figure 6.
T86 11117-11226 Sentence denotes Our neural network model is capable of prediction orders of magnitudes faster than the full simulation model.
T87 11227-11407 Sentence denotes More precisely, predictions were on average computed in 0.027 s, with minimum and maximum times of 0.025 and 0.039 s, respectively, in a simulation of 1000 random input variations.
T88 11408-11460 Sentence denotes Distribution of the results can be seen in Figure 7.
T89 11461-11606 Sentence denotes Notice that the time scale we achieve here is on the order of milliseconds, whereas previously we required minutes to produce these same results.
T90 11607-11733 Sentence denotes We achieve an improvement of over 3000× in terms of speed, largely due to the computation cost of simulating the entire event.
T91 11734-11815 Sentence denotes Furthermore, we do all this locally as opposed to the need for cloud computation.
T92 11817-11819 Sentence denotes 4.
T93 11820-11830 Sentence denotes Discussion
T94 11831-12023 Sentence denotes The drive-through model developed in this study is a meta-model (or models of models) that as described by Obsie et al. [33], represents a deterministic proxy for stochastic simulation models.
T95 12024-12283 Sentence denotes However, developing this type of meta-models requires running complex simulation for many combinations and then using the data generated by each run to create a simpler machine learning model that is a reasonable approximation of the initial simulation model.
T96 12284-12362 Sentence denotes The meta-models can then provide quick predictions for different input values.
T97 12363-12505 Sentence denotes While simulation-based or meta-modeling is not a new concept [34], its application in different fields and simulation types are still limited.
T98 12506-12590 Sentence denotes In this study, we applied meta-modeling approach in the context of mass vaccination.
T99 12591-12732 Sentence denotes The AI-based drive-through model developed here has been deployed as a web-application that can be used by potential users (www.adersim.org).
T100 12733-12866 Sentence denotes The approach serves our purpose of providing a general decision supporting tool for planning a large-scale drive-through vaccination.
T101 12867-13084 Sentence denotes A general decision unit, such as a municipal government or a public health region, may simply input its unique parameters, and our model will generate accurate prediction of waiting time, capacity, etc., very quickly.
T102 13085-13304 Sentence denotes Since our AI model is much faster than the simulation model, municipal governments or public health regions may compare multiple decisions, regarding the number of lanes, staffing levels, etc., and make the best choice.
T103 13305-13403 Sentence denotes Further, our AI model is as small as 10 Mb, which can be implemented on any mobile device as well.
T104 13404-13576 Sentence denotes Our drive-through simulation has been developed and parameterized based on the past and current best practices available for drive-through design, settings, and operations.
T105 13577-13786 Sentence denotes However, because the details of a potential SARS-Cov-2 vaccination protocols are not available yet, our drive-through mass vaccination simulation tool can be further tuned as new information becomes available.
T106 13787-13975 Sentence denotes For example, the vaccination and recovery times may be very much influenced by the specific immunization protocols for SARS-Cov-2 vaccine, which may be different from previous vaccination.
T107 13976-14081 Sentence denotes In the absence of real drive-through cases, such simulations can play a significant role in pre-planning.
T108 14082-14321 Sentence denotes Another potential application of our method in the mass immunization case would be an extension of this approach for other mass vaccination clinics such as large temporary walk-in clinics that is currently under development by the authors.
T109 14322-14442 Sentence denotes In future, we would like to further refine our neural network to further reduce its complexity and enhance its accuracy.
T110 14443-14570 Sentence denotes More importantly, it will be interesting to see whether the same network will serve more general cases as they arise in future.
T111 14572-14574 Sentence denotes 5.
T112 14575-14586 Sentence denotes Conclusions
T113 14587-14803 Sentence denotes Drive-through clinics may be able to play a significant role in SARS-Cov-2 mass vaccination process as they provide a faster and safer options, as proven to be effective for SARS-Cov-2 testing in different countries.
T114 14804-14901 Sentence denotes As such, drive-through option can complement other traditional and new mass immunization methods.
T115 14902-15087 Sentence denotes In this paper, we demonstrated how data generated from a simulation can be used to develop a new and better predictive machine learning model for drive-through mass vaccination clinics.
T116 15088-15268 Sentence denotes The generated model can be used to obtain quick predictions of the number of people to be vaccinated and the average time it takes for vaccination under various parameter settings.
T117 15269-15433 Sentence denotes The results show promising outcomes in applying this method in other aspects of pandemic management where large numbers of simulations are developed for prediction.
T118 15434-15639 Sentence denotes Most of these simulations have the potential to be further enhanced and turned into artificial intelligence models that can help end users and policy makers to assess the impacts of various policy options.
T119 15641-15764 Sentence denotes Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
T120 15766-15786 Sentence denotes Author Contributions
T121 15787-15826 Sentence denotes J.W., A.A. and M.C. conceived the idea.
T122 15827-15874 Sentence denotes A.A. and M.M.N. developed the simulation model.
T123 15875-15937 Sentence denotes S.Z.V. developed the AI model with feedback from M.C. and A.A.
T124 15938-15978 Sentence denotes S.Z.V. and A.A. drafted the manuscripts.
T125 15979-16047 Sentence denotes J.W. and M.M.N. reviewed the manuscript and made intellectual input.
T126 16048-16124 Sentence denotes All authors have read and agreed to the published version of the manuscript.
T127 16126-16133 Sentence denotes Funding
T128 16134-16290 Sentence denotes Public Health Agency of Canada; Canadian Institute of Health Research, Ontario; Research Funds, National Science and Engineering Research Council of Canada.
T129 16291-16416 Sentence denotes The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
T130 16418-16454 Sentence denotes Institutional Review Board Statement
T131 16455-16470 Sentence denotes Not applicable.
T132 16472-16498 Sentence denotes Informed Consent Statement
T133 16499-16514 Sentence denotes Not applicable.
T134 16516-16543 Sentence denotes Data Availability Statement
T135 16544-16632 Sentence denotes The data presented in this study are available on request from the corresponding author.
T136 16633-16690 Sentence denotes The data are not publicly available due to its huge size.
T137 16692-16713 Sentence denotes Conflicts of Interest
T138 16714-16758 Sentence denotes The authors declare no conflict of interest.
T139 16760-16939 Sentence denotes Figure 1 The 2D layout (a) and 3D (b) of the drive-through mass vaccination simulation tool (available at: https://cloud.anylogic.com/model/583c2075-6a8b-41be-8a03-d692eba71683).
T140 16940-17032 Sentence denotes Figure 2 The service lanes flowchart of the drive-through mass vaccination simulation tool.
T141 17033-17222 Sentence denotes Figure 3 Sample simulation results for number of cars and passengers using the drive-through (a) and average time spent in the drive-through (b) based on drive-through settings in Table 1.
T142 17223-17334 Sentence denotes Figure 4 Each layer in our network model consists of a fully connected tensor with a Relu activation function.
T143 17335-17441 Sentence denotes The first four all utilize a 20% dropout layer for regularization, while the final feature layer does not.
T144 17442-17546 Sentence denotes Figure 5 Test (blue) and validation (yellow) set errors over the course of training through 100 epochs.
T145 17547-17603 Sentence denotes Validation error seems to level off around 50 epochs in.
T146 17604-17677 Sentence denotes Figure 6 Example of our model predictions for a set of input parameters.
T147 17678-17778 Sentence denotes All time data is sampled on 10 min intervals but can easily be adjusted for finer grain information.
T148 17779-17868 Sentence denotes Wait time distributions align with pre-observed results directly from the AnyLogic model.
T149 17869-17904 Sentence denotes Red line represents mean wait time.
T150 17905-18011 Sentence denotes Figure 7 Computation times across 1000 random input samples using our fully trained neural network model.
T151 18012-18091 Sentence denotes Table 1 Parameter settings for the sample drive-through simulation experiment.
T152 18092-18109 Sentence denotes Parameters Value
T153 18110-18129 Sentence denotes Lane 1-Lane10 Open
T154 18130-18169 Sentence denotes Minimum number of passengers in cars 1
T155 18170-18209 Sentence denotes Maximum number of passengers in cars 5
T156 18210-18259 Sentence denotes Number of cars coming to drive through per min 5
T157 18260-18297 Sentence denotes Average registration time (min) 4.24
T158 18298-18334 Sentence denotes Average vaccination time (min) 3.26
T159 18335-18365 Sentence denotes Average recovery time (min) 4
T160 18366-18417 Sentence denotes Number of staff in each station in lane 1-lane10 4
T161 18418-18458 Sentence denotes Assign High Occupancy Lanes (HOV) FALSE
T162 18459-18516 Sentence denotes Fraction of cars pre-registered for vaccination (0–1) 50
T163 18517-18558 Sentence denotes Pre-registration impact factor (0–1) 0.5
T164 18559-18590 Sentence denotes Consider pre-registration TRUE
T165 18591-18619 Sentence denotes Low occupancy vehicle FALSE
T166 18620-18658 Sentence denotes Fraction of non-adult passengers 0.15
T167 18659-18703 Sentence denotes Fraction of cars rejected at screening 0.01
T168 18704-18718 Sentence denotes Shift hours 8
T169 18719-18738 Sentence denotes Number of shifts 3
T170 18739-18766 Sentence denotes Average screening time 0.5
T171 18767-18795 Sentence denotes Minimum screening time 0.25
T172 18796-18823 Sentence denotes Maximum screening time 0.5
T173 18824-18882 Sentence denotes Dynamically learn and adjust cars going to each lane TRUE
T174 18883-18920 Sentence denotes Use schedule for incoming cars FALSE
T175 18921-18975 Sentence denotes Table 2 Parameter sampling ranges for model training.
T176 18976-18999 Sentence denotes Parameter Range Notes
T177 19000-19033 Sentence denotes Average recovery time (min) 5–10
T178 19034-19070 Sentence denotes Average registration time (min) 2–7
T179 19071-19101 Sentence denotes Average screening time 0.25–1
T180 19102-19134 Sentence denotes Maximum screening time 1 fixed
T181 19135-19170 Sentence denotes Minimum screening time 0.25 fixed
T182 19171-19206 Sentence denotes Average vaccination time (min) 2–7
T183 19207-19262 Sentence denotes Maximum number of passengers in cars 5–7 integer only
T184 19263-19309 Sentence denotes Minimum number of passengers in cars 1 fixed
T185 19310-19353 Sentence denotes Fraction of non-adult passengers 0.15–0.30
T186 19354-19410 Sentence denotes Number of cars coming to drive through per minute 0.5–5
T187 19411-19462 Sentence denotes Fraction of cars rejected at screening 0.01 fixed
T188 19463-19486 Sentence denotes Shift hours 6, 8 or 12
T189 19487-19567 Sentence denotes Number of shifts 2, 3 or 4 Depending on shift hours such that total hours = 24
T190 19568-19582 Sentence denotes Days 1 fixed
T191 19583-19640 Sentence denotes Lanes 1–10 True or False At least 1 lane is always open
T192 19641-19713 Sentence denotes Number of staff in each lane 1–4 Can differ lane to lane, integer only
T193 19714-19754 Sentence denotes Consider pre-registration True or False
T194 19755-19803 Sentence denotes Assign High Occupancy Lanes (HOV) True or False
T195 19804-19904 Sentence denotes Dynamically learn and adjust cars going to each lane True or False Can only be true if HOV is open
T196 19905-19974 Sentence denotes Low occupancy vehicle True or False Can only be true if HOV is open