# Notebook from k-konst/COVID-Vaccine-App
Path: .ipynb_checkpoints/COVID_Vaccine_Model-checkpoint.ipynb

# import libraries
import tellurium as te
import numpy as np
import matplotlib.pyplot as plt
import lmfit
import csv
from ipywidgets import *
from IPython.display import clear_output_____no_output_____

# A Vaccine Model for COVID-19_____no_output_____## Biological Background
The COVID-19 pandemic, caused by the novel coronavirus SARS-CoV-2, quickly spread across the globe following the first known case reported in Wuhan, China on November 17, 2019 \[1\]. The first case in the US was reported in Washington state on January 21, 2020, the same day the virus was confirmed to transmit person-to-person. Despite travel restrictions and quarantines around the world, the epidemic continued to spread, and the World Health Organization (WHO) declared COVID-19 a pandemic on March 11, 2020 \[2\]. By December 18, 2020, there were 17,459,296 cumulative cases and 313,588 deaths in the US alone \[3\].

The time from infection by SARS-CoV-2 and onset of symptoms is called the incubation period, and cases are asymptomatic during this period. It is estimated that the incubation period for COVID-19 is 4-5 days \[4\]. Once this period is over, a symptomatic case of COVID-19 displays symptoms like fever, cough, shortness of breath, and loss of taste and smell \[4\]. In general, symptomatic cases of a disease are more infectious than asymptomatic cases, since symptoms like a cough allow for more of the virus to be spread through droplet and airborne transmission.

A COVID-19 vaccine has largely been seen as a sort of holy grail for ending the pandemic. Traditionally, a vaccine contains a dead or weakened strain of the virus. The injection of this strain through vaccination stimulates the body’s adaptive immune system to develop antibodies against the virus, which will then help prevent future illness. However, messenger RNA (mRNA) vaccines have been developed for SARS-CoV-2. The mRNA in the vaccine codes for a protein on the surface of the SARS-CoV-2 virus. The ribosomes in our cells transcribe the mRNA to make the protein. Then the protein is broken down and the pieces are displayed on our cells' surfaces, triggering an immune response \[5\]. Regardless of the type of vaccine, the end goal is the same: to generate the antibodies needed to combat infection should exposure to the real virus occur.

This app primarily serves as an educational tool and aims to develop a vaccination model for the COVID-19 pandemic in the United States. You'll investigate three epidemiological compartmental models of increasing complexity, run pandemic simulations, and see what happens when different variables are changed. Then you'll use these models along with real US COVID-19 pandemic data to answer some questions about a US vaccine strategy. This app is best experienced by completing each exercise in order.
_____no_output_____## The Mathematical Representation
The models in this app are in the form of compartmental network models and based off the paper ["An SVEIR model for assessing potential impact of an imperfect anti-SARS vaccine"](https://www.researchgate.net/publication/41823768_An_SVEIR_model_for_assessing_potential_impact_of_an_imperfect_anti-SARS_vaccine) by Gumel, McCluskey, and Watmough, which models the introduction of a SARS vaccine with less than 100% efficacy \[6\].
### What is a Compartmental Model?
A compartmental model divides the total population under study into various categories and models the movement of individuals between categories using differential questions. The rate of transition between categories is often dependent on a rate constant and the size of one or both categories involved in the transition. For example, if we wanted to model the infection of currently uninfected, but susceptible individuals, we may write

$\frac{dS}{dt} = -\beta SI$

$\frac{dI}{dt} = \beta SI$

where $\beta$ is the contact rate between susceptible and infected individuals (often described as the number of personal encounters per day multiplied by the probability of infection), S is the number of susceptible individuals, and I is the number of infected individuals. The inclusion of both the susceptible and individual populations in this transition is logical. The more susceptible individuals there are, the more opportunities for infection. The more infected individuals there are, the more the infection can spread. We can also model this transition through an arrow diagram, with the rate of transformation $\beta SI$ written above the arrow:

It's important to note that these models assume parameters like $\beta$ are constant with time. In reality, if a stay-at-home order were put in place or a mask-wearing mandate were implemented, these policies would decrerase the number of encounters a person would have per day and the probability of infection. Either or both of these effects would in turn decrease $\beta$. _____no_output_____### The Basic Reproduction Number ($R_0$)
One more important concept we'll be using in this app is the basic reproduction number, or $R_0$ value. $R_0$ is the number of individuals each infected person will infect during the duration of illness. The greater $R_0$ is, the more the disease spreads. In its most basic form, $R_0$ can be calculated as

$R_0 = \frac{\beta}{\mu}$

where $\beta$ is once again the contact rate and $\mu$ is the recovery rate, equal to the inverse of the illness duration. As more modes of infection and removal from the susceptible population are introduced, the expression for calculated $R_0$ can become increasingly complicated. Generally speaking, an outbreak will occur if $R_0$ is greater than 1._____no_output_____## Introducing a Vaccine: The SVIRD Model
Let's start with the most basic epidemiological model including a vaccine: the SVIRD model. The SVIRD model contains the following categories: Susceptible (S), Vaccinated (V), Infected (I), Recovered (R), and Dead (D). The Dead category only includes deaths from the illness under study. Susceptible individuals can either become vaccinated or infected. Vaccinated individuals can also become infected, but at a decreased rate dependent on the vaccine efficacy. Infected individuals can either recover from the illness or die from it. For every day in the simulation, there are a number of births into the Susceptible population and deaths from other causes unrelated to the illness under study (natural mortality). The arrow diagram and differential equations for this model are below:

where $\Pi$ is the number of births per day, $\mu$ is the natural mortality rate (proportion of the population per day that dies of non-illness causes), $\beta$ is the contact rate (here defined as the probability of infection following exposure to SARS-CoV-2, multiplied by the number of encounters with others per day, and normalized by the total population $S_0$), $\xi$ is the vaccination coverage rate (proportion of the susceptible population that is vaccinated per day), $\tau$ is the vaccine efficacy, $\delta$ is the recovery rate (inverse of the duration of illness), and $d$ is the illness death rate (proportion of infected individuals that die from the illness per day).

There are a few key assumptions in this model. First, recovered individuals are assumed to be immune and cannot become reinfected. Second, we assume that the vaccine does not expire, so there is no need for re-vaccination. Finally, there is the assumption that births and deaths are the only major contributors to total population change, as immigration, emigration, and travel are not included in the model.

For this first exercise, we'll be using the values of these paramters from Table 1 in the Gumel et al. paper (displayed below), assuming a population of 4 million and beginning with 1 infected person \[6\].

The parameters were estimated from statistics on a SARS outbreak in the Greater Toronto Area in 2003 \[6\]. Take a look at the two vaccine-related parameters, the vaccination coverage rate and vaccine efficacy. The vaccination coverage rate (proportion of susceptible people vaccinated every day) is 1/7. This is an unreasonably large proportion. If 1/7 (approximately 14.3%) of the US population were vaccinated every day, the entire population could be vaccinated in less than a week. In contrast, as of Jan. 4, 2020, the US has administered around 4.23 million total COVID-19 vaccines, which amounts to only about 0.099% of the population \[7\]. The table's vaccine efficacy, on the other hand, is more realistic. Vaccine efficacies can vary widely: the flu vaccine can range from 40 - 60% efficacy, while recently-developed COVID-19 vaccines have reached 95% efficacy after two doses \[8, 9\]. Therefore, a vaccine efficacy of 0.8, or 80% is not unreasonable. _____no_output_____

# This Antimony string defines the SVIRD model
SVIRD_model = '''
// Equations
E1: S -> V ; vaccination_coverage_rate*S ;
E2: V -> I ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E3: S -> I ; contact_rate*S*(I/Ptotal) ; // contact_rate*S*(I/Ptotal)*S
E4: I -> R ; recovery_rate*I ;
E5: -> S ; births ;
E6: S -> ; natural_mortality_rate*S ;
E7: V -> ; natural_mortality_rate*V ;
E8: I -> ; natural_mortality_rate*I ;
E91: I -> D ; disease_mortality_rate*I ;
E10: R -> ; natural_mortality_rate*R ;

// Parameters
natural_mortality_rate = 3.65E-5 // natural mortality rate, per day ;
contact_rate = 0.4 // effective contact rate, per day ; // beta = contact_rate*S0
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
recovery_rate = .04 // recovery rate ;
disease_mortality_rate = 0.008 // disease-induced mortality rate, per day ;
births = 146 // recruitment rate of suceptible humans, per day ;
//R0 := (alpha*beta*pi1)/(mu*(mu + alpha)*(delta + mu + d)) ;

// Initial Conditions
S0 = 4E6 // total initial population ;
I =  1 // initial infected people ;
S = S0 - I - R - D// susceptible population ;
R = 0 // recovered people ;
D = 0 ;
V = 0 ;
Ptotal := S + V + I + R ; // total current population ;
'''_____no_output_____

### Examining the Impact of Changing Parameters
We need to make sure that our model behaves logically. Let's first look at a scenario where there is no vaccine. We can do this by setting the vaccination coverage rage to zero. Then we can sweep through our other parameters to see if our model displays the behavior we expect. Later, we'll examine parameters that are fitted to the US COVID-19 pandemic, but for now let's use the parameter values from the paper, assume an initial population of 1 million, and start with 1 infected person.

Let's start with the contact rate. As the contact rate increases, probability of infection increases. Therefore, we expect to see an increase in cases as the contact rate increases. Observe this for yourself by moving the slider to simulate with different contact rates. As you increase the contact rate, notice that not only do the number of cases increase, but a peak will emerge. As the contact rate is further increased, the maximum value of this peak will increase, and the peak will shift to the left, signifying an earlier uptick in cases._____no_output_____

# This function simulates an outbreak for 500 days and plots the number of infected individuals over time.
# App users can user the slider widget to adjust the contact rate variable and observe how the plot changes.
def plot_contact_rate_SVIRD(contact_rate):
    m = te.loada(SVIRD_model)
    m.resetAll()
    m.contact_rate = contact_rate
    s = m.simulate(0,500,2000,['time', 'I'])
    m.plot(title = 'Cases (I) Over Time', xtitle = 'Days', ytitle = 'Cases')
    print("Contact rate: " + str(m.contact_rate))

interact(plot_contact_rate_SVIRD, contact_rate = FloatSlider(min=0,max=1,step=.01,value=0), continuous_update=False);_____no_output_____

Now let's look at the recovery rate. As the recovery rate increases, we expect to see a decrease in peak cases and an increase in recoveries. Use the slider to change the recovery rate and observe that the model exhibits this behavior._____no_output_____

# This function simulates an outbreak for 500 days and plots the number of infected and recovered individuals over time.
# App users can user the slider widget to adjust the recovery rate variable and observe how the plot changes.
def plot_recovery_rate_SVIRD(recovery_rate):
    m = te.loada(SVIRD_model)
    m.resetAll()
    m.recovery_rate = recovery_rate
    s = m.simulate(0,500,2000,['time','I', 'R'])
    m.plot(title = "Infected and Recovered Individuals Over Time", xtitle = "Days", ytitle ="Infected/Recovered")
    print("Recovery rate: " + str(m.recovery_rate))

interact(plot_recovery_rate_SVIRD, recovery_rate = FloatSlider(min=0,max=.1,step=.01,value=0), continuous_update=False);_____no_output_____

Next is the natural mortality rate. This is the mortality rate from causes other than the disease. Notice that as the natural mortality rate is increased, the total population decreases faster._____no_output_____

# This function simulates an outbreak for 500 days and plots the total population over time.
# App users can user the slider widget to adjust the natural mortality rate variable and observe how the plot changes.
def plot_natural_mortality_rate_SVIRD(natural_mortality_rate):
    m = te.loada(SVIRD_model)
    m.resetAll()
    m.natural_mortality_rate = natural_mortality_rate
    s = m.simulate(0,500,2000,['time','Ptotal'])
    m.plot(title = "Total Population Over Time", xtitle = "Days", ytitle = "Total Population")
    print("Natural mortality rate: " + str(m.natural_mortality_rate))

interact(plot_natural_mortality_rate_SVIRD, natural_mortality_rate = FloatSlider(min=0,max=.01,step=.001,value=0), continuous_update=False);_____no_output_____

Our final non-vaccine parameter is the disease mortality rate. This is specifically the mortality rate for the disease of interest. Use the slider to observe that as the disease mortality rate increases, the total deaths from the disease also increase._____no_output_____

# This function simulates an outbreak for 500 days and plots the number of disease-related deaths overr time.
# App users can user the slider widget to adjust the disease mortality rate variable and observe how the plot changes.
def plot_disease_mortality_rate_SVIRD(disease_mortality_rate):
    m = te.loada(SVIRD_model)
    m.resetAll()
    m.disease_mortality_rate = disease_mortality_rate
    s = m.simulate(0,500,2000,['time', 'D'])
    m.plot(title = "Disease Deaths Over Time", xtitle = "Days", ytitle = "Deaths")
    print("Disease mortality rate: " + str(m.disease_mortality_rate))

interact(plot_disease_mortality_rate_SVIRD, disease_mortality_rate = FloatSlider(min=0,max=.01,step=.001,value=0), continuous_update=False);_____no_output_____

### Exploring Vaccination Rate and Efficacy
Now that we've established that our model behaves as we'd expect, let's see what happens when we introduce a vaccine by changing the vaccination coverage rate and vaccine efficacy parameters. As you move the slider to increase the vaccination coverage rate, you'll notice that the peak for vaccinated individuals increases in height, which is logical since if we vaccinate more susceptible people per day, there will be more vaccinated individuals. As you move the slider to increase the vaccine efficacy, you'll notice that the peak for infected individuals decreases slightly. This is also expected, since an increased vaccine efficacy lowers the probability that vaccinated individuals will become infected. As both parameters are increased simultaneously, you should observe both of these effects simultaneously, with an even more pronounced decrease in the height of the peak for infected individuals. As the vaccine becomes more effective and more people are vaccinated, there are noticeably fewer cases.

Also notice that the peak for vaccinated individuals always precedes the peak for infected individuals. Since vaccinated individuals have a decreased probability of infection, there is a slower increase in infected individuals._____no_output_____

# This function simulates an outbreak for 1000 days and plots the number of susceptible, infected, and vaccinated individuals over time.
# App users can user the slider widget to adjust the vaccination coverage rate and vaccine efficacy and observe how the plot changes.
def plot_vaccine_params_SVIRD(vaccination_coverage_rate, vaccine_efficacy):
    m = te.loada(SVIRD_model)
    m.resetAll()
    m.vaccination_coverage_rate = vaccination_coverage_rate
    m.vaccine_efficacy = vaccine_efficacy
    s = m.simulate(0,1000,10000,['time','S', 'I','V'])
    m.plot(title = "Susceptible, Infected, and Vaccinated Individuals Over Time", xtitle = "Days", ytitle = "Individuals")
    print("Vaccination coverage rate: " + str(m.vaccination_coverage_rate))
    print("Vaccine efficacy: " + str(m.vaccine_efficacy))
interact(plot_vaccine_params_SVIRD, vaccination_coverage_rate = FloatSlider(min=0,max=.01,step=0.001,value=.001), vaccine_efficacy = FloatSlider(min=0,max=1,step=0.1,value=.1), continuous_update=False);_____no_output_____

## Introducing Asymptomatic Cases: The SVEIRD Model
The SVEIRD model adds the category of asymptomatic individuals E to the SVIRD model above. Now, susceptible or vaccinated individuals who become infected enter the asymptomatic E category and stay there throughout the duration of the incubation period before transitioning to the symptomatic category I. The transition between the E and I categories is determined by a new variable called the rate symptom developent ($\alpha$). The rate of symtom development is the inverse of the incubation period. All other variables are the same as in the SVIRD model. The arrow diagram and equations for the SVEIRD model are below:

This is the model appearing in the paper by Gumel et al., with the addition of a D category which contains disease-related deaths. Including the assumptions noted for the SVIRD model, the SVEIRD model assumes that asymptomatic individuals are noninfectious. Only individuals in the symptomatic category I are capable of infecting susceptible indiviuals. Though in reality asymptomatic individuals can spread the virus, they generally are not as infectious as symptomatic individuals, who may spread the virus in the air and in droplets via symptoms such as coughing.

We will still be using the parameter values from the paper's Table 1 above for the next exercises._____no_output_____

# This Antimony string describes the SVEIRD model
SVEIRD_model = '''
// Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: I -> D ; disease_mortality_rate*I ;

// Parameters
natural_mortality_rate = .0003 // natural mortality rate, per year ;
contact_rate = 0.4 // effective contact rate, per day ;
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0 // vaccine efficacy ;
symptom_development_rate = .125 // rate of development of clinical symptoms, per day ;
recovery_rate = .04 // recovery rate ;
disease_mortality_rate = 0.008 // disease-induced mortality rate, per day ;
births = 146 // recruitment rate of suceptible humans, per day ;

// Initial Conditions
S0 = 4E6 // total population ;
I =  1 // initial infected people ;
E = 0 ;
S = S0 - I - R - D - E// susceptible population ;
R = 0 // recovered people ;
D = 0 ;
Ptotal := S + V + E + I + R ;
'''_____no_output_____

### Examining the Impact of Changing Parameters
As with the SVIRD model above, in this exercise, we'll sweep our non-vaccination parameters to make sure our model displays reasonable behavior.

Let's again start with the contact rate. As you move the slider to increase the contact rate, you should notice that the maximum peak value for the number of asymptomatic cases increases. This is logical because if the probability of infection increases, there should be more cases._____no_output_____

# This function simulates an outbreak for 500 days and plots the number of asymptomatically infected individuals over time.
# App users can user the slider widget to adjust the contact rate variable and observe how the plot changes.
def plot_contact_rate_SVEIRD(contact_rate):
    m = te.loada(SVEIRD_model)
    m.resetAll()
    m.contact_rate = contact_rate
    s = m.simulate(0,500,2000,['time','E'])
    m.plot(title = "Asymptomatic Individuals Over Time", xtitle = "Days", ytitle = "Individuals")
    print("Contact rate: " + str(m.contact_rate))
interact(plot_contact_rate_SVEIRD, contact_rate = FloatSlider(min=0,max=1,step=.01,value=0), continuous_update=False);_____no_output_____

Next, let's look at the recovery rate. As with the SVIRD model, we expect that as the recovery rate increases, we should see a lower peak number of symptomatically infected individuals and an increase in recoveries. This is what you should observe as you use the slider to increase the recovery rate._____no_output_____

# This function simulates an outbreak for 500 days and plots the number of infected and recovered individuals over time.
# App users can user the slider widget to adjust the recovery rate variable and observe how the plot changes.
def plot_recovery_rate_SVEIRD(recovery_rate):
    m = te.loada(SVEIRD_model)
    m.resetAll()
    m.recovery_rate = recovery_rate
    s = m.simulate(0,500,2000,['time','I', 'R'])
    m.plot(title = "Symptomatic and Recovered Individuals Over Time", xtitle = "Days", ytitle = "Individuals")
    print("Recovery rate: " + str(m.recovery_rate))

interact(plot_recovery_rate_SVEIRD, recovery_rate = FloatSlider(min=0,max=.01,step=.001,value=0), continuous_update=False);_____no_output_____

Let's go on to the symptom development rate, a new parameter for the SVEIRD model. Since this variable controls the transition from asymptomatic to symptomatic, we'll examine how the curves for these categories change with the symptom development rate. Remember that the symptom development rate is the inverse of the incubation period. For example, if the indcubation period of a disease is 5 days, the symptom developmemt rate would be 1/5, or 0.2. If the incubation period is longer, and the symptom development rate smaller, we would not expect people to become infectious as quickly. In the opposite case with a short incubation period and larger symptom development rate, we'd expect infectious cases to grow more quickly. You should observe these effects as you move the slider for the symptom development rate to increase its value. As the symptom development rate increases (incubation period shortens), the number of symptomatic individuals increases more quickly. _____no_output_____

# This function simulates an outbreak for 500 days and plots the number of asymptomatically infected individuals over time.
# App users can user the slider widget to adjust the symptom development rate variable and observe how the plot changes.
def plot_symptom_development_rate_SVEIRD(symptom_development_rate):
    m = te.loada(SVEIRD_model)
    m.resetAll()
    m.symptom_development_rate = symptom_development_rate
    s = m.simulate(0,500,2000,['time','E', 'I'])
    m.plot(title = "Asymptomatic and Symptomatic Populations Over Time", xtitle = "Days", ytitle = "Individuals")
    print("Symptom Development Rate: " + str(m.symptom_development_rate))

interact(plot_symptom_development_rate_SVEIRD, symptom_development_rate = FloatSlider(min=0,max=.2,step=.02,value=.02), continuous_update=False);

_____no_output_____

Finally, we look at the two mortality rates. Using the slider for natural mortality rate, observe that the total population decreases as the natural mortality rate increases. This is the behavior we'd expect, since an increase in mortality rate translates to more deaths._____no_output_____

# This function simulates an outbreak for 500 days and plots the total population over time.
# App users can user the slider widget to adjust the natural mortality rate variable and observe how the plot changes.
def plot_natural_mortality_rate_SVEIRD(natural_mortality_rate):
    m = te.loada(SVEIRD_model)
    m.resetAll()
    m.natural_mortality_rate = natural_mortality_rate
    s = m.simulate(0,500,2000,['time','Ptotal'])
    m.plot(title = "Total Population Over Time", xtitle = "Days", ytitle = "Total Population")
    print("Natural Mortality Rate: " + str(m.natural_mortality_rate))

interact(plot_natural_mortality_rate_SVEIRD, natural_mortality_rate = FloatSlider(min=0,max=.01,step=.001,value=0), continuous_update=False);_____no_output_____

Also observe as you move the slider to increase the disease mortality rate, the number of disease-related deaths increases._____no_output_____

# This function simulates an outbreak for 500 days and plots the number of disease-related deaths over time.
# App users can user the slider widget to adjust the disease mortality rate variable and observe how the plot changes.
def plot_disease_mortality_rate_SVEIRD(disease_mortality_rate):
    m = te.loada(SVEIRD_model)
    m.resetAll()
    m.disease_mortality_rate = disease_mortality_rate
    s = m.simulate(0,500,2000,['time','D'])
    m.plot(title = "Disease Deaths Over Time", xtitle = "Days", ytitle = "Deaths from Disease")
    print("Disease Mortality Rate: " + str(m.disease_mortality_rate))

interact(plot_disease_mortality_rate_SVEIRD, disease_mortality_rate = FloatSlider(min=0,max=.01,step=.001,value=0), continuous_update=False);_____no_output_____

### Exploring Vaccination Rate and Efficacy
Now that we've established our model behavior makes sense, let's take a look at the vaccination variables: vaccination coverage rate and vaccine efficacy. We can make the same observations here as we could with the SVIRD model. As the vaccination coverage rate increases, the maximum number of vaccinated individuals increases. As you move the slider to increase the vaccine efficacy, the maximum number of infected individuals decreases slightly. As both parameters are increased simultaneously, both of these effects are observed simultaneously and in a more pronounced way. We again observe that the time of the peak for vaccinated individuals precedes the time of the peak for infected individuals._____no_output_____

# This function simulates an outbreak for 1000 days and plots the number of susceptible, asymptomatic, and vaccinated individuals over time.
# App users can user the slider widget to adjust the vaccination coverage rate and vaccine efficacy and observe how the plot changes.
def plot_vaccine_params_SVEIRD(vaccination_coverage_rate, vaccine_efficacy):
    m = te.loada(SVEIRD_model)
    m.resetAll()
    m.vaccination_coverage_rate = vaccination_coverage_rate
    m.vaccine_efficacy = vaccine_efficacy
    s = m.simulate(0,1000,10000,['time','S','E','V'])
    m.plot(title = "Susceptible, Vaccinated, and Asymptomatic Individuals Over Time", xtitle = "Days", ytitle = "Individuals")
    print("Vaccination coverage rate: " + str(m.vaccination_coverage_rate))
    print("Vaccine efficacy: " + str(m.vaccine_efficacy))

interact(plot_vaccine_params_SVEIRD, vaccination_coverage_rate = FloatSlider(min=0,max=.01,step=0.001,value=.001), vaccine_efficacy = FloatSlider(min=0,max=1,step=0.1,value=.1), continuous_update=False);_____no_output_____

### Key Results from Gumel et al.
In addition to examining how changing different parameters impacts model behavior, replicating the results from the paper on which the model is based is an indicator that the model is implemented correctly. Let's try replicating some results from the Gumel et al. paper. This paper's model does not include a D category, but this should not impact the behavior of the remaining categories. First, let's replicate Figure 3 from the paper, reproduced below, showing the number of susceptible, infected, and recovered individuals over a 300 day period. \[6\]. Here, the "Infected" category includes both asymptomatic and symptomatic individuals. We'll use the parameter values from Table 1 as well.

_____no_output_____

# This Antimony string contains the SVEIRD model with parameters from Figure 3 in the Gumel et al. paper
SVEIRD_model_paper = '''
// Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: I -> D ; disease_mortality_rate*I ;

// Parameters
natural_mortality_rate = .0003 // natural mortality rate, per year ;
contact_rate := 4E-7*Ptotal // effective contact rate, per day ;
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .125 // rate of development of clinical symptoms, per day ;
recovery_rate = .04 // recovery rate ;
disease_mortality_rate = 0.008 // disease-induced mortality rate, per day ;
births = 146 // recruitment rate of suceptible humans, per day ;
R0 := (symptom_development_rate*contact_rate*births)/(natural_mortality_rate*(natural_mortality_rate + symptom_development_rate)*(recovery_rate + natural_mortality_rate + disease_mortality_rate))

// Initial Conditions
I = 10 // initial infected people ;
S = 5E5 // susceptible population ;
V = 0 // vaccinated people ;
R = 0 // recovered people ;
E = 0 // value needs to change
Ptotal := S + V + E + I + R ;
'''_____no_output_____# This simulation replicates Figure 3 from the paper
m = te.loada(SVEIRD_model_paper)
s = m.simulate(0, 300, 1000,['time', 'S', 'I', 'E', 'R'])
plt.plot(s['time'], s['S'])
plt.plot(s['time'], s['R'])
plt.plot(s['time'], s['I']+s['E'])
plt.title("Susceptible, Total Infected, and Recovered Populations")
plt.xlabel("Days")
plt.ylabel("Individuals")
plt.legend(['Susceptible', 'Recovered', 'Infected'])
plt.show()_____no_output_____

Notice that this graph matches Figure 3, indicating that we've replicated the results successfully.

Next we'll replicate Figure 4 from the paper (shown below), which involves the introduction of a vaccine with a vaccination coverage rate of 1/7 and an efficacy of 80% \[6\]. This result is produced by using a contact rate of $4.8E-8$, but all other variables retain their values from Table 1. The number of susceptible, infected, and vaccinated individuals over time is tracked over an 80 day period. As with Figure 3 above, the "Infected" category includes both asymptomatic and symptomatic cases.

_____no_output_____

# This Antimony string contains the SVEIRD model with parameters from Figure 4 in the Gumel et al. paper
SVEIRD_model_paper = '''
// Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: I -> D ; disease_mortality_rate*I ;

// Parameters
natural_mortality_rate = 3.65E-5 // natural mortality rate, per year ;
contact_rate := 4.8E-8*Ptotal // effective contact rate, per day ;
vaccination_coverage_rate = 1/7 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .125 // rate of development of clinical symptoms, per day ;
recovery_rate = .04 // recovery rate ;
disease_mortality_rate = 0.008 // disease-induced mortality rate, per day ;
births = 146 // recruitment rate of suceptible humans, per day ;
R0 := (symptom_development_rate*contact_rate*births)/(natural_mortality_rate*(natural_mortality_rate + symptom_development_rate)*(recovery_rate + natural_mortality_rate + disease_mortality_rate))

// Initial Conditions
I = 10 // initial infected people ;
S = 4E6 // susceptible population ;
V = 0 // vaccinated people ;
R = 0 // recovered people ;
E = 0 // value needs to change
Ptotal := S + V + E + I + R ;
'''_____no_output_____# This simulation replicates Figure 4 from the Gumel et al. paper
m = te.loada(SVEIRD_model_paper)
s = m.simulate(0, 80, 1000,['time', 'S', 'I', 'V', 'E'])

plt.plot(s['time'], s['S']/100000)
plt.plot(s['time'], s['V']/100000)
plt.plot(s['time'], s['I']+s['E'])
plt.title("Susceptible, Infected, and Recovered Populations with Vaccine")
plt.xlabel("Days")
plt.ylabel("Individuals")
plt.legend(['Susceptible (x 100,000)', 'Recovered (x 100,000)', 'Infected'])
plt.show()_____no_output_____

This graph matches Figure 4 above, so we have successfully replicated the paper's results!_____no_output_____## When Immunity Does Not Last Forever: The SVEIRSD Model
There is one final addition to the model to make before we can start answering some questions about COVID-19. In our previous models, we assumed that people who had recovered from the illness could not get re-infected. However, for COVID-19, there is evidence of re-infection (though it is thought to be rare) \[10\]. The SVEIRSD model includes a transition from R to S dependent on a re-susceptibility rate sigma and the population of recovered individuals, as shown below:

In this model, the re-susceptibility rate sigma is defined as the inverse of the time it takes for a recovered individual to become susceptible again. For example, if individuals become susceptible to a disease again 30 days after recovering, sigma would be 1/30. _____no_output_____

# This Antimony string contains the SVEIRSD model
SVEIRSD_model = '''
// Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: R -> S ; resusceptible_rate*R ;
E13: I -> D ; disease_mortality_rate*I ;

// Parameters
resusceptible_rate = .1 // re-susceptibility rate ;
natural_mortality_rate = .0003 // natural mortality rate, per year ;
contact_rate = 0.4 // effective contact rate, per day ;
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0 // vaccine efficacy ;
symptom_development_rate = .125 // rate of development of clinical symptoms, per day ;
recovery_rate = .04 // recovery rate ;
disease_mortality_rate = 0.008 // disease-induced mortality rate, per day ;
births = 146 // recruitment rate of suceptible humans, per day ;

// Initial Conditions
S0 = 4E6 // total population ;
I = 1 // initial infected people ;
S = S0 - I - V - R - E // susceptible population ;
V = 0 // vaccinated people ;
R = 0 // recovered people ;
E = 0 // value needs to change ;
Ptotal := S + V + E + I + R ;
'''_____no_output_____

### Examining the Impact of Changing Parameters
Since the only difference between this model and the SVEIRD model is the addition of the R --> S transition, we only need to examine the re-susceptibility rate parameter. Notice that as you increase the re-susceptible rate using the slider, the the maximum number of recovered individuals decreases and the downslope of the curve becomes more steep. We expect this behavior because as the re-susceptibility rate increases, a greater proportion of recovered individuals should leave the Recovered category and become susceptible again._____no_output_____

# This function simulates an outbreak for 500 days and plots the number of recovered individuals over time.
# App users can user the slider widget to adjust the re-susceptibility rate variable and observe how the plot changes.
def plot_resusceptible_rate_SVEIRSD(resusceptible_rate):
    m = te.loada(SVEIRSD_model)
    m.resetAll()
    m.resusceptible_rate = resusceptible_rate
    s = m.simulate(0,500,10000,['time','R'])
    m.plot(title = "Recovered Population Over Time", xtitle = "Days", ytitle = "Individuals")
    print("Resusceptible rate: " + str(m.resusceptible_rate))

interact(plot_resusceptible_rate_SVEIRSD, resusceptible_rate = FloatSlider(min=0,max=.01,step=.001,value=0), continuous_update=False);_____no_output_____

### Exploring Vaccination Rate and Efficacy
Now, as before, let's look at the vaccine coverage rate and vaccine efficacy. As with the SVIRD and SVEIRD models, the behavior observed while changing the vaccination coverage rate and vaccine efficacy remains the same: Increasing the vaccination coverage rate leads to an increased vaccination curve peak, and increasing the vaccination efficacy decreases the asymptomatic curve peak. _____no_output_____

# This function simulates an outbreak for 1000 days and plots the number of susceptible, asymptomatic, and vaccinated individuals over time.
# App users can user the slider widget to adjust the vaccination coverage rate and vaccine efficacy and observe how the plot changes.
def plot_vaccine_params_SVEIRSD(vaccination_coverage_rate, vaccine_efficacy):
    m = te.loada(SVEIRSD_model)
    m.resetAll()
    m.vaccination_coverage_rate = vaccination_coverage_rate
    m.vaccine_efficacy = vaccine_efficacy
    s = m.simulate(0,1000,2000,['time','S', 'E', 'V'])
    m.plot(title = "Susceptible, Vaccinated, and Asymptomatic Populations Over Time", xtitle = "Days", ytitle = "Individuals")
    print("Vaccination coverage rate: " + str(m.vaccination_coverage_rate))
    print("Vaccine efficacy: " + str(m.vaccine_efficacy))
interact(plot_vaccine_params_SVEIRSD, vaccination_coverage_rate = FloatSlider(min=0,max=.01,step=0.001,value=.001), vaccine_efficacy = FloatSlider(min=0,max=1,step=0.1,value=.1), continuous_update=False);_____no_output_____

## Answering Questions
Now that we have a working model, we can use it to try to answer some questions related to a US SARS-CoV-2 vaccine strategy. But first, we need to fit the model parameters to actual pandemic data.
### Parameter Fitting
To fit our parameters, we'll be using data from the 2019 Novel Coronavirus Visual Dashboard by the Johns Hopkins University Center for Systems Science and Engineering (JHU CSSE). The US data on cumulative confirmed cases, deaths, and recovered cases was extracted from the global time series data found [here](https://github.com/CSSEGISandData/COVID-19/tree/master/csse_covid_19_data/csse_covid_19_time_series) \[3\]. It is worthwhile to note that recovered cases were estimated from local reports and therefore may not comprise an exact count. There may also be a negligible amount of error in the cumulative deaths data from COVID-19 patients dying of non-COVID-19 related causes. The data provided in this section are from January 22, 2020 to November 28, 2020. If you'd like to download the data files for yourself, the files are provided in the Appendix.

We need to look at active case data in order to fit the model. An active case is a person who is currently infected with the SARS-CoV-2 virus. This does not include cases that have already recovered. The data we have is cumulative: it counts the total number of cases since data collection began. We can estimate the active cases by subtracting the cumulative deaths and recovered cases from the cumulative cases:

Active Cases = Cumulative Cases - Cumulative Deaths - Cumulative Recovered
_____no_output_____

# Arrays containing the cumulative confirmed cases, deaths, and recoveries from COVID-19 in the US
confirmed = np.array([1, 1, 2, 2, 5, 5, 5, 6, 6, 8, 8, 8, 11, 11, 11, 12, 12, 12, 12, 12, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 16, 16, 16, 16, 16, 16, 17, 17, 25, 32, 55, 74, 107, 184, 237, 403, 519, 594, 782, 1147, 1586, 2219, 2978, 3212, 4679, 6512, 9169, 13663, 20030, 26025, 34855, 46086, 56698, 68773, 86613, 105293, 124900, 143779, 165861, 192177, 218060, 248447, 280417, 313432, 341629, 371802, 403212, 435407, 469989, 503474, 532782, 559709, 585518, 614082, 644247, 675648, 708317, 736244, 761933, 790353, 816413, 845727, 878911, 912662, 944234, 971078, 994265, 1018926, 1046737, 1076224, 1110464, 1138228, 1162685, 1186067, 1210577, 1235666, 1263402, 1290151, 1315099, 1333970, 1353397, 1376122, 1397085, 1424243, 1449498, 1473514, 1491829, 1513816, 1534871, 1557933, 1583798, 1607109, 1628212, 1648158, 1666505, 1685956, 1704489, 1727357, 1751612, 1775428, 1794465, 1811393, 1832782, 1852788, 1874156, 1899505, 1920910, 1938572, 1956064, 1974425, 1995376, 2018418, 2043332, 2068600, 2087619, 2107075, 2130768, 2157267, 2185134, 2216115, 2248032, 2274000, 2304676, 2340884, 2375310, 2415666, 2461019, 2502378, 2542787, 2582844, 2628824, 2680312, 2735900, 2787741, 2833290, 2882870, 2927032, 2987787, 3047527, 3110094, 3177953, 3237989, 3297074, 3355869, 3423296, 3490931, 3568033, 3639705, 3702174, 3762890, 3824368, 3888599, 3960522, 4029038, 4102241, 4167616, 4222480, 4278427, 4344212, 4416044, 4483643, 4551554, 4609390, 4655662, 4700229, 4757512, 4811945, 4871426, 4929748, 4984196, 5030566, 5079299, 5126224, 5183020, 5234762, 5299526, 5346302, 5387483, 5424000, 5469037, 5516408, 5560495, 5608748, 5652401, 5686858, 5723343, 5763322, 5808842, 5853962, 5900891, 5946245, 5981032, 6016230, 6057907, 6098858, 6142639, 6192768, 6235908, 6267419, 6290964, 6317861, 6351756, 6387822, 6435261, 6476236, 6510541, 6544411, 6583796, 6622256, 6667178, 6715928, 6758397, 6796841, 6848379, 6887861, 6926442, 6972152, 7020229, 7064532, 7101812, 7134060, 7176856, 7218052, 7263347, 7317749, 7366423, 7402515, 7441432, 7485381, 7536022, 7594351, 7650345, 7704352, 7750186, 7790890, 7843769, 7903242, 7967600, 8036049, 8092924, 8142220, 8209226, 8270314, 8333340, 8408675, 8489662, 8572449, 8634210, 8700789, 8776790, 8855474, 8945642, 9044161, 9133045, 9208607, 9292091, 9416709, 9519582, 9645772, 9771480, 9899480, 10009386, 10130752, 10268446, 10411994, 10573242, 10750810, 10917519, 11053304, 11211070, 11371585, 11541986, 11729949, 11925959, 12104032, 12246766, 12418228, 12591163, 12772653, 12883264, 13088821, 13244417])
deaths = np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 6, 7, 11, 12, 14, 17, 21, 22, 28, 33, 43, 51, 58, 70, 97, 132, 191, 264, 365, 464, 574, 762, 1003, 1329, 1741, 2259, 2892, 3484, 4180, 5272, 6452, 7955, 9348, 10889, 12457, 14233, 16803, 18957, 21176, 23403, 25527, 27342, 29280, 31734, 34337, 36506, 38599, 40568, 42499, 44679, 47221, 49659, 52125, 54285, 55988, 57322, 58787, 61028, 63555, 65971, 67848, 69535, 70638, 71956, 74255, 76599, 78523, 80275, 81817, 82717, 83719, 85333, 87082, 88856, 90520, 91792, 92547, 93714, 95240, 96758, 97969, 99214, 100348, 100964, 101521, 102185, 103693, 104876, 106031, 106985, 107674, 108451, 109500, 110494, 111499, 112405, 113102, 113543, 114040, 114970, 115869, 116723, 117543, 118316, 118629, 119012, 119833, 120564, 121272, 121934, 122540, 122833, 123217, 124059, 124815, 125360, 125981, 126483, 126761, 127139, 127787, 128479, 129199, 129858, 130127, 130416, 130777, 131984, 132847, 133845, 134666, 135359, 135820, 136188, 137108, 138080, 139030, 139963, 140825, 141278, 141802, 142903, 144123, 145215, 146345, 147233, 147716, 148835, 150204, 151635, 152851, 154093, 155202, 155613, 156152, 157526, 158902, 160150, 161388, 162470, 162986, 163513, 164581, 166087, 167151, 168494, 169524, 170099, 170545, 171876, 173207, 174285, 175396, 176380, 176835, 177279, 178509, 179737, 180841, 181813, 182777, 183089, 183684, 184749, 185801, 186868, 187838, 188609, 189019, 189295, 189741, 190923, 191830, 193043, 193753, 194139, 194563, 195851, 196838, 197704, 198634, 199349, 199590, 200015, 201038, 202140, 203057, 203996, 204742, 205024, 205345, 206257, 207199, 208050, 208978, 209657, 210006, 210468, 211170, 212100, 213064, 214029, 214656, 215078, 215415, 216212, 217213, 218031, 218936, 219681, 220109, 220576, 221510, 222665, 223515, 224466, 225402, 225780, 226272, 227240, 228247, 229214, 230242, 231127, 231535, 232067, 233638, 234728, 235878, 237073, 238081, 238554, 239242, 240623, 242036, 243245, 244403, 245663, 246290, 247024, 248716, 250613, 252600, 254517, 255946, 256866, 257779, 259925, 262222, 263454, 264858, 266047])
recovered = np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 5, 5, 5, 5, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 12, 12, 12, 12, 17, 17, 105, 121, 147, 176, 178, 178, 348, 361, 681, 869, 1072, 2665, 5644, 7024, 8474, 9001, 9707, 14652, 17448, 19581, 21763, 23559, 25410, 28790, 31270, 32988, 43482, 47763, 52096, 54703, 58545, 64840, 70337, 72329, 75204, 77366, 80203, 99079, 100372, 106988, 111424, 115936, 120720, 153947, 164015, 175382, 180152, 187180, 189791, 189910, 195036, 198993, 212534, 216169, 232733, 230287, 243430, 246414, 250747, 268376, 272265, 283178, 289392, 294312, 298418, 350135, 361239, 366736, 379157, 384902, 391508, 399991, 406446, 416461, 444758, 458231, 463868, 479258, 485002, 491706, 500849, 506367, 518522, 524855, 533504, 540292, 547386, 556606, 561816, 576334, 583503, 592191, 599115, 606715, 617460, 622133, 640198, 647548, 656161, 663562, 670809, 679308, 685164, 705203, 720631, 729994, 781970, 790404, 894325, 906763, 924148, 936476, 953462, 969111, 983185, 995576, 1006326, 1031939, 1049098, 1075882, 1090645, 1107204, 1122720, 1131121, 1160087, 1182018, 1210849, 1233269, 1261624, 1279414, 1297863, 1325804, 1355363, 1389425, 1414155, 1438160, 1461885, 1468689, 1513446, 1528979, 1577851, 1598624, 1623870, 1643118, 1656864, 1670755, 1714960, 1753760, 1774648, 1796326, 1818527, 1833067, 1865580, 1898159, 1925049, 1947035, 1965056, 1985484, 1997761, 2020774, 2053699, 2084465, 2101326, 2118367, 2140614, 2153939, 2184825, 2202663, 2231757, 2266957, 2283454, 2302187, 2315995, 2333551, 2359111, 2387479, 2403511, 2417878, 2434658, 2451406, 2474570, 2495127, 2525573, 2540334, 2556465, 2577446, 2590671, 2615949, 2646959, 2670256, 2710183, 2727335, 2750459, 2766280, 2794608, 2813305, 2840688, 2860650, 2873369, 2897322, 2911699, 2935142, 2952390, 2999895, 3021252, 3039089, 3062983, 3075077, 3106728, 3124593, 3155794, 3177397, 3197539, 3220573, 3234138, 3272603, 3295148, 3323354, 3353056, 3375427, 3406656, 3422878, 3460455, 3487666, 3518140, 3554336, 3578452, 3612478, 3630579, 3674981, 3705130, 3743527, 3781751, 3810791, 3851465, 3881491, 3928845, 3961873, 3997175, 4051256, 4095146, 4148444, 4174884, 4244811, 4293640, 4350789, 4410709, 4457930, 4529700, 4526513, 4633600, 4696664, 4835956, 4871203, 4947446, 5023063])

# Plot the confirmed cases, deaths, recovered, and active cases over time
active = confirmed - deaths - recovered
num_days = np.arange(0, len(active), 1)
plt.plot(num_days, confirmed)
plt.title("Cumulative Confirmed Cases")
plt.xlabel("Days")
plt.ylabel("Confirmed Cases")
plt.show()
plt.plot(num_days, deaths)
plt.title("Cumulative Deaths")
plt.xlabel("Days")
plt.ylabel("Deaths")
plt.show()
plt.plot(num_days, recovered)
plt.title("Cumulative Recovered Cases")
plt.xlabel("Days")
plt.ylabel("Recovered Cases")
plt.show()
plt.plot(num_days, active)
plt.title("Active Cases")
plt.xlabel("Days")
plt.ylabel("Active Cases")
plt.show()_____no_output_____

Using this data and a little bit of external research, we can estimate the parameters of the model before fitting. According to the [National Center of Health Statistics](https://www.cdc.gov/nchs/data/vsrr/vsrr-8-508.pdf), there were 3,745,540 births in the US in 2019 \[11\]. That's around 10,261 births per day for our births parameter. According to the [Center for Disease Control and Prevention (CDC)](https://www.cdc.gov/nchs/data/databriefs/db355-h.pdf), there were 2,839,205 deaths in the US in 2018 \[12\]. That's around 7,778 deaths per day, normalized by the total US population in 2018 of 327.2 million gives us a natural mortality rate constant of approximately 0.00002377. Also according to the [CDC](https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-guidance-management-patients.html), the median time of symptom onset following exposure to SARS-CoV-2 is 4-5 days, so let's say that it takes 5 days for symptoms to develop and the symptom development rate is 1/5 = 0.2 \[4\].

Some remaining parameters can be estimated from the data. First, we need to determine which portion of the data to use to make estimations. In general, linear areas are better for estimation. The Active Case graph shows the different waves of the pandemic. The most recent wave of COVID-19, starting around day 289, probably best represents the conditions under which a vaccine would be introduced, so let's look there:_____no_output_____

# Plot data only from day 289 on
num_days = np.arange(289, 312, 1)
plt.plot(num_days, active[289:313])
plt.title("Active Cases")
plt.xlabel("Days")
plt.ylabel("Active Cases")
plt.show()
plt.plot(num_days, deaths[289:313])
plt.title("Deaths")
plt.xlabel("Days")
plt.ylabel("Deaths")
plt.show()
plt.plot(num_days, recovered[289:313])
plt.title("Recovered Cases")
plt.xlabel("Days")
plt.ylabel("Recovered Cases")
plt.show()_____no_output_____

Since we have no way of knowing how many active cases are symptomatic or asymptomatic, we will assume that all active cases are symptomatic. We can estimate the COVID mortality rate by looking by dividing the increase in deaths per day by the number of active cases on that day:_____no_output_____

# Estimate the COVID-19 mortality rate by looking at the daily difference in cumulative deaths
daily_death_increase = np.diff(deaths[289:312])
COVID_mortality = np.mean(daily_death_increase/active[289:311])
print(COVID_mortality)_____no_output_____

We can similarly estimate the recovery rate:_____no_output_____

# Estimate the COVID-19 recovery rate by looking at the daily difference in recoveries
daily_recovered_increase = np.diff(recovered[289:312])
recovery_rate = np.mean(daily_recovered_increase/active[289:311])
print(recovery_rate)_____no_output_____

Now we can try to estimate the number of asymptomatic cases, assuming that all active cases are symptomatic. We can do this by dividing the daily increase in active cases by our estimate for the symptom development rate 0.2:_____no_output_____

# Estimate the number of asymptomatic cases over time by looking at the difference in cumulative cases not related to death or recovery
daily_active_change = np.diff(active[289:312])
asymptomatic = (daily_active_change + active[289:311]*(COVID_mortality + recovery_rate + 0.00002377))/0.2
plt.plot(np.arange(289, 311, 1), asymptomatic)
plt.show()
print("starting_E: " + str(asymptomatic[0]))_____no_output_____

These estimations are imperfect, but let's use them for now. Let's determine the rest of our starting values, defined by their values on day 289 of the data:_____no_output_____

# Get starting number of symptomatic cases, recoveries, and deaths.
starting_I = active[289]
print(starting_I)
starting_R = recovered[289]
print(starting_R)
starting_D = deaths[289]
print(starting_D)_____no_output_____

Now we can move on to fitting the contact rate and re-susceptibility rate. We will also fit for the number of asymptomatic individuals, since that is difficult to estimate from the data we have. We'll accomplish this fitting using an optimizer that minimizes the difference between the data points for active cases and our model values. First we will fit the SVEIRSD model, and then the SVEIRD model._____no_output_____

# This Antimony string defines a new SVEIRSD model with the estimated parameters
SVEIRSD_COVID_model = '''
//Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: R -> S ; resusceptible_rate*R ;
E13: I -> D ; disease_mortality_rate*I ;

// Parameters
resusceptible_rate = .1 // re-susceptibility rate ;
natural_mortality_rate = 0.00002377 // natural mortality rate, per day ;
contact_rate = 0.4 // effective contact rate, per day ;
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .2 // rate of development of clinical symptoms, per day ;
recovery_rate = .00799 // recovery rate ;
disease_mortality_rate = 0.00019 // disease-induced mortality rate, per day ;
births = 10261 // recruitment rate of suceptible humans, per day ;
R0 := (symptom_development_rate*contact_rate*births)/(natural_mortality_rate*(natural_mortality_rate + symptom_development_rate)*(recovery_rate + natural_mortality_rate + disease_mortality_rate))

// Initial Conditions
S0 = 328E6 // total population ;
I = 5723616 // initial infected people ;
S = S0 - I - V - R - E // susceptible population ;
V = 0 // vaccinated people ;
R = 3810791 // recovered people ;
E = 666468 // asymptomatic cases ;
D = 237073 ;
Ptotal := S + V + E + I + R ;
'''_____no_output_____# This is the optimizer function that tries to minimize the difference between the data points and the values generated by the simulation
def cost_fun(p):
    # load model string and parameters to fit
    m = te.loada(SVEIRSD_COVID_model)
    pp = p.valuesdict()
    for i in range(nParameters):
        m.model[toFit[i]] = pp[toFit[i]]
    try:
        # Run an epidemic simulation
        s = m.simulate(int(num_days[0]), int(num_days[-1]), len(num_days)*10, ['time','I'])
        # diff will hold the total difference between data values and simulation values
        diff = 0
        # for every day simulated
        for i in range(len(num_days)):
            # find the minimum difference between the data value for a day
            # and all model values
            j = (np.abs(s['time'] - num_days[i])).argmin()
            # add this value to the total difference
            diff += (active[i+289] - s['I'][j]) ** 2
        # Take the square root of the difference
        diff = np.sqrt(diff)
        clear_output(wait=True)
        # Plot the simulated model and the data together on the same graph
        plt.plot(num_days, active[289:313],'.',label='Data')
        plt.plot(s['time'], s['I'],label='Model')
        plt.legend()
        plt.title("Fitting: Active Cases, SVEIRSD Model")
        plt.show()
    # stop the fitting if the runtime becomes too long
    except RuntimeError:
        diff = 1E21
    return(diff)
_____no_output_____# We are fitting the contact rate, resusceptible rate, and number of asymptomatic individuals
num_days = np.arange(0, len(active[289:313]), 1)
toFit = ['contact_rate', 'resusceptible_rate','E' ]
contact_rate = 0.4
resusceptible_rate = 0.0001
E = 666468

# Here we define the minimum and maximum values each variable could have. This limits our parameter exploration space
nParameters = len(toFit)
params = lmfit.Parameters()
params.add('contact_rate', value=contact_rate, min=contact_rate/1000.0, max=1)
params.add('resusceptible_rate', value=resusceptible_rate, min=resusceptible_rate/100.0, max=resusceptible_rate*100.0)
params.add('E', value=E, min=E/100.0, max=E*100.0)

cost_fun(params)_____no_output_____# Perform the fitting and print the final parameters
minimizer = lmfit.Minimizer(cost_fun, params)
result = minimizer.minimize(method='nelder')
lmfit.report_fit(result.params, min_correl=0.5)_____no_output_____# This Antimony string defines a new SVEIRD model with the estimated parameters
SVEIRD_COVID_model = '''
//Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: I -> D ; disease_mortality_rate*I ;

// Parameters
natural_mortality_rate = 0.00002377 // natural mortality rate, per day ;
contact_rate = 0.4 // effective contact rate, per day ;
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .2 // rate of development of clinical symptoms, per day ;
recovery_rate = .00799 // recovery rate ;
disease_mortality_rate = 0.00019 // disease-induced mortality rate, per day ;
births = 10261 // recruitment rate of suceptible humans, per day ;

// Initial Conditions
S0 = 328E6 // total population ;
I = 5723616 // initial infected people ;
S = S0 - I - V - R - E // susceptible population ;
V = 0 // vaccinated people ;
R = 3810791 // recovered people ;
E = 666468 // asymptomatic cases ;
D = 237073 ;
Ptotal := S + V + E + I + R ;
'''_____no_output_____num_days = np.arange(0, len(active[289:313]), 1)
# This is the optimizer function that tries to minimize the difference between the data points and the values generated by the simulation
def cost_fun(p):
    # load model string and parameters to fit
    m = te.loada(SVEIRD_COVID_model)
    pp = p.valuesdict()
    for i in range(nParameters):
        m.model[toFit[i]] = pp[toFit[i]]
    try:
        # Run an epidemic simulation
        s = m.simulate(int(num_days[0]), int(num_days[-1]), len(num_days)*10, ['time','I'])
        # diff will hold the total difference between data values and simulation values
        diff = 0
        # for every day simulated
        for i in range(len(num_days)):
            # find the minimum difference between the data value for a day
            # and all model values
            j = (np.abs(s['time'] - num_days[i])).argmin()
            # add this value to the total difference
            diff += (active[i+289] - s['I'][j]) ** 2
        # Take the square root of the difference
        diff = np.sqrt(diff)
        clear_output(wait=True)
        # Plot the simulated model and the data together on the same graph
        plt.plot(num_days, active[289:313],'.',label='Data')
        plt.plot(s['time'], s['I'],label='Model')
        plt.legend()
        plt.title("Fitting: Active Cases, SVEIRD Model")
        plt.xlabel("Days")
        plt.ylabel("Number of Individuals")
        plt.show()
    # stop the fitting if the runtime becomes too long
    except RuntimeError:
        diff = 1E21
    return(diff)

_____no_output_____# We are fitting the contact rate and number of asymptomatic individuals
toFit = ['contact_rate','E' ]
contact_rate = 0.4
E = 666468

# Here we define the minimum and maximum values each variable could have. This limits our parameter exploration space
nParameters = len(toFit)
params = lmfit.Parameters()
params.add('contact_rate', value=contact_rate, min=contact_rate/1000.0, max=1)
params.add('E', value=E, min=E/100.0, max=E*100.0)

cost_fun(params)_____no_output_____# Perform the fitting and print the final parameters
minimizer = lmfit.Minimizer(cost_fun, params)
result = minimizer.minimize(method='nelder')
lmfit.report_fit(result.params, min_correl=0.5)_____no_output_____

Notice that after fitting both the SVEIRSD and SVEIRD models, the parameters for the contact rate and initial number of asymptomatic cases are the same. From this fitting, the re-susceptibility does not significantly impact the model, as demonstrated by its extremely small value in the SVEIRSD fitting result. This is because we only simulated over a period of 24 days. If the resusceptibility rate were any longer than that, it would not be observable.

But what if we simulate for longer? Use the slider below to adjust the re-susceptibility rate on the three-year COVID-19 simulation below with our other fitted parameters. Remember that since the re-susceptibility rate is the inverse of the time it takes for a recovered person to become susceptible again. Therefore, a smaller re-susceptibility rate implies a longer time period before a recovered person becomes susceptible. The values for the re-susceptibility rate on the slider range from a period of approximately 30 days to 360 days._____no_output_____

# This is the final SVEIRSD fitted model
SVEIRSD_COVID_model = '''
//Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: R -> S ; resusceptible_rate*R ;
E13: I -> D ; disease_mortality_rate*I ;

// Parameters
resusceptible_rate = 1E-6 // re-susceptibility rate ;
natural_mortality_rate = 0.00002377 // natural mortality rate, per day ;
contact_rate = 0.0257 // effective contact rate, per day ;
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .2 // rate of development of clinical symptoms, per day ;
recovery_rate = .00799 // recovery rate ;
disease_mortality_rate = 0.00019 // disease-induced mortality rate, per day ;
births = 10261 // recruitment rate of suceptible humans, per day ;

// Initial Conditions
S0 = 328E6 // total population ;
I = 5723616 // initial infected people ;
S = S0 - I - V - R - E // susceptible population ;
V = 0 // vaccinated people ;
R = 3810791 // recovered people ;
E = 752565 // asymptomatic cases ;
D = 237073 ;
Ptotal := S + V + E + I + R ;
'''_____no_output_____def plot_resusceptible_rate_SVEIRSD_COVID(resusceptible_rate):
    m = te.loada(SVEIRSD_COVID_model)
    m.resetAll()
    m.resusceptible_rate = resusceptible_rate
    s = m.simulate(0,1095,100000,['time','R', 'S'])
    m.plot(title = "Susceptible and Recovered Population Over Time", xtitle = "Days", ytitle = "Individuals")
    print("Resusceptible rate: " + str(m.resusceptible_rate))

interact(plot_resusceptible_rate_SVEIRSD_COVID, resusceptible_rate = FloatSlider(min=0.0001,max=.008,step=.0001,value=0.0001), continuous_update=False);_____no_output_____

Notice that as you increase the re-susceptibility rate (and hence decrease the time it takes for a recovered person to become susceptible again), the final number of recovered individuals decreases and the final number of susceptible individuals increases. This is to be expected. Also notice that these two values approach a steady-state by the end of the simulation as the re-susceptibility rate is increased. That is, while there are oscillations in the curves, the two curves more quickly approach constant values at greater re-susceptibility rates. _____no_output_____

# This is the final SVEIRD fitted model
SVEIRD_COVID_model = '''
//Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: I -> D ; disease_mortality_rate*I ;

// Parameters
natural_mortality_rate = 0.00002377 // natural mortality rate, per day ;
contact_rate = 0.0257 // effective contact rate, per day ;
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .2 // rate of development of clinical symptoms, per day ;
recovery_rate = .00799 // recovery rate ;
disease_mortality_rate = 0.00019 // disease-induced mortality rate, per day ;
births = 10261 // recruitment rate of suceptible humans, per day ;

// Initial Conditions
S0 = 328E6 // total population ;
I = 5723616 // initial infected people ;
S = S0 - I - V - R - E // susceptible population ;
V = 0 // vaccinated people ;
R = 3810791 // recovered people ;
E = 752565 // asymptomatic cases ;
D = 237073 ;
Ptotal := S + V + E + I + R ;
'''_____no_output_____

### Question 1: What fraction of the population needs to be vaccinated?
To answer this question, we first need to define the conditions that signify the end of the pandemic. In general, an epidemic will die out if the basic reproduction number $R_0$ < 1. For a model with no vaccination, Gumel et al. define the $R_0$ value as follows \[6\]:

$R_0 = \frac{\alpha\beta\Pi}{\mu(\alpha + \mu)(\delta + \mu + d)}$

However, for a case where a vaccine is introduced, we need to modify this expression slightly. Instead of using the basic reproduction number, we will use the control reproduction number $R_v$. $R_v$ is similar to $R_0$, but specifically applies when a vaccination program is in place \[6\].

$R_v = \frac{\alpha\beta\Pi[\mu+(1-\tau)\xi]}{\mu(\xi+\mu)(\alpha+\mu)(\delta+\mu+d)}$

However, because we are working with a large population, with a large number of births in the numerator and small parameter values in the denominator, our $R_v$ value is unlikely to fall below 1 even if we meet conditions that would end the pandemic. This is not ideal for conceptualizing the value, but we can still work with it. We just need to use a different value in place of 1. Under our parameters for the US, the threshold that we need to fall under was found to be 425,930,847. Above this $R_v$ value, the number of cases blows up. Below this value, the pandemic ends.

Use the slider to adjust values of vaccination coverage rate and vaccine efficacy and see what conditions lead to the end of the pandemic. Since some COVID-19 vaccinations require multiple dosages, we can conceptualize the Vaccinated (V) category as including individuals who have received all dosages._____no_output_____

# The SVEIRD model string is rewritten here
SVEIRD_COVID_model = '''
// Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: I -> D ; disease_mortality_rate*I ;
E13: S -> Vtot ; vaccination_coverage_rate*S ;

// Parameters
natural_mortality_rate = 0.00002377 // natural mortality rate, per day ;
contact_rate = 0.0257 // effective contact rate, per day ;
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .2 // rate of development of clinical symptoms, per day ;
recovery_rate = .00799 // recovery rate ;
disease_mortality_rate = 0.00019 // disease-induced mortality rate, per day ;
births = 10261 // recruitment rate of suceptible humans, per day ;
Rv := (symptom_development_rate*contact_rate*births*(natural_mortality_rate + (1 - vaccine_efficacy)*vaccination_coverage_rate))/(natural_mortality_rate*(vaccination_coverage_rate + natural_mortality_rate)*(natural_mortality_rate + symptom_development_rate)*(recovery_rate + natural_mortality_rate + disease_mortality_rate))

// Initial Conditions
S0 = 328E6 // total population ;
I = 5723616 // initial infected people ;
S = S0 - I - E - V - R  // susceptible population ;
V = 0 // vaccinated people ;
R = 3810791 // recovered people ;
E = 65533831 // asymptomatic cases ;
D = 237073 ;
Vtot = 0 // cumulative vaccine count ;
Ptotal := S + V + E + I + R ;
'''_____no_output_____# This function calculates the Rv value and prints a message for the epidemic ending or remaining ongoing, dependong on the value of Rv
def find_Rv(vacc_eff, vacc_cov_rate):
    m = te.loada(SVEIRD_COVID_model)
    m.resetAll()
    m.vaccination_coverage_rate = vacc_cov_rate
    m.vaccine_efficacy = vacc_eff
    s = m.simulate(0, 300, 2000,['time', 'E', 'I', 'V'])
    m.plot(title = "Infections and Vaccinations Over Time", xtitle = "Days", ytitle = "Individuals")
    if m.Rv < 425930847:
        print("Epidemic ended: Rv = " + str(m.Rv) + ", vaccine efficacy = " + str(m.vaccine_efficacy) + ", vaccination coverage rate = " + str(m.vaccination_coverage_rate))
        print("Percent vaccinated: " + str(m.Vtot/m.Ptotal))
    else:
        print("Epidemic ongoing: Rv = " + str(m.Rv) + ", vaccine efficacy = " + str(m.vaccine_efficacy) + ", vaccination coverage rate = " + str(m.vaccination_coverage_rate))
        print("Percent of population vaccinated: " + str(m.Vtot/m.Ptotal))
_____no_output_____interact(find_Rv, continuous_update = False, vacc_cov_rate = FloatSlider(min=0,max=0.01,step=0.0001,value=.0001), vacc_eff = FloatSlider(min=0,max=1,step=0.01,value=.1));_____no_output_____

Start by setting the vaccination coverage rate to 0.0003. 0.03% of the susceptible population would be vaccinated every day. Then increase the vaccine efficacy until you see a message that the epidemic ended. You should see that this occurs when the vaccine efficacy is 0.74, or 74% effective. In this model, around 10 million people are eventually vaccinated. Now try increasing the vaccination coverage rate to 0.001. Now you should notice that the epidemic will end if the vaccine efficacy is only 0.71, or 71% effective. In this model, around 20 million people are eventually vaccinated. This makes sense: if the vaccine efficacy is lower, we need to vaccinate more people. Notice that in either case, however, the percentage of the population that needs to be vaccinated is very low and the peak number of symptomatic cases is still very high. We would expect that more people would need to be vaccinated. Our standard for the end of the pandemic in the US is simply that cases will drop at a sufficiently quick rate so as not to remain at high levels for a prolonged period of time. This is evident as the downslope of the curves for asymptomatic and symptomatic individuals becomes steeper in cases when the pandemic ends in the US._____no_output_____### Question 2: If there is a limited amount of vaccines, what is the optimal allocation?
If there is a limited amount of vaccine, we need to determine the timeline for giving them. This could be done in 2 ways:

1. Continuously distributing the vaccines over a certain period of time
2. Distributing the vaccines in bursts

A continuous vaccination strategy would involve a constant vaccination coverage rate over a period of time. A burst strategy would involve a constant vaccination coverage rate during the burst of vaccinations followed by a non-burst interval of time with no vaccinations (vaccination coverage rate = 0). Let's look at the continuous strategy first and simulate over a year._____no_output_____

# This is the SVEIRSD model string used to simulate a continuous vaccination strategy. Vtot contains the total number
# of vaccines administered, and total_vaccines contains the maximum, finite number of vaccines. If Vtot grows
# equal to total_vaccines, the vaccination coverage rate drops to zero and no more vaccinations can occur.
SVEIRSD_continuous_vaccine = '''
// Equations
E1: S -> E ; contact_rate*S*I/Ptotal ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*I/Ptotal ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: R -> S ; resusceptible_rate*R ;
E13: I -> D ; disease_mortality_rate*I ;
E14: total_vaccines -> Vtot ; vaccination_coverage_rate*S ;
E15: at (total_vaccines <= 0): vaccination_coverage_rate = 0 ;

// Parameters
resusceptible_rate = 1E-6 // re-susceptibility rate ;
natural_mortality_rate = 0.00002377 // natural mortality rate, per day ;
contact_rate = 0.0257 // effective contact rate, per day ;
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .2 // rate of development of clinical symptoms, per day ;
recovery_rate = .00799 // recovery rate ;
disease_mortality_rate = 0.00019 // disease-induced mortality rate, per day ;
births = 10261 // recruitment rate of suceptible humans, per day ;

// Initial Conditions
S0 = 328E6 // total population ;
I = 5723616 // initial infected people ;
S1 = S0 - I // susceptible population ;
S = S1
V = 0 // vaccinated people ;
Vtot = 0 // cumulative vaccine count ;
total_vaccines = 50E6 // total number of vaccines available ;
R = 3810791 // recovered people ;
E = 65533831 // asymptomatic cases ;
D = 237073 ;
Ptotal := S + V + E + I + R ;
'''_____no_output_____# This function runs a yearlong simulation under a continuous vaccination strategy
def continuous_vaccine(total_vaccines, vacc_cov_rate, vacc_eff):
    m = te.loada(SVEIRSD_continuous_vaccine)
    m.resetAll()
    m.total_vaccines = total_vaccines
    m.vaccination_coverage_rate = vacc_cov_rate
    m.vaccine_efficacy = vacc_eff
    s = m.simulate(0, 365, 365*10,['time', 'I', 'V', 'D'])
    m.plot(title = "Continuous Vaccine Distribution", xtitle = "Days", ytitle = "Individuals")
    print("Total initial vaccines: " + str(total_vaccines))
    print("Vaccination coverage rate: " + str(m.vaccination_coverage_rate))
    print("Vaccine efficacy: " + str(m.vaccine_efficacy))
    print("Final symptomatic population: " + str(s['I'][-1]))
    print("Total deaths: " + str(s['D'][-1]))
    print("Percent of population vaccinated: " + str(m.Vtot/m.Ptotal))

interact(continuous_vaccine, continuous_update = False, total_vaccines = FloatSlider(min=10**7,max=10**8,step=10**7,value=10**7), vacc_cov_rate = FloatSlider(min=0,max=.01,step=.0001,value=.0001), vacc_eff = FloatSlider(min = 0, max = 1, step = 0.01, value = 0.01));_____no_output_____

Let's once again focus on the vaccination coverage rate and vaccine efficacy, since the total vaccines parameter is mostly there to ensure we don't use more than our available vaccines. Set the total number of vaccines to its maximum value using the slider. Then you'll notice that as you increase either the vaccination coverage rate or vaccine efficacy, the final symptomatic population decreases, the final vaccinated population increases, and the total deaths decrease. _____no_output_____Now let's look at the burst method. Here, there are more parameters to consider: number of bursts, number of days in a burst, and number of days between bursts. _____no_output_____

# This is the SVEIRSD model string used to simulate a burst vaccination strategy. Vtot contains the total number
# of vaccines administered during a burst, and total_vaccines contains the maximum, finite number of vaccines that
# can be administered in a burst. If Vtot grows equal to total_vaccines, the vaccination coverage rate drops to
# zero and no more vaccinations can occur.
SVEIRSD_burst_vaccine = '''
// Equations
E1: S -> E ; contact_rate*S*I/Ptotal ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*I/Ptotal ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: R -> S ; resusceptible_rate*R ;
E13: I -> D ; disease_mortality_rate*I ;
E14: total_vaccines -> Vtot ; vaccination_coverage_rate*S ;
E15: at (total_vaccines <= 0): vaccination_coverage_rate = 0 ;

// Parameters
resusceptible_rate = 1E-6 // re-susceptibility rate ;
natural_mortality_rate = 0.00002377 // natural mortality rate, per day ;
contact_rate = 0.0257 // effective contact rate, per day ;
vaccination_coverage_rate = 0 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .2 // rate of development of clinical symptoms, per day ;
recovery_rate = .00799 // recovery rate ;
disease_mortality_rate = 0.00019 // disease-induced mortality rate, per day ;
births = 10261 // recruitment rate of suceptible humans, per day ;

S0 = 328E6 // total population ;
I = 5723616 // initial infected people ;
S = S0 - I // susceptible population ;
V = 0 // vaccinated people ;
Vtot = 0 // total vaccines given ;
R = 3810791 // recovered people ;
E = 65533831 // asymptomatic cases ;
D = 237073 ;
Ptotal := S + V + E + I + R ;
total_vaccines = 50E6 // total number of available vaccines ;
'''_____no_output_____# This function runs a simulation under a burst vaccination strategy. The length of the simulation depends on the
# number of bursts, number of days in a burst, and number of days between bursts.
def burst_vaccine(total_vaccines, non_burst_interval, num_bursts, days_per_burst, vacc_cov_rate, vacc_eff):
    # load model string
    m = te.loada(SVEIRSD_burst_vaccine)
    # create lists to store simulation values for the susceptible population, vaccinated
    # population, asymptomatic population, symptomatic population, recovered population,
    # deaths from COVID-19, and total number of vaccines given.
    S = np.empty((0,0))
    V = np.empty((0,0))
    E = np.empty((0,0))
    I = np.empty((0,0))
    R = np.empty((0,0))
    D = np.empty((0,0))
    m.resetAll()
    # set total number of vaccines per burst, vaccination coverage rate, and vaccine efficacy
    m.total_vaccines = total_vaccines/num_bursts
    m.vaccination_coverage_rate = vacc_cov_rate
    m.vaccine_efficacy = vacc_eff
    # for every burst
    for n in range(num_bursts):
        # run a simulation for a single vaccination burst
        s = m.simulate(0, days_per_burst, days_per_burst*10,['time', 'S', 'V', 'E', 'I', 'R', 'D', 'Vtot'])
        # append the simulation results for each population category to the appropriate list
        S = np.append(S, s['S'].reshape(1, s['S'].shape[0]))
        V = np.append(V, s['V'].reshape(1, s['V'].shape[0]))
        E = np.append(E, s['E'].reshape(1, s['E'].shape[0]))
        I = np.append(I, s['I'].reshape(1, s['I'].shape[0]))
        R = np.append(R, s['R'].reshape(1, s['R'].shape[0]))
        D = np.append(D, s['D'].reshape(1, s['D'].shape[0]))
        m.resetAll()
        # set the vaccination coverage rate to 0 in preparation for a non-burst simulation
        m.vaccination_coverage_rate = 0
        # set initial values for the non-burst simulation based on the final values in each category
        # from the previous simulation
        m.S = s['S'][-1]
        m.V = s['V'][-1]
        m.E = s['E'][-1]
        m.I = s['I'][-1]
        m.R = s['R'][-1]
        m.D = s['D'][-1]
        m.Vtot = s['Vtot'][-1]
        # run the non-burst simulation
        s = m.simulate(0, non_burst_interval, non_burst_interval*10,['time', 'S', 'V', 'E', 'I', 'R', 'D', 'Vtot'])
        # append the simulation results for each population category to the appropriate list
        S = np.append(S, s['S'].reshape(1, s['S'].shape[0]))
        V = np.append(V, s['V'].reshape(1, s['V'].shape[0]))
        E = np.append(E, s['E'].reshape(1, s['E'].shape[0]))
        I = np.append(I, s['I'].reshape(1, s['I'].shape[0]))
        R = np.append(R, s['R'].reshape(1, s['R'].shape[0]))
        D = np.append(D, s['D'].reshape(1, s['D'].shape[0]))
        m.resetAll()
        # set total number of vaccines per burst, vaccination coverage rate, and vaccine efficacy
        # in preparation for burst simulation
        m.total_vaccines = total_vaccines/num_bursts
        m.vaccination_coverage_rate = vacc_cov_rate
        m.vaccine_efficacy = vacc_eff
        # set initial values for the non-burst simulation based on the final values in each category
        # from the previous simulation
        m.S = s['S'][-1]
        m.V = s['V'][-1]
        m.E = s['E'][-1]
        m.I = s['I'][-1]
        m.R = s['R'][-1]
        m.D = s['D'][-1]
        m.Vtot = s['Vtot'][-1]
    # create an array for the time period of the simulation
    T = np.arange(0, (num_bursts*days_per_burst)+(num_bursts*int(non_burst_interval)), .1)
    # plot the number of vaccinated population, symptomatic population, and COVID-19 deaths over time
    Vaccinated = plt.plot(T, V)
    Symptomatic = plt.plot(T, I)
    Deaths = plt.plot(T, D)
    plt.legend(['Vaccinated', 'Symptomatic', 'Deaths'])
    plt.title("Burst Vaccine Distribution")
    plt.xlabel("Days")
    plt.ylabel("Individuals")
    plt.show()
    # Print key parameters
    print("Total initial vaccines: " + str(total_vaccines))
    print("Non-burst time interval: " + str(non_burst_interval))
    print("Number of bursts: " + str(num_bursts))
    print("Days per burst: " + str(days_per_burst))
    print("Vaccination coverage rate: " + str(m.vaccination_coverage_rate))
    print("Vaccine efficacy: " + str(m.vaccine_efficacy))
    print("Final symptomatic population: " + str(I[-1]))
    print("Total deaths: " + str(D[-1]))
    print("Percent of population vaccinated: " + str(m.Vtot/m.Ptotal))

# This line creates the sliders that allow users to adjust parameter values
interact(burst_vaccine, continuous_update = False, total_vaccines = FloatSlider(min=10**7,max=10**8,step=10**7,value=10**7), num_bursts = IntSlider(min = 1, max = 12, step = 1, value = 1), days_per_burst = IntSlider(min = 1, max = 60, step = 1, value = 30), vacc_cov_rate = FloatSlider(min=0,max=.01,step=.0001,value=.0001), vacc_eff = FloatSlider(min = 0, max = 1, step = 0.01, value = 0.01), non_burst_interval = IntSlider(min = 0, max = 60, step = 1, value = 30));_____no_output_____

As you increase the number of bursts, vaccination coverage rate, or vaccine efficacy, you'll notice that the number of symptomatic cases decreases as time goes on. You'll also begin to notice oscillations in the curve for the vaccinated population. This occurs because the number of vaccinated individuals increases during a burst and decreases during a non-burst period. Because no vaccinations occur durin the non-burst period, there is time for the overall number of vaccinated people drop as they become infected.

As with the continuous vaccination method, let's simulate the burst method over a year (approximately). Using the sliders, set the total number of initial vaccines to its maximum value, both the non-burst interval (days between bursts) and the days per burst to 30, with 6 total bursts. As with continuous vaccination you'll notice that overall cases and deaths will decrease as vaccination coverage rate and vaccine efficacy are increased. Previously, we found that a vaccination coverage rate of 0.0003 coupled with a vaccine efficacy of 0.74 should lead to the end of the pandemic in the US. Set the vaccination coverage rate and vaccine efficacy to these values for both the continuous and burst vaccine models. You'll notice that the continuous model leads to fewer cases, deaths, and a higher percentage of vaccines distributed, compared to the burst model. This makes sense since continuously vaccinating every day offsets the number of vaccinated individuals who later become infected. This is not the case in the burst model, since no vaccinations occur in the non-burst interval, allowing for a decrease in the number of vaccinated individuals as they become infected. In general, continuous vaccination is cheaper because year-round vaccinations mostly consist of people who would need booster doses of vaccinations anyway. The burst model works for seasonal illnesses like the flu or if the normal vaccination coverage for a disease is low but a localized outbreak occurs.

_____no_output_____### Question 3: What if re-vaccination is required?
The immunity provided by a vaccine may not last for a lifetime. In this case, re-vaccination would be required. We can model this by introducing a new V --> S transition in which vaccinated individuals become susceptible again over time. We'll call this re-susceptibility rate for vaccinated individuals $\theta$ and define it as the inverse of the time it takes for a vaccinated individual to become susceptible again. For example, if a vaccinated individual becomes susceptible to infection by SARS_CoV-2 again after 6 months (approximately 180 days), $\theta$ would be 1/180. The arrow diagram and equations for this adjusted SVEIRSD model are below:

Use the slider below to examine what happens as we adjust the rate at which vaccinated individuals become susceptible again. This simulation will cover a 3-year period and assumes a vaccination coverage rate of 0.003 and a vaccine efficacy of 0.8. The remaining parameters are based on the parameter values found when fitting the model._____no_output_____

SVEIRSD_revaccinate_COVID_model = '''
//Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: R -> S ; resusceptible_rate*R ;
E13: I -> D ; disease_mortality_rate*I ;
E14: V -> S ; vaccinated_resusceptible_rate*V ;

// Parameters
resusceptible_rate = 1E-6 // re-susceptibility rate for recovered individuals;
natural_mortality_rate = 0.00002377 // natural mortality rate, per day ;
contact_rate = 0.0257 // effective contact rate, per day ;
vaccination_coverage_rate = 0.003 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .2 // rate of development of clinical symptoms, per day ;
recovery_rate = .00799 // recovery rate ;
disease_mortality_rate = 0.00019 // disease-induced mortality rate, per day ;
births = 10261 // recruitment rate of suceptible humans, per day ;
vaccinated_resusceptible_rate = 0.008 ; // re-susceptibility rate for vaccinated individuals ;

// Initial Conditions
S0 = 328E6 // total population ;
I = 5723616 // initial infected people ;
S = S0 - I - V - R - E // susceptible population ;
V = 0 // vaccinated people ;
R = 3810791 // recovered people ;
E = 752565 // asymptomatic cases ;
D = 237073 ;
Ptotal := S + V + E + I + R ;
'''_____no_output_____def plot_resusceptible_rate_vaccinated_COVID(vacc_resusceptible_rate):
    m = te.loada(SVEIRSD_revaccinate_COVID_model)
    m.resetAll()
    m.vaccinated_resusceptible_rate = vacc_resusceptible_rate
    s = m.simulate(0,1095,100000,['time','V', 'S', 'E', 'I', 'R'])
    m.plot(title = "Populations Over Time", xtitle = "Days", ytitle = "Individuals")
    print("Resusceptible rate for vaccinated individuals: " + str(m.vaccinated_resusceptible_rate))
    print("Final number of total cases (asymptomatic and symptomatic): " + str(s['I'][-1] + s['E'][-1]))
    print("Final number of vaccinated individuals: " + str(s['V'][-1]))
interact(plot_resusceptible_rate_vaccinated_COVID, vacc_resusceptible_rate = FloatSlider(min=0.0001,max=.008,step=.0001,value=0.0001), continuous_update=False);_____no_output_____

Notice that as you increase the re-susceptibility rate for vaccinated individuals (and hence decrease the time it takes for the the vaccine's protection to wear off), something interesting happens. The final number of total cases (both asymptomatic and symptomatic) increases up until reaching a re-susceptibility rate of 0.0029, or 345 days until becoming susceptible again. After this point, the final number of total cases decreases again. Why might this be? Remember that our re-susceptibility rate for recovered individuals was negligibly small at 0.000001. As the re-susceptibility rate for vaccinated individuals increases, vaccinated people become susceptible again at higher rates. These susceptible people can then either become vaccinated again or infected. As we see the number of infected individuals grow, we expect to see more recoveries follow. Since recovered individuals return to the Susceptible category at such small rates, we eventually see an accumulation of recovered individuals and a decrease in susceptible individuals. This decrease in susceptible individuals then leads to a decrease in total cases later.

But what if the re-susceptibility rate for recovered individuals were not negligible? The simulation below assumes that recovered individuals become susceptible again after 180 days, or approximately 6 months, giving a re-susceptibility rate of 1/180, or 0.0056. In this case, as you use the slider to increase the re-susceptibility rate for vaccinated individuals, there is a strict increase in the final number of cases (both asymptomatic and symptomatic) since the return of recovered individuals to the Susceptible category does not allow for an accumulation of recovered individuals. _____no_output_____

SVEIRSD_revaccinate_COVID_model_180 = '''
//Equations
E1: S -> E ; contact_rate*S*(I/Ptotal) ;
E2: E -> I ; symptom_development_rate*E ;
E3: I -> R ; recovery_rate*I ;
E4: S -> V ; vaccination_coverage_rate*S ;
E5: V -> E ; contact_rate*(1 - vaccine_efficacy)*V*(I/Ptotal) ;
E6: -> S ; births ;
E7: S -> ; natural_mortality_rate*S ;
E8: V -> ; natural_mortality_rate*V ;
E9: E -> ; natural_mortality_rate*E ;
E10: I -> ; natural_mortality_rate*I ;
E11: R -> ; natural_mortality_rate*R ;
E12: R -> S ; resusceptible_rate*R ;
E13: I -> D ; disease_mortality_rate*I ;
E14: V -> S ; vaccinated_resusceptible_rate*V ;

// Parameters
resusceptible_rate = 1/180 // re-susceptibility rate for recovered individuals;
natural_mortality_rate = 0.00002377 // natural mortality rate, per day ;
contact_rate = 0.0257 // effective contact rate, per day ;
vaccination_coverage_rate = 0.003 // vaccination coverage rate, per day ;
vaccine_efficacy = 0.8 // vaccine efficacy ;
symptom_development_rate = .2 // rate of development of clinical symptoms, per day ;
recovery_rate = .00799 // recovery rate ;
disease_mortality_rate = 0.00019 // disease-induced mortality rate, per day ;
births = 10261 // recruitment rate of suceptible humans, per day ;
vaccinated_resusceptible_rate = 0.008 ; // re-susceptibility rate for vaccinated individuals ;

// Initial Conditions
S0 = 328E6 // total population ;
I = 5723616 // initial infected people ;
S = S0 - I - V - R - E // susceptible population ;
V = 0 // vaccinated people ;
R = 3810791 // recovered people ;
E = 752565 // asymptomatic cases ;
D = 237073 ;
Ptotal := S + V + E + I + R ;
'''_____no_output_____def plot_resusceptible_rate_vaccinated_COVID_180(vacc_resusceptible_rate):
    m = te.loada(SVEIRSD_revaccinate_COVID_model_180)
    m.resetAll()
    m.vaccinated_resusceptible_rate = vacc_resusceptible_rate
    s = m.simulate(0,1095,100000,['time','V', 'S', 'E', 'I', 'R'])
    m.plot(title = "Populations Over Time", xtitle = "Days", ytitle = "Individuals")
    print("Resusceptible rate for vaccinated individuals: " + str(m.vaccinated_resusceptible_rate))
    print("Final number of total cases (asymptomatic and symptomatic): " + str(s['I'][-1] + s['E'][-1]))
    print("Final number of vaccinated individuals: " + str(s['V'][-1]))
interact(plot_resusceptible_rate_vaccinated_COVID_180, vacc_resusceptible_rate = FloatSlider(min=0.0001,max=.008,step=.0001,value=0.0001), continuous_update=False);_____no_output_____

Thank you for using this app! Hopefully now you have a better understanding of epidemiological modeling and how it can be used to answer key questions on epidemics and pandemics. _____no_output_____## Appendix_____no_output_____Here are the files containing the US COVID-19 data used in this app. It is global data, but the US-specific data can be extracted from them.

[Cumulative Case Data](global_confirmed_data.rtf)

[COVID-19 Death Data](global_death_data.rtf)

[Recovered Case Data](global_recovered_data.rtf)

The files can be opened as CSV files._____no_output_____

data = []
with open('global_confirmed_data.rtf', newline = '') as csvfile:
    lines = csv.reader(csvfile, delimiter = ',')
    for l in lines:
        data.append(l)
confirmed = data[10:]
for i in range(len(confirmed)):
    if confirmed[i][1] == 'US':
        confirmed = confirmed[i]
        break
confirmed = confirmed[4:]

for j in range(len(confirmed)):
    if '\\' in confirmed[j]:
        confirmed[j] = confirmed[j].replace('\\', '')
    confirmed[j] = int(confirmed[j])

_____no_output_____data = []
with open('global_death_data.rtf', newline = '') as csvfile:
    lines = csv.reader(csvfile, delimiter = ',')
    for l in lines:
        data.append(l)
deaths = data[10:]
for i in range(len(deaths)):
    if deaths[i][1] == 'US':
        deaths = deaths[i]
        break
deaths = deaths[4:]

for j in range(len(deaths)):
    if '\\' in deaths[j]:
        deaths[j] = deaths[j].replace('\\', '')
    deaths[j] = int(deaths[j])

_____no_output_____data = []
with open('global_recovered_data.rtf', newline = '') as csvfile:
    lines = csv.reader(csvfile, delimiter = ',')
    for l in lines:
        data.append(l)
recovered = data[10:]
for i in range(len(recovered)):
    if recovered[i][1] == 'US':
        recovered = recovered[i]
        break
recovered = recovered[4:]

for j in range(len(recovered)):
    if '\\' in recovered[j]:
        recovered[j] = recovered[j].replace('\\', '')
    recovered[j] = int(recovered[j])

_____no_output_____

## References
\[1\] Georgiou, A. (2020, November 17). It’s Been Exactly One Year Since the First Cse of COVID Was Found in China. Newsweek. https://www.newsweek.com/one-year-first-covid-case-china-1547200

This source contains information on the first reported case of COVID-19.

\[2\] AJMC Staff. (2020, November 25). A Timeline of COVID-19 Developments in 2020. AJMC. https://www.ajmc.com/view/a-timeline-of-covid19-developments-in-2020

This source is a timeline of the major events of the COVID-19 pandemic worldwide.

\[3\] Dong, E., Hongru Du, and Lauren Gardner. (2020). An Interactive Web-Based Dashboard to Track COVID-19 in Real Time. In The Lancet Infectious Diseases. Lancet Publishing Group. doi, 10, S1473-3099.

This is the data source used in this app. It contains US-specific and global data for cumulative COVID-19 cases, deaths, and recoveries.

\[4\] Clinical Care Guidance. (2020, December 8). CDC. https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-guidance-management-patients.html

This CDC source contains clinical information on COVID-19, including symptoms and the median incubation period.

\[5\] Understanding mRNA COVID-19 Vaccines. (2020, December 18). CDC. https://www.cdc.gov/coronavirus/2019-ncov/vaccines/different-vaccines/mrna.html

This source includes information on mRNA vaccines, particularly in relation to COVID-19 vaccines.

\[6\] Gumel, A. B., McCluskey, C. C., & Watmough, J. (2006). An SVEIR model for assessing potential impact of an imperfect anti-SARS vaccine. Mathematical Biosciences & Engineering, 3(3), 485.

This is the paper on which the models are based. The SVEIR model here was modified to include a Death (D) category for COVID-19 deaths. The Asymptomatic (E) category was omitted to create the SVIRD model, and a new transition from Recovered (R) to Susceptible (S) was added to create the SVEIRSD model.

\[7\] Ritchie, H., Ortiz-Ospina, E., Beltekian, D., Mathieu, E., Hasell, J., Macdonald, B., Giattino, C., Roser, M., Yunits, B., van Woerden, E., Gavrilov, D., Bergel, M., Ahmad, S., & Crawford, J. (2021). Coronavirus (COVID-19) Vaccinations. Our World in Data. https://ourworldindata.org/covid-vaccinations

This data source shows the number of total COVID-19 vaccinations and daily COVID-19 vacinations given in multiple countries.

\[8\] How effective is the flu vaccine? (2020, December 16). CDC. https://www.cdc.gov/flu/vaccines-work/vaccineeffect.htm

This source provides information on the efficacy of the flu vaccine.

\[9\] Polack, F. P., Thomas, S. J., Kitchin, N., Absalon, J., Gurtman, A., Lockhart, S., Perez, J. L., Marc, G. P., Moreira, E. D., Zerbini, C., Bailey, R., Swanson, K.A., et al. (2020, December 31). Safety and Efficacy of the BNT162b2 mRNA COVID-19 Vaccine. The New England Journal of Medicine, 383(27), 2603-2615.

This source discusses the efficacy of the mRNA vaccine developed by Pfizer and BioNTech.

\[10\] Reinfection. (2020, October 27). CDC. https://www.cdc.gov/coronavirus/2019-ncov/your-health/reinfection.html

This source discusses the potential for reinfection by SARS-CoV-2.

\[11\] Hamilton, B.E., Martin J.A., and Osterman, M.J.K. (2020). Births: Provisional Data for 2019 (Report No. 008). National Center for Health Statistics. https://www.cdc.gov/nchs/data/vsrr/vsrr-8-508.pdf

The data from this source was used to estimate the average daily births parameter for the models.

\[12\] Xu, J., Murphy, S.L., Kochanek, K.D., and Arias, E. (2020). Mortality in the United States, 2018 (Report No. 355). Center for Disease Control and Prevention. https://www.cdc.gov/nchs/data/databriefs/db355-h.pdf.

This source was used to estimate the daily natural death rate for the models.

_____no_output_____