Predicting Early Stages of the SARS-CoV-2 Pandemic in the United States Using an Expert Judgment Model

The emergence of the SARS-CoV-2 pandemic in December of 2019, was the beginning of a public health crisis. With limited information on the transmission and spatial spread of the virus at the onset of the outbreak, public health officials were faced with many difficult decisions on how to proceed. 

Infectious disease experts began to form their own predictions and modeling frameworks during the pandemic. By utilizing existing disease data and analyzing trends in spread, mathematical models are helpful tools to have available when outbreaks occur. During the beginning of the SARS-CoV-2 pandemic, while many disease models were in development, input from these experts was another important source of information for predictions about the virus. Researchers identified numerous questions that needed to be answered regarding Coronavirus: predicted number of cases, number of deaths, how the decisions of state governments would impact number of cases, and the percentage of infections reported as cases. 

A study conducted between February and May of 2020 surveyed epidemiological experts’ predictions in these areas, while the pandemic was still in its initial phases. Forty-one infectious disease experts contributed to these surveys to offer their expert opinions, including experts from the Center for the Ecology of Infectious Diseases. The inputs from each expert were then combined into a “linear pool”, examining the expert’s predictions holistically, rather than looking at individual responses. Within this combined data pool, the overall predictions were more accurate for the indicated measures, compared to individual predictions. For example, in 36 out of the 44 total predictions, the linear pool was among the top 50% most accurate forecasts, while individual responses were more subject to variation. 

Experts were asked to predict the number of SARS-CoV-2 related deaths that would occur in the U.S. by the end of 2020. Although the experts underestimated the true number of deaths on average, they recognized that it was highly likely that there would be over 250,000 deaths due to SARS-CoV-2. In March of 2020, experts predicted around a 50% chance of over 250,000 deaths occurring. At the end of 2020, there were 336,802 reported deaths due to SARS-CoV-2. When experts were asked to predict the weekly SARS-CoV-2 reported cases, they tended to underestimate the number of cases early on, but the expert’s accuracy of predicting weekly cases began to greatly improve over the surveyed period. Experts were also asked to predict the percentage of SARS-CoV-2 infections out of all SARS-CoV-2 cases in the U.S. The median percentage of infections detected in expert predictions in the linear pool was between 6% and 16%, though this outcome can never truly be measured. Experts were further tasked with predicting the number of confirmed cases that would occur if a state 1) continued with reopening or 2) if the state did not start to reopen. In Georgia, 1,044 cases were predicted if the state decided to reopen, compared to 487 cases if lock-down restrictions remained in place. After Georgia decided to keep restrictions in place in April of 2020, the expert predictions were revealed to be fairly accurate. 

The results of this study highlight the importance of mathematical modelers and their expert opinions in providing insight into the trajectory of SARS-CoV-2 early on in the pandemic. Utilizing an expert linear pool model, such as the model used in this study, can help provide answers surrounding outbreaks, supplementing other mathematical models to predict pandemics reliably. 

For more information about this study, please click here

By: Brenna Daly