Modeling the pandemic for prediction and policymaking
February 1, 2021
“The outbreak should follow the same process in every community,” says Los Alamos scientist Ben McMahon. “At least in theory. It should get worse and worse until the community realizes they have to get serious about isolating, and then it should fall away quickly. The epidemic curve is really a learning curve.”
But the COVID-19 outbreak is far from a textbook event, and McMahon, a key player in Los Alamos’s comprehensive effort to model the pandemic, is knee-deep in all the ways the learning curve can be distorted. In a joint enterprise to model the pandemic for better-informed policymaking, Los Alamos shares detailed weekly reports with three other national laboratories, and every single week—even after the better part of a year—surprising, fundamental new information is still coming to light. For a disease that stubbornly carves out an exception to nearly every rule the experts try to attach to it—from the symptoms it produces to the effectiveness of the antibodies its survivors retain—McMahon and his colleagues strive to assemble the most believable set of “facts” possible and feed them to a computer to answer one question: What is likely to happen next?
“The trouble is”—McMahon has to interrupt himself here—“well, one of the many troubles is: Susceptibility varies greatly depending on age, sex, and certain preexisting conditions. That means some people are substantially less likely to die, get tested, or even show any symptoms, even though they may be every bit as likely to transmit the virus.” With something like Ebola, everyone is suitably terrified, young and old, and the learning curve is very steep: isolate or die. With COVID-19, the weight of the message is considerably more fragmented.
Uncertainties about both the contagion itself and the personal and societal behaviors that contribute to either its spread or its containment greatly complicate researchers’ efforts to predict the course of the pandemic. But that information is absolutely crucial. If policymakers know the potential landscape of tomorrow, they will have a much better idea of what to do about it today.
What will happen
Los Alamos has a number of COVID-modeling efforts underway. The most widely shared of these is on its public website and featured on the Centers for Disease Control (CDC) website as well, due to its track record for accuracy. The model spans the globe, country by country, and the United States, state by state. It is produced and managed by a team of about 20 Los Alamos specialists, including computer scientists, bioscientists, mathematicians, economists, and others; statistician Dave Osthus leads the team.
“Our model produces forecasts, not projections,” Osthus explains. “Whereas a projection predicts what would happen if various strategies were put in place or various circumstances came to pass, a forecast directly predicts what will happen based on what is already happening.” That doesn’t mean it ignores policy interventions, such as stay-at-home orders—far from it. But rather than trying to figure out how much of a difference they ought to make, the model examines how much of a difference they are already making or how much difference they have already made elsewhere. The result is an ultimate best-guess at the future—cumulative confirmed cases and deaths—driven by real-world data.
Unlike flu, there is no historical data on COVID-19—no benefit of hindsight.
Real-world data, however, are not especially straightforward. Actual cases are sharply different from confirmed cases; confirmed cases result from testing, and testing is not uniformly accurate. And even if all COVID-19 tests were perfectly accurate, there would still be a huge question mark when it comes to who is getting tested. How many people? Which ones? People who are already sick? People who visit a clinic for some other reason? Or a cross section of the public at large? There is tremendous variation in procedures from state to state and even county to county, since much of this data is obtained by public health departments at the county level. The Los Alamos statistical model has to deal with these challenges and generate the most reliable prediction possible anyway.
To do that, the model has to learn; it has to assimilate large amounts of data and figure out how to recognize trends, broken down by region. It also has to learn from its mistakes. As events unfold and new data are gathered from one week to the next, the model must attempt to improve itself.
Fortunately, Osthus had already been working with just such a machine-learning model, called Dante, to predict recent flu seasons. In a contest sponsored by the CDC for the 2018–2019 flu season, 24 teams submitted model output, and Dante’s predictions came closest to matching reality. Osthus and others reworked it for the COVID-19 pandemic.
However, COVID-19 and flu have two important differences, in terms of modeling. The first is the fact that people have been dealing with the flu for ages, and there is a lot of valuable historical data to work with, but not for COVID-19—there’s no benefit of hindsight. All the data on COVID-19 comes from the current pandemic in real time. To put it bluntly, the forecast gets more accurate if more people get sick and die.
The other major difference between the current pandemic and the flu stems from individual behavior. Because flu is so familiar, the range of human behavior is not very wide. A relatively consistent fraction of infected people will go to work anyway, despite feeling sick. A relatively consistent fraction of people will see a doctor. A relatively consistent fraction of people will get a flu vaccine each year. It is through this similarity from season to season that a gigantic source of uncertainty—human behavior—can be tamed. But with COVID-19, individual behaviors are critical, and there is no historical basis to justify anything modelers might assume. Hand washing, face masks, social distancing, restricted travel—such things vary to a large degree and are extraordinarily difficult to predict or even assess after the fact. How often did residents of Hawaii or Ohio wash their hands in the past month? How seriously did they adhere to social distancing mandates?
Without knowing the answers to these kinds of questions, it’s difficult to predict the future. It’s even more difficult to determine which interventions would be the most effective. But just because individual behavior is difficult to quantify doesn’t mean Los Alamos scientists can’t find a way to model it.
What would happen
A trio of cause-and-effect, rather than statistical, Los Alamos models is intended to address what-if questions. What would happen if schools ramp up onsite learning? Or if non-pharmaceutical interventions, such as face masks, social distancing, and hygiene measures, were intensified (or reduced)? Or if a vaccine were distributed in a particular way?
Perhaps the most straightforward of these models is EpiGrid, an epidemiological model that tracks the geographic spread of a disease by breaking the landscape into a connected grid of 10-kilometer-square regions, rather than administrative units like countries, states, or counties. Originally developed as a risk-assessment tool for bioterror attacks and natural pandemics, EpiGrid is comprehensive and versatile, making do with imperfect data. Scenarios have been developed for many countries, pathogens, and assumed responses.
EpiGrid accounts for details of the infectious agent itself (How long does it incubate? How is it transmitted—droplets, contaminated water, mosquitoes, etc.? Are asymptomatic or pre-symptomatic people contagious? Can people who have recovered be infected again?), the progression of the disease (How many people are susceptible? Exposed? Infected? Seriously ill or hospitalized? How many have recovered? How many have died?), the modes of treatment (Antivirals? Vaccines? Other treatments?), and societal actions (Are quarantines in place? Are masks required? Are schools open?).
“Los Alamos has been doing epidemiological modeling for decades, starting with HIV,” says Paul Fenimore, EpiGrid project leader. “It’s a capability we were very wise to develop.” For the sudden emergence of COVID-19, Fenimore and his colleagues strive to make EpiGrid as reliable as it already is for infections like plague or cholera. So they work the problem in both directions: in January, they forecast February, and in February, they retroactively assess what did and didn’t work in the forecast in January.
Another key model, EpiCast, has similarly deep roots—but from a completely different kind of soil. Rather than being built from the ground up for epidemiology, EpiCast was adapted from an earlier materials-science model designed to support nuclear weapons technology. Just as individual atoms contribute to the nature of a material, individual infected people contribute to the progression of an epidemic, and the model is structured to treat each element (atoms or people) in an agent-based fashion, tracking its influence and that of its neighbors to their ultimate global effects. Whereas EpiGrid typically covers large regions in aggregate (e.g., the eastern half of the country) with medium-grain resolution, EpiCast resolves down to the census-tract level, consisting of only about 2000 individuals, capturing their contact networks and daily travels, as well as any pandemic-related policy restrictions on either.
Not surprisingly, operating a model with such resolution requires a powerful computer. While EpiGrid can run on a laptop, EpiCast requires a supercomputer—and Los Alamos has several. In fact, Los Alamos has long been a key player nationally in high-performance computing (HPC) across the board, always keeping up with cutting-edge hardware, expert personnel, and scientists studying both the complex systems that require HPC for their simulations (e.g., climate models) and the science of HPC itself (such as minimizing error rates and applying different algorithmic approaches). Los Alamos HPC capabilities are currently being shared across a broad consortium of national laboratories and government agencies, universities, and technology companies to make supercomputers—which are normally prohibitively expensive for smaller organizations—freely available to researchers working to combat the virus with computationally intensive tasks such as drug discovery.
With Los Alamos’s own agent-based HPC pandemic model, the results are especially credible, since the “agents” are essentially actual Americans: EpiCast incorporates real census counts combined with accurate demographics, school and workforce participation, and public-transit commuter information, among other key parameters. In addition, a key differentiator between EpiCast and other similar efforts is its ability to categorize workers within different industry sectors. This feature proved critical in understanding and projecting the pandemic in the United States by taking into account the variability in work-from-home policies affecting different segments of the workforce. The effects of changing mitigation strategies or individual behaviors thus percolate through an uncommonly realistic representation of the American populace. It is here that Los Alamos scientists Tim Germann, Carrie Manore, and Sara Del Valle can model those difficult-to-model human behaviors and analyze which ones are most effective in slowing the pandemic. As a result, EpiCast has been able to meaningfully assess the impact of reopening schools and workplaces.
What does happen
Inferring the movement of the virus from epidemiological data, such as interviews with infected people to pinpoint where they have been and with whom they have had contact, results in an incomplete picture, making it difficult to calibrate models with real-world data. Los Alamos scientists Emma Goldberg, Ethan Romero-Severson, and Thomas Leitner are therefore tracking the movement of the virus with direct analyses of its genome as it migrates through the human population. Small, natural mutations are always happening to individual viral particles, and they happen at a fairly steady rate of approximately one or two nucleotides (basic elements of genetic code) every one or two weeks. That stream of inherited changes makes it possible to draw conclusions along the lines of whether this person could have acquired SARS-CoV-2 from that source (person, hospital, city, etc.) over such and such a timeframe when the viral genomes are so different.
The epidemic curve is really a learning curve.
By tracing what the mutations show about the relatedness of infections, i.e., the phylogenetics—a capability Los Alamos previously advanced to address the evolution of HIV infections—the scientists can help identify how and when the virus traveled from one region to another. This makes it possible to reliably tease apart whether a resurgence of cases in one area was caused by community spread within that area or by reinfection from the outside. The answer matters: if it’s the former, then it might make sense to double down on isolation measures, such as closures and social distancing; if it’s the latter, it might be more consequential to restrict interstate travel. In this way, real-world genomic data can be used to identify what happened in specific regions at specific times—and also validate (or contradict) models such as EpiCast, allowing them to more accurately extrapolate and predict the direction of the pandemic across the country.
“Of course, we need up-to-date genome data to make up-to-date inferences,” says Goldberg. “That’s why we’re coordinating with the University of New Mexico, TriCore Reference Laboratories, and the New Mexico Department of Health to continue to get viral genomes as more infections are confirmed in state.” She and Romero-Severson are performing sophisticated statistical analyses to pull patterns from this in-state data, combined with other publicly available genomic data shared from across the globe. Such patterns reveal actionable characteristics of the movement of the virus—for example, which groups of cases trace back to a single introduction into New Mexico and how the number of such introductions is changing over time.
Meanwhile, Leitner is comparing current SARS-CoV-2 phylogenetics with those of other recent coronavirus outbreaks, including SARS-CoV and MERS-CoV, and with other types of resident coronavirus infections in animals, such as bats. In addition, a user-friendly web interface for genomic science, built by Los Alamos bioinformatics specialist Patrick Chain and his colleagues, is now being used to help automate the reconstruction of SARS-CoV-2 genomes for inclusion in phylogenetic trees and public genome repositories. The system analyzes the population of viral genomes found in a sample from a COVID-19 patient and identifies specific mutations and their prevalence. There is also a feature for evaluating how effective current high-quality viral-RNA-based COVID-19 diagnostic tests are at recognizing emerging genetic variants. And all of this work—phylogenetic analysis, pattern extraction, comparative studies, genome reconstruction, and diagnostic-test validation—capitalizes on Los Alamos computing technology and expertise.
In addition to geographic, phylogenetic, and behavioral aspects, a final key element of the Los Alamos modeling effort is systemic and capitalizes on a major research initiative from the previous decade. From 2003 to 2010, Los Alamos scientists modeled the nation’s critical infrastructure—things like power, transportation, and, of particular relevance now, public health—to expose their interdependencies and learn how to maintain them in a crisis. When the COVID-19 pandemic struck, Los Alamos scientist Jeanne Fair and fellow researchers Rene LeClaire, and Lori Dauelsberg—all of whom were key players in the critical-infrastructure study—responded quickly to restore that capability and adapt it to the current pandemic.
Models have to accept flawed data and generate the best possible prediction anyway.
As part of this process, they had to rework an earlier epidemiological model of an influenza pandemic scenario so that it would properly account for the very different scenario brought on by a coronavirus. The result, known as MEDIAN (Modeling Epidemics for Decision support with Infrastructure ANalysis), is a suite of system-dynamics models designed to identify the key drivers of the pandemic. It explores the large uncertainties pertaining to the disease itself—things like incubation period and mortality rates—together with the way society’s infrastructure systems function to make things better or worse.
For example, one often hears about the danger of simply “overwhelming the healthcare system,” but the healthcare system is a complicated animal. People are routed among home care, physicians’ offices, hospitals, intensive-care units, emergency rooms, and long-term care facilities. Medical services can include multiple types of COVID-19 testing and treatment, and the selection of services could have significant impacts on the trajectory of the pandemic. The MEDIAN team is looking at which knobs to turn to most affect the outcome, and it has been tasked in particular with understanding the uncertainties associated with testing and diagnostics to help identify an optimal testing strategy.
What should happen
COVID-19 is a killer, and Los Alamos is doing everything it can to provide life-saving scientific guidance for policymakers. The four-lab collaboration between Los Alamos, Argonne, Sandia, and Oak Ridge national laboratories has been fruitful in this regard. Just as Los Alamos is particularly well positioned to provide expansive modeling and diagnostics, partner labs have their own specialties that collectively contribute to overall situational awareness. Ben McMahon, who continues to learn everything he can to help accelerate the nation’s learning curve, is paying close attention.
“Weekly reports between partner labs reveal an ever-expanding, ever-sharpening picture,” says McMahon, “but they also deliver a healthy dose of humility. They increase what we know and refocus our attention on everything we don’t.”
Within Los Alamos’s home state, this knowledge—incomplete though it may be—is making a big difference. Throughout the crisis, Laboratory experts have been in regular contact with New Mexico state officials, hospital representatives, mental health specialists, regional economists, and other policy professionals. Typically, two or three conference calls per week allow vital information to be shared as soon as it is discovered. Additionally, state officials can get scientific evaluations from Los Alamos on the questions that arise day to day, such as whether a new cluster of cases is likely to represent a “real” problem or a statistical blip, or how best to distribute the available COVID-19 tests. Major policy announcements or changes are made only after extensive discussions with a diverse set of experts, including Los Alamos scientists from many disciplines.
“Los Alamos serves the entire nation with its resources, capabilities, and expertise, but the partnership between Los Alamos and the state of New Mexico has been extraordinarily productive for everyone involved as well,” says Kirsten McCabe of the Lab’s National Security and Defense Program Office. “We are fortunate to be able to interact with the state government and Presbyterian Healthcare Services and to have a proactive governor making informed decisions to manage the crisis. Critical information flows freely in both directions.”
Exponential change goes in both directions, too. If one infected person infects five more, then 25, then 125, the cases will skyrocket. But if one infected person infects one-tenth as many—0.5 on average, say—then exponential growth reverses and becomes exponential decay: 20 cases become ten, ten become five, and any new flare-up dwindles away. If model-informed policies can put the population firmly in the exponential-decay domain, then careful, controlled attempts to restore particular elements of normal life can be attempted relatively safely. With great vigilance to rapidly isolate and contact trace new cases as they appear, the prevailing condition of exponential decay can be relied upon to do its thing.
“The math works with us or against us,” says McMahon, “but it’s a very fine line. It all hinges on having extremely accurate models and acting on the best possible information.”
Like many of his pandemic-modeling colleagues at Los Alamos and around the world, McMahon feels frazzled. But there is no rest. Until scientists know much more about this virus, the weight of the world will continue to hang on a select few, including healthcare workers, elected leaders, and yes, modelers, who continuously reshape shifting uncertainties into the most likely truths. They are, after all, the ones specifically entrusted with advancing our learning curve. LDRD