Figuring Out Excess COVID Deaths Is Tarot Card Reading with Math
Two weeks ago I and others had figured out that we likely had a COVID death attribution problem. Meaning that based on CDC data we were attributing too many deaths to COVID. A commenter brought up a New York Times article that made the claim that 377,000 people had actually died of the novel coronavirus. So how did they figure this out? Is this more or less likely to be accurate. Let’s dig in.
A Video Primer
I created the video above so that you can follow along. It’s got everything in the blog text, but more commentary from me to help make sure you understand the concepts.Request a Regenexx Appointment
What Are Excess Deaths?
The idea behind excess COVID deaths is both simple and fraught with problems. You take the actual number of deaths from any cause and you subtract out the number of people who should have died (see above). What’s left are the excess deaths due to COVID-19. The problem? Figuring out the number of people who should have died is a guess.
The New York Times Article
This is what the NYT reported earlier this month (1):
“Deaths nationwide were 19 percent higher than normal from March 15 to Dec. 5. Altogether, the analysis shows that 377,000 more people than normal have died in the United States during that period, a number that may be an undercount since recent death statistics are still being updated.”
Let’s look at why this analysis is at best highly inaccurate and at worst just tarot card reading with math.
Our First Challenge: What Is Normal?
How many people are supposed to die in any given week? From my diagram above, you can see that this is a key concept in excess mortality as, without that guess, we can’t figure the number of COVID deaths. So let’s examine how the New York Times or Harvard or anyone would figure that out.
Here I downloaded the 2014-2018 and 2018-2020 All-Cause Mortality tables from the CDC website and graphed those in excel (2,3):
The yellow line represents the number of deaths by week, with weeks being on the x-axis below. The y-axis is the number of deaths in ten thousand increments. I’ve labeled all of the winters, where you see a peak in deaths of various magnitudes. That makes sense, as all-cause deaths peak around January every year. Also note that when that happens, the U.S. loses about 55-70,000 people even without COVID-19. The lull in deaths between the peaks represents the summers, where fewer people die. The spike in the yellow line off to the right represents our Winter 2019 1st wave peak (that actually occurred in spring 2020). The two lines on the bottom of the graph (red and blue) that peak off to the right represent the number of COVID deaths in two different categories.
One of the first issues we see is the HUGE variability in how many people are supposed to die each week if COVID never existed. In fact in the summer lull of 2015, we get down to 46,000 people dying, while in the winter high of 2017, that number is 69,000:
Hence, if we were going to chose how many people should have died in any given week it would probably be as low as 46,000 to as high as 69,000. If we predict low, we can make more COVID deaths appear and if we chose high, fewer deaths will be considered due to COVID.
Using Past Trends to Make a Future Prediction
You’ve probably seen this disclaimer in your 401K plan:
“Please remember that past performance may not be indicative of future results.”
Given that the above yellow line looks like the stock market, you might guess that trying to predict where that yellow squiggly line will go in the future is hard. In fact, it’s impossible to do accurately. For example, the winter of 2017 was bad. However, far fewer peak deaths occurred just a year later in 2018.
So let’s try to predict where the yellow line will go by using two different scenarios. The first will be using the 2017 peak and superimposing it onto the winters of 2019 and 2020. This makes sense, as we are in a bad respiratory virus year, so we should model based on a prior bad year. To keep this model balanced, we’ll also take the best summer lull in deaths in the summer of 2017. I’ll superimpose that best-case onto the summer of 2020. Hence, our first choice of the winter of 2017 will have the effect of decreasing COVID deaths while our best-case summer scenario will have the effect of increasing COVID deaths:
Above you see the two 2017 winter peaks in red placed into the winters of 2019 and 2020. The blue area is the number of deaths that didn’t happen in winter of 2019 (the difference between the yellow line and the red 2019 peak) that needs to be subtracted out of the COVID death count (see video for a more detailed explanation). The yellow color is the excess death count in this model, which is under 200,000. Not surprisingly, this is very similar to this numbers on the CDC website:
That’s between the COVID and non-COVID pneumonia deaths. That makes sense, as the main cause of death due to COVID-19 is pneumonia. The number to the left at 271K, which is the one reported on the news is with our without COVID-19. The number to the right at 128K is the number of pneumonia deaths with a confirmed COVID diagnosis.
In this model, we will use past death patterns to make the COVID deaths increase. In this case, we’ll take the low number of peak deaths that happened in the winter of 2018 and move them forward to the winters of 2019 and 2020 and keep the low death counts of the summer of 2018. Below you can see we have more yellow color now (COVID excess deaths) and only a tiny bit of blue to subtract out. So by choosing these past trends we have increased COVID deaths. We could make the COVID numbers go even higher by using previous past trends.
We Can Increase or Decrease COVID Deaths at Will by Making Different Guesses About the Future
Think about how many COVID deaths would result if I used the death rate from the summer of 2015 when the lowest point was 46,000! So the claim by the New York Times that COVID caused 377K deaths as of early December is at best a wild guess and at worst, just playing with these numbers to find the scenario that will make the best headline to attract the most eyeballs.
I could also decrease the COVID death toll below 200K quite easily by using the previous worst estimates for the summer and winter. Meaning, I would take a prior summer and winter with the worse deaths and superimpose those onto 2019/2020 as our baselines.
In fact, at best, this exercise is as accurate as trying to predict the stock market, which is usually no better than chance. So it doesn’t matter how much math you apply, accurately predicting the future is not possible. Hence, calculating a COVID excess death count will always be somebody’s best guesstimate.
Epidemiologists and Predicting the Future?
You only need to look at the Imperial College of London disaster whose coronavirus model predicted millions of first wave COVID-19 deaths in the US to see that epidemiologists are awful at predicting the future. In fact, if epidemiologists were good at predicting the future, there would be no poor ones, as they would all be able to predict the movements of the stock market and get filthy rich in months. Whole university departments would be as rich as Crassus (a rich Roman general who was executed by the Persians by having molten gold poured down this throat). Since that doesn’t happen, making guesses about the future with math and a Ph.D. still amounts to glorified guesswork.
The upshot? I put no more credence in the New York Times estimate of excess COVID deaths than I do the result of the local Tarot Card reader. I hope that between the video and my blog above, you too can see why you should ignore these guesstimates concerning excess COVID deaths.
(1) The New York Times. True Pandemic Toll in the U.S. Reaches 377,000. https://www.nytimes.com/interactive/2020/us/covid-death-toll-us.html. Accessed 1/1/21.
(2) Centers for Disease Control and Prevention. Weekly Counts of Deaths by State and Select Causes, 2019-2020. https://data.cdc.gov/NCHS/Weekly-Counts-of-Deaths-by-State-and-Select-Causes/muzy-jte6 Accessed 12/31/20.
(3) Centers for Disease Control and Prevention. Weekly Counts of Deaths by State and Select Causes, 2014-2018. https://data.cdc.gov/widgets/3yf8-kanr Accessed 12/31/20.