Missing States in USA

[TRM13]

I downloaded the CSV file and was going over it and noticed a bunch of states in the USA missing:

Alabama,Alaska,Arizona,Arkansas,California,Connecticut,Delaware,Florida,Georgia,Hawaii,Idaho,Indiana,Iowa,Kansas,Kentucky,Maine,Minnesota,Mississippi,Missouri,Montana,Nebraska,Nevada,New_Hampshire,New_Mexico,North_Carolina,North_Dakota,Ohio,Oklahoma,Oregon,Pennsylvania,Rhode_Island,South_Carolina,South_Dakota,Tennessee,Texas,Utah,Vermont,Virginia,West_Virginia,Wisconsin,Wyoming

If it is "total deaths" you want and a baseline you could try the "Pneumonia and Influenza" page which has downloads. I've done that and parsed it out using a 4 year average instead of 5 like you but similar idea.

For 2020 the USA has an "excess death" rate about 5.5% (50,331) higher than the previous 4 year average for weeks 1 to 16. As a comparison I checked the first 16 weeks of 2018 compared to the previous 4 year average and it was 7.2% (63,260).

The script and all related files are here if you want to kick the tires:
https://www.dropbox.com/sh/fh9x5fngmfbeiiu/AAAH-OtOMqiY_R9qqG6YccCRa?dl=0

[MichaelChirico]

Also came here looking for North Carolina -- the NCHS dashboard has "insufficient data" for the NC map but (1) also Ohio shows this & Ohio is included in the repo data and (2) a data download from that page seems fine...

[TRM13]

North Carolina finally got past the 20% limit to be included. I'm sure there will be more updates to follow. I also did a first grab of weeks 1-21 and it is in at 6.8% but 8 weeks are missing. West Virginia (1), Connecticut(3) and of course North Carolina(4). LOL.

I get it from the CDC site: https://gis.cdc.gov/grasp/fluview/mortality.html

It says "Pnuemonia and Influenza" but it does have a "Total Deaths" which is what I was looking for. I gave up on trying to get stuff off the various state web sites. All are laid out differently, with different years available. It was a total mess. Hard to make good decisions with bad data.

[LEMOINE-1955]

Good evening, sir, In fact, I was able to cross-check your estimate of mortality in the United States with national (not state) data from your csv file as the attached file attests. Like you and a few units aside, I estimate the total excess mortality between week 10 (when the first Covid-19 deaths were recorded there) and week 17 last known result to be 70515 deaths, a figure comparable to the one you published in the summary table (71700 and +26%). Perhaps it is the result of chance, but as of April 25, less than half of the deaths known to date had been identified by the health authorities, and the study must be completed in order to have an exhaustive view of the total. For many other countries, at least those that have long provided reliable mortality data, which are in the terminal phase of the pandemic, we have a fairly accurate picture of the exact excess mortality and the gap between the data displayed on the lethality due to Covid-19 by each country and the raw reality/there are some surprises. Yours sincerely, Le mar. 2 juin 2020 à 15:59, TRM13 <notifications@github.com> a écrit :

…

I downloaded the CSV file and was going over it and noticed a bunch of states in the USA missing: Alabama,Alaska,Arizona,Arkansas,California,Connecticut,Delaware,Florida,Georgia,Hawaii,Idaho,Indiana,Iowa,Kansas,Kentucky,Maine,Minnesota,Mississippi,Missouri,Montana,Nebraska,Nevada,New_Hampshire,New_Mexico,North_Carolina,North_Dakota,Ohio,Oklahoma,Oregon,Pennsylvania,Rhode_Island,South_Carolina,South_Dakota,Tennessee,Texas,Utah,Vermont,Virginia,West_Virginia,Wisconsin,Wyoming If it is "total deaths" you want and a baseline you could try the "Pneumonia and Influenza" page which has downloads. I've done that and parsed it out using a 4 year average instead of 5 like you but similar idea. For 2020 the USA has an "excess death" rate about 5.5% (50,331) higher than the previous 4 year average for weeks 1 to 16. As a comparison I checked the first 16 weeks of 2018 compared to the previous 4 year average and it was 7.2% (63,260). The script and all related files are here if you want to kick the tires: https://www.dropbox.com/sh/fh9x5fngmfbeiiu/AAAH-OtOMqiY_R9qqG6YccCRa?dl=0 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APZVM5TANQT2OYL4CB5SMNTRUUAT7ANCNFSM4NQYDEGA> .

[TRM13]

Your 71,700 up to week 17 sounds about right. Up to week 21 I have it at 79,816 so close and with 8 more weeks of data still to be posted it will undoubtedly go up.

It would be interesting to see what excess deaths there are in other countries. I have enough trouble just finding things on the CDC site and it's in English (my native language). LOL.

Thanks for the feedback.

[LEMOINE-1955]

Hi, In fact, I am trying to keep statistics on all countries that have reliable, long-term mortality statistics that can be used to provide a more comprehensive and accurate picture of the Covid-19 epidemic. It is known that the mortality data posted by the health authorities of all the countries of the world and presented every night contain a certain bias that makes it difficult to compare them. However, it is clear that debates of a political nature, at the national or international level, are being organized around the issue on erroneous bases. Through the study of excess mortality, I believe, like your editorial staff, that it is important to achieve a more accurate vision of the pandemic. My method consists of: - calculate the mortality of each country from the week the first Covid deaths were reported to the last week of data provided by each country of appropriate data (see below) - compare the 2020 mortality to the average of historical data (most often calculated over the years 2015-2019), - determine the excess mortality and compare it with each country's weekend announcement of known data. In this way, it is possible to determine, by difference, a kind of "grey" excess mortality on which investigation and questioning of public authorities could be undertaken. I consider, in all subjectivity and subject to contradictory data presented by the health authorities, that it is reasonable to believe that 80% of this excess mortality is attributable to Covid-19 and I have reviewed the figures for certain countries (number of deaths, number of deaths per million inhabitants) accordingly. A final methodological clarification: it is important to end the "grey" excess mortality calculation sequence at the strict peak period, because I have observed a phenomenon of under-mortality (compared to historical data) after the wave which may artificially minimise the phenomenon. This is the case in France where I have daily mortality data which shows that since the beginning of May 2020 we would die less than in previous years. *The origin and reliability of the data * Like you, I was quickly discouraged from looking for data on the sites dedicated to the issue by each of the countries under consideration. I stopped at letter A, like Austria, where my knowledge of the German language, which is a school language and therefore ancient, was a formidable obstacle despite all the efforts of Google Translate. I have therefore relied on the cvs mega file prepared by your editorial staff, which I would be most grateful if you would send me regular updates. I am also using mortality data from the Human Mortality Database ( https://www.mortality.org), a cooperative database established by the University of Berkeley, the Max Planck Institute and the United Nations. Of course, I cross-checked the data from these two databases, which in the vast majority of cases are identical or show very small differences. On the other hand, we must be vigilant (or patient) about the recently filed data: the last two weeks of 2020 published in HMD on the United States and France are clearly wrong. *My conclusions (all provisional)* As far as I am concerned, I identify three groups of countries: - *Countries where excess mortality is non-existent or almost non-existent. * Very largely, this confirms the tables that you have recently published, which essentially consist of a Scandinavian block: Denmark, Finland, Norway and Iceland. - *Countries with excess mortality fairly consistent with their public announcements of deaths attributed to Covi* Belgium (estimated "grey" excess mortality of 625), Switzerland (0), Denmark (0), Germany (0), France (1200), Sweden (620), - *Countries with a significant difference between the "official" figures and the excess mortality observed* The United Kingdom, whose situation has been detailed in lengthy articles in the Financial Times, has, according to my assessment, an excess of 24,000 grey deaths, which should lead to a total death toll attributable to Covid-19 of 62,000. Italy, with an estimated grey over-mortality of 8200 deaths, is expected to reach a total death toll of over 40,000. Finally, Spain, whose confusion over the figures makes estimates more difficult, presents, subject to inventory, an excess grey mortality of between 17900 and 19200 deaths according to the data, which leads to an overall balance sheet above the 40000 deaths, quite far from the official figure which has been invariably constant for the last ten days or so. Austria, with a low mortality rate, nevertheless has a total excess mortality equal to twice the official Covid balance sheet. The Netherlands also has a grey excess mortality of 3800 deaths, which, if taken into account, would place them closer to their Belgian neighbours than to their Scandinavian neighbours, except for Sweden. *The specific case of the United States* I confirm your trend of 79,816 deaths in week 21, since my last calculation at the end of week 18 puts the death toll at 73,458, but it is still too early to make a definitive assessment. Moreover, what I have called the grey excess mortality, which I estimate at this date at 5800 deaths is very low in view of the size of the country, which should enable them to join the group of countries whose excess mortality is close to the Covid balance sheet. Anyway, I am at your disposal to send you all the data on which I base these opinions and to take a critical look at the method. Best regards N.B. I am waiting to have more complete data from Brazil and to verify the data from Portugal, in order to deliver conclusions (never definitive)on these two countries. Le jeu. 11 juin 2020 à 20:06, TRM13 <notifications@github.com> a écrit :

…

Your 71,700 up to week 17 sounds about right. Up to week 21 I have it at 79,816 so close and with 8 more weeks of data still to be posted it will undoubtedly go up. It would be interesting to see what excess deaths there are in other countries. I have enough trouble just finding things on the CDC site and it's in English (my native language). LOL. Thanks for the feedback. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APZVM5R4JYPP3K6VJ2QMH5DRWEMLBANCNFSM4NQYDEGA> .

[TRM13]

Good work. I hope the Financial Times takes you up on your offer. You could put your data & scripts on Dropbox or here on Github when you are ready.

The one change I would make to your method is in regards to this:
"calculate the mortality of each country from the week the first Covid deaths were reported"

That would miss early cases where it could be confused with influenza. In the case of the USA I was probably in circulation in Jan/Feb but the CDC discouraged testing during that period. The athletes returning from the World Military Games most likely brought it back in early November.

I start mine at Jan 1st because that is where the CDC starts their yearly mortality statistics from. It will make comparisons to previous years better. Unlike the pneumonia and influenza deaths that they list from week 40 to week 39 the following year.

Tracking month by month is an interesting idea. That could be a way to track any "second wave" effects (although due to lockdowns it is really just a delayed first wave).

[LEMOINE-1955]

What is interesting about this case is that in sharing the concern to avoid disruptive events, I come to a different conclusion. Let me explain: In Europe, the seasonal flu episode in 2020 was characterised by its earliness and lack of virulence. This has resulted in one of the most significant under-mortalities in recent years, as shown on the Momo website https://www.euromomo.eu/graphs-and-maps/, which concatenates mortality data from 24 countries. This under-mortality seems to have lasted until week 10 or 11, precisely when the lethality of the Covid begins to occur. In my mind, taking into account the data from the first 8 or 9 weeks of the year would have led to mechanically and artificially minimizing Covid-19's sub-mortality. Certainly, the virus was already circulating, but it does not seem to me that its lethal effects manifested themselves during this period. Moreover, relying on averages established over several years makes it possible to smooth out the effects specific to each of them (in particular flu or gastroenteritis more or less late or more or less virulent, depending on the year). In this respect, I expected to see in the figures the effects of the last strong flu in 2017, which caused 13400 deaths in France (45% of deaths 19 on this subject, and with general indifference) and well not all the year 2017 is the year in which mortality is the lowest of the years taken into account to establish the historical average, a phenomenon that seems to be repeated in all the European countries studied. Finally, I would like to strictly align the study period with the period of the pandemic peak and to stop it as soon as the curves return to "normal" mortality or even to under-mortality (which is spectacularly the case in France, starting from week 18). See you soon. I will follow up on your proposal. Have a good weekend. Gilles LEMOINE Le ven. 12 juin 2020 à 15:13, TRM13 <notifications@github.com> a écrit :

…

Good work. I hope the Financial Times takes you up on your offer. You could put your data & scripts on Dropbox or here on Github when you are ready. The one change I would make to your method is in regards to this: "calculate the mortality of each country from the week the first Covid deaths were reported" That would miss early cases where it could be confused with influenza. In the case of the USA I was probably in circulation in Jan/Feb but the CDC discouraged testing during that period. The athletes returning from the World Military Games most likely brought it back in early November. I start mine at Jan 1st because that is where the CDC starts their yearly mortality statistics from. It will make comparisons to previous years better. Unlike the pneumonia and influenza deaths that they list from week 40 to week 39 the following year. Tracking month by month is an interesting idea. That could be a way to track any "second wave" effects (although due to lockdowns it is really just a delayed first wave). — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APZVM5XZ6TQSFPQX4HQPFP3RWISZFANCNFSM4NQYDEGA> .

[TRM13]

Now you got me going down the rabbit hole again. Love it. Thanks.

After you mentioned it, I think the month by month statistics could be very handy so I did them up. It will also be very useful for seeing if the re-opened states are having a second wave or not.

In short January and February were pretty much in line with the previous 4 year average. March surprised me and the huge hit appears to be in April. Looking forward to weeks 19-22 (May).

Using the same data I downloaded on 2020-06-05 I ran it for each month. The CDC tracks by week not month so it is a bit of a kludge but these are the weeks that approximately correlate to the months:
Weeks 1 to 5 ~= January (Entire USA was -0.8%)
Weeks 6 to 9 ~= February (Entire USA was +0.3%)
Weeks 10 to 13 ~= March (Entire USA was +3.9%)
Weeks 14 to 18 ~= April (Entire USA was +28.7%)

After that the data is still too incomplete to be of use and even week 18 is missing too much for my liking.

Interestingly weeks 10 to 13 (March) show very little except for New York City at +49.7% and only a few others breaking double digits (Montana=10.1 ; South Carolina=11.7 ; Louisiana=11.7 ; New Jersy=12). Also noteable were the states well below the 4 year average (Connecticut= -12.6 ; Pennsylvania= -19.2%). Pennsylvania was -14, -29 and -19 for the first 3 months (I got to look into that).

Weeks 14 to 18 (April) is where the biggest hit took place.
New York City = +415.9%
New Jersey = +169%
New York State = +87.7%
Massachusetts = +85.5

I'll update my dropbox with the monthly script & data once I check it again and clean it up this weekend. In the mean time I've attached the reports spreadsheet for the monthly stats.
reports.xlsx

https://www.dropbox.com/sh/fh9x5fngmfbeiiu/AAAH-OtOMqiY_R9qqG6YccCRa?dl=0

[LEMOINE-1955]

Hi First of all, I learned a new expression: "Going down the rabbit hole". It's very uplifting. So, before I even read your interesting new e-mail, I had started digging my own rabbit hole even deeper... In this case, in line with a study of the parallel evolution of Covid-19 and the mortality in specific time slices that I could describe as follows; before, during and after the epidemic wave, there are very significant phenomena. I have chosen three European countries (Austria, Italy, and France) with different epidemic profiles, but which show significant trends: - the beginning of a year of more or less significant under-mortality compared to the historical trend, - a rather important gap, during the hard core of the epidemic crisis, between the deaths attributed to Covid-19 by the health authorities of each country and Excess deaths: a gap of 40% in Italy and which even rises to 56% in a country as little affected as Austria? - a return to a sub-mortality after the passage of the wave (which I did not really characterize in Italy for lack of sufficient statistical data). This mini-study (see attached file) leads me to confirm my idea on the need to focus strictly on the acute period of the crisis, avoiding the integration of mortality data, before and after the emergence of the lethality of the virus, which waters down the phenomenon. A very quick reading of the data could lead one to think, by positioning the cursor at the end of May, that the excess mortality in Austria or France is much lower than it actually was. The only question is that the approximately 700 Austrians, 14800 Italians and 1900 French who died during this period beyond the usual standards without being classified as Covid-19 deaths are not resurrected in the following weeks, which a quick reading of the data might suggest. Or the Catholic and Roman tradition of the respective history of these countries. Lol. Best regards, Gilles LEMOINE N.B. My next playgrounds are Spain and the Netherlands; I'm waiting for the crisis to stabilize definitively in the United Kingdom and the United States (even if it's likely that you'll have beaten me to it). Le ven. 12 juin 2020 à 23:55, TRM13 <notifications@github.com> a écrit :

…

Now you got me going down the rabbit hole again. Love it. Thanks. After you mentioned it, I think the month by month statistics could be very handy so I did them up. It will also be very useful for seeing if the re-opened states are having a second wave or not. In short January and February were pretty much in line with the previous 4 year average. March surprised me and the huge hit appears to be in April. Looking forward to weeks 19-22 (May). Using the same data I downloaded on 2020-06-05 I ran it for each month. The CDC tracks by week not month so it is a bit of a kludge but these are the weeks that approximately correlate to the months: Weeks 1 to 5 ~= January (Entire USA was -0.8%) Weeks 6 to 9 ~= February (Entire USA was +0.3%) Weeks 10 to 13 ~= March (Entire USA was +3.9%) Weeks 14 to 18 ~= April (Entire USA was +28.7%) After that the data is still too incomplete to be of use and even week 18 is missing too much for my liking. Interestingly weeks 10 to 13 (March) show very little except for New York City at +49.7% and only a few others breaking double digits (Montana=10.1 ; South Carolina=11.7 ; Louisiana=11.7 ; New Jersy=12). Also noteable were the states well below the 4 year average (Connecticut= -12.6 ; Pennsylvania= -19.2%). Pennsylvania was -14, -29 and -19 for the first 3 months (I got to look into that). Weeks 14 to 18 (April) is where the biggest hit took place. New York City = +415.9% New Jersey = +169% New York State = +87.7% Massachusetts = +85.5 I'll update my dropbox with the monthly script & data once I check it again and clean it up this weekend. In the mean time I've attached the reports spreadsheet for the monthly stats. reports.xlsx <https://github.com/Financial-Times/coronavirus-excess-mortality-data/files/4773409/reports.xlsx> https://www.dropbox.com/sh/fh9x5fngmfbeiiu/AAAH-OtOMqiY_R9qqG6YccCRa?dl=0 — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APZVM5VKUR54PEE32FKRYIDRWKP3PANCNFSM4NQYDEGA> .

[TRM13]

It was a reference to "Alice's Adventures in Wonderland" by Lewis Carroll. He was a brilliant mathematician as well as a writer. A phrase used when weird stuff starts happening. Seemed appropriate.

I like the time slice idea and the finest granularity we can get is weekly so I'll run those. As of this morning only North Carolina is missing week 18. To get to the end of May we'll need 22 weeks and a lot is still missing from those.

Thanks and TTYL

[LEMOINE-1955]

I knew of course "Alice in Wonderland Adventures" and Lewis Carroll but not his background as a mathematician. I do think that a time slice analysis provides a granularity that can give us a more accurate description of events and I'm glad to see that this seems to have convinced you. It seems to me that the method is all the more suited to a state-continent such as the United States, where it seems that the epidemic is setting in with a much less homogeneous rhythm than in European countries. Clearly the North-Eastern states (New York and New Jersey) are already out of the acute phase of the epidemic and it should be possible to have a panoramic view of what happened there before, during and after. On the other hand, it seems that in the Midwest, the South (Louisiana except perhaps) and the West, the relative maturity of the epidemic will require a little time to get a more accurate picture of the situation. The method has the merit, in my opinion, of being able to look at exactly what is happening per state (or group of states) and to adapt the examination of mortality to more homogeneous territories and to the precise moment when the epidemic is producing its effects, rather than drowning the whole thing in the great national cauldron. And also, to question some of the States, particularly in the south, which seem, from what we hear, to be taking some liberties with the sincere and exhaustive feedback of information. Moreover, since the epidemic was earlier in Europe, I had all the data I needed to examine what happened just after the end of the acute phase of the disease: it is clear, particularly in Austria and France (I reserve my prognosis for Italy) that we are entering a phase of under-mortality which somehow smoothes out the effects of the crisis, at least in the statistics. In this respect, I have a theory, which is not necessarily followed by eminent epidemiologists, which is that in the case of diseases affecting mainly elderly and frail people who are often affected by co-morbidities, the disease acts as an accelerator to an inevitable end. This phenomenon is also reflected in the statistics I have sent you in 2017, the year of a very virulent influenza in Europe, where you see that the first two months of the year show the highest mortality rate in the historical series (2015-2019) and often the lowest in the months that follow. Best regards, Gilles Le lun. 15 juin 2020 à 16:04, TRM13 <notifications@github.com> a écrit :

…

It was a reference to "Alice's Adventures in Wonderland" by Lewis Carroll. He was a brilliant mathematician as well as a writer. A phrase used when weird stuff starts happening. Seemed appropriate. I like the time slice idea and the finest granularity we can get is weekly so I'll run those. As of this morning only North Carolina is missing week 18. To get to the end of May we'll need 22 weeks and a lot is still missing from those. Thanks and TTYL — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APZVM5RXZ6BAIG4OFV3I6ZTRWYS5HANCNFSM4NQYDEGA> .

[TRM13]

I'm going to do up a "Weekly" to go along with the "Year To Date" and "Monthly". That is the finest granularity I can get with the CDC data and it should be fine. I don't think we'll see much difference from the monthly but it will be interesting to see how the supposed "second wave" works out. In reality it is the first wave just delayed by shutdowns.

Looking at things across different time spans is useful. For the overall effect the full year statistics are good and to see when things happened and how fast the monthly/weekly are good.

As luck would have it I'm quite busy for the next 7-10 days but given that the data isn't complete past week 17 it shouldn't really matter. Once a month updates should be fine.

TTYL

[LEMOINE-1955]

Hi, Okay, and when do you plan to do an article on the conclusions that can be drawn from these statistics? Another question: Has anyone at F.T. had the idea of carrying out an international study on Covid-19 victims among health care workers (infected or even dead people by category; doctors/nurses and places: hospital, nursing or care homes, in town, etc.)? Regards, GL Le sam. 20 juin 2020 à 14:45, TRM13 <notifications@github.com> a écrit :

…

I'm going to do up a "Weekly" to go along with the "Year To Date" and "Monthly". That is the finest granularity I can get with the CDC data and it should be fine. I don't think we'll see much difference from the monthly but it will be interesting to see how the supposed "second wave" works out. In reality it is the first wave just delayed by shutdowns. Looking at things across different time spans is useful. For the overall effect the full year statistics are good and to see when things happened and how fast the monthly/weekly are good. TTYL — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APZVM5QZ6JVL72TDOY4F3MLRXSVOZANCNFSM4NQYDEGA> .

[TRM13]

I don't work for FT or anyone else. I just reported an error on selection in their report in this thread and we sort of hijacked it. LOL.

Specific causes of death won't be available for 2-3 years from the CDC unfortunately. It would be interesting to see how the health care workers are fairing.