def(prepare cases) function not working....
Opened this issue · 19 comments
when you run the block where posteriors are being calculated for all counties it gives you this error:
TypeError: cannot do slice indexing on <class 'pandas.core.indexes.multi.MultiIndex'> with these indexers [[29]] of <class 'numpy.ndarray'>
the indexing is not correct for smoothed
Please take a look at https://github.com/sanzgiri/covid-19-dashboards/blob/master/_notebooks/Realtime_R0_USA_By_County.ipynb
That is a working version of the notebook that runs as a scheduled job every day.
The smoothing function requires a minimum threshold for positive case counts. In the original KS repo (for states), it is 25 or 10, For counties, you have to lower it a bit in order to get a sufficient range of dates.
I am not able to recreate this problem. What state are you running for?
I am running your code only same as it is that is not working because of smoothed curve it generates....
Also, i need a little help from your side can we execute this whole process on all counties regardless of how many cases they have or for a long time they are having stable cases?? is there any possiblity??
No, there are several counties for which it won't run. What state are you trying to analyze?
I am trying it for State Iowa but i need all counties.. for example Iowa state has 99 counties in total but i get result for 50 counties. what is the reason and specific logic behind that i cannot get it for all 99 counties??
Have you looked at the data for the counties that fail? How many cases do they have? Do they have lots of days with 0 cases?
In the starting days yes there are lots of 0's we can say as it is for all counties of all states...
Talking about daily cases in past 7 days i think that we are doing we are rolling 7 days backward to see if there are daily cases more than 1..
For example lets say i talk about state - Alabama and county - Clay it is having 27 cases as cumulative since 11th may 2020. the cases in this county for Alabama has not increased since 11th may 2020. so why this logic fails here?? i want to ask that?? if you can help me understand this?? why this logic fails when there is no rise in cases for past days???
Yes, you are correct. It does fail for Clay even though there are several days with non-zero cases. Not sure why. Will dig into this when I get a chance.
Iowa_Rt.xlsx
Attached is a full fledged example for State-IOWA..
The feilds marked in orange are those for which the logic fails...
Although i tried this:
for country_name, cases in countries_to_process.groupby(level='state_county'):
#clear_output(wait=True)
print(f'Processing {country_name}')
new, smoothed = prepare_cases(cases)
if (len(smoothed) < 1):
print(f"Skipping {country_name}, too few cases from smoothing algorithm")
failed_countries.append(country_name)
i have made change in len(smoothed) as less than 1 so i get the maximum counties result but still counties marked in orange fails....
i would really appreciate if you can look why this fails
When I ran this for Iowa, with the reduced filtering, prepare cases failed only for Clay and Cleburne
HDI failed for these: ['Autauga', 'Barbour', 'Bibb', 'Blount', 'Chambers', 'Clarke', 'Coosa', 'Covington', 'Crenshaw', 'Cullman', 'Dallas', 'Escambia', 'Franklin', 'Henry', 'Jackson', 'Lauderdale', 'Limestone', 'Macon', 'Marengo', 'Marion', 'Perry', 'Pickens', 'Randolph', 'Walker', 'Washington']
Sorry, that's Alabama
Even though you ran for State Alabama why are the prepare cases getting failed and why we get HDI errors. And what filtering have you reduced??
I want to understand why logic fails for several counties. Help me understsnd this please ...
Yes, I see that for Iowa, smoothing fails for about 16 counties and HDI for 29. The filters I meant were on the positive case threshold and number of rows for smoothing. With the sparse data the intervals on R_t are very wide, so the calculation is not very useful.
Any solution to this??
No, sorry, I can't help you with this.
Ok. No problem. Can you just let me know the whole logic on which we are doing the smoothing.. as in what does this functio def prepare cases do??
As in i want to understand the whole logic of this code. If you can let me know that would do the needful.. please...