Broken Code Issues

[tanaytrivedi]

Hi all,
Don't mean to be a pain, as I know there have been data issues recently that broke a few components here. I have been keeping up with the recent commits and issues, and it seems two things are breaking the recent code:

1. format_ecdc_cases from NCoVUtils outputs a dataframe with columns:
   'date'
   'region'
   'import_status'
   'cases'
   but EpiNow::regional_rt_pipeline requires a dataframe with columns:
   'date'
   'region'
   'import_status'
   'confirm'.
   This immediately broke line 70 in EpiNow::regional_rt_pipeline. A simple fix on my side here pushed this issue aside.
2. The recent switch to lognorm instead of gamma in EpiNow::dist.R doesn't seem to work as yet, when reaching line 75 in EpiNow::rt_pipeline I get a failure in the usage of "rep":
   
   Don't know how to begin debugging this. I believe the "samples" variable may be missing some casting to make it an integer, and therefore isn't running correctly but I could be wrong.

Thanks for your help, please let me know if you need any more information to help with these issues.

[rhelpacc]

Hi

We have exactly the same issue.
@Author/maintainer - I am wondering if you have a plan to rectify the issue? Thank you.

[seabbs]

Hi @tanaytrivedi,

Thanks for flagging this. We made some changes to the approach recently that have yet to be pushed to the website and so aren't immediately obvious.

The error you are seeing is likely caused by not pre-specifying the delay distribution which is now external to the pipeline (as you do not always want to refit it) and is used to control the number of samples.

An example of the new interface (the data being fit for the reporting delay here is very much an example only and should not be used if wanting useful estimates) is here: https://github.com/epiforecasts/EpiNow/blob/master/R/regional_rt_pipeline.R

Let me know if that doesn't help (and feel free to reopen). The code and estimates @epiforecasts/covid will be updated shortly once we have nailed down a memory leak we are seeing.

Sam

P.S a working version of the global script is here (where we are exploring the memory leak we are seeing so data is restricted to the first 2000 rows): https://gist.github.com/seabbs/5a9a22950357d569c482535a12645c2c

[tanaytrivedi]

Hi Sean, thanks for the recommendations. I am running the working version of the global script, runs fine on my infrastructure no memory issues.

I have found an interesting trend that I want to confirm with you. Here are two runs on the United States, with the code being as it was in the GitHub on both dates:

This is the average of the upper lower band. I found the upturn at the end a tad worrying, so I wanted to check across the cross-section of countries on both days.

On 5/3, 45/95 countries had a week-on-week increasing Rt at the end of timeseries. On 5/11, 67/95 countries had increasing Rt. Given the current news cycle and attitude about dropping cases, we just want to know whether this is something you're seeing yourself as well.

I know that some adjustments (negative binomial upscaling) in the methodologies elevate the estimates at the end of the time series on purpose, but it seemed out of place that it elevated so much on countries where the current narrative is a slowing epidemic.