Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rt estimates early in outbreaks are low #42

Open
batson opened this issue Apr 23, 2020 · 8 comments
Open

Rt estimates early in outbreaks are low #42

batson opened this issue Apr 23, 2020 · 8 comments

Comments

@batson
Copy link

batson commented Apr 23, 2020

Most estimates of R0 for COVID19 are north of 2. (eg Los Alamos paper on Wuhan.)

I would expect most the Rt estimates for most states to begin around R0 (before any responses were made), and then decrease as societal and personal behavior changed.

In epiforecasts.io, this is the case: the 50% credible interval is above 2 for the first week in each of the states they highlight, and the median prediction on March 9 in New Jersey, Illinois, Pennsylvania, is around 2.

In the current rt.live model, all point estimates for Rt are below 2, including on March 9.

To diagnose this, it may be worth simulating some data for which Rt begins at 2.5 and then decreases steadily over a month to 1 or below. There are a myriad of potential causes, including the value of sigma used in the brownian motion and edge effects. Besides, a simulation or two will make a good sanity check for the model in general!

Very cool to see all the progress in the last week, btw.

@k-sys
Copy link
Owner

k-sys commented Apr 24, 2020

Looking into this, thanks for the note. I'm in contact with the epiforecasts folks. I can't tell if it's a prior issue or something else. Generally we converge on the most recent values though. Btw I am assuming this is for the mcmc version?

@batson
Copy link
Author

batson commented Apr 24, 2020

Awesome, glad to hear you and the epiforecasts folks are working together 🤝!

This was for the MCMC version posted yesterday. (@mnielsen actually flagged the issue on twitter and I confirmed.)

@nicholasjma
Copy link

nicholasjma commented May 3, 2020

The prior for the serial interval is set too low, may want to try something like alpha=12, beta=2 (since the mean is alpha/beta) for that gamma distribution. May also want to adjust the prior for theta to be a bit higher. I tried making these adjustments, and it actually does make a difference in the final values. If you start with a higher prior, the final estimates are 0.1 to 0.2 higher.

@gkossakowski
Copy link

I have ran into the same issue with the MCMC model adapted to European data. R_t's are underestimated early into the outbreak for all countries. I've reproduced the problem on synthetic data so I suspect it's a bug.

@bastienboutonnet
Copy link

@gkossakowski same here I adapted to European data and things are looking a little low.

Do you mind sharing how you generated your toy data? And when you tested on generated data Rt was underestimated also?

@SaulML
Copy link

SaulML commented May 28, 2020

Looking into this, thanks for the note. I'm in contact with the epiforecasts folks. I can't tell if it's a prior issue or something else. Generally we converge on the most recent values though. Btw I am assuming this is for the mcmc version?

The model under estimates R0 due to not including asymptomatic and undetected cases into the model. I created graphs and estimation figures for NSW, Australia.

@SaulML
Copy link

SaulML commented May 28, 2020

I have ran into the same issue with the MCMC model adapted to European data. R_t's are underestimated early into the outbreak for all countries. I've reproduced the problem on synthetic data so I suspect it's a bug.

There is certainly under estimation of R0 of the current model in early stages of the outbreak which is quite concerning. My model resolves this issue my modeling undetected cases as well. More details are in https://github.com/https-seyhan/COVID-19/wiki/Real-time-COVID-19-Infection-Prediction-of-Australian-NSW-population-using-Bayesian-Approach-and-Improved-Poisson-Likelihood-Function

The model, I introduced, improves the Bayesian approach significantly. The model solves the omitted variable bias by modeling uncertainty of undetected and asymptomatic cases as well.

Best,

Seyhan

@gkossakowski
Copy link

@bastienboutonnet the MCMC model trained on European data: https://github.com/gkossakowski/covid-19/blob/master/Realtime%20Rt%20mcmc.ipynb

The generated data was coming from a simple SIR model with R predefined (and fixed for periods of time to simplify analysis). I expected that the model would recover R from case numbers and we found that R was underestimated. Upon further investigation, we realized that case -> onset tranformation was the culprit. Surprisingly, what's happening in the far tail end of pdelay distribution (as defined in the notebook) has outsized impact on R estimates.

Have you been digging more into this yourself?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants