Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update recommendations for responding to convergence warnings #617

Merged
merged 2 commits into from
Apr 10, 2024
Merged

Update recommendations for responding to convergence warnings #617

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f5d8298
update recommendations for responding to convergence warnings
jgabry Mar 28, 2024
c720a43
Update rstanarm.Rmd
jgabry Mar 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 17 additions & 5 deletions vignettes/rstanarm.Rmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,15 +334,19 @@ the relevant issues.

### Markov chains did not converge

__Recommendation:__ run the chains for more iterations. <br>
__Recommendation:__ run the chains for more iterations (in certain cases, see qualification below). <br>

By default, all __rstanarm__ modeling functions will run four randomly
initialized Markov chains, each for 2000 iterations (including a warmup period
of 1000 iterations that is discarded). All chains must converge to the target
distribution for inferences to be valid. For most models, the default settings
are sufficient, but if you see a warning message about Markov chains not
converging, the first thing to try is increasing the number of iterations. This
can be done by specifying the `iter` argument (e.g. `iter = 3000`).
can be done by specifying the `iter` argument. However, if all parameters have
proper priors (no priors were set to `NULL`), and you used the default values
for iterations (2000) and chains (4), and Rhats (explained below) are greater
than 2, then increasing the number of iterations alone is unlikely to solve to
the problem.

One way to monitor whether a chain has converged to the equilibrium distribution
is to compare its behavior to other randomly initialized chains. This is the
Expand Down Expand Up @@ -388,6 +392,7 @@ any(summary(good_rhat)[, "Rhat"] > 1.1)
Details on the computation of Rhat and some of its limitations can be found in
[Stan Modeling Language User's Guide and Reference Manual](https://mc-stan.org/users/documentation/).


### Divergent transitions

__Recommendation:__ increase the target acceptance rate `adapt_delta`. <br>
Expand Down Expand Up @@ -418,7 +423,8 @@ the slower sampling is a minor cost.

### Maximum treedepth exceeded

__Recommendation:__ increase the maximum allowed treedepth `max_treedepth`. <br>
__Recommendation:__ increase the maximum allowed treedepth `max_treedepth` unless
all other convergence diagnostics are ok. <br>

Configuring the No-U-Turn-Sampler (the variant of HMC used by Stan) involves
putting a cap on the depth of the trees that it evaluates during each iteration.
Expand All @@ -427,11 +433,17 @@ maximum allowed tree depth is reached it indicates that NUTS is terminating
prematurely to avoid excessively long execution time. If __rstanarm__ prints a
warning about transitions exceeding the maximum treedepth you should try
increasing the `max_treedepth` parameter using the optional `control` argument.
For example, to increase `max_treedepth` to 20 (the default used __rstanarm__ is
15) you can provide the argument `control = list(max_treedepth = 20)` to any of
For example, to increase `max_treedepth` to 16 (the default used __rstanarm__ is
15) you can provide the argument `control = list(max_treedepth = 16)` to any of
the __rstanarm__ modeling functions. If you do not see a warning about hitting
the maximum treedepth (which is rare), then you do not need to worry.

With the models __rstanarm__ is capable of fitting, when you get a warning about
max treedepth you will typically also get warnings about other diagnostics. However,
if you see a max treedepth warning but all other convergence diagnostics
are fine, you can typically ignore the warning. In that case the warning
likely indicates efficiency issues but not that the results are invalid to analyze.

# Conclusion

In this vignette, we have gone through the four steps of a Bayesian analysis.
Expand Down
Loading