stan-dev/loo

Add R2

sims1253 opened this issue · 6 comments

sims1253 commented

It would be nice if the loo objects weren't hardcoded to use elpd. For a study I wanted to include rmse and R2 and had to build fake loo objects in order to use the loo_compare functionality. Code is here.

avehtari commented

I had missed this issue
Thanks for the code!
PRs #202 and #203 are adding more utilities/losses, but we still need to add options for loo_compare to know which utility/loss to use
brms part of the code needs to go to brms package

avehtari commented

@sims1253 do you need something else than what is in the PRs mentioned above, or can we close this issue?

avehtari commented

We do have rmse, but we're still missing R2 (which is available in brms and rstanarm, but not as plain loo package version)

sims1253 commented

...which is available in brms

At least the brms version does not use psis, which is why I wrote the version from my pr.

sims1253 commented

True. Sorry, I'm currently on vacation and only have my phone and only found the Bayes r2 function.
Maybe it was about the pointwise estimate for the SE then that we built the new version. I don't quite remember to be honest.