Dependence aware tests for ppc*ecdf#428
Dependence aware tests for ppc*ecdf#428florence-bockting wants to merge 170 commits intostan-dev:masterfrom
ppc*ecdf#428Conversation
…improve input validation checks
|
As mentioned in the PR introduction text, the idea would be migration in multiple stages. Informing the user in the first stage is currently implemented as follows: |
jgabry
left a comment
There was a problem hiding this comment.
Here are a few comments and questions. So far I've only looked at some of the code, so I may have more, but I thought I would give you these now so we could start discussing.
… corresponding unittests
…e-bockting/bayesplot into dependence-aware-LOO-PIT
|
@florence-bockting The changes look good, but I think there are the same issues (with the |
|
@jgabry I have refactored the code such that it removes duplicate code and documentation by creating new helper functions or template files. |
|
@florence-bockting please also officially add yourself to the DESCRIPTION file as part of this PR! |
Description
The current approach in
ppc_loo_pit_ecdfandppc_pit_ecdfassumes independence of LOO-PIT values which is not valid (Marhunenda et al., 2005). The corresponding graphical test yields an envelope that is too wide, reducing the test's ability to reveal model miscalibration.Tesso & Vehtari (2026, see preprint) propose three testing procedures that can handle any dependent uniform values and provide an updated graphical representation that uses color coding to indicate influential regions or most influential points of the ECDF. This PR implements the new development, by adding the updated approach (method = "correlated") additionally to the previous approach (method = "independent").
TODOs
ppc_loo_pit_ecdf()function inppc-loo.Rppc_pit_ecdf()andppc_pit_ecdf_grouped()function inppc-distributions.Rmethodargumentmethodargument