WAIC: Widely Applicable Information Criterion
Implementation of the WAIC for model comparison.
WAIC( fittedModel, n.adapt = 1000, n.chains = 3, n.iter = 10000, n.thin = 1, summarize = TRUE )
fittedModel |
|
n.adapt |
number of adaptation samples. |
n.chains |
number of chains (no parallel computation). |
n.iter |
number of iterations after burnin. |
n.thin |
Thinning rate. |
summarize |
whether to summarize the WAIC values which are computed separately for each observed node. |
WAIC provides an approximation of predictive accuracy with respect to out-of-sample deviance.
The uncertainty of the WAIC for the given number of observed nodes
(i.e., number of free categories times the number of participants)
is quantified by the standard error of WAIC "se_waic" (cf. Vehtari et al., 2017).
In contrast, to assess whether the approximation uncertainty due to MCMC sampling
(not sample size) is sufficiently low, it is a good idea to fit each model twice and compute WAIC
again to assess the stability of the WAIC values.
For more details, see Vehtari et al. (2017) and the following discussion about the JAGS implementation (which is currently an experimental feature of JAGS 4.3.0):
If summarize=TRUE (default), a vector containing the WAIC penalty term
"p_waic", "deviance", "waic", and the corresponding
standard error "se_waic".
If summarize=FALSE, a list containing three vectors p_waic,
deviance, and waic with separate values for each observed node
(i.e., for all combinations of persons and free categories).
Vehtari, A., Gelman, A., & Gabry, J. (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27(5), 1413–1432. doi:10.1007/s11222-016-9696-4
## Not run: #### WAIC for a latent-trait MPT model: fit <- traitMPT(...) WAIC(fit) #### pairwise comparison of two models: # (1) compute WAIC per model waic1 <- WAIC(fit1, summarize = FALSE) waic2 <- WAIC(fit2, summarize = FALSE) # (2) WAIC differences (per observation!) waic_diff <- waic1$waic - waic2$waic # (3) standard error of the WAIC differences: n_obs <- length(waic_diff) c(diff = sum(waic_diff), se_diff = sqrt(n_obs) * sd(waic_diff)) ## End(Not run)
Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.