Paired t-Tests for Model Comparisons
Paired t-test comparisons of resampled performance metrics from different models.
## S3 method for class 'PerformanceDiff' t.test(x, adjust = "holm", ...)
x |
performance difference result. |
adjust |
p-value adjustment for multiple statistical comparisons as
implemented by |
... |
arguments passed to other methods. |
PerformanceDiffTest class object that inherits from
array. p-values and mean differences are contained in the lower and
upper triangular portions, respectively, of the first two dimensions. Model
pairs are contined in the third dimension.
## Requires prior installation of suggested package gbm to run ## Numeric response example fo <- sale_amount ~ . control <- CVControl() gbm_res1 <- resample(fo, ICHomes, GBMModel(n.trees = 25), control) gbm_res2 <- resample(fo, ICHomes, GBMModel(n.trees = 50), control) gbm_res3 <- resample(fo, ICHomes, GBMModel(n.trees = 100), control) res <- c(GBM1 = gbm_res1, GBM2 = gbm_res2, GBM3 = gbm_res3) res_diff <- diff(res) t.test(res_diff)
Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.