Computes the Ratio Between The RMSE of Two Experiments
Calculates the ratio of the RMSE for two forecasts of the same observations.
The ratio RMSE(ens, obs) / RMSE(ens.ref, obs) is output.
The p-value is provided by a two-sided Fischer test.
.RatioRMS provides the same functionality but taking two matrices of
ensemble members (ens and ens.ref) as input.
RatioRMS(var_exp1, var_exp2, var_obs, posRMS = 1, pval = TRUE) .RatioRMS(exp, exp_ref, obs, pval = TRUE)
var_exp1 |
Array of experimental data 1. |
var_exp2 |
Array of experimental data 2. |
var_obs |
Array of observations. |
posRMS |
Dimension along which the RMSE are to be computed = the position of the start dates. |
pval |
Whether to compute the p-value of Ho : RMSE1/RMSE2 = 1 or not. TRUE by default. |
exp |
Matrix of experimental data 1. |
exp_ref |
Matrix of experimental data 2. |
obs |
Vector of observations. |
RatioRMS:
Matrix with the same dimensions as var_exp1/var_exp2/var_obs except along
posRMS where the dimension has length 2 if 'pval = TRUE', or 1 otherwise.
The dimension of length 2 corresponds to the ratio between the RMSE
(RMSE1/RMSE2) and the p-value of the two-sided Fisher test with Ho:
RMSE1/RMSE2 = 1.
.RatioRMS:
ratiormsThe ratio of the RMSE of the two experimental datasets
p_valThe p-value
History:
0.1 - 2011-11 (V. Guemas, vguemas@bsc.es) - Original code
1.0 - 2013-09 (N. Manubens, nicolau.manubens@bsc.es) - Formatting to R CRAN
1.1 - 2017-02 (A. Hunter, alasdair.hunter@bsc.es) - Adapted to veriApply()
# See examples on Load() to understand the first lines in this example
## Not run:
data_path <- system.file('sample_data', package = 's2dverification')
expA <- list(name = 'experiment', path = file.path(data_path,
'model/$EXP_NAME$/$STORE_FREQ$_mean/$VAR_NAME$_3hourly',
'$VAR_NAME$_$START_DATE$.nc'))
obsX <- list(name = 'observation', path = file.path(data_path,
'$OBS_NAME$/$STORE_FREQ$_mean/$VAR_NAME$',
'$VAR_NAME$_$YEAR$$MONTH$.nc'))
# Now we are ready to use Load().
startDates <- c('19851101', '19901101', '19951101', '20001101', '20051101')
sampleData <- Load('tos', list(expA), list(obsX), startDates,
output = 'lonlat', latmin = 27, latmax = 48,
lonmin = -12, lonmax = 40)
## End(Not run)
# Compute DJF seasonal means and anomalies.
leadtimes_dimension <- 4
initial_month <- 11
mean_start_month <- 12
mean_stop_month <- 2
sampleData$mod <- Season(sampleData$mod, leadtimes_dimension, initial_month,
mean_start_month, mean_stop_month)
sampleData$obs <- Season(sampleData$obs, leadtimes_dimension, initial_month,
mean_start_month, mean_stop_month)
clim <- Clim(sampleData$mod, sampleData$obs)
ano_exp <- Ano(sampleData$mod, clim$clim_exp)
ano_obs <- Ano(sampleData$obs, clim$clim_obs)
# Generate two experiments with 2 and 1 members from the only experiment
# available in the sample data. Take only data values for a single forecast
# time step.
ano_exp_1 <- Subset(ano_exp, 'member', c(1, 2))
ano_exp_2 <- Subset(ano_exp, 'member', c(3))
ano_exp_1 <- Subset(ano_exp_1, c('dataset', 'ftime'), list(1, 1), drop = 'selected')
ano_exp_2 <- Subset(ano_exp_2, c('dataset', 'ftime'), list(1, 1), drop = 'selected')
ano_obs <- Subset(ano_obs, c('dataset', 'ftime'), list(1, 1), drop = 'selected')
# Compute ensemble mean and provide as inputs to RatioRMS.
rrms <- RatioRMS(Mean1Dim(ano_exp_1, 1),
Mean1Dim(ano_exp_2, 1),
Mean1Dim(ano_obs, 1))
# Plot the RatioRMS for the first forecast time step.
PlotEquiMap(rrms[1, , ], sampleData$lon, sampleData$lat,
toptitle = 'Ratio RMSE')
# The following example uses veriApply combined with .RatioRMS instead of RatioRMS
## Not run:
require(easyVerification)
# The name of the function has to end in 'ss' in order for veriApply() to
# detect it as a skill score.
RatioRMSss <- s2dverification:::.RatioRMS
rrms2 <- veriApply("RatioRMSss", ano_exp_1,
# see ?veriApply for how to use the 'parallel' option
Mean1Dim(ano_obs, 1),
ano_exp_2,
tdim = 2, ensdim = 1)
## End(Not run)Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.