spaMM: dopar – R documentation

Become an expert in R — Interactive courses, Cheat Sheets, certificates and more!

dopar

Interface for parallel computations

Description

An interface to apply some function fn in parallel on columns of a matrix. dopar is used internally in spaMM, but is not logically restricted to mixed-effect applications, hence it can be used more widely. Depending on the nb_cores argument, parallel or serial computation is performed. A socket cluster is used by default for parallel computations, but a fork cluster can be requested on linux and alike operating systems by using argument cluster_args=list(type="FORK").

Usage

dopar(newresp, fn, nb_cores = NULL, fit_env, control = list(), 
      cluster_args = NULL, debug. = FALSE, iseed = NULL, 
      showpbar = eval(spaMM.getOption("barstyle")), 
      pretest_cores =NULL, ...)

Arguments

`newresp`	A matrix on whose columns `fn` will be applied (e.g., as used internally in spaMM, the return value of a `simulate.HLfit()` call).
`fn`	Function, whose first argument is named `y`, to be applied to each column of `newresp`.
`nb_cores`	Integer. Number of cores to use for parallel computations. If >1, a cluster of `nb_cores` nodes is used. Otherwise, no parallel computation is performed.
`fit_env`	(for socket clusters only:) An environment, or a list, containing variables to be exported on the nodes of the cluster (by `parallel::clusterExport`); e.g., `list(bar=bar)` to pass object `bar` to each node. The argument `control(.errorhandling = "pass")`, below, is useful to find out missing variables.
`control`	A list following the `foreach` control syntax, even if `foreach` is not used. In particular, for socket clusters, with `doSNOW` attached, `foreach` is called with default arguments including `i = 1:ncol(newresp), .combine = "cbind", .inorder = TRUE, .errorhandling = "remove", .packages = "spaMM"`. `control` may be used to provide non-default values of these arguments. For example, `.errorhandling = "pass"` is useful to get error messages from the nodes, and therefore strongly recommended when first experimenting with this function. for socket clusters, with `doSNOW` not attached, `control$.packages` is still handled. The result is still in the format returned by `foreach` with default `.combine="cbind"` or possible non-default `.combine="rbind"`. if a fork cluster is used, `control$mc.silent` can be used to control the `mc.silent` argument of `mclapply`.
`cluster_args`	A list of arguments passed to `parallel::makeCluster`. E.g., `outfile="log.txt"` may be useful to collect output from the nodes, and `type="FORK"` to force a fork cluster on linux(-alikes).
`debug.`	(for socket clusters only:) For debugging purposes. Effect, if any, is to be defined by the `fn` as provided by the user.
`iseed`	(all parallel contexts:) Integer, or NULL. If an integer, it is used as the `iseed` argument of `clusterSetRNGStream` to initialize `"L'Ecuyer-CMRG"` random-number generator (see Details). If `iseed` is `NULL`, the default generator is selected on each node, where its seed is not controlled.
`showpbar`	(for socket clusters only:) Controls display of progress bar. See `barstyle` option for details.
`pretest_cores`	(for socket clusters only:) A function to run on the cores before running `fn`. It may be used to check that all arguments of the `fn` can be evaluated in the cores' environments (the internal function `.pretest_fn_on_cores` provides an example).
`...`	Further arguments to be passed (unevaluated) to `fn`.

Details

Control of random numbers through the "L'Ecuyer-CMRG" generator and the iseed argument is not sufficient for consistent results when the doSNOW parallel backend is used, so if you really need such control in a fn using random numbers, do not use doSNOW. Yet, it is fine to use doSNOW for bootstrap procedures in spaMM, because the fitting functions do not use random numbers: only sample simulation uses them, and it is not performed in parallel.

Value

The result of calling foreach, pbapply or mclapply, as dependent on the control argument. A side-effect of dopar is to show a progress bar that informs about the type of parallelisation performed: a default "=" character for fork clusters, and otherwise"P" for parallel computation via foreach and doSNOW, "p" for parallel computation via pbapply, and "s" for serial computation via pbapply.

Examples

## See source code of spaMM_boot()

## Not run: 
# Useless function, but requiring some argument beyond the first
foo <- function(y, somearg, ...) {
  if ( is.null(somearg) || TRUE ) length(y)
}

# Whether FORK can be used depends on OS and whether Rstudio is used:
dopar(matrix(1,ncol=4,nrow=3), foo, fit_env=list(), somearg=NULL, 
  nb_cores=2, cluster_args=list(type="FORK"))

## End(Not run)

spaMM

Mixed-Effect Models, with or without Spatial Random Effects

v3.10.0

CeCILL-2

Authors

François Rousset [aut, cre, cph] (<https://orcid.org/0000-0003-4670-0371>), Jean-Baptiste Ferdy [aut, cph], Alexandre Courtiol [aut] (<https://orcid.org/0000-0003-0637-2959>), GSL authors [ctb] (src/gsl_bessel.*)

Initial release

2022-02-06