targets: tar_make_future – R documentation

Become an expert in R — Interactive courses, Cheat Sheets, certificates and more!

tar_make_future

Run a pipeline of targets in parallel with transient future workers.

Description

This function is like tar_make() except that targets run in parallel with transient future workers. It requires that you declare your future::plan() inside the target script file (default: _targets.R). future is not a strict dependency of targets, so you must install future yourself.

Usage

tar_make_future(
  names = NULL,
  shortcut = targets::tar_config_get("shortcut"),
  reporter = targets::tar_config_get("reporter_make"),
  workers = targets::tar_config_get("workers"),
  callr_function = callr::r,
  callr_arguments = targets::callr_args_default(callr_function, reporter),
  envir = parent.frame(),
  script = targets::tar_config_get("script"),
  store = targets::tar_config_get("store")
)

Arguments

`names`	Names of the targets to build or check. Set to `NULL` to check/build all the targets (default). Otherwise, you can supply `tidyselect` helpers like `all_of()` and `starts_with()`. Because `tar_make()` and friends run the pipeline in a new R session, if you pass a character vector to a tidyselect helper, you will need to evaluate that character vector early with `!!`, e.g. `tar_make(names = all_of(!!your_vector))`. Applies to ordinary targets (stem) and whole dynamic branching targets (patterns) but not to individual dynamic branches.
`shortcut`	Logical of length 1, how to interpret the `names` argument. If `shortcut` is `FALSE` (default) then the function checks all targets upstream of `names` as far back as the dependency graph goes. `shortcut = TRUE` increases speed if there are a lot of up-to-date targets, but it assumes all the dependencies are up to date, so please use with caution. It relies on stored metadata for information about upstream dependencies. `shortcut = TRUE` only works if you set `names`.
`reporter`	Character of length 1, name of the reporter to user. Controls how messages are printed as targets run in the pipeline. Defaults to `tar_config_get("reporter_make")`. Choices: `"silent"`: print nothing. `"summary"`: print a running total of the number of each targets in each status category (queued, started, skipped, build, canceled, or errored). Also show a timestamp (`"%H:%M %OS2"` `strptime()` format) of the last time the progress changed and printed to the screen. `"timestamp"`: same as the `"verbose"` reporter except that each .message begins with a time stamp. `"timestamp_positives"`: same as the `"timestamp"` reporter except without messages for skipped targets. `"verbose"`: print messages for individual targets as they start, finish, or are skipped. `"verbose_positives"`: same as the `"verbose"` reporter except without messages for skipped targets.
`workers`	Positive integer, maximum number of transient `future` workers allowed to run at any given time.
`callr_function`	A function from `callr` to start a fresh clean R process to do the work. Set to `NULL` to run in the current session instead of an external process (but restart your R session just before you do in order to clear debris out of the global environment). `callr_function` needs to be `NULL` for interactive debugging, e.g. `tar_option_set(debug = "your_target")`. However, `callr_function` should not be `NULL` for serious reproducible work.
`callr_arguments`	A list of arguments to `callr_function`.
`envir`	An environment, where to run the target R script (default: `_targets.R`) if `callr_function` is `NULL`. Ignored if `callr_function` is anything other than `NULL`. `callr_function` should only be `NULL` for debugging and testing purposes, not for serious runs of a pipeline, etc. The `envir` argument of `tar_make()` and related functions always overrides the current value of `tar_option_get("envir")` in the current R session just before running the target script file, so whenever you need to set an alternative `envir`, you should always set it with `tar_option_set()` from within the target script file. In other words, if you call `tar_option_set(envir = envir1)` in an interactive session and then `tar_make(envir = envir2, callr_function = NULL)`, then `envir2` will be used.
`script`	Character of length 1, path to the target script file. Defaults to `tar_config_get("script")`, which in turn defaults to `_targets.R`. When you set this argument, the value of `tar_config_get("script")` is temporarily changed for the current function call. See `tar_script()`, `tar_config_get()`, and `tar_config_set()` for details about the target script file and how to set it persistently for a project.
`store`	Character of length 1, path to the `targets` data store. Defaults to `tar_config_get("store")`, which in turn defaults to `_targets/`. When you set this argument, the value of `tar_config_get("store")` is temporarily changed for the current function call. See `tar_config_get()` and `tar_config_set()` for details about how to set the data store path persistently for a project.

Details

To configure tar_make_future() with a computing cluster, see the future.batchtools package documentation.

Value

NULL except if callr_function = callr::r_bg(), in which case a handle to the callr background process is returned. Either way, the value is invisibly returned.

Examples

if (identical(Sys.getenv("TAR_EXAMPLES"), "true")) {
tar_dir({ # tar_dir() runs code from a temporary directory.
tar_script({
  future::plan(future::multisession, workers = 2)
  list(
    tar_target(x, 1 + 1),
    tar_target(y, 1 + 1)
  )
}, ask = FALSE)
tar_make_future()
})
}

targets

Dynamic Function-Oriented 'Make'-Like Declarative Workflows

v0.10.0

MIT + file LICENSE

Authors

William Michael Landau [aut, cre] (<https://orcid.org/0000-0003-1878-3253>), Matthew T. Warkentin [ctb], Samantha Oliver [rev] (<https://orcid.org/0000-0001-5668-1165>), Tristan Mahr [rev] (<https://orcid.org/0000-0002-8890-5116>), Eli Lilly and Company [cph]

Initial release