drake (version 7.5.2)

predict_workers: Predict the load balancing of the next call to make() for non-staged parallel backends.

Description

Take the past recorded runtimes times from build_times() and use them to predict how the targets will be distributed among the available workers in the next make(). Predictions only include the time it takes to run the targets, not overhead/preprocessing from drake itself.

Usage

predict_workers(config, targets = NULL, from_scratch = FALSE,
  targets_only = NULL, jobs = 1, known_times = numeric(0),
  default_time = 0, warn = TRUE)

Arguments

config

Optional internal runtime parameter list of produced by both make() and drake_config().

targets

Character vector, names of targets. Predict the runtime of building these targets plus dependencies. Defaults to all targets.

from_scratch

Logical, whether to predict a make() build from scratch or to take into account the fact that some targets may be already up to date and therefore skipped.

targets_only

Deprecated.

jobs

The jobs argument of your next planned make(). How many targets to do you plan to have running simultaneously?

known_times

A named numeric vector with targets/imports as names and values as hypothetical runtimes in seconds. Use this argument to overwrite any of the existing build times or the default_time.

default_time

Number of seconds to assume for any target or import with no recorded runtime (from build_times()) or anything in known_times.

warn

Logical, whether to warn the user about any targets with no available runtime, either in known_times or build_times(). The times for these targets default to default_time.

Value

A data frame showing one likely arrangement of targets assigned to parallel workers.

See Also

predict_runtime(), build_times(), make()

Examples

# NOT RUN {
isolate_example("Quarantine side effects.", {
if (suppressWarnings(require("knitr"))) {
load_mtcars_example() # Get the code with drake_example("mtcars").
make(my_plan) # Run the project, build the targets.
config <- drake_config(my_plan)
known_times <- rep(7200, nrow(my_plan))
names(known_times) <- my_plan$target
known_times
# Predict the runtime
if (requireNamespace("lubridate", quietly = TRUE)) {
predict_runtime(
  config = config,
  jobs = 7,
  from_scratch = TRUE,
  known_times = known_times
)
predict_runtime(
  config,
  jobs = 8,
  from_scratch = TRUE,
  known_times = known_times
)
balance <- predict_workers(
  config,
  jobs = 7,
  from_scratch = TRUE,
  known_times = known_times
)
balance
}
}
})
# }