A BatchJobs multicore future is an asynchronous multiprocess future that will be evaluated in a background R session.
We highly recommend using multisession
(sic!) futures of the future package instead of
multicore BatchJobs futures.
batchjobs_multicore(
expr,
envir = parent.frame(),
substitute = TRUE,
globals = TRUE,
label = "BatchJobs",
workers = availableCores(constraints = "multicore"),
job.delay = FALSE,
...
)The R expression to be evaluated
The environment in which global environment should be located.
Controls whether expr should be
substitute():d or not.
(optional) a logical, a character vector, a named list, or a Globals object. If TRUE, globals are identified by code inspection based on expr and tweak searching from environment envir. If FALSE, no globals are used. If a character vector, then globals are identified by lookup based their names globals searching from environment envir. If a named list or a Globals object, the globals are used as is.
(optional) Label of the future (where applicable, becomes the job name for most job schedulers).
The number of multicore processes to be available for concurrent BatchJobs multicore futures.
(optional) Passed as is to submitJobs().
Additional arguments passed
to BatchJobsFuture().
An object of class BatchJobsFuture.
BatchJobs multicore futures rely on the BatchJobs backend set
up by makeClusterFunctionsMulticore().
The BatchJobs multicore backend only works on operating systems
supporting the ps command-line tool, e.g. Linux and OS X.
However, they are not supported on neither Windows nor Solaris
Unix (because ps -o ucomm= is not supported). When not
supported, it falls back to batchjobs_local.
Warning: For multicore BatchJobs, the BatchJobs
package uses a built-in algorithm for load balancing based on
other processes running on the same machine. This is done
in order to prevent the machine's CPU load to blow up.
Unfortunately, the BatchJobs criteria for handling this often
results in starvation, that is, long waiting times before
launching jobs. The risk for this is particularly high if
there are other R processes running on the same machine
including those by other users.
See also https://github.com/tudo-r/BatchJobs/issues/99.
Conclusion: We highly recommend using
multisession futures of the
future package instead of multicore BatchJobs futures.
Also, despite the name, BatchJobs multicore futures are in function closer related to multisession futures than multicore futures, both provided by the future package. This is because BatchJobs spawns off background R sessions rather than forking the current R process as the name otherwise might imply (at least that is how the term "multicore processing" is typically used in the R world).