parallelly (version 1.41.0)

makeClusterMPI: Create a Message Passing Interface (MPI) Cluster of R Workers for Parallel Processing

Description

The makeClusterMPI() function creates an MPI cluster of R workers for parallel processing. This function utilizes makeCluster(..., type = "MPI") of the parallel package and tweaks the cluster in an attempt to avoid stopCluster() from hanging (1). WARNING: This function is very much in a beta version and should only be used if parallel::makeCluster(..., type = "MPI") fails.

Usage

makeClusterMPI(
  workers,
  ...,
  autoStop = FALSE,
  verbose = getOption2("parallelly.debug", FALSE)
)

Value

An object of class c("RichMPIcluster", "MPIcluster", "cluster") consisting of a list of "MPInode" workers.

Arguments

workers

The number workers (as a positive integer).

...

Optional arguments passed to makeCluster(workers, type = "MPI", ...).

autoStop

If TRUE, the cluster will be automatically stopped using stopCluster() when it is garbage collected, unless already stopped. See also autoStopCluster().

verbose

If TRUE, informative messages are outputted.

Alternative usage

In R (>= 4.4.0), an alternatively to using cl <- parallelly::makeClusterMPI(workers) is:

cl <- parallel::makeCluster(workers, type = parallelly::MPI)

Details

Creating MPI clusters requires that the Rmpi and snow packages are installed.

References

1. R-sig-hpc thread Rmpi: mpi.close.Rslaves() 'hangs' on 2017-09-28.

See Also

makeClusterPSOCK() and parallel::makeCluster().

Examples

if (FALSE) {
if (requireNamespace("Rmpi") && requireNamespace("snow")) {
  cl <- makeClusterMPI(2, autoStop = TRUE)
  print(cl)
  y <- parLapply(cl, X = 1:3, fun = sqrt)
  print(y)
  rm(list = "cl")
}
}