HelpersMG (version 5.1)

universalmclapply: Run the function FUN on X using parallel computing

Description

Return the results of the function FUN applied to X. It uses forking in unix system and not in windows system.

Usage

universalmclapply(
  X,
  FUN,
  ...,
  mc.cores = getOption("mc.cores", parallel::detectCores()),
  mc.preschedule = TRUE,
  clusterExport = list(),
  clusterEvalQ = list(),
  forking = ifelse(.Platform$OS.type == "windows", FALSE, TRUE),
  progressbar = FALSE
)

Arguments

X

A vector (atomic or list) or an expressions vector. Other objects (including classed objects) will be coerced by as.list.

FUN

The function to be applied to each element of X

...

Optional arguments to FUN

mc.cores

The number of cores to use, i.e. at most how many child processes will be run simultaneously.

mc.preschedule

if set to TRUE then the computation is first divided to (at most) as many jobs are there are cores and then the jobs are started, each job possibly covering more than one value. If set to FALSE then one job is forked for each value of X. The former is better for short computations or large number of values in X, the latter is better for jobs that have high variance of completion time and not too many values of X compared to mc.cores.

clusterExport

List of clusterExport parameters as list

clusterEvalQ

List of clusterEvalQ parameters as list

forking

If TRUE will use forking

progressbar

If pbapply package is installed, show a progressbar

Value

The results of the function FUN applied to X

Details

universalmclapply runs the function FUN on X using parallel computing

Examples

# NOT RUN {
library(HelpersMG)
x <- 1:1000
funx <- function(y) {
  mint <- rep(NA, length(y))
  for (i in seq_along(y)) {
    k <- rnorm(runif(n = 1, 50, 50), mean=10, sd=2)
    mint[i] <- mean(k)
  }
  mint
}
# Note that parallel computing is not always the best solution !
(tp <- system.time({
   m <- lapply(X=x, FUN=funx)
}))
(tp <- system.time({
   m <- universalmclapply(X=x, FUN=funx, forking=FALSE)
}))
(tp <- system.time({
   m <- universalmclapply(X=x, FUN=funx, forking=TRUE)
}))

### An example using clusterExport
# Here no error is generated because environment was exported
# However forking is not possible in windows and non parallel code is ran
pp <- runif(100)
x <- 1:100
funx1 <- function(y) {pp[y]*10}
u <- universalmclapply(x, FUN=funx1, forking=TRUE)

# Here an error is generated because environment was not exported when parLapplyLB is used
pp <- runif(100)
x <- 1:100
u <- universalmclapply(x, FUN=funx1, forking=FALSE)

# here no error is generated because the variable pp is exported
pp <- runif(100)
x <- 1:100
u <- universalmclapply(x, FUN=funx1, forking=FALSE, 
                       clusterExport=list(varlist=c("pp"), envir=environment()))
                       
### An example using clusterEvalQ
asc("a") # asc() is a function from packages HelpersMG
funx2 <- function(y) {asc("a")*10}
# In unix, the loaded packages are visible from all cores
x <- 1:100
u <- universalmclapply(x, FUN=funx2, forking=TRUE)
# In windows, the loaded packages are not visible from all cores
x <- 1:100
u <- universalmclapply(x, FUN=funx2, forking=FALSE)
# In windows, the loaded packages are not visible from all cores
x <- 1:100
u <- universalmclapply(x, FUN=funx2, forking=FALSE, 
clusterEvalQ=list(expr=expression(library(HelpersMG)))
)

### If package pbapply is available, progress bar can be shown
m <- universalmclapply(X=x, FUN=funx, forking=FALSE, progressbar=TRUE)
m <- universalmclapply(X=x, FUN=funx, forking=TRUE, progressbar=TRUE)
# }