Learn R Programming

BatchJobs (version 1.9)

Batch Computing with R

Description

Provides Map, Reduce and Filter variants to generate jobs on batch computing systems like PBS/Torque, LSF, SLURM and Sun Grid Engine. Multicore and SSH systems are also supported. For further details see the project web page.

Copy Link

Version

Install

install.packages('BatchJobs')

Monthly Downloads

1,414

Version

1.9

License

BSD_2_clause + file LICENSE

Issues

41

Pull Requests

7

Stars

85

Forks

20

Maintainer

Bernd Bischl

Last Published

March 21st, 2022

Functions in BatchJobs (1.9)

batchMapResults

Maps a function over the results of a registry by using batchMap.
addRegistrySourceFiles

Add source files to registry.
batchMap

Maps a function over lists or vectors, adding jobs to a registry.
batchMapQuick

Combination of makeRegistry, batchMap and submitJobs.
batchExport

Export R object to be available on the slaves.
addRegistrySourceDirs

Add source dirs to registry.
callFunctionOnSSHWorkers

Call an arbitrary function on specified SSH workers.
addRegistryPackages

Add packages to registry.
BatchJobs

The BatchJobs package
batchReduceResults

Reduces results via a binary function and adds jobs for this to a registry.
dbCreateJobDefTable

ONLY FOR INTERNAL USAGE.
dbGetJobs

ONLY FOR INTERNAL USAGE.
batchReduce

Reduces via a binary function over a list adding jobs to a registry.
getConfig

Returns a list of BatchJobs configuration settings
batchQuery

Manually query the BatchJobs database
findDone

Find jobs depending on computional state.
getJobNr

Get number of jobs in registry.
getJobLocation

Get the physical location of job files on the hard disk.
checkIds

Check job ids.
cfBrewTemplate

Cluster functions helper: Brew your template into a job description file.
cfReadBrewTemplate

Cluster functions helper: Read in your brew template file.
configuration

BatchJobs configuration.
copyRequiredJobFiles

ONLY FOR INTERNAL USAGE.
getJob

Get job from registry by id.
grepLogs

Grep log files for a pattern.
findJobs

Finds ids of jobs that match a query.
getErrorMessages

Get error messages of jobs.
filterResults

Find all results where a specific condition is true.
makeClusterFunctionsSSH

Create an SSH cluster to execute jobs.
loadConfig

Load a specific configuration file.
killJobs

Kill some jobs on the batch system.
makeClusterFunctionsLSF

Create cluster functions for LSF systems.
installPackagesOnSSHWorkers

Install packages on SSH workers.
applyJobFunction

applyJobFunction ONLY FOR INTERNAL USAGE.
batchUnexport

Unload exported R objects.
getJobs

Get jobs from registry by id.
makeClusterFunctionsTorque

Create cluster functions for torque-based systems.
sourceRegistryFiles

Source registry files
reduceResults

Reduce results from result directory.
removeRegistry

Remove a registry object.
makeClusterFunctionsLocal

Create cluster functions for sequential execution on local host.
submitJobs

Submit jobs or chunks of jobs to batch system via cluster function.
getJobIds

Get ids of jobs in registry.
setJobFunction

Sets the job function for already existing jobs.
getJobInfo

Get computational information of jobs.
makeClusterFunctionsMulticore

Use multiple cores on local Linux machine to spawn parallel jobs.
getLogFiles

Get log file paths for jobs.
loadResult

Loads a specific result file.
batchExpandGrid

Map function over all combinations.
loadRegistry

Load a previously saved registry.
loadExports

Load exported R data objects.
makeClusterFunctionsSLURM

Create cluster functions for SLURM-based systems.
sweepRegistry

Sweep obsolete files from the file system.
removeRegistrySourceDirs

Remove packages from registry.
removeRegistryPackages

Remove packages from registry.
makeClusterFunctionsSGE

Create cluster functions for Sun Grid Engine systems.
waitForJobs

Wait for termination of jobs on the batch system.
syncRegistry

Syncronize staged queries into the registry.
makeJob

Creates a job description.
loadResults

Loads result files for id vector.
resetJobs

Reset computational state of jobs.
removeRegistrySourceFiles

Remove source files from registry.
makeClusterFunctionsOpenLava

Create cluster functions for OpenLava systems.
setJobNames

Set job names.
showStatus

Retrieve or show status information about jobs.
makeRegistry

Construct a registry object.
showLog

Display the contents of a log file.
cfHandleUnknownSubmitError

Cluster functions helper: Handle an unknown error during job submission.
cfKillBatchJob

Cluster functions helper: Kill a batch job via OS command
setRegistryPackages

Set packages for a registry.
debugSSH

Helper function to debug SSH mode.
debugMulticore

Helper function to debug multicore mode.
sanitizePath

Sanitize a path
setConfig

Set and overwrite configuration settings
getJobResources

Function to get the resources that were submitted for some jobs.
getJobParamDf

Retrieve Job Parameters.
makeClusterFunctionsInteractive

Create cluster functions for sequential execution in same session.
getResources

Function to get job resources in job function.
makeSSHWorker

Create SSH worker for SSH cluster functions.
getSSHWorkersInfo

Print and return R installation and other information for SSH workers.
showClusterStatus

Show information about available computational resources on cluster.
makeClusterFunctions

Create a ClusterFuntions object.
makeSubmitJobResult

Create a SubmitJobResult object.
updateRegistry

ONLY FOR INTERNAL USAGE.
testJob

Tests a job by running it with Rscript in a new process.