flowr (version 0.9.11)

queue-class: A queue object defines details regarding how a job is submitted

Description

Internal function (used by to_flow), to define the format used to submit a job.

Usage

queue(
  object,
  platform = c("local", "lsf", "torque", "sge", "moab", "test", "slurm"),
  format = "",
  queue = "long",
  walltime,
  memory,
  cpu = 1,
  extra_opts = "",
  submit_exe,
  nodes = "1",
  jobname = "name",
  email = Sys.getenv("USER"),
  dependency = list(),
  server = "localhost",
  verbose = FALSE,
  cwd = "",
  stderr = "",
  stdout = "",
  ...
)

Arguments

object

this is not used currently, ignore.

platform

Required and important. Currently supported values are 'lsf' and 'torque'. [Used by class job]

format

[advanced use] We have a default format for the final command line string generated for 'lsf' and 'torque'.

queue

the type of queue your group usually uses 'bsub' etc.

walltime

max walltime of a job.

memory

The amount of memory reserved. Units depend on the platform used to process jobs

cpu

number of cpus you would like to reserve [Used by class job]

extra_opts

[advanced use] Extra options to be supplied while create the job submission string.

submit_exe

[advanced use] Already defined by 'platform'. The exact command used to submit jobs to the cluster example 'qsub'

nodes

[advanced use] number of nodes you would like to request. Or in case of torque name of the nodes.optional [Used by class job]

jobname

[debug use] name of this job in the computing cluster

email

[advanced use] Defaults to system user, you may put you own email though may get tons of them.

dependency

[debug use] a list of jobs to complete before starting this one

server

[not used] This is not implemented currently. This would specify the head node of the computing cluster. At this time submission needs to be done on the head node of the cluster where flow is to be submitted

verbose

[logical] TRUE/FALSE

cwd

[debug use] Ignore

stderr

[debug use] Ignore

stdout

[debug use] Ignore

...

other passed onto object creation. Example: memory, walltime, cpu

Details

Resources: Can be defined **once** using a queue object and recycled to all the jobs in a flow. If resources (like memory, cpu, walltime, queue) are supplied at the job level they overwrite the one supplied in queue Nodes: can be supplied to extend a job across multiple nodes. This is purely experimental and not supported.

Server: This a hook which may be implemented in future.

Submission script The 'platform' variable defines the format, and submit_exe; however these two are available for someone to create a custom submission command.

Examples

# NOT RUN {
qobj <- queue(platform='lsf')
# }