flowr (version 0.9.11)

run_pipe_v2: Run automated Pipelines

Description

Run complete pipelines, by wrapping several steps into one convenient function.

NOTE: please use flowr version 0.9.8.9010 or higher.

In summary, this function performs the following steps:

• the argument x defines the name of the pipeline. Say, for example sleep_pipe.

• fetch_pipes: finds the pipeline definition (sleep_pipe.R, sleep_pipe.def and sleep_pipe.conf files)

• sleep_pipe(…): Create all the required commands (flowmat)

• to_flow: Use flowmat and sleep_pipe.def to create a flow object.

• submit_flow: Submit the flow to the cluster.

Usage

run_pipe_v2(
  pipe_func,
  pipe_src,
  flow_def,
  flow_conf,
  flowname,
  platform,
  flow_run_path = opts_flow$get("flow_run_path"),
  rerun_wd,
  start_from,
  execute = FALSE,
  ...
)

Arguments

pipe_func

name of the pipeline function in `pipe_src`

pipe_src

path to pipeline script

flow_def

flow definition file

flow_conf

flow conf file with various parameters used by the flow

flowname

name for the flow for submission.

platform

what platform to use, overrides flowdef

flow_run_path

passed onto to_flow. Default it picked up from flowr.conf. Typically this is ~/flowr/runs

rerun_wd

if you need to re-run, supply the previous working dir

start_from

the step to start a rerun from. Intuitively, this is ignored in a fresh run and only used in re-running a pipeline.

execute

TRUE/FALSE

...

passed onto the pipeline function as specified in x