Learn R Programming


output: html_document: keep_md: yes packagedocs: toc: false navpills: |

Streamlining Computing workflows

Latest documentation: flow-r.github.io/flowr

Flowr framework allows you to design and implement complex pipelines, and deploy them on your institution's computing cluster. This has been built keeping in mind the needs of bioinformatics workflows. However, it is easily extendable to any field where a series of steps (shell commands) are to be executed in a (work)flow.

Highlights

  • No new syntax or language. Put all shell commands as a tsv file called flow mat.
  • Define the flow of steps using a simple tsv file (serial, scatter, gather, burst...) called flow def.
  • Works on your laptop/server or cluster (/cloud).
  • Supports multiple cluster computing platforms (torque, lsf, sge, slurm ...), cloud (star cluster) OR a local machine.
  • One line installation (install.packages("flowr"))
  • Reproducible and transparent, with cleanly structured execution logs
  • Track and re-run flows
  • Lean and Portable, with easy installation
  • Fine grain control over resources (CPU, memory, walltime, etc.) of each step.

Example

Copy Link

Version

Install

install.packages('flowr')

Monthly Downloads

273

Version

0.9.11

License

MIT + file LICENSE

Issues

Pull Requests

Stars

Forks

Maintainer

Last Published

March 2nd, 2021

Functions in flowr (0.9.11)

check_args

Assert none of the arguments of a function are null.
run_pipe_v2

Run automated Pipelines
run

Run automated Pipelines
fetch

Two generic functions to search for pipelines and configuration files.
get_resources

Extract resources used by each job of a flow get_resources currently this only works on LSF
get_resources_lsf

get_resources_lsf
job

Describing details of the job object
kill

Kill all jobs submitted to the computing platform, for one or multiple flows
status

Monitor status of flow(s)
setup

Setup and initialize flowr
submit_flow

Submit a flow to the cluster
create_jobs_mat

create_jobs_mat
error

Error Handler
submit_job

Submit a step of a flow
to_flowmat

Create a flowmat using a list a commands.
flow-class

Describing the flow class
verbose

Verbose levels, defining verboseness of messages
get_unique_id

get_unique_id
plot_flow

Plot a clean and scalable flowchart describing the (work)flow
flowopts

Default options/params used in flowr and ngsflows
generate_flow_code

generate_flow_code
queue-class

A queue object defines details regarding how a job is submitted
get_wds

Get all the (sub)directories in a folder
to_flowdef

Flow Definition defines how to stitch steps into a (work)flow.
test_queue

test_queue
submit_run

Submit several flow objects, limit the max running concurrently
to_flowdet

Create a flow's submission detail file
whisker_render

Wrapper around whisker.render with some additional checks
write_flow_details

Write files describing this flow
to_df.status

to_df.status
rerun

Re-run a pipeline in case of hardware or software failures.
to_flow

Create flow objects
replace_slots

replace slots in a S4 object
check

Check consistency of flowdef and flowmat