A Distributed Worker Launcher Framework
Description
In computationally demanding analysis projects,
statisticians and data scientists asynchronously
deploy long-running tasks to distributed systems,
ranging from traditional clusters to cloud services.
The 'NNG'-powered 'mirai' R package by Gao (2023)
is a sleek
and sophisticated scheduler that
efficiently processes these intense workloads.
The 'crew' package extends 'mirai' with a unifying
interface for third-party worker launchers.
Inspiration also comes from packages.
'future' by Bengtsson (2021) ,
'rrq' by FitzJohn and Ashton (2023) ,
'clustermq' by Schubert (2019) ),
and 'batchtools' by Lang, Bischel, and Surmann (2017)
.