improve slurm arguments handling
This is still the right place - only bilby-non-pipe has moved to github so far, right?
Here I do a slight revamp of how the slurm arguments are handled before being written into the submit script: instead of immediately building strings, it first uses dicts which can then be updated with the custom --scheduler-args
and also from node to node. This fixes two problems:
- So far, no
mem
was assigned to the slurm master job itself. This is however a required argument for all submitted jobs on some clusters, like the RES node Picasso in Malaga that the UIB group is often using. - If the user put something like
mem
ortime
into the custom--scheduler-args
that already has a hard-coded value, then the old string concatenation approach would result in duplicate settings for the individual job nodes, which apparently does not trigger errors but left it open to guessing which value would actually be used.
In principle, this also lays the groundwork for being able to add extra user options to the parser in case people need to customise the memory allocation for individual nodes beyond the main analysis
one, e.g. sometimes the generation or post-processing may need more memory then bilby_pipe assigns by default. But I'll leave that to a separate MR.