Follow-up from "Draft: Add support for sampler plugins"
The following discussion from !1299 (merged) should be addressed:
-
@michael.williams started a discussion: (+1 comment) Thinking about this in connection with bilby_pipe, specifically the logic here: https://git.ligo.org/lscsoft/bilby_pipe/-/blob/master/bilby_pipe/job_creation/nodes/analysis_node.py#L125. As it stands, this can prevent samplers from being used out-of-the-box with bilby pipe unless you set
transfer-files=False
.What would folks think about adding one or two attributes to the sampler classes that specify the files and/or directories the sampler will create? Something like:
expected_output_files = [] expected_output_directories = []
or a function that returns them based on the labels.
bilby_pipe could then use these attributes to create the directories and placeholder files, rather than having to hard code things. It would then fall on the sampler plugins to maintain this rather than the bilby_pipe maintainers
I'm happy to implement something like this in a separate MR if there's interest.