Skip to content

ENH: allow outdir to change for file transfer

Colm Talbot requested to merge specify-outdir-for-saving into master

When running through HTCondor with the file transfer system there is a possible situation where:

  • a result file is written on cluster X
  • it is transferred back to the main filesystem due to some failure
  • it is transferred to a different cluster Y using the resume dag

When we attempt to resave the result file on cluster Y it fails because it tries to write the file to cluster X which it no longer has access to.

This MR explicitly makes it always save to the user-specified outdir and so is immune to these problems. For vanilla use, there should be no change to the behaviour.

Merge request reports