Fix condor file transfer

This MR fixes the condor file transfer set up for the analysis job, which was quite broken. Short summary:

• need to transfer the prior file
• need to use basenames of all input files in DAG ARGS since that's how they will appear on the execute node

added 2 commits

• 7db37206 - appease black
• 50a7ab50 - bilby_pipe.main: use private variable

Compare with previous version

Hmmm... this doesn't work very well, it seems.

@gregory.ashton, does it make more sense to add an option to the ArgumentsString constructor to inform it that it will be built using condor file transfer? Something like:

class ArgumentString(object):
    def __init__(self, condor_file_transfer=False):
        self.argument_list = []
        self.condor_file_transfer = condor_file_transfer

    ...

    def add(self, argument, value):
        self.append("--{}".format(argument))
        if self.condor_file_transfer and argument.endswith("-file"): 
            self.append(os.path.basename(str(value)))
        else:
            self.append(str(value))

I'm not sure this will catch arbitrary file arguments given on the command line, so you probably want a more robust method than that.

I'd be happy with that solution. I am slightly confused as to why this is breaking, I had thought I'd checked that things were working okay. I'll try to investigate.

If the issue is only the prior file, then an alternative option that I've been wanting for a while is to write the prior file into the ini file. This reduces the amount of files required. We now work by taking the users ini file and writing a "working copy" of the ini file which is actually used by the generation and analysis script. So, I'd envision this basically writing the contents of the prior file into the working copy of the ini file and then using that.

@duncanmmacleod I've run with and without file transfer and things are running fine as far as I can tell.

I think that we resolved the issue of moving files using condor's file transfer mechanism by also searching locally for the files. I'll close this for now as I don't think we need to the changes to solve any issues. Please ping me if I'm missing something!

closed