Skip to content

Add current working directory to comment row of process table.

Luke Davis requested to merge ldavis__pwd_in_coinc into spiir-O4-EW-development

Certain IGWN consumers require access to the working directory of a pipeline in order to fetch pipeline artifacts that are not uploaded with the coinc.xml to gracedb.

This MR will add metadata to the 'comment' row of the 'process' table which is normally left blank.
The 'process' row will need to be parsed with json.loads().

At the moment the only metadata added is the current working directory under the 'pwd' key.

Here's an example process table:

<Table Name="process:table">
         <Column Name="process:comment" Type="lstring"/>
         <Column Name="process:cvs_entry_time" Type="int_4s"/>
         <Column Name="process:cvs_repository" Type="lstring"/>
         <Column Name="process:domain" Type="lstring"/>
         <Column Name="process:end_time" Type="int_4s"/>
         <Column Name="process:ifos" Type="lstring"/>
         <Column Name="process:is_online" Type="int_4s"/>
         <Column Name="process:jobid" Type="int_4s"/>
         <Column Name="process:node" Type="lstring"/>
         <Column Name="process:process_id" Type="ilwd:char"/>
         <Column Name="process:program" Type="lstring"/>
         <Column Name="process:start_time" Type="int_4s"/>
         <Column Name="process:unix_procid" Type="int_4s"/>
         <Column Name="process:username" Type="lstring"/>
         <Column Name="process:version" Type="lstring"/>
         <Stream Delimiter="," Name="process:table" Type="Local">
             "{\"pwd\": \"/home/luke.davis/singularity/spiir-scripts/runs/run-ldavis__pwd_in_coinc-tdavies__require_min_chisq_for_upload-1694663864-655ac5bd6\"}",,,,,"H1,L1,V1",0,0,"node283","process:process_id:13","gstlal_inspiral_postcohspiir_online",1378706657,1682454,"luke.davis",,
         </Stream>
</Table>

Once merged this code can be added to appropriate consumers:

        comment_row = xml_tables['process']['comment'].item()

        # pwd not in coinc, can't read marginilized state.
        if comment_row is None:
            logger.debug("process table comment row empty.")
            return

        comment_row = json.loads(comment_row)
        if 'pwd' not in comment_row:
            logger.debug("pwd not in process table comment row.")
            return

        pwd = comment_row['pwd']
Edited by Luke Davis

Merge request reports