Skip to content

Offline workflow: allow local frame caches, file transfer fixes, cleanup in config

Patrick Godwin requested to merge feature-workflow_config into master

This merge request has fixes/additions/cleanup for the offline workflow:

  • Merge "min instrument" like settings so only one is needed (removed the min-instruments setting in filter section)
  • config.yml: ifos -> instruments, min-ifos -> min-instruments
  • config.Config:
    • Normalize key-value pairs in condor config before reading (like every other section)
    • factor out condor submit options parsing into its own method
  • Add ability to pass in a frame cache for offline workflows. This is not allowed currently if using Condor file transfer and/or flocking (e.g. OSG) for simplicity
  • Add simple validation in Config to check some settings are consistent
  • Rearrange, cleanup in filter_injections_online_layer in dags/layers/inspiral.py
  • Fix issue in PSD jobs where file transfer setting wasn't being passed in causing issues when it was disabled

This has the effect where the schema within the config has changed in a backwards-incompatible way, but will be fixed in the relevant repos.

Edited by Patrick Godwin

Merge request reports

Loading