Skip to content

Add Monitor & apply loop hooks

Daniel Williams requested to merge 107-monitor-loop-hooks into review

What?

This MR introduces two new ways that code from outside the asimov package can interact with the asimov codebase in the form of plugins. There are two new entrypoints for these plugins:

  1. monitor loop: when asimov runs asimov.monitor it conducts a number of tasks, including checking whether jobs are still running, and beginning any required post-processing. This hook is run at the end of the monitor loop, and passes the asimov ledger (the database of all analyses and their states) to the plugin. This was added for cbcflow, which can now check the contents of the asimov ledger using this entrypoint, and is called automatically at the stage where relevant changes will be made by asimov.
  2. application plugins: these allow for asimov to use a plugin as a source of input data in place of a blueprint YAML file. By running e.g. asimov apply -p cbcflow -e GW150914 data can be collected from a CBCFlow library, and used to create a new event in the ledger. With a future modification of the application code this would also allow the settings to be updated, but at present this is not supported in the code.

Some lightweight documentation for these two endpoints has been added, however it would be wise to revisit this when there is more time in the near future.

Who?

This MR was prepared by Daniel Williams.

Where?

Modifications are made to the commandline interface for the asimov apply and asimov monitor applications. An additional update is made to the Bayeswave integration to prevent an error if results are not yet available when reporting to CBCFlow.

Why?

This was primarily created to allow CBCFlow to access asimov data, and asimov to access CBCFlow data without the need to integrate CBCFlow code directly into the codebase.

When?

These changes are scheduled to be included in asimov 0.5 on the current release roadmap. The contents of this MR will be merged into the v0.5-preview branch, and made available in pre-releases where feasible to simplify testing and review.

Suggested testing

This MR does not provide any new unit tests at present, and none are currently proposed. Since this MR was created for use with CBCFlow, I'd recommend that we test that an end-to-end testing run with as many pipelines as feasible is run with the CBCFlow plugin enabled, and that the information is written-out as expected.

In order to allow asimov to communicate with CBCFlow, first CBCFlow must be installed in the same environment as asimov.

This allows CBCFlow to be accessed via the asimov apply interface, e.g. by running asimov apply -p cbcflow -e GW150914, however it's necessary to make sure that the interface is set up properly in the asimov.conf file so that asimov can find the CBCFlow library.

Asimov can also write to CBCFlow by specifically adding the CBCFlow hook to the ledger. This can be done with a configuration blueprint, for example:

kind: configuration
hooks:
   postmonitor: 
     cbcflow:
       library location: /path/to/library
       schema section: ParameterEstimation

Checklist

  • Code changes
  • Unit tests
  • Documentation
  • Release scheduling
Edited by Daniel Williams

Merge request reports