Skip to content

WIP: Update the analysis interface

What?

This merge request implements a considerable refactoring of analyses and how they're handled by asimov. In versions prior to v0.6.0 asimov referred to all analyses as "productions", and these were invariably runs on a single event by a single pipeline with simple dependencies. This merge request retains this form of production, which has been renamed a SimpleAnalysis, and adds two more.

  • SubjectAnalysis, which is an analysis which can work with results of multiple SimpleAnalysis jobs, and can be automatically rerun if the set of SimpleAnalysis jobs has changed upstream and new results have been produced.
  • ProjectAnalysis, which is an analysis which can work with results from multiple subjects (events) and multiple analyses from each subject. These have been designed to allow e.g. populations analyses which often require multiple events' analyses to be combined.

We have also introduced a new syntax for specifying the requirements of an analysis. In previous versions of asimov these were provided using the needs section of the analysis blueprint, e.g.

needs: 
 - Prod1
 - Prod3

where the name of each analysis was explicitly required. This MR retains this functionality, while introducing the potential to specify a characteristic for a run, which will allow asimov to automatically use the appropriate productions at the time the job is run. For example, all runs created with the bayeswave pipeline can be specified as needs: [pipeline:bayeswave], or all runs which are reviewedcan be specified with needs: [review: approved]. In the case of the needs keyword the union of all productions which meet are currently returned, but if used with the analyses keyword (for ProjectAnalysis jobs) the intersection is returned. We will continue to develop this before completing the MR in order to have a uniform and more flexible approach.

Who?

This feature was developed by Daniel Williams (daniel.williams@ligo.org) at the University of Glasgow.

Why?

This feature is designed to make running complex analyses which require PE results easier.

How?

This feature does not introduce any new dependencies.

Timeline

This MR has been scheduled for inclusion in v0.6.0 of asimov.

To do list

  • Unit tests
  • Code completion
  • Code review
  • Documentation
  • Release scheduling
Edited by Daniel Williams

Merge request reports