... | ... | @@ -4,28 +4,21 @@ This page exists to collect ideas for the development of asimov for the O4 analy |
|
|
|
|
|
### New web interface
|
|
|
|
|
|
In O3 asimov was able to generate web pages to show the ongoing progress of runs, but adding new runs required access to its CLI on the cluster which was a major bottleneck for setting up new runs.
|
|
|
One solution to this is to create a web interface which will allow communication directly with the asimov process, and allowing us to remove the dependence on gitlab.
|
|
|
In O3 asimov was able to generate web pages to show the ongoing progress of runs, but adding new runs required access to its CLI on the cluster which was a major bottleneck for setting up new runs. One solution to this is to create a web interface which will allow communication directly with the asimov process, and allowing us to remove the dependence on gitlab.
|
|
|
|
|
|
This is a major project, and will require development of a new database for storing the asimov ledger, back-end interface work, and front-end interface work.
|
|
|
However, it has the potential to substantially improve the process of setting-up a run, and making it easier for non-pipeline experts to rapidly produce results.
|
|
|
This is a major project, and will require development of a new database for storing the asimov ledger, back-end interface work, and front-end interface work. However, it has the potential to substantially improve the process of setting-up a run, and making it easier for non-pipeline experts to rapidly produce results.
|
|
|
|
|
|
### Improved error logging
|
|
|
|
|
|
A major problem we encountered in O3 was tracking down the sources of errors and runtime problems.
|
|
|
We need an easier way of accessing errors which arise within asimov, as well as collecting errors from the pipelines it is monitoring in a more efficient manner.
|
|
|
A major difficulty for this, at least naively, is a pipeline like RIFT which generates thousands of separate error files.
|
|
|
A major problem we encountered in O3 was tracking down the sources of errors and runtime problems. We need an easier way of accessing errors which arise within asimov, as well as collecting errors from the pipelines it is monitoring in a more efficient manner. A major difficulty for this, at least naively, is a pipeline like RIFT which generates thousands of separate error files.
|
|
|
|
|
|
### Condor improvements
|
|
|
|
|
|
The process for tracking jobs progress in condor was hacky and unreliable in O3. This problem was partially solved when producing the post-processing pipeline, and the code from that should be merged back in to the main project.
|
|
|
In addition, to improve stability we should develop a "heartbeat" process for asimov which queries the condor scheduler at regular intervals (e.g. 15 minutes) in order to reduce load on condor, provide more detailed statistics about e.g. run duration and stability, and allow us to retire the current solution which uses a cronjob.
|
|
|
The process for tracking jobs progress in condor was hacky and unreliable in O3. This problem was partially solved when producing the post-processing pipeline, and the code from that should be merged back in to the main project. In addition, to improve stability we should develop a "heartbeat" process for asimov which queries the condor scheduler at regular intervals (e.g. 15 minutes) in order to reduce load on condor, provide more detailed statistics about e.g. run duration and stability, and allow us to retire the current solution which uses a cronjob.
|
|
|
|
|
|
### "Offline" running
|
|
|
|
|
|
In O3 we used asimov only for production analyses, however if we wish to allow it to be used for exploratory analyses it would be useful to allow any user to set-up a run.
|
|
|
To allow this we could develop an interface whereby an analysis can be "cloned", with configurations and results easily pulled and packaged from the main asimov process, edited, and runs set up.
|
|
|
I'd envisage this working very similarly to how the O3 runs worked, but it should be possible to easily push results back to a main asimov server so that these results can be easily found by the collaboration.
|
|
|
In O3 we used asimov only for production analyses, however if we wish to allow it to be used for exploratory analyses it would be useful to allow any user to set-up a run. To allow this we could develop an interface whereby an analysis can be "cloned", with configurations and results easily pulled and packaged from the main asimov process, edited, and runs set up. I'd envisage this working very similarly to how the O3 runs worked, but it should be possible to easily push results back to a main asimov server so that these results can be easily found by the collaboration.
|
|
|
|
|
|
### Additional pipelines
|
|
|
|
... | ... | @@ -33,8 +26,7 @@ It would be good to add some additional pipelines for O4, e.g. pycbc (work under |
|
|
|
|
|
### Detector characterisation
|
|
|
|
|
|
In O3 some detchar information was added to asimov by Derek, in a fairly manual process.
|
|
|
It would be good to investigate if we can improve this, either by interfacing with whatever automation approach detchar takes, or by providing automation to them.
|
|
|
In O3 some detchar information was added to asimov by Derek, in a fairly manual process. It would be good to investigate if we can improve this, either by interfacing with whatever automation approach detchar takes, or by providing automation to them.
|
|
|
|
|
|
### Improvements to prior specification
|
|
|
|
... | ... | @@ -44,52 +36,55 @@ At present the syntax for specifying priors is a mess and created in an ad-hoc m |
|
|
|
|
|
### v0.4.0
|
|
|
|
|
|
| *Release target* | January 2023 |
|
|
|
| *Milestone* | https://git.ligo.org/groups/asimov/-/milestones/1 |
|
|
|
| *Preview Branch* | [infrastructure-updates](https://git.ligo.org/asimov/asimov/-/tree/infrastructure-updates) |
|
|
|
| *Preview release*| v0.4.0b5 |
|
|
|
| *Review status* | Under review |
|
|
|
| | |
|
|
|
|--|--|
|
|
|
| _Release target_ | January 2023 |
|
|
|
| _Milestone_ | https://git.ligo.org/groups/asimov/-/milestones/1 |
|
|
|
| _Preview Branch_ | [infrastructure-updates](https://git.ligo.org/asimov/asimov/-/tree/infrastructure-updates) |
|
|
|
| _Preview release_| v0.4.0b5 |
|
|
|
| _Review status_ | Under review |
|
|
|
|
|
|
This version is intended to support the CBC-PE group's sampler review exercise, and should be feature-complete for this purpose.
|
|
|
This version overhauls the codebase from the O3 analyses, aiming to provide more stability and a simpler interface for analysts.
|
|
|
As of 20 January 2023 primary development is now complete, and undergoing beta testing. Once review and testing have been completed we will release v0.4.0.
|
|
|
This version is intended to support the CBC-PE group's sampler review exercise, and should be feature-complete for this purpose. This version overhauls the codebase from the O3 analyses, aiming to provide more stability and a simpler interface for analysts. As of 20 January 2023 primary development is now complete, and undergoing beta testing. Once review and testing have been completed we will release v0.4.0.
|
|
|
|
|
|
### v0.5.0
|
|
|
|
|
|
|*Release target* | February 2023 |
|
|
|
|*Milestone* | https://git.ligo.org/groups/asimov/-/milestones/3 |
|
|
|
|*Preview Branch* | |
|
|
|
|*Preview release* | |
|
|
|
|*Review status* | Pending |
|
|
|
| | |
|
|
|
|--|--|
|
|
|
|_Release target_ | February 2023 |
|
|
|
|_Milestone_ | https://git.ligo.org/groups/asimov/-/milestones/3 |
|
|
|
|_Preview Branch_ | |
|
|
|
|_Preview release_ | |
|
|
|
|_Review status_ | Pending |
|
|
|
|
|
|
New features:
|
|
|
|
|
|
| Feature | MR | Status | Review |
|
|
|
|---------|-------|--------|--------|
|
|
|
| Bilby OSG Integration | | Waiting | |
|
|
|
| RIFT O4 Integration | !51 | Ongoing | |
|
|
|
|
|
|
|---------|----|--------|--------|
|
|
|
| Bilby OSG Integration | | Waiting | |
|
|
|
| RIFT O4 Integration | !51 | Ongoing | |
|
|
|
|
|
|
### v0.6.0
|
|
|
|
|
|
| *Release target* | Late March 2023 |
|
|
|
| *Milestone* | https://git.ligo.org/groups/asimov/-/milestones/2 |
|
|
|
| *Preview Branch* | |
|
|
|
| *Preview release* | |
|
|
|
| *Review status* | Pending |
|
|
|
| | |
|
|
|
|--|--|
|
|
|
| _Release target_ | Late March 2023 |
|
|
|
| _Milestone_ | https://git.ligo.org/groups/asimov/-/milestones/2 |
|
|
|
| _Preview Branch_ | |
|
|
|
| _Preview release_ | |
|
|
|
| _Review status_ | Pending |
|
|
|
|
|
|
Planned New features:
|
|
|
|
|
|
| Feature | MR | Status | Review |
|
|
|
|---------|-------|--------|--------|
|
|
|
| Multi-event analyses | !46 | Ongoing | :construction: |
|
|
|
| Integration with online PE | | Waiting | :construction: |
|
|
|
| Integration with CBC Workflow | | To Do | :construction: |
|
|
|
| Overhaul of Postprocessing | | To Do | :construction: |
|
|
|
| Support for multibanded likelihoods | | To Do | :construction: |
|
|
|
| Support for ROQ Likelihoods | | To Do | :construction: |
|
|
|
| Improved calibration envelope handling | | To Do | :construction: |
|
|
|
| Further prior interface improvements | | To Do | :construction: |
|
|
|
|---------|----|--------|--------|
|
|
|
| Multi-event analyses | !46 | Ongoing | :construction: |
|
|
|
| Integration with online PE | | Waiting | :construction: |
|
|
|
| Integration with CBC Workflow | | To Do | :construction: |
|
|
|
| Overhaul of Postprocessing | | To Do | :construction: |
|
|
|
| Support for multibanded likelihoods | | To Do | :construction: |
|
|
|
| Support for ROQ Likelihoods | | To Do | :construction: |
|
|
|
| Improved calibration envelope handling | | To Do | :construction: |
|
|
|
| Further prior interface improvements | | To Do | :construction: |
|
|
|
|
|
|
## Priorities
|
|
|
|
... | ... | @@ -104,7 +99,8 @@ My current suggestion of the prioritisation of development tasks is as follows, |
|
|
|
|
|
## Available work
|
|
|
|
|
|
| Person | FTE | Priority |
|
|
|
| ------ | --- | -------- |
|
|
|
| daniel.williams | 0.2 | Infrastructure; web interface & condor. Dev lead |
|
|
|
| richard.udall | 0.2 | condor, priors, offline/cloned | |
|
|
\ No newline at end of file |
|
|
| Person | FTE | Priority |
|
|
|
|--------|-----|----------|
|
|
|
| daniel.williams | 0.2 | Infrastructure; web interface & condor. Dev lead |
|
|
|
| richard.udall | 0.2 | condor, priors, offline/cloned |
|
|
|
|