Reproducibility between LALInference and bilby

For review and comparison, we want to have documented how one runs identical analysis with LALInference and Bilby

This is something the Monash group discussed in person yesterday. We're working towards having bilby_pipe up and running with a reproducible workflow and data provenance (see !273 (merged) and #236 for example). Hopefully this will be up and running in the next few days, at which point we'll freeze a new version of the code (e.g., 0.4) and can start doing these comparisons. There are one or two extra MRs we'd like to get in before doing this (e.g. !257 (merged)).

Hi @paul-lasky, I've looked at bilby_pipe and I think your work is great! I'm looking forward to see the finished version of the code and run it myself. I've been running a few bilby tutorials and I am writing an "advanced" tutorial for GW150914. The idea is to explicitly write out and describe the various option for the PE code (e.g. sampling_frequency, nlive etc) so that they can be set equal to a lalinference ini file. It will be interesting to see how the posterior distributions look for lalinference and bilby having used the same settings.

@virginia.demilio great to hear! In case you are keen, I've just finished updating bilby_pipe and it is ready for people to start installing and trying it out (for GW150914 there is an example ini file). I'm working on the documentation, but more hands are needed for testing (feel free to chuck issues on the issue-tracker).

EDIT: There is a doc page which may be of interest https://lscsoft.docs.ligo.org/bilby_pipe/running.html

Hi @gregory.ashton, awesome! I’ll install it and have a go as soon as possible. Thank you for the doc link as well.

changed milestone to %Future

added Testing label

marked this issue as related to #228 (closed)

added 1 deleted label and removed Documentation label

changed milestone to %1.0.0

added 1 deleted label

mentioned in issue #311 (closed)

assigned to @simon-stevenson

I've created a wiki page with some of the work I've done on reproducing GW150914. Please take a look here https://git.ligo.org/lscsoft/bilby_pipe/wikis/GW150914_Virginia_runs. On the wiki there are also links to the actual data from bilby and lalinference runs.

Closing this as this will be done as part of the review (see https://git.ligo.org/lscsoft/bilby_pipe/wikis/O3-review)

closed

unassigned @simon-stevenson and @virginia.demilio