02 July 2019
Review-ready items in the review table
- Is the information sufficient?
- Can any of these get a tick?
- Not-ready results are being worked
Known event comparisons (Isobel)
Charlie, Colm, Greg, Isobel, Matt, Moritz, Paul, Shanika, Simon, Sylvia, Virginia
Zero Likelihood tests
Zero Likelihood test
- Part of nightly unit tests
- Makes sure that nothing gets stuck weird modes
- Should return the prior https://git.ligo.org/lscsoft/bilby_pipe/wikis/O3-review/zero-likelihood
Some people can't see the test initially, Greg had to add them as Developer to bilby
Matt: Is there a reason to change the prior in the test?
Greg: Works either way, we can always reweight the posterior
Matt: We might want to add the prior cdf test for all priors
Greg: Good idea
Matt: Ticks off test as done
Greg: Nothing to review, informational purpose
Matt: Simon and I have to go through them
Greg: They're all the same, except for chirp_mass due to different ROQ bases, and luminosity_distance due to different SNR distribution.
Matt: The files look fine. a_1/2 don't go beyond 0.8 due to ROQ basis?
Matt: Maximum spin is 0.1 in 128 s example?
Greg: That's some old setting, that we might need to discuss.
Matt: We might want to increase this to 0.8
ACTION: Greg should change the upper limit of the spin prior in the 128 s file to 0.8
Simon: Are prior files specific to IMRPhenomD?
Greg: You can use any other waveform in LAL if they make sense. We focus on IMRPhenomPv2 in the review. We would need a waveform expert to help us define priors.
Simon: Reason for the question is because I had trouble running with different waveforms.
Colm: We should add a test to run with aligned spins
Sylvia: Can we return likelihood -infinity if we hit forbidden parts of the parameter space?
Matt: We can just use IMRPhenomPv2 for review. Can we compare waveforms between LALSuite and bilby?
Greg: Yes, Nikhil is working on it. He uses PyCBC. Is that sufficient? Who should we ask for LALSuite?
Matt: PyCBC is fine, but there should be a way to call LALInference in a way to output the waveform.
Greg: Sylvia open an issue for that.
Fiducial BBH tests
Greg: Injection in H1L1
Greg: Walk through 1D pdf/cdf page, there is a bug in the priors that has since been fixed
Greg: @reviewers, is this what you want to see?
Matt: This does look like it is.
ACTION: Make sure cdfs are loading on the webpage
Greg: review table - fiducial BBH 128 s is equivalent to a LALInference run, let's skip fiducial runs for now
Greg: PP-tests, there should be no bias
Greg: Need to be careful about SNR distributions
Greg: Sampling times are included as well because this is the best place to look at them
Matt: looks good, similar to LALInference review, Simon and I have to go through it in our own time eventually.
Greg: For 128 s we used low spin prior, we have to check this
Greg: We did this for 0.5.2, we want to redo this for newer version
Greg: back to the table
Greg: There are other ongoing efforts, do calibration checks sound reasonable
15 D Gaussian
Matt: I might have pointed you to an older page, plots look correct and . I'll have to look for a more up to date page.
Moritz/Matt: We should use bilby default settings
Moritz: there are some technical issues with running this on a bimodal distribution.
Colm: It would be nice to include the sampler's quoted uncertainty in the evidence in your summary.
Known event comparison
Isobel: Uses IMRPhenomPv2, posteriors have been downsampled, comparison plots look good
Greg: a_1/2 look slightly different, Carl said there was a waveform redefinition which might explain differences.
Isobel: phi_jl looks significantly different, could be a pi/2 shift
Colm: Why is this not just a shift
Greg: Might be something more than just a shift.
Colm: We might ask somebody to convert them back to LALInference O2 definitions or vice versa
Matt: Either would be good. Analytic waveform comparison should be a good check. Maybe this is a problem with LALInferenceNest, it might not show up in LALInferenceMCMC because there were similar issues.
Greg: We could re-run LALInference using the O3 branch. This should not have the spin issues in there. We could rerun it of GW150914. @Reviewers, what do you think.
Matt: GW150914, GW151226, one of the events with support for positive chi_eff
Colm: Boxing day event gives us positive chi_eff
ACTION: Re-run LALInference with O3 evemts
Simon: If there is an easy way to convert, we should try, otherwise re-run LALInference with new, O3, convention. O1/O2 might have to be re-run with new convention anyway.
Greg: Let's ask Carl if there is an easy fix.
ACTION: Let's talk to Carl
Charlie: Phi_12 also seems to be different. They have different ranges.
Colm/Greg: There might be a difference
Matt: There are some difference between LALInferenceNest, LALInferenceMCMC if you look at a1/a2. Might be worth running with LALInferenceMCMC on GW150914
Virginia: Are bilby devs going to the LVC meeting. I might want to run a bilby tutorial, can bilby devs support?
Greg, Moritz: Yes to both
Colm: Yes to being at LVC
Next meeting: Same time next week.
<08:01:52> "Greg Ashton [h]": https://git.ligo.org/lscsoft/bilby_pipe/wikis/O3-review/minutes/190702
<08:02:31> "Paul Lasky": Moritz has agreed to take minutes as well. Thanks!
<08:07:19> "Sylvia Biscoveanu": I can't see the latest run
<08:07:25> "Simon Stevenson": I get 404 not found
<08:07:26> "Charlie Hoy": I have a 404 when I click on this link: https://git.ligo.org/lscsoft/bilby/pipeline_schedules
<08:07:42> "Colm Talbot": I can see it
<08:07:46> "Paul Lasky": I'll addd you to the wiki while we talk
<08:07:59> "Sylvia Biscoveanu": Being a reporter is not enough
<08:08:03> "Matt Pitkin [h]": Nope, I can't see it either
<08:08:12> "Isobel Romero-Shaw": I can see it
<08:08:18> "Virginia d'Emilio [s]": I can't see it either
<08:08:21> "Sylvia Biscoveanu": So that means developer access at minimum
<08:09:01> "Sylvia Biscoveanu": Can you add me too please?
<08:09:14> "Charlie Hoy": I still get 404
<08:09:15> "Paul Lasky": Ive added Virginia
<08:09:26> "Matt Pitkin [h]": Still 404 here too
<08:09:33> "Virginia d'Emilio [s]": Thank you!
<08:09:33> "Simon Stevenson": Yeah
<08:09:34> "Charlie Hoy": ohh no its fine
<08:09:37> "Charlie Hoy": takes a while to update
<08:10:19> "Shanika Galaudage": me also, cannot see it
<08:10:52> "Paul Lasky": I've just updated a bunch of people
<08:11:01> "Paul Lasky": please hit refresh and let us know....
<08:11:12> "Shanika Galaudage": works now, cheers!
<08:13:56> "Paul Lasky": woohoo!!!
<08:14:05> "Simon Stevenson": I'm good
<08:20:08> "Simon Stevenson": hand up
<08:22:11> "Colm Talbot": Hand up
<08:23:20> "Sylvia Biscoveanu": Hand up
<08:23:27> "Simon Stevenson": yup it was
<08:23:37> "Simon Stevenson": I'd appreciate that
<08:24:47> "Sylvia Biscoveanu": Not yet
<08:25:00> "Moritz Hübner": Sorry, Sylvia was cut off for me, can someone summarize the question?
<08:26:15> "Matt Pitkin [h]": hand up
<08:29:34> "Sylvia Biscoveanu": There's an issue, not a MR
<08:29:41> "Charlie Hoy": You can easily use pesummary to display results (pesummary can handle pycbc result files)
<08:32:15> "Paul Lasky": I have to run off. I'll catch up with the Monash folk later today. Thanks all!!
<08:34:50> "Moritz Hübner": ok
<08:35:14> "Charlie Hoy": The cdfs can be seen on pesummary if you want to look at them
<08:38:32> "Sylvia Biscoveanu": Need to run, bye!
<08:48:24> "Matt Pitkin [h]": yep
<08:48:43> "Matt Pitkin [h]": got them
<08:48:44> "Greg Ashton [h]": I've opened them
<08:48:45> "Simon Stevenson": got it
<08:49:56> "Isobel Romero-Shaw": https://ldas-jobs.ligo.caltech.edu/~isobel.romero-shaw/known_events/GW150914/GW150914.prior
<08:54:04> "Simon Stevenson": possibly also mass ratio
<08:55:28> "Colm Talbot": Hand up
<08:55:51> "Greg Ashton [h]": Link to the git issue: bilby#385 (closed)
<08:58:09> "Simon Stevenson": I guess we want what will be used going forward
<08:58:44> "Charlie Hoy": hand up after this discussion
<09:02:14> "Colm Talbot": Boxing day gives us that
<09:06:05> "Charlie Hoy": about events
<09:06:25> "Virginia d'Emilio [s]": hand up after Charlie very quickly
<09:06:48> "Colm Talbot": I have no idea how these parameters are constructed...
<09:07:24> "Charlie Hoy": it is in the bilby .dat files?
<09:07:32> "Matt Pitkin [h]": Just for info, here's the lalinference_nest vs lalinference_mcmc comparison https://ldas-jobs.ligo.caltech.edu/~carl-johan.haster/LALInference_review/ProperMaster_9april19_comparisons/fiducialBBH/ (linked from https://git.ligo.org/pe/lalinference-review/issues/1) - as Greg suspected the differences between the two aren't as pronounced as those between lalinference_nest and bilby
<09:08:32> "Charlie Hoy": ok thanks!
<09:08:42> "Matt Pitkin [h]": Actually I take that back https://ldas-jobs.ligo.caltech.edu/~carl-johan.haster/LALInference_review/ProperMaster_9april19_comparisons/fiducialBBH/a2_90.png show's a reasonable difference
<09:09:27> "Charlie Hoy": @colm phi1 and phi2 are in this .dat file: https://git.ligo.org/gregory.ashton/bilby_pe_event_samples/blob/master/GW150914/GW150914_dynesty_IMRPhenomPV2_reweighted_posterior_samples.dat
<09:10:23> "Virginia d'Emilio [s]": just one quick question
<09:10:48> "Colm Talbot": I am
<09:10:55> "Moritz Hübner": I'm going
<09:11:45> "Moritz Hübner": Sure
<09:11:53> "Virginia d'Emilio [s]": Awesome thank you
<09:13:24> "Isobel Romero-Shaw": thanks, bye!
<09:13:24> "Simon Stevenson": bye!
<09:13:25> "Virginia d'Emilio [s]": Bye!
<09:13:28> "Charlie Hoy": bye
<09:14:01> "Shanika Galaudage": bye