O3-review.md

@@ -35,7 +35,7 @@ To run all tests in the review, we have a [review script](https://git.ligo.org/l
 | Task         | Reviewer | Status  | Comments |
 | ------       | ------   | ------  | ------   |
 | [PP Tests](./O3-review/PP-tests)     | @simon-stevenson | reviewed | :heavy_check_mark: |
-| [known event comparison](./O3-review/known-events)| | 150914, 151226, 170814 passed, others ongoing | :construction_site: |
+| [known event comparison](./O3-review/known-events)| | 150914, 151226, 170814, 170817 passed (required by review), others ongoing :construction_site: (not required for review) | :heavy_check_mark: |
 | [fiducial BBH - high_mass](./O3-review/fiducial-BBH-high_mass) | @simon-stevenson | reviewed | :heavy_check_mark:       |
 | [fiducial BBH - 4s](./O3-review/fiducial-BBH-4s) | @simon-stevenson | reviewed |  :heavy_check_mark: |
 | [fiducial BBH - 128s](./O3-review/fiducial-BBH-128s) |  @simon-stevenson @matthew-pitkin | reviewed  |  :heavy_check_mark:         |
@@ -60,4 +60,22 @@ To run all tests in the review, we have a [review script](https://git.ligo.org/l
 
 # Final Review Statement
 
+The review of the parameter-estimation code Bilby has been completed. Full details and minutes of meetings are recorded at https://git.ligo.org/lscsoft/bilby_pipe/wikis/O3-review. Here we provide a summary of the salient points of the review.
 
+Review items pertaining to simulations
+
+1. Runs on fiducial simulated events in Gaussian noise for IMRPhenomPv2
+1. PP tests demonstrating the Dynesty sampler works without bias for IMRPhenomPv2 with spins up to 0.8 across a range of typical masses
+1. Various sanity checks, including waveform outputs, 15-dimensional Gaussians, and confirming the sampler returns the prior when the likelihood is uninformative.
+
+Review items pertaining to like-for-like comparison of GW150914, GW151226, GW170814, and GW170817:
+
+1. Tests of the the agreement in posteriors between Bilby and LALInference. The criteria for passing this test is that the difference between the LALInference Nest and MCMC samplers is comparable to the difference between Bilby with either of these samplers.
+1. Differences are noted for the spin-angles,which is due to the change in the LALSuite waveforms interface between O2 and O3. This was confirmed by looking at S190521r and by point 5 below.
+1. For GW170817, the comparison was made using a custom-built ROQ on the fixed-sky low-spin case, future analysis of the high-spin case is planned.
+1. A like-for-like comparison of IMPhenomD, IMRPhenomC, and SEOBNRv3_ROM on GW150914, demonstrating that Bilby is comparable across waveforms.
+1. A comparison of both Bilby and LALInference run using IMRPhenomPv2 with and without calibration on GW150914, demonstrating similar changes in the source-parameter posteriors.
+
+Note: only four known events were selected for closer examination, the other seven events from GWTC1 were not inspected. A PE-group publication is planned producing results for all events in GWTC1. 
+
+Matt Pitkin (@matthew-pitkin) & Simon Stevenson (@simon-stevenson)