O3a-Spin-Quadrupole-BH-mimicker-test-results/o3a-review-report.md

@@ -17,7 +17,7 @@
 |7.|[GW190707a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190707q)<br> person responsible: Krishnendu| Prod0 OK. <br>Anuradha: OK.| Archisman: No mtot railing. SIQM very light railing. **SIGNED OFF!** <br><br> Anuradha: BBH Comparison_All.html page does not open for me. Please fix. Otherwise looks good. <br><br> <br> Krishnendu: BayesCompos added for all the events. <br> comparison_all page works for me <br><br> Archisman: sym 4 chains OK. pos 4 chains OK. <br><br> Anuradha: LogL is a bit different in BBH case. **SIGNED OFF!**| | **SIGNED OFF!**| Yes, reruns finished and results are uploaded|
 |8.|[GW190708a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190708ap)<br> person responsible:Saleem | Prod0/5 OK. <br>Anuradha: OK.|Anuradha: LogL is a bit different in BBH case. Otherwise looks good. **SIGNED OFF!**<br><br> Archisman: No mtot railing. SIQM OK. sym 4 chains OK. pos 3 chains OK. **SIGNED OFF!** | | **SIGNED OFF!**|Yes, reruns finished and results are uploaded for all runs. Ready to be reviewed|
 |9.|[GW190720a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190720a)<br> person responsible: Krishnendu| Prod0 seems OK. Should we check with PE if L1 flow is finally 20 and not 40 Hz? <br><br> Krishnendu: fixed. [reply PE channel](https://chat.ligo.org/ligo/channels/parameter-estimation/1bypq6rnmpngtjetm8ssw17t3e) <br><br> Archisman: OK. <br>Anuradha: OK.|Anuradha: <br> why t0, cos(alpha), azimuth are different in all the 3 cases from prod run? <br> BBH: The **ACF** for tc, t0, theta_JN, cos(alpha), azimuth, delta, DL, iota look suspicious. <br> pos: The **ACF** for azimuth look suspicious. <br> The **ACF** for cos(alpha), azimuth, a1, cos(theta1) look suspicious. <br><br> <br> Krishnendu: [BH comparison pages](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_prod_c01/psd_rerun/cbcBayesCompos/comp_bh/) looks fine for me, for example [theta_JN](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/psd_rerun/comp_PE_Prod0/bh_bh/plots/combined_1d_posterior_theta_jn.png) <br> Not sure if we need reruns for ACF issues <br><br> Archisman: So mtot railing. SIQM OK. sym 4 chains; chain 4 may be problematic. pos only 1 chain! **SIGNED OFF** <br><br>Anuradha: The difference in t0, alpha, azimuth is due to the different choice of the first detector. **SIGNED OFF!**| | **SIGNED OFF!**|Yes, reruns finished and results are uploaded|
-|10.|[GW190728a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190728q)<br> person responsible: Saleem| Prod0 OK. <br>Anuradha: OK.| Anuradha: LogL is a bit different in BBH case. Otherwise looks good. <br><br> Archisman: sym mtot bimodal; no railing. SIQM only negative; pos consistent with sym. sym 2 chains; sampling non-identical regions. pos only 1 chain. **NOT OK** <br><br> Archisman (2020-Aug-14): No mtot railing, but sharp fall-off at lower end. pos OK. **Chirp mass bimodal for sym.** The bad samples are coming from a single chain, which can be removed. <br><br> Anuradha: Mc is bimodal but the WFs are sane for the secondary peak and this does not go away with stronger setting. Could be due to either correlation between dks and other parameters or du to noise to both. Saleem has done some investigations [here](https://git.ligo.org/muhammed.saleem/o3siqm-updates/-/blob/master/script/explore_mc_bimodality_injections.ipynb) which look okay. **SIGNED OFF**|The runs already have high specs like srate = 2048, nlive = 2048, maxmcmc = 10000, nparallel = 8. Chains broken due to calibration files removed. Resubmitted the dag (16/7/2020) which seems to be converging (as of 20/7/2020). Reruns started already to compensate in case the resumed runs didn't go well. ||Results from PSD reruns uploaded and reviewed. Waiting for results with more samples.|
+|10.|[GW190728a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190728q)<br> person responsible: Saleem| Prod0 OK. <br>Anuradha: OK.| Anuradha: LogL is a bit different in BBH case. Otherwise looks good. <br><br> Archisman: sym mtot bimodal; no railing. SIQM only negative; pos consistent with sym. sym 2 chains; sampling non-identical regions. pos only 1 chain. **NOT OK** <br><br> Archisman (2020-Aug-14): No mtot railing, but sharp fall-off at lower end. pos OK. **Chirp mass bimodal for sym.** The bad samples are coming from a single chain, which can be removed. <br><br> Anuradha: Mc is bimodal but the WFs are sane for the secondary peak and this does not go away with stronger setting. Could be due to either correlation between dks and other parameters or du to noise to both. Saleem has done some investigations [here](https://git.ligo.org/muhammed.saleem/o3siqm-updates/-/blob/master/script/explore_mc_bimodality_injections.ipynb) which look okay. **SIGNED OFF** <br><br> Archisman (2020-09-15): Chirp mass still bimodal for sym. however this feature seems robust across different runs. So OK. **SIGNED OFF**|The runs already have high specs like srate = 2048, nlive = 2048, maxmcmc = 10000, nparallel = 8. Chains broken due to calibration files removed. Resubmitted the dag (16/7/2020) which seems to be converging (as of 20/7/2020). Reruns started already to compensate in case the resumed runs didn't go well. ||Results from PSD reruns uploaded and reviewed. Waiting for results with more samples.|
 |11.|[GW190814a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190814bv) <br> person responsible: Krishnendu| Archisman: base config file info not available. <br><br> Krishnendu: fixed. <br><br> Archisman: Abhirup, could you please check the base config for this event? |Anuradha: Why t0, azimuth and cos(alpha) are different from Prod run in all the 3 cases? In 'result summary' ks_pos has the link of ks_sym, **please fix.** <br><br> Krishnendu: fixed. <br><br> Anuradha: In pos run, **ACF** for all the spin parameters, all the mass parameters, alpha, delta, tc, iota, azimuth look suspicious. <br><br> Archisman: PE comparison t0, cosalph, and azimuth are different. Mtot bimodal for pos. SIQM mostly neg. sym 4 chains (1 chain very short) OK. pos 2 chains sampling different regions. **NOT OK** |Reruns started ||Yes, reruns finished and results are uploaded|
 |12.|[GW190828a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190828j) <br> person responsible: Saleem| Archisman: Mass priors have been increased between Prod0 and Prod5, but no railing with SIQM settings. OK. <br>Anuradha: OK.|Anuradha: LogL is a bit different in BBH case. Which might be okay. **ACFs** for time, iota, ra, dec, and theta_JN are suspicious in the 3 runs. **SIGNED OFF!** <br><br> Archisman: No mtot railing. SIQM OK. sim 4 chains OK. pos 4 chains OK. **SIGNED OFF!**| | **SIGNED OFF!**|Yes, PSD reruns finished and  reviewed.|
 |13.|[GW190828b](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190828l) <br> person responsible: Saleem| Prod4 OK. **Configs and diffs not found!** <br> Saleem: **Link to configs and diffs have been fixed (16/7/20)**  <br>Anuradha: OK. <br><br> Archisman: OK.| Anuradha:<br>BBH: Awaiting results. <br> pos/sym: **ACF** for time, ra, dec, DL are suspicious. <br><br> Archisman: No mtot railing. SIQM samples mostly neg; pos consistent with sym. sym 4 chains OK; chain 2 may be a bit dirty. pos 4 chains OK; chains 2 & 3 may be a bit dirty. **SIGNED OFF!** <br>Anuradha: **SIGNED OFF!**|||Yes, Reruns finished for ks_pos and ks_sym whose results are ready to be reviewed. BBH runs failed and reruns started (14/7/2020)|