Skip to content

Changes

Page history
Update o3a review report authored by NV Krishnendu's avatar NV Krishnendu
Show whitespace changes
Inline Side-by-side
O3a-Spin-Quadrupole-BH-mimicker-test-results/o3a-review-report.md
View page @ 93731733
......@@ -8,18 +8,18 @@
|No.| NAME | Comments on config ini file|Comments on results from the reviewer | Reviewee comments |Final status|PSD issue affected or not|
|---| ---- | ---- | ---- | --- |--- |----|
|1. |[GW190408a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190408a)<br> person responsible: Krishnendu| Prod0/5 OK. | Archisman: No mtot railing. SIQM light railing against prior which is an usual feature. **SIGNED OFF!** <br><br> Anuradha: **SIGNED OFF!**<br><br> Archisman: sym 3 chains OK. pos 4 chains OK. | ||Yes, reruns finished and results are uploaded|
|1. |[GW190408a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190408a)<br> person responsible: Krishnendu| Prod0/5 OK. | Archisman: No mtot railing. SIQM light railing against prior which is an usual feature. **SIGNED OFF!** <br><br> Anuradha: **SIGNED OFF!**<br><br> Archisman: sym 3 chains OK. pos 4 chains OK. | | **SIGNED OFF!**|Yes, reruns finished and results are uploaded|
|2. |[GW190412m](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190412m)<br> person responsible: Krishnendu| Prod0 OK. | Archisman: No mtot railing. SIQM **sym and pos result pages appear to have been swapped. <br><br> <br> Krishnendu: fixed ** <br><br> Anuradha: sym and pos comparison pages are not opening, **please fix**. dkappas_pos plot is **not provided**. In sym run, **ACF** for m1, m2, mtot, eta, iota, theta_JN, a, theta1, s1z, chi_eff look suspicious.<br><br>Krishnendu: dkappas_pos plot is **not provided**: please see the cbcBayesPostProc page linked from the event page. From example, [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190412m/pv2_c01_prod/psd_rerun/ks_pos/1239082262.17-0/V1H1L1/posplots.html) <br> ACF issues affected or not? <br><br> Archisman: sym 4 chains; chain 3 samples below q=0.2. pos 3 chains; not very identical for some params. |||Yes, reruns finished and results are uploaded|
|3. |[GW190425z](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190425z)<br> person responsible: Krishnendu| || | event removed from paper |Not affcted, RA from the prvious runs|
|4.|[GW190512b](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190512at)<br> person responsible: Krishnendu| Prod0 OK. | Archisman: No mtot railing; masses bimodal (particularly for sym). SIQM no neg railing; pos unconstrained. **SIGNED OFF!** <br><br> Anuradha: <br> BBH: tc, t0, \rho_ML, logL, mtot, cos(alpha), Mchirp are different. Do we understand why? <br> sym/pos: Why azimuth, t0, cos(alpha) different? <br> In sym run, the **ACF** for Mc, m2, mtot, a, S1z, S2z, chi_eff, chi_p look suspicious. <br><br>Krishnendu: Differences are expected especially on masses as the total mass prior is different from the PE-Prod run <br> ACF issues affected or not? <br><br> Archisman: sym only 1 chain! pos 4 chains OK. <br><br>Anuradha: The difference in t0, alpha, azimuth is due to the different choice of the first detector. The difference in mass is due different prior choice. **SIGNED OFF!**| ||Yes, reruns finished and results are uploaded|
|5.|[GW190521r](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190521r) <br> person responsible: Saleem| Prod0 OK. |Anuradha: looks good. **SIGNED OFF!**<br><br> Archisman: No mtot railing. SIQM OK. sym 4 chains OK. pos 4 chains OK. **SIGNED OFF!** | |Yes, reruns finished and results are uploaded |
|3. |[GW190425z](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190425z)<br> person responsible: Krishnendu| || | **event removed from paper** |Not affcted, RA from the prvious runs|
|4.|[GW190512b](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190512at)<br> person responsible: Krishnendu| Prod0 OK. | Archisman: No mtot railing; masses bimodal (particularly for sym). SIQM no neg railing; pos unconstrained. **SIGNED OFF!** <br><br> Anuradha: <br> BBH: tc, t0, \rho_ML, logL, mtot, cos(alpha), Mchirp are different. Do we understand why? <br> sym/pos: Why azimuth, t0, cos(alpha) different? <br> In sym run, the **ACF** for Mc, m2, mtot, a, S1z, S2z, chi_eff, chi_p look suspicious. <br><br>Krishnendu: Differences are expected especially on masses as the total mass prior is different from the PE-Prod run <br> ACF issues affected or not? <br><br> Archisman: sym only 1 chain! pos 4 chains OK. <br><br>Anuradha: The difference in t0, alpha, azimuth is due to the different choice of the first detector. The difference in mass is due different prior choice. **SIGNED OFF!**| | **SIGNED OFF!**|Yes, reruns finished and results are uploaded|
|5.|[GW190521r](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190521r) <br> person responsible: Saleem| Prod0 OK. |Anuradha: looks good. **SIGNED OFF!**<br><br> Archisman: No mtot railing. SIQM OK. sym 4 chains OK. pos 4 chains OK. **SIGNED OFF!** | **SIGNED OFF!**|Yes, reruns finished and results are uploaded |
|6.|[GW190630a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190630ag)<br> person responsible: Krishnendu| Prod0/5 OK. | Archisman: No mtot railing. SIQM sym kink at zero; neg some railing; pos no railing; OK. **pos results currently linking to sym.** <br><br> <br> Krishnendu: fixed <br><br>Anuradha: The Comparison_All.html pages do not open for me. No dkappaS plots. <br><br> <br> Krishnendu: Please see the updated pages. <br><br> Archisman: no mtot railing. sym 4 chains OK. pos 4 chains OK. **Why are there no SIQM plots?** <br><br> Anuradha: Looks good. **SIGNED OFF**| || Yes, reruns finished and results are uploaded |
|7.|[GW190707a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190707q)<br> person responsible: Krishnendu| Prod0 OK. | Archisman: No mtot railing. SIQM very light railing. **SIGNED OFF!** <br><br> Anuradha: BBH Comparison_All.html page does not open for me. Please fix. Otherwise looks good. <br><br> <br> Krishnendu: BayesCompos added for all the events. <br> comparison_all page works for me <br><br> Archisman: sym 4 chains OK. pos 4 chains OK. <br><br> Anuradha: LogL is a bit different in BBH case. **SIGNED OFF!**| || Yes, reruns finished and results are uploaded|
|8.|[GW190708a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190708ap)<br> person responsible:Saleem | Prod0/5 OK. |Anuradha: LogL is a bit different in BBH case. Otherwise looks good. **SIGNED OFF!**<br><br> Archisman: No mtot railing. SIQM OK. sym 4 chains OK. pos 3 chains OK. **SIGNED OFF!** | ||Yes, reruns finished and results are uploaded for all runs. Ready to be reviewed|
|9.|[GW190720a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190720a)<br> person responsible: Krishnendu| Prod0 seems OK. Should we check with PE if L1 flow is finally 20 and not 40 Hz? <br><br> Krishnendu: fixed. [reply PE channel](https://chat.ligo.org/ligo/channels/parameter-estimation/1bypq6rnmpngtjetm8ssw17t3e) <br><br> Archisman: OK. |Anuradha: <br> why t0, cos(alpha), azimuth are different in all the 3 cases from prod run? <br> BBH: The **ACF** for tc, t0, theta_JN, cos(alpha), azimuth, delta, DL, iota look suspicious. <br> pos: The **ACF** for azimuth look suspicious. <br> The **ACF** for cos(alpha), azimuth, a1, cos(theta1) look suspicious. <br><br> <br> Krishnendu: [BH comparison pages](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_prod_c01/psd_rerun/cbcBayesCompos/comp_bh/) looks fine for me, for example [theta_JN](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/psd_rerun/comp_PE_Prod0/bh_bh/plots/combined_1d_posterior_theta_jn.png) <br> Not sure if we need reruns for ACF issues <br><br> Archisman: So mtot railing. SIQM OK. sym 4 chains; chain 4 may be problematic. pos only 1 chain! **SIGNED OFF** <br><br>Anuradha: The difference in t0, alpha, azimuth is due to the different choice of the first detector. **SIGNED OFF!**| ||Yes, reruns finished and results are uploaded|
|7.|[GW190707a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190707q)<br> person responsible: Krishnendu| Prod0 OK. | Archisman: No mtot railing. SIQM very light railing. **SIGNED OFF!** <br><br> Anuradha: BBH Comparison_All.html page does not open for me. Please fix. Otherwise looks good. <br><br> <br> Krishnendu: BayesCompos added for all the events. <br> comparison_all page works for me <br><br> Archisman: sym 4 chains OK. pos 4 chains OK. <br><br> Anuradha: LogL is a bit different in BBH case. **SIGNED OFF!**| | **SIGNED OFF!**| Yes, reruns finished and results are uploaded|
|8.|[GW190708a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190708ap)<br> person responsible:Saleem | Prod0/5 OK. |Anuradha: LogL is a bit different in BBH case. Otherwise looks good. **SIGNED OFF!**<br><br> Archisman: No mtot railing. SIQM OK. sym 4 chains OK. pos 3 chains OK. **SIGNED OFF!** | | **SIGNED OFF!**|Yes, reruns finished and results are uploaded for all runs. Ready to be reviewed|
|9.|[GW190720a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190720a)<br> person responsible: Krishnendu| Prod0 seems OK. Should we check with PE if L1 flow is finally 20 and not 40 Hz? <br><br> Krishnendu: fixed. [reply PE channel](https://chat.ligo.org/ligo/channels/parameter-estimation/1bypq6rnmpngtjetm8ssw17t3e) <br><br> Archisman: OK. |Anuradha: <br> why t0, cos(alpha), azimuth are different in all the 3 cases from prod run? <br> BBH: The **ACF** for tc, t0, theta_JN, cos(alpha), azimuth, delta, DL, iota look suspicious. <br> pos: The **ACF** for azimuth look suspicious. <br> The **ACF** for cos(alpha), azimuth, a1, cos(theta1) look suspicious. <br><br> <br> Krishnendu: [BH comparison pages](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_prod_c01/psd_rerun/cbcBayesCompos/comp_bh/) looks fine for me, for example [theta_JN](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/psd_rerun/comp_PE_Prod0/bh_bh/plots/combined_1d_posterior_theta_jn.png) <br> Not sure if we need reruns for ACF issues <br><br> Archisman: So mtot railing. SIQM OK. sym 4 chains; chain 4 may be problematic. pos only 1 chain! **SIGNED OFF** <br><br>Anuradha: The difference in t0, alpha, azimuth is due to the different choice of the first detector. **SIGNED OFF!**| | **SIGNED OFF!**|Yes, reruns finished and results are uploaded|
|10.|[GW190728a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190728q)<br> person responsible: Saleem| Prod0 OK. | Anuradha: LogL is a bit different in BBH case. Otherwise looks good. <br><br> Archisman: sym mtot bimodal; no railing. SIQM only negative; pos consistent with sym. sym 2 chains; sampling non-identical regions. pos only 1 chain. **NOT OK** |The runs already have high specs like srate = 2048, nlive = 2048, maxmcmc = 10000, nparallel = 8. Chains broken due to calibration files removed. Resubmitted the dag (16/7/2020) which seems to be converging (as of 20/7/2020). Reruns started already to compensate in case the resumed runs didn't go well. ||Results from PSD reruns uploaded and reviewed. Waiting for results with more samples.|
|11.|[GW190814a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190814bv) <br> person responsible: Krishnendu| Archisman: base config file info not available. <br><br> Krishnendu: fixed. <br><br> Archisman: Abhirup, could you please check the base config for this event? |Anuradha: Why t0, azimuth and cos(alpha) are different from Prod run in all the 3 cases? In 'result summary' ks_pos has the link of ks_sym, **please fix.** <br><br> Krishnendu: fixed. <br><br> Anuradha: In pos run, **ACF** for all the spin parameters, all the mass parameters, alpha, delta, tc, iota, azimuth look suspicious. <br><br> Archisman: PE comparison t0, cosalph, and azimuth are different. Mtot bimodal for pos. SIQM mostly neg. sym 4 chains (1 chain very short) OK. pos 2 chains sampling different regions. **NOT OK** |Reruns started ||Yes, reruns finished and results are uploaded|
|12.|[GW190828a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190828j) <br> person responsible: Saleem| Archisman: Mass priors have been increased between Prod0 and Prod5, but no railing with SIQM settings. OK. |Anuradha: LogL is a bit different in BBH case. Which might be okay. **ACFs** for time, iota, ra, dec, and theta_JN are suspicious in the 3 runs. **SIGNED OFF!** <br><br> Archisman: No mtot railing. SIQM OK. sim 4 chains OK. pos 4 chains OK. **SIGNED OFF!**| ||Yes, PSD reruns finished and reviewed.|
|12.|[GW190828a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190828j) <br> person responsible: Saleem| Archisman: Mass priors have been increased between Prod0 and Prod5, but no railing with SIQM settings. OK. |Anuradha: LogL is a bit different in BBH case. Which might be okay. **ACFs** for time, iota, ra, dec, and theta_JN are suspicious in the 3 runs. **SIGNED OFF!** <br><br> Archisman: No mtot railing. SIQM OK. sim 4 chains OK. pos 4 chains OK. **SIGNED OFF!**| | **SIGNED OFF!**|Yes, PSD reruns finished and reviewed.|
|13.|[GW190828b](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190828l) <br> person responsible: Saleem| Prod4 OK. **Configs and diffs not found!** <br> Saleem: **Link to configs and diffs have been fixed (16/7/20)** | Anuradha:<br>BBH: Awaiting results. <br> pos/sym: **ACF** for time, ra, dec, DL are suspicious. <br><br> Archisman: No mtot railing. SIQM samples mostly neg; pos consistent with sym. sym 4 chains OK; chain 2 may be a bit dirty. pos 4 chains OK; chains 2 & 3 may be a bit dirty. **SIGNED OFF!** |||Yes, Reruns finished for ks_pos and ks_sym whose results are ready to be reviewed. BBH runs failed and reruns started (14/7/2020)|
|14.|[GW190924a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190924h)<br> person responsible: Saleem| Prod0/5 OK. | Anuradha: There were no PSD_reruns for this event and I have signed off on the results from the existing runs (see table below). **ACF:** For BBH run the phase, theta2 and S2z and for sym run, the phase and Theta_JN look weird to me.| Saleem: Reruns started with stronger settings (16/7/20). The postplots pages for previous runs are produced (was missing). ||No, Results are uploaded from the previous set of runs. |
......