Skip to content

Changes

Page history
Update o3a review report authored by NV Krishnendu's avatar NV Krishnendu
Show whitespace changes
Inline Side-by-side
O3a-Spin-Quadrupole-BH-mimicker-test-results/o3a-review-report.md
View page @ 645b3edf
......@@ -2,30 +2,30 @@ Main results review page: https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3
|No.| NAME | Comments on config ini file|Comments on results from the reviewer | Reviewees comments |Final status|
|---| ---- | ---- | ---- | --- |--- |
|1. |[GW190408a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190408a)| |<br> Anuradha:<br>BBH run: In think there is a typo in labeling, it should be BBH not ks_pos.<br> ks_pos/ks_sym: The logL is different. S1z and S2z are broader than BBH run, and hence effective spin is broader than BBH run. Mtotal and Mc are shifted towards lower values. <br> <br> Archisman: looks fine. **SIGNED OFF!** <br> Anuradha: **SIGNED OFF!**|<br> Difference in SNR and logL are expected. [Discussion on matter-]most(https://chat.ligo.org/ligo/pl/n1rhn8sdt7g89muqe45iumdixa) <br> In order to make sure that the logL difference not affected our analsyis we can check the BFs and compare the one from PE Prod run and the one obtained from our BBH run. <br> In this case, log(BF)-PE PROD is 94.99 and SIQM-BH run is 95.2, hence no issues.| **SIGNED OFF!** |
|2. |[GW190412m](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190412m)||<br> Anuradha: <br> BBH run: Why log L is so different?<br>ks_pos: Why log L is so different? cos(alpha) is very different. chi_eff, q and Mc have shifted towards lower values. Mtotal and m1 are railing at the lower side. <br>ks_sym: Why log L is so different? Mtotal and m1 are railing at the lower side. q has shifted to higher values. <br> <br> Archisman: needs rerun with wider mtot prior.||Rerun|
|3. |[GW190425z](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190425z)|| Archisman: still running ||
|4.|[GW190512b](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190512at)||<br> <br> Anuradha: BBH: Why log L is different? <br> <br> Archisman: whys is t0 different? Is the BH run used for comparison a non-spinning run? <br> <br> Archisman (following Krishnendu's first comment): This looks better, thanks! There is a minor mtot railing. We need to decide what to do for runs such as these. <br> <br> Archisman: for symmetric, mtotal is bimodal. **ATTENTION:** Rerun is necessary for this event. Railing of mtotal is clear on this [postproc](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/sym/ks/1241719652.42-0/V1H1L1/posplots.html) page. | <br> Krishnendu <br> Yes, this is now being addressed. I used Prod4 results for comparison and which was a non-spinning run. Now it is fixed. <br> <br> Krishnendu: <br> logL difference will not affect this case as well (BFs 53.17 [bh case](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/bh/1241719652.42-0/V1H1L1/posplots.html) and 52.14 [ks_pos](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/pos/ks/1241719652.42-0/V1H1L1/posplots.html) )<br> <br> Krishnendu: Currently, I am passing the following in the [config file](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/config/Prod0_IMRPP_S190512at_Ks_pos.ini) for the mass parameters. The config file for [Prod0 config file](https://git.ligo.org/pe/O3/S190512at/-/tree/master/C01_offline) also uses the same q-min = 0.125, mtotal-min = 40, mtotal-max = 65. In [this](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_bh/), the BH run using our settings is compared with the PE-Prod0 results. Both the runs give almost same mtotal, mchirp posteriors (no prior railing) but for [ks_pos](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_ks_pos/) and[ks_sym](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_ks_sym/) cases there is a difference in the mchip estimate and the total mass shifted to lower mass end. <br> Any suggestion for rerun will be useful <br> <br> Rerun started with extending mtotal from mtotal-min = 20 to mtotal-max = 65 |Rerun|
|5.|[GW190521r](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190521r) | | <br>Anuradha:<br> BBH: Why log L is different? <br> ks_pos: Why log L is different? q is shifted toward lower side. 1D ks plot is missing on the posplots.html page. <br> ks_sym: Why log L is different? 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: Spins and spin angles are different (expected). Looks good. **SIGNED OFF!** <br><br> Anuradha: looks good. **SIGNED OFF** || **SIGNED OFF!**|
|6.|[GW190630a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190630ag)| |<br>Anuradha: <br>BBH: Why is log L different? <br> ks_pos: Why is log L different? a1, a2 and chi_eff are less constrained. <br> Why is log L different?<br> ks_sym: no comparison page <br> <br> Archisman: where are the comparison pages? I could not find them. <br> <br> Archisman: Sorry, I still cannot find the comparison plots. I get to the following summary page, but I do not see any plot: https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_ks_pos/home.html <br> <br> Is there an old-school post-processing page for an SIQM run for this event? <br>Anuradha: looks good. **SIGNED OFF!** |<br> Krishnendu <br> <br> Please see this page for [PE results](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/) and [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/) are the comparison page with PE Prod runs <br><br> Please find the cbcBayesCompPos page [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/) <br><br> Krishnendu (answering comments from Anuradha): <br> BBH logL difference again can be ignored everything else inclusing the BFs match here (PE Prod BF 98.82 and SIQM-BBH BFs 98.46 see [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_bh/home.html)) <br> compared to the BH estimate there is a slight difference in the [chirp-mass](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_ks_sym/html/Comparison_chirp_mass.html), [chi_eff](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_ks_sym/html/Comparison_chi_eff.html) and I think this is expected.|Waiting for comments|
|7.|[GW190707a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190707q)||<br> Anuradha: All the comparison pages have 'Prod0' vs 'Prod4' legends, so I am not sure if I am looking are correct plots. But I do see evidence for Mtot posterior trailing [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/comp_PE_Prod0/bh_ks_pos/html/Comparison_All.html) <br> <br> Archisman: same question as Anuradha above. Also prod 4 seems to be an aligned spin run. Something rails against the mtot bound, but it is not clear what. Also it seems to slightly fall off near the bound, doesn't it? <br> <br> Archisman: Thanks the comparison page seems fixed! Both the PE and the SIQM run neem to be railing against the prior, but on closer inspection, it seems more likely that the posterior falls off just near the prior edge. The behavior is clearer on the source frame plot. I am happy to sign this off -- are the other reviewers fine with it? <br> <br> mtot prior goes down to 20 but samples only go down to 22-ish; so fine! <br> Anuradha: Looks good. **SIGNED OFF!** | <br> Krishnendu: <br> <br> Not sure if we need a rerun for this event or not. <br> Base config is the Prod0 ini and there are [no open issues](https://git.ligo.org/search?utf8=%E2%9C%93&snippets=&scope=issues&repository_ref=&search=S190707q&project_id=3816&button=) with this event <br>[bh-comparison page with the Prod results](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_bh/) <br><br> Issues regarding the comparison pages are fixed now. <br> Krishnendu: Not hitting the prior boundary [mtotal](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/pos/ks/1246527224.18-0/H1L1/1Dpdf/mtotal.pdf) and [mtotal_source](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/pos/ks/1246527224.18-0/H1L1/1Dpdf/mtotal_source.pdf)|**SIGNED OFF!**|
|8.|[GW190708a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190708ap) ||<br> Anuradha: <br> BBH: Why log L, a1, a1z, and a2z are different? <br>ks_pos: a1z, and a2z are different. a1, a2 are totally unconstrained. 1D ks plot is missing on the posplots.html page. <br> ks_sym: a1z, a2z, a1, a2 are unconstrained. 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: spins are significantly different. Why are spins estimated better for the SIQM run than for the PE run? Otherwise fine. In particular, no mtot railing. <br> <br> Archisman: PE comparison has been performed with Prod 2 which is a non-spinning run. Please fix this. <br><br> Anuradha: ks plots are provided which look good but the comments have not been answered.|| |
|9.|[GW190720a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190720a)||<br> Anuradha: There is one comparison page given, the label says 'BH comparison page' but when you go inside it has BH and ks_sym legends. So, I am not sure what am I looking at. Since there are no comparison plots, I am not sure if the results are alright. <br> There are evidence for Mtot posterior trailing [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_prod_c01/comp_ks_bh/) and [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/sym/ks/1247616534.7-0/V1H1L1/posplots.html) <br> <br> Archisman: Spins and spin angles are different (expected). Why is chirp mass so different? There is a minor railing of mtot against the prior. <br> <br> Archisman: This looks fine to me now. Chirp mass is fine. No mtot railing. <br> <br> Archisman: mtot prior goes down to 20 while samples go down to only about 23. **Krishnendu**, why do you want to rerun this? <br> Anuradha: looks good. **SIGNED OFF!**|<br> <br> Posteriors not hitting the prior boundary[mtotal_det](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/sym/ks/1247616534.7-0/V1H1L1/1Dpdf/mtotal.pdf) <bt> [mtotal_source](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/sym/ks/1247616534.7-0/V1H1L1/1Dpdf/mtotal_source.pdf) <br> After discussion on 29.05.2020 we decided to have no rerun | **SIGNED OFF!**|
|10.|[GW190728a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190728q)|| Anuradha: Log L is a bit different in BH vs Prod case. Please confirm that the Bayes factors are similar. Otherwise looks good. | Runs are over and Results are added|
|11.|[GW190814a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190814bv) ||<br>Anuradha: No plots, comparison or otherwise, to look at. <br> <br> Archisman: where are the comparison pages? <br> <br> Archisman: no mtot railing. Looks good. <br> <br> Anuradha: Base config does not look up-to-date. <br> BBH:The comparison is actually with ks_pos. Please fix. <br> ks_pos/ks_sym: ks plots missing.|Please see the updated wiki page|Waiting for comments|
|12.|[GW190828a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190828j) ||<br> Anuradha: BBH: All parameters are different. You BBH runs is actually better. Prod results are bad, why? <br> ks_pos: Since Prod results are bad, I can not compare. 1D ks plot is missing on the posplots.html page. <br> ks_sym: Since Prod results are bad, I can not compare. 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: something very incorrect in the comparison pages which I see as of May 29 -- everything is estimated differently. Good that this is being rerun. |Saleem: Comparison pages are updated |Waiting for comments|
|13.|[GW190924a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190924h)||<br> Anuradha: <br> BBH: m1 and mtotal are railing at the boundary. <br>ks_pos: I hope it is a typo, it says BBH vs PE run. m1, m2 and mtotal are railing at the boundary. 1D ks plot is missing on the posplots.html page. <br> ks_sym: mtotal is railing at the boundary. 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: is the PE run used for comparison a production PE run? PE run seems to have very restrictive distance priors. Why is everything estimated better for the SIQM run then for the PE run? PE run shows mtot railing. SIQM run shows a very marginal mtot railing. <br> Anuradha: looks good. **SIGNED OFF**| Saleem: The results are now updated with the latest runs. earlier runs used an older lalsuite installation. The results are not expected to be different. The typos in labels are fixed and posteriors are now made from more number of samples. From manually plotting the posteriors, there is no railing against prior for Mtotal. Also, no Mtot prior is imposed in the config for this event. |Rerun|
|1. |[GW190408a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190408a)<br> person responsible: Krishnendu| |<br> Anuradha:<br>BBH run: In think there is a typo in labeling, it should be BBH not ks_pos.<br> ks_pos/ks_sym: The logL is different. S1z and S2z are broader than BBH run, and hence effective spin is broader than BBH run. Mtotal and Mc are shifted towards lower values. <br> <br> Archisman: looks fine. **SIGNED OFF!** <br> Anuradha: **SIGNED OFF!**|<br> Difference in SNR and logL are expected. [Discussion on matter-]most(https://chat.ligo.org/ligo/pl/n1rhn8sdt7g89muqe45iumdixa) <br> In order to make sure that the logL difference not affected our analsyis we can check the BFs and compare the one from PE Prod run and the one obtained from our BBH run. <br> In this case, log(BF)-PE PROD is 94.99 and SIQM-BH run is 95.2, hence no issues.| **SIGNED OFF!** |
|2. |[GW190412m](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190412m)<br> person responsible: Krishnendu||<br> Anuradha: <br> BBH run: Why log L is so different?<br>ks_pos: Why log L is so different? cos(alpha) is very different. chi_eff, q and Mc have shifted towards lower values. Mtotal and m1 are railing at the lower side. <br>ks_sym: Why log L is so different? Mtotal and m1 are railing at the lower side. q has shifted to higher values. <br> <br> Archisman: needs rerun with wider mtot prior.||Rerun|
|3. |[GW190425z](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190425z)<br> person responsible: Krishnendu|| Archisman: still running ||
|4.|[GW190512b](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190512at)<br> person responsible: Krishnendu||<br> <br> Anuradha: BBH: Why log L is different? <br> <br> Archisman: whys is t0 different? Is the BH run used for comparison a non-spinning run? <br> <br> Archisman (following Krishnendu's first comment): This looks better, thanks! There is a minor mtot railing. We need to decide what to do for runs such as these. <br> <br> Archisman: for symmetric, mtotal is bimodal. **ATTENTION:** Rerun is necessary for this event. Railing of mtotal is clear on this [postproc](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/sym/ks/1241719652.42-0/V1H1L1/posplots.html) page. | <br> Krishnendu <br> Yes, this is now being addressed. I used Prod4 results for comparison and which was a non-spinning run. Now it is fixed. <br> <br> Krishnendu: <br> logL difference will not affect this case as well (BFs 53.17 [bh case](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/bh/1241719652.42-0/V1H1L1/posplots.html) and 52.14 [ks_pos](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/pos/ks/1241719652.42-0/V1H1L1/posplots.html) )<br> <br> Krishnendu: Currently, I am passing the following in the [config file](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/config/Prod0_IMRPP_S190512at_Ks_pos.ini) for the mass parameters. The config file for [Prod0 config file](https://git.ligo.org/pe/O3/S190512at/-/tree/master/C01_offline) also uses the same q-min = 0.125, mtotal-min = 40, mtotal-max = 65. In [this](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_bh/), the BH run using our settings is compared with the PE-Prod0 results. Both the runs give almost same mtotal, mchirp posteriors (no prior railing) but for [ks_pos](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_ks_pos/) and[ks_sym](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190512at/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_ks_sym/) cases there is a difference in the mchip estimate and the total mass shifted to lower mass end. <br> Any suggestion for rerun will be useful <br> <br> Rerun started with extending mtotal from mtotal-min = 20 to mtotal-max = 65 |Rerun|
|5.|[GW190521r](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190521r) <br> person responsible: Krishnendu| | <br>Anuradha:<br> BBH: Why log L is different? <br> ks_pos: Why log L is different? q is shifted toward lower side. 1D ks plot is missing on the posplots.html page. <br> ks_sym: Why log L is different? 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: Spins and spin angles are different (expected). Looks good. **SIGNED OFF!** <br><br> Anuradha: looks good. **SIGNED OFF** || **SIGNED OFF!**|
|6.|[GW190630a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190630ag)<br> person responsible: Krishnendu| |<br>Anuradha: <br>BBH: Why is log L different? <br> ks_pos: Why is log L different? a1, a2 and chi_eff are less constrained. <br> Why is log L different?<br> ks_sym: no comparison page <br> <br> Archisman: where are the comparison pages? I could not find them. <br> <br> Archisman: Sorry, I still cannot find the comparison plots. I get to the following summary page, but I do not see any plot: https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_ks_pos/home.html <br> <br> Is there an old-school post-processing page for an SIQM run for this event? <br>Anuradha: looks good. **SIGNED OFF!** |<br> Krishnendu <br> <br> Please see this page for [PE results](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/) and [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/) are the comparison page with PE Prod runs <br><br> Please find the cbcBayesCompPos page [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/) <br><br> Krishnendu (answering comments from Anuradha): <br> BBH logL difference again can be ignored everything else inclusing the BFs match here (PE Prod BF 98.82 and SIQM-BBH BFs 98.46 see [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_bh/home.html)) <br> compared to the BH estimate there is a slight difference in the [chirp-mass](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_ks_sym/html/Comparison_chirp_mass.html), [chi_eff](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_ks_sym/html/Comparison_chi_eff.html) and I think this is expected.|Waiting for comments|
|7.|[GW190707a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190707q)<br> person responsible: Krishnendu||<br> Anuradha: All the comparison pages have 'Prod0' vs 'Prod4' legends, so I am not sure if I am looking are correct plots. But I do see evidence for Mtot posterior trailing [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/comp_PE_Prod0/bh_ks_pos/html/Comparison_All.html) <br> <br> Archisman: same question as Anuradha above. Also prod 4 seems to be an aligned spin run. Something rails against the mtot bound, but it is not clear what. Also it seems to slightly fall off near the bound, doesn't it? <br> <br> Archisman: Thanks the comparison page seems fixed! Both the PE and the SIQM run neem to be railing against the prior, but on closer inspection, it seems more likely that the posterior falls off just near the prior edge. The behavior is clearer on the source frame plot. I am happy to sign this off -- are the other reviewers fine with it? <br> <br> mtot prior goes down to 20 but samples only go down to 22-ish; so fine! <br> Anuradha: Looks good. **SIGNED OFF!** | <br> Krishnendu: <br> <br> Not sure if we need a rerun for this event or not. <br> Base config is the Prod0 ini and there are [no open issues](https://git.ligo.org/search?utf8=%E2%9C%93&snippets=&scope=issues&repository_ref=&search=S190707q&project_id=3816&button=) with this event <br>[bh-comparison page with the Prod results](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_bh/) <br><br> Issues regarding the comparison pages are fixed now. <br> Krishnendu: Not hitting the prior boundary [mtotal](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/pos/ks/1246527224.18-0/H1L1/1Dpdf/mtotal.pdf) and [mtotal_source](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/pos/ks/1246527224.18-0/H1L1/1Dpdf/mtotal_source.pdf)|**SIGNED OFF!**|
|8.|[GW190708a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190708ap)<br> person responsible:Saleem ||<br> Anuradha: <br> BBH: Why log L, a1, a1z, and a2z are different? <br>ks_pos: a1z, and a2z are different. a1, a2 are totally unconstrained. 1D ks plot is missing on the posplots.html page. <br> ks_sym: a1z, a2z, a1, a2 are unconstrained. 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: spins are significantly different. Why are spins estimated better for the SIQM run than for the PE run? Otherwise fine. In particular, no mtot railing. <br> <br> Archisman: PE comparison has been performed with Prod 2 which is a non-spinning run. Please fix this. <br><br> Anuradha: ks plots are provided which look good but the comments have not been answered.|| |
|9.|[GW190720a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190720a)<br> person responsible: Krishnendu||<br> Anuradha: There is one comparison page given, the label says 'BH comparison page' but when you go inside it has BH and ks_sym legends. So, I am not sure what am I looking at. Since there are no comparison plots, I am not sure if the results are alright. <br> There are evidence for Mtot posterior trailing [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_prod_c01/comp_ks_bh/) and [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/sym/ks/1247616534.7-0/V1H1L1/posplots.html) <br> <br> Archisman: Spins and spin angles are different (expected). Why is chirp mass so different? There is a minor railing of mtot against the prior. <br> <br> Archisman: This looks fine to me now. Chirp mass is fine. No mtot railing. <br> <br> Archisman: mtot prior goes down to 20 while samples go down to only about 23. **Krishnendu**, why do you want to rerun this? <br> Anuradha: looks good. **SIGNED OFF!**|<br> <br> Posteriors not hitting the prior boundary[mtotal_det](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/sym/ks/1247616534.7-0/V1H1L1/1Dpdf/mtotal.pdf) <bt> [mtotal_source](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/sym/ks/1247616534.7-0/V1H1L1/1Dpdf/mtotal_source.pdf) <br> After discussion on 29.05.2020 we decided to have no rerun | **SIGNED OFF!**|
|10.|[GW190728a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190728q)<br> person responsible: Saleem|| Anuradha: Log L is a bit different in BH vs Prod case. Please confirm that the Bayes factors are similar. Otherwise looks good. | Runs are over and Results are added|
|11.|[GW190814a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190814bv) <br> person responsible: Krishnendu||<br>Anuradha: No plots, comparison or otherwise, to look at. <br> <br> Archisman: where are the comparison pages? <br> <br> Archisman: no mtot railing. Looks good. <br> <br> Anuradha: Base config does not look up-to-date. <br> BBH:The comparison is actually with ks_pos. Please fix. <br> ks_pos/ks_sym: ks plots missing.|Please see the updated wiki page|Waiting for comments|
|12.|[GW190828a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190828j) <br> person responsible: Saleem||<br> Anuradha: BBH: All parameters are different. You BBH runs is actually better. Prod results are bad, why? <br> ks_pos: Since Prod results are bad, I can not compare. 1D ks plot is missing on the posplots.html page. <br> ks_sym: Since Prod results are bad, I can not compare. 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: something very incorrect in the comparison pages which I see as of May 29 -- everything is estimated differently. Good that this is being rerun. |Saleem: Comparison pages are updated |Waiting for comments|
|13.|[GW190924a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190924h)<br> person responsible: Saleem||<br> Anuradha: <br> BBH: m1 and mtotal are railing at the boundary. <br>ks_pos: I hope it is a typo, it says BBH vs PE run. m1, m2 and mtotal are railing at the boundary. 1D ks plot is missing on the posplots.html page. <br> ks_sym: mtotal is railing at the boundary. 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: is the PE run used for comparison a production PE run? PE run seems to have very restrictive distance priors. Why is everything estimated better for the SIQM run then for the PE run? PE run shows mtot railing. SIQM run shows a very marginal mtot railing. <br> Anuradha: looks good. **SIGNED OFF**| Saleem: The results are now updated with the latest runs. earlier runs used an older lalsuite installation. The results are not expected to be different. The typos in labels are fixed and posteriors are now made from more number of samples. From manually plotting the posteriors, there is no railing against prior for Mtotal. Also, no Mtot prior is imposed in the config for this event. |Rerun|
**Details of O1/O2 events**
|No.| NAME | Comments on config ini file|Comments on results | Reviewees comments |Final status|
|---| ---- | ---- | ---- | --- |--- |
|1. |[GW150914](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/GW150914) | |<br>Anuradha: No comparison-with-prod-run-plots, no ks plots.<br> ks_pos/ks_sym: Why is log L different? why a1 and a2 are oppositely unconstrained? Why is optimal (or matched-filter) SNR so different than BBH case? <br> Anuradha: <br> BBH: Why are ra, dec, DL, MF-SNR, OP-SNR are different? Is comparison with L1 samples? <br> ks_pos/ks_sym: Why are ra, dec, DL differnt? ks plots missing.|<br>Krishnendu: Comparison pages added now <br> As Anuradha pointed out the coherence test was not done on this event and there are still some chains found to be not-converged. <br> rerun would be necessary and started these runs| Rerun|
|2. |[GW151226](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/GW151226) | |<br>Anuradha: No comparison-with-prod-run-plots, no ks plots.<br> ks_pos/ks_sym: Why is log L very very different? why a1 and a2 are totally unconstrained? Why is optimal (or matched-filter) SNR so different than BBH case? Mc, Mtot and m1 are railing at the boundary. <br> <br> Archisman: spin angles are different (generic feature); mtot railing. |<br>Krishnendu: Comparison pages added now <br><br> Krishnendu (on the other comments): <br> [compare mtotal_source and mtotal_detector](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW151226_G211117/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_bh/) <br> [config](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW151226_G211117/config/GW151226_C02_IMRPv2_bh.ini) <br> mtotal should not just bounded bcz of the mass prior ```distance-max=1500 chirpmass-min=9.5 chirpmass-max=10.5 q-min=0.0555555555555556 q-max=1.0 comp-max=54.398 comp-min=3.022 ```` <br> Any suggestion on the rerun settings?|Rerun? Need to decide|
|3. |[GW170104](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/GW170104) | |<br>Anuradha: No comparison-with-prod-run-plots, no ks plots. <br> ks_pos: Why is log L different? Why a1 and a2 are totally unconstrained? <br> ks_sym: DL and q go beyond prior limits. Why Mc is so different? <br> <br> Archisman: does distance hit the upper bound or is that just the shape of the posterior? |<br>Krishnendu: Comparison pages added now <br><br> No distance prior issue on the [BH run](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW170104_G268556/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_ks_pos/) <br> When we compare the BH and ks_sym case [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW170104_G268556/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_ks_sym/), when we include ks distance, mchirp (hence mtotal, m1, m2), chi_eff are over-estimated and both optimal and matched filter SNR are peaked to a lower value <br> [base config](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW170104_G268556/config/GW170104_C02_IMRPv2.ini) uses no distance prior hence the sharp boundary at the right end of the distance posterior should also coming from the introduction of this extra parameter <br><br> these effects are not present with ks_sym <br><br> rerun only needed for [ks_sym case](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW170104_G268556/pv2_c01_prod/post-plots/)? |Results ready for review|
|4. |[GW170608](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/GW170608) | |<br> Anuradha: <br> BBH: Why log L is different? <br>ks_pos: Why log L is different? a2 is unconstrained. m1, m2, and mtotal are railing at the boundary. 1D ks plot is missing on the posplots.html page. <br>ks_sym: Why log L is different? a2 is unconstrained. m1, m2 and mtotal are railing at the boundary. 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: spin angles are different (generic); mtot severely railing. | Saleem: There is no Mtot prior imposed in the config and there is no railing too (it appears to have so due to the small number of bins). The same applies for m1 and m2 railing as well. The priors on m1 and m2 are [1, 40] and the so-called railing is way too inside this limits. So, a rerun may not be required. <br> <br> Compared to the PE ini, the mc prior is wider for siqm: [8,9] vs [5,20], with the latter being the same as that used for TIGER. For the given mass ratio prior, this allows wider range of mtot to be sampled. <br> <br> The 1D pdf is available in the pesummary page which is linked from the event page.|Results ready for review|
|5. |[GW170814](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/GW170814) | |<br>Anuradha: No comparison-with-prod-run-plots, no ks plots.<br> ks_pos/ks_sym: Why a1 and a2 are totally unconstrained? <br> <br> Archisman: spin angles are different (generic); mtot is fine. Why is dL so different? Why are sky angles not estimated in the SIQM run? Was a 2 detector run launched by mistake? <br> Anuradha: same question about DL and sky angles. Ks plots missing. |<br>Krishnendu: Comparison pages added now|Results ready for review|
|1. |[GW150914](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/GW150914) <br> person responsible: Krishnendu| |<br>Anuradha: No comparison-with-prod-run-plots, no ks plots.<br> ks_pos/ks_sym: Why is log L different? why a1 and a2 are oppositely unconstrained? Why is optimal (or matched-filter) SNR so different than BBH case? <br> Anuradha: <br> BBH: Why are ra, dec, DL, MF-SNR, OP-SNR are different? Is comparison with L1 samples? <br> ks_pos/ks_sym: Why are ra, dec, DL differnt? ks plots missing.|<br>Krishnendu: Comparison pages added now <br> As Anuradha pointed out the coherence test was not done on this event and there are still some chains found to be not-converged. <br> rerun would be necessary and started these runs| Rerun|
|2. |[GW151226](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/GW151226) <br> person responsible: Krishnendu| |<br>Anuradha: No comparison-with-prod-run-plots, no ks plots.<br> ks_pos/ks_sym: Why is log L very very different? why a1 and a2 are totally unconstrained? Why is optimal (or matched-filter) SNR so different than BBH case? Mc, Mtot and m1 are railing at the boundary. <br> <br> Archisman: spin angles are different (generic feature); mtot railing. |<br>Krishnendu: Comparison pages added now <br><br> Krishnendu (on the other comments): <br> [compare mtotal_source and mtotal_detector](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW151226_G211117/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_bh/) <br> [config](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW151226_G211117/config/GW151226_C02_IMRPv2_bh.ini) <br> mtotal should not just bounded bcz of the mass prior ```distance-max=1500 chirpmass-min=9.5 chirpmass-max=10.5 q-min=0.0555555555555556 q-max=1.0 comp-max=54.398 comp-min=3.022 ```` <br> Any suggestion on the rerun settings?|Rerun? Need to decide|
|3. |[GW170104](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/GW170104) <br> person responsible: Krishnendu| |<br>Anuradha: No comparison-with-prod-run-plots, no ks plots. <br> ks_pos: Why is log L different? Why a1 and a2 are totally unconstrained? <br> ks_sym: DL and q go beyond prior limits. Why Mc is so different? <br> <br> Archisman: does distance hit the upper bound or is that just the shape of the posterior? |<br>Krishnendu: Comparison pages added now <br><br> No distance prior issue on the [BH run](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW170104_G268556/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_ks_pos/) <br> When we compare the BH and ks_sym case [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW170104_G268556/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_ks_sym/), when we include ks distance, mchirp (hence mtotal, m1, m2), chi_eff are over-estimated and both optimal and matched filter SNR are peaked to a lower value <br> [base config](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW170104_G268556/config/GW170104_C02_IMRPv2.ini) uses no distance prior hence the sharp boundary at the right end of the distance posterior should also coming from the introduction of this extra parameter <br><br> these effects are not present with ks_sym <br><br> rerun only needed for [ks_sym case](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/O1_O2/GW170104_G268556/pv2_c01_prod/post-plots/)? |Results ready for review|
|4. |[GW170608](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/GW170608)<br> person responsible: Krishnendu | |<br> Anuradha: <br> BBH: Why log L is different? <br>ks_pos: Why log L is different? a2 is unconstrained. m1, m2, and mtotal are railing at the boundary. 1D ks plot is missing on the posplots.html page. <br>ks_sym: Why log L is different? a2 is unconstrained. m1, m2 and mtotal are railing at the boundary. 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: spin angles are different (generic); mtot severely railing. | Saleem: There is no Mtot prior imposed in the config and there is no railing too (it appears to have so due to the small number of bins). The same applies for m1 and m2 railing as well. The priors on m1 and m2 are [1, 40] and the so-called railing is way too inside this limits. So, a rerun may not be required. <br> <br> Compared to the PE ini, the mc prior is wider for siqm: [8,9] vs [5,20], with the latter being the same as that used for TIGER. For the given mass ratio prior, this allows wider range of mtot to be sampled. <br> <br> The 1D pdf is available in the pesummary page which is linked from the event page.|Results ready for review|
|5. |[GW170814](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/GW170814)<br> person responsible: Krishnendu | |<br>Anuradha: No comparison-with-prod-run-plots, no ks plots.<br> ks_pos/ks_sym: Why a1 and a2 are totally unconstrained? <br> <br> Archisman: spin angles are different (generic); mtot is fine. Why is dL so different? Why are sky angles not estimated in the SIQM run? Was a 2 detector run launched by mistake? <br> Anuradha: same question about DL and sky angles. Ks plots missing. |<br>Krishnendu: Comparison pages added now|Results ready for review|
**Misc**
......