O3a-Spin-Quadrupole-BH-mimicker-test-results/o3a-review-report.md

@@ -9,7 +9,7 @@ Main results review page: https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3
 |5.|[GW190521r](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190521r) <br> person responsible: Krishnendu|  | <br>Anuradha:<br> BBH: Why log L is different? <br> ks_pos: Why log L is different? q is shifted toward lower side. 1D ks plot is missing on the posplots.html page. <br> ks_sym: Why log L is different? 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: Spins and spin angles are different (expected). Looks good. **SIGNED OFF!** <br><br> Anuradha: looks good. **SIGNED OFF** || **SIGNED OFF!**|
 |6.|[GW190630a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190630ag)<br> person responsible: Krishnendu| |<br>Anuradha: <br>BBH: Why is log L different? <br> ks_pos: Why is log L different? a1, a2 and chi_eff are less constrained. <br> Why is log L different?<br> ks_sym: no comparison page <br> <br> Archisman: where are the comparison pages? I could not find them. <br> <br> Archisman: Sorry, I still cannot find the comparison plots. I get to the following summary page, but I do not see any plot: https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_ks_pos/home.html <br> <br> Is there an old-school post-processing page for an SIQM run for this event? <br>Anuradha: looks good. **SIGNED OFF!** |<br> Krishnendu <br> <br> Please see this page for [PE results](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/) and [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/) are the comparison page with PE Prod runs <br><br> Please find the cbcBayesCompPos page [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/) <br><br> Krishnendu (answering comments from Anuradha): <br> BBH logL difference again can be ignored everything else inclusing the BFs match here (PE Prod BF 98.82 and SIQM-BBH BFs 98.46 see [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_bh/home.html)) <br> compared to the BH estimate there is a slight difference in the [chirp-mass](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_ks_sym/html/Comparison_chirp_mass.html), [chi_eff](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190630ag/pv2_c01_prod/comp_PE_Prod0/bh_ks_sym/html/Comparison_chi_eff.html) and I think this is expected.|Waiting for comments|
 |7.|[GW190707a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190707q)<br> person responsible: Krishnendu||<br> Anuradha: All the comparison pages have 'Prod0' vs 'Prod4' legends, so I am not sure if I am looking are correct plots. But I do see evidence for Mtot posterior trailing [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/comp_PE_Prod0/bh_ks_pos/html/Comparison_All.html) <br> <br> Archisman: same question as Anuradha above. Also prod 4 seems to be an aligned spin run. Something rails against the mtot bound, but it is not clear what. Also it seems to slightly fall off near the bound, doesn't it? <br> <br> Archisman: Thanks the comparison page seems fixed! Both the PE and the SIQM run neem to be railing against the prior, but on closer inspection, it seems more likely that the posterior falls off just near the prior edge. The behavior is clearer on the source frame plot. I am happy to sign this off -- are the other reviewers fine with it? <br> <br> mtot prior goes down to 20 but samples only go down to 22-ish; so fine! <br> Anuradha: Looks good. **SIGNED OFF!** | <br> Krishnendu: <br>  <br> Not sure if we need a rerun for this event or not. <br> Base config is the Prod0 ini and there are [no open issues](https://git.ligo.org/search?utf8=%E2%9C%93&snippets=&scope=issues&repository_ref=&search=S190707q&project_id=3816&button=) with this event <br>[bh-comparison page with the Prod results](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/comp_PE_Prod0_cbcBayesCompPos/bh_bh/) <br><br> Issues regarding the comparison pages are fixed now. <br> Krishnendu: Not hitting the prior boundary [mtotal](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/pos/ks/1246527224.18-0/H1L1/1Dpdf/mtotal.pdf) and [mtotal_source](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190707q/pv2_c01_prod/pos/ks/1246527224.18-0/H1L1/1Dpdf/mtotal_source.pdf)|**SIGNED OFF!**|
-|8.|[GW190708a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190708ap)<br> person responsible:Saleem ||<br> Anuradha: <br> BBH: Why log L, a1, a1z, and a2z are different? <br>ks_pos: a1z, and a2z are different. a1, a2 are totally unconstrained.  1D ks plot is missing on the posplots.html page. <br> ks_sym: a1z, a2z, a1, a2 are unconstrained. 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: spins are significantly different. Why are spins estimated better for the SIQM run than for the PE run? Otherwise fine. In particular, no mtot railing. <br> <br> Archisman: PE comparison has been performed with Prod 2 which is a non-spinning run. Please fix this. <br><br> Anuradha: ks plots are provided which look good but the comments have not been answered.| Saleem:<br> I could not find the Prod0 postplots page for making the comparison page. The path linked from Prod0 run directory currently contains the Prod2 postplots as Archisman mentioned. Please see the summary pages (linked from event page) where the comparison with prod0 has been made.)  | |
+|8.|[GW190708a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190708ap)<br> person responsible:Saleem ||<br> Anuradha: <br> BBH: Why log L, a1, a1z, and a2z are different? <br>ks_pos: a1z, and a2z are different. a1, a2 are totally unconstrained.  1D ks plot is missing on the posplots.html page. <br> ks_sym: a1z, a2z, a1, a2 are unconstrained. 1D ks plot is missing on the posplots.html page. <br> <br> Archisman: spins are significantly different. Why are spins estimated better for the SIQM run than for the PE run? Otherwise fine. In particular, no mtot railing. <br> <br> Archisman: PE comparison has been performed with Prod 2 which is a non-spinning run. Please fix this. <br><br> Anuradha: ks plots are provided which look good but the comments have not been answered.| Saleem:<br> I could not find the Prod0 postplots page for making the comparison page. The path linked from Prod0 run directory currently contains the Prod2 postplots as Archisman mentioned. Please see the pesummary pages (linked from the event page) where the comparison with prod0 PE run has been made.) <br> <br> The 1d-pdf can be seen in the pesummary page or are also linked directly from the table | |
 |9.|[GW190720a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190720a)<br> person responsible: Krishnendu||<br> Anuradha: There is one comparison page given, the label says 'BH comparison page' but when you go inside it has BH and ks_sym legends. So, I am not sure what am I looking at. Since there are no comparison plots, I am not sure if the results are alright. <br> There are evidence for Mtot posterior trailing [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_prod_c01/comp_ks_bh/) and [here](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/sym/ks/1247616534.7-0/V1H1L1/posplots.html) <br> <br> Archisman: Spins and spin angles are different (expected). Why is chirp mass so different? There is a minor railing of mtot against the prior. <br> <br> Archisman: This looks fine to me now. Chirp mass is fine. No mtot railing. <br> <br> Archisman: mtot prior goes down to 20 while samples go down to only about 23. **Krishnendu**, why do you want to rerun this? <br> Anuradha: looks good. **SIGNED OFF!**|<br> <br> Posteriors not hitting the prior boundary[mtotal_det](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/sym/ks/1247616534.7-0/V1H1L1/1Dpdf/mtotal.pdf) <bt> [mtotal_source](https://ldas-jobs.ligo.caltech.edu/~nv.krishnendu/LSC/lalinference/O3/o3_a_tgr/S190720a/pv2_c01_prod/sym/ks/1247616534.7-0/V1H1L1/1Dpdf/mtotal_source.pdf) <br> After discussion on 29.05.2020 we decided to have no rerun | **SIGNED OFF!**|
 |10.|[GW190728a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190728q)<br> person responsible: Saleem|| Anuradha: Log L is a bit different in BH vs Prod case. Please confirm that the Bayes factors are similar. Otherwise looks good. | Runs are over and Results are added|Waiting for comments|
 |11.|[GW190814a](https://git.ligo.org/nv.krishnendu/lalsuite/-/wikis/O3a-Spin-Quadrupole-BH-mimicker-test-results/S190814bv) <br> person responsible: Krishnendu||<br>Anuradha: No plots, comparison or otherwise, to look at. <br> <br> Archisman: where are the comparison pages? <br> <br> Archisman: no mtot railing. Looks good. <br> <br> Anuradha: Base config does not look up-to-date. <br> BBH:The comparison is actually with ks_pos. Please fix. <br> ks_pos/ks_sym: ks plots missing.|Please see the updated wiki page|Waiting for comments|