Skip to content

Discuss the state of pastro

Action item:

Chad, Jolien, Anarya and Shio will get together to discuss the state of pastro in GstLAL on Friday, April 28th at 10:00 am EDT (9:00 am for Anarya and Jolien). The zoom info is:

Things to discuss

  • Is the MDC12 pastro model effective enough? (See issue 472.)
    • Are we satisfied that the BNS + NSBH vs BBH category is mixed up minimally?
    • Many BNS events are misclassified as NSBH. Is this a concern?
    • Many terrestrial events are classified as BBH. Is this a concern? We'll have to revisit this once the MDC12 FARs are fixed.
  • Is implementing the injection rates patch the right move?
    • Right now, we're using different pastro models for inj and non-inj to get a pp-plot where the data points lie on the diagonal. In O4, is it our highest priority to get such pp-plots?
    • If the R&P injections are not ready at the start of O4 (see issue 473 for discussion on the path to O4 production), how should we phase in the injection rates patch?

Summary of the situation (Copied from GstLAL code review.)

Link to slides presented at the April 4 GstLAL West Call: slides

Background

The pastro model used in MDC 11 was miscategorizing BNS and NSBH injections as shown in this plot (also edited screenshot below) presented by the MBTA group during the March 2023 LVK meeting.

Screen_Shot_2023-04-20_at_6.35.25_PM

The template weights of the pastro model used in MDC11 was not symmetric along chi_eff = 0, as seen in slide 4 of the slides presented at the April 4 GstLAL West Call and in the screenshot below.

Screen_Shot_2023-04-20_at_6.40.49_PM

Fixes for MDC12

There was a bug in the pastro code where pastro models were computed using s1z, s2z, whereas our template bank is generated using chi_eff. Chad and Anarya have fixed this bug as well as fixing the "Nan to Num" issues that were seen. We are now using a simplified version of Heather's thesis. (Details on slide 5 of the slides presented at the April 4 GstLAL West Call.)

MDC12 uses this fixed pastro model.

Template weights

The template weights of the MDC12 pastro model is m1 m2 plot and mchirp chi_eff plot (also see screenshots below).

Screen_Shot_2023-04-20_at_6.36.52_PM Screen_Shot_2023-04-20_at_6.37.47_PM

pp plots to compare MDC11 pastro model and MDC12 pastro model on 1 month of MDC11 data

We made pp plots to compare the MDC11 pastro model and MDC12 pastro model on 1 month (Feb18 - Mar24) of MDC11 data. The MDC11 pastro model results were queried and obtained from GraceDB, whereas results for MDC12 pastro model results were obtained from recomputing the pastros using the MDC12 pastro model on MDC11 triggers.

The MDC 11 pastro model result for FAR < 1/year is: Screen_Shot_2023-04-20_at_6.38.36_PM

The MDC12 pastro model result for FAR < 1/year is: Screen_Shot_2023-04-20_at_6.38.57_PM

pp plots and Sankey diagrams to compare MDC11 pastro model on 1 week of MDC11 data and MDC12 pastro model on 1 week of MDC12 data

We made pp plots and Sankey diagrams to compare the MDC11 pastro model on 1 week of MDC11 data and the MDC12 pastro model on 1 week of MDC12 data (that corresponds to that in MDC11). Both results were obtained by querying GraceDB instead of recomputing pastros using pastro models. The FARs in MDC12 are not reliable as of the beginning of April, so we have used LR to query GraceDB. Based on MDC11 results, Shio and Anarya agreed that lnLR = 5.5 corresponds to the GraceDB upload FAR threshold and lnLR = 16.5 corresponds to FAR = 1/year.

For the max lnLR event per superevent where lnLR >= 5.5 (the top row is mdc12, the bottom row is MDC11): Screen_Shot_2023-04-20_at_6.39.22_PM

For the max lnLR event per superevent where lnLR >= 16.5 (the top row is mdc12, the bottom row is MDC11): Screen_Shot_2023-04-20_at_6.39.39_PM

Summary

  • MDC12 pastro has fewer incidences where potentially EM-bright inj are misclassified as BBH. For events with lnLR >= 5.5, no BBH events are misclassified as BNS/NSBH.
  • For events with lnLR >= 5.5, both MDC12 pastro and MDC11 pastro miscategorize terrestrial events as BNS/NSBH/BBH, but especially BBH. MDC12 has a larger ratio of terrestrial events being miscategorized as BNS or NSBH than mdc11. For events with lnLR >= 16.6, MDC12 pastro miscategorizes terrestrial events as BNS/NSBH/BBH whereas MDC11 pastro miscategorizes them only as NSBH.

To Do

  • Check back in after the MDC12 FAR is fixed.
  • Discuss the injection rates patch that was applied at the end of MDC10.
Edited by Shio Sakon