Only one preliminary alert sent after early warning event for replay event
The review team flagged S230423bg as a superevent with behavior we should understand. Specifically, S230423bg triggered one early warning notice and then triggered only one preliminary notice. @amanda.baylor, @naresh.adhikari, @pratyusava.baral, and I investigated this a bit live on zoom. For context, the version running on playground at the time was Mothman.
The early warning notice appears to work as intended. One thing we noticed that surprised me though is that the GCN_PRELIM_SENT
label is applied after sending EarlyWarning alerts here. An additional observation is that SIGNIF_LOCKED
is not applied at that time, as it's not applied for early warning events (code here).
So it seems like what happened is that the next thing the orchestrator did after sending the EW notice was start the workflow that sends the second preliminary alert (here) because it received an IGWN-Alert indicating that the GCN_PRELIM_SENT
label had been applied. I don't understand why this ever works, but so far this is the only superevent I've seen that only sent one preliminary notice after the early warning notice.
Another odd thing about this superevent is that the preferred event is an early warning event. The Superevent cleaned up after first preliminary alert
message, which is part of the workflow kicked off by the GCN_PRELIM_SENT
label, appears here in the logs
And comparing that to G1020047, which should have been made the preferred event at this time, you can see that the last of required labels was added several minutes before the superevent cleanup.
To summarize, there are three primary things we identified that we don't understand.
- Why does this not have the
SIGNIF_LOCKED
label? Where should this have happened? Is this possibly related to whatever reason G1020047 did not get picked as the preferred event? - Why is the
GCN_PRELIM_SENT
label applied after sending early warning alerts, and follow-up: how does this ever work normally and produce two preliminary notices? - Why is G1020047 not selected as the preferred event when cleaning up the superevent when sending what is supposed to be the second preliminary notice?