clean up superevent on SIGNIFICANT_LABEL

Port over part of !1051 (merged) to fix #537 (closed)

The reason for #537 (closed) is as follows:

• EM_Selected freezes superevent. But it does not launch less-significant alerts yet. That will be done in !1051 (merged).
• In this patch, we port over the part of !1051 (merged) which cleans up the superevent on EM_SelectedConfident before sending the significant alert.
• Without this change, superevents that start out with a less significant G-event will have the alert not be sent due to failure to satisfy publishability condition: https://git.ligo.org/emfollow/gwcelery/-/blob/main/gwcelery/tasks/orchestrator.py#L700-707
• Another subtle but important change is needed in !1051 (merged) but refactored here: the superevents.keyfunc should explicitly order significant events higher than less-significant events i.e. a FAR=1e-5, SNR=12 is lower ranked than FAR=1e-10, SNR=11.

assigned to @deep.chatterjee

added 1 commit

• cb060f10 - clean up superevent on SIGNIFICANT_LABEL

Compare with previous version

requested review from @geoffrey.mo

Let me do a quick test on emfollow-test. And report back.

marked this merge request as draft

added 1 commit

• 2e095829 - rank publishability condition of significant events above less-significant ones

Compare with previous version

I made a slight modification to first2years uploads for this particular superevent to recreate the case that is failing on production: https://gracedb-test.ligo.org/superevents/MS230223e/view/

Let me know if this makes sense @geoffrey.mo

marked this merge request as ready

added 1 commit

• a68f1bf9 - clean up superevent on SIGNIFICANT_LABEL, rank significant event higher

Compare with previous version

changed the description

I'm looking at this but I want to note that your URL linked above (https://git.ligo.org/emfollow/gwcelery/-/blob/master/gwcelery/tasks/orchestrator.py#L577-588) links to the deprecated master branch, not the current main branch. I think I found what you were referring to in the live main branch, but we should consider deleting or otherwise hiding the master branch to avoid this sort of confusion in the future.

Good point. I have updated the description now. Yes, I have no problem in remove the master branch. Thanks for taking a look.

changed the description

added 2 commits

• 455860b0 - 1 commit from branch emfollow:main
• 86a2256b - clean up superevent on SIGNIFICANT_LABEL, rank significant event higher

Compare with previous version

I think this looks good from a code point of view, but I think adding some more comments in the code to indicate what's happening would be helpful for future debugging. I also think that this is a big enough change that a short changelog entry is worth adding.

added 1 commit

• 76f151e1 - clean up superevent on SIGNIFICANT_LABEL, rank significant event higher

Compare with previous version

Thank you @geoffrey.mo. Made the changes requested, added comments, and also updated our sphinx docs. Please have a look.

mentioned in merge request !1051 (merged)