Describe criteria for HIGH_PROFILE label

Currently the user guide does not state under what circumstances a HIGH_PROFILE label is applied to a public event. Without this information, it may be difficult for outside observers to determine why a particular event is high profile.

@max.trevor GraceDB's public-facing view hides all the labels from a superevent, including the HIGH_PROFILE label. That change has been active since ER15. We made this decision (@roberto.depietri, @leo-singer) specifically to not justify HIGH_PROFILE to non-LVK members.

Please open tonight's superevent (https://gracedb.ligo.org/superevents/S240830gn/view/) in a private window to confirm.

I can't see the label on the superevent page itself but I can see it on the list of latest events. I was asked to open this ticket by @jennne.driggers after myself and others reported that the HIGH_PROFILE label was visible without logging in.

To show what I was looking at I've attached a screenshot from me looking at the 'Latest' tab on GraceDB without logging in! gracedb_labels_nologin

At the time of taking this screenshot and writing this message, S240830gn has fallen off the latest events page, but I was previously able to see it's high profile label along with the other labels shown for the events in the screenshot.

changed title from Describeria criteria for HIGH_PROFILE label to Describe criteria for HIGH_PROFILE label

I believe @sushant.sharma-chaudhary mentioned he could also see the HIGH_PROFILE label without logging in.

@alexander.pace, should this be a GraceDB server issue rather than a User Guide issue?

I would think that by showing the labels for events this far into O4 we have effectively let the cat out of the bag. Maybe this should be discussed on an Ops call? @jenne-driggers @nicolas.arnaud

I don't know about that. In any case the HIGH_PROFILE label just means whether it passes the threshold for a significant alert, right? And that threshold is public knowledge.

No, HIGH_PROFILE means that it passes the significant alert threshold, and fulfills some additional criteria, such as high chance of being a BNS or NSBH, or having a localization area below some threshold. The intention of the label was to flag events that need low-latency vetting from experts because they may be of interest for low-latency follow-up. The criteria are not currently described anywhere in the user guide.

OK. Patches welcome!

Hi @max.trevor, "The criteria are not currently described anywhere."

Let me set the record straight. The criteria have been explicitly described in O4 RRT Policy https://dcc.ligo.org/L2100046 since before O4. The companion chart https://dcc.ligo.org/L2100045 and the Responder Guide https://emfollow.docs.ligo.org/followup-advocate-guide/index.html that are used to put the policy into actionable items have been available since before O4, too. The criteria were advertised to external observers at https://wiki.gw-astronomy.org/OpenLVEM/Telecon20230427. Prior to coming to the current definition there were multiple discussions in opsdiv and at least one in DAC.

Sorry, I meant to say "not currently described anywhere in the user guide"

Oh, I see Max, sorry for misunderstanding.

@max.trevor @sushant.sharma-chaudhary:

Removing the public visibility of labels was just for the superevent page itself, not for search results (and by proxy, the latest page). I know through email inquiries that there are outside (non-LVK) observing partners that query GraceDB specifically for labels, so hiding them on the search results isn't the best solution. They're also visible in the public API view. If there's a new push to hide labels there too, please make a GraceDB server issue. And get the buy-in from at least @jenne-driggers, @keita.kawabe, and @michael-coughlin.

@leo-singer:

There's already a one-line description in the GraceDB public documentation. If there's further explanation needed that we feel like the public should be aware of, then I think it belongs in the userguide.

As a compromise, if there's publicly-available RRT documents that have a description of what is and isn't HIGH_PROFILE, then I can edit the GraceDB documentation to include a link.

Here's an idea: I made an GWCelery issue before (here: gwcelery#836 (closed)) suggesting that GWCelery create a log message when adding HIGH_PROFILE that describes why it got the label (since even LVK members occasionally ask that). If you implemented that feature, and then made the log message public, then everyone would know what's going on. Just a thought.

The push was not to hide the labels, but rather to give an explanation of what makes an event HIGH_PROFILE.

I understood your previous message as meaning that there was no point in explaining because the labels weren't visible to the public, but they are visible in the search results.

I would strongly support adding a log message to explain why events got a HIGH_PROFILE label, since that was the first question I asked when responding to S240830gn.

I suggest that you first wait for @jenne-driggers and @keita.kawabe to clarify the point of policy of whether the HIGH_PROFILE label should remain public. If their answer is 'yes' then would you please submit a MR to document the label?

• I wouldn't hide in the middle of a run an information that has been public so far. Though, we may want to revisit what is public and what is not after O4: most labels are for internal use and so I see no reason why they should be public.
• I like the idea of adding a log message documenting why an event is tagged as HIGH_PROFILE, as that question arises publicly at least once per RRT vetting of such events -- and certainly many people ask that question themselves when joining the discussion.
• Then, I don't know what's best/simplest to describe the HIGH_PROFILE label, between making a log message automatically or manually public and updating the user guide. As I'm not aware of such request from external users (that discussion was triggered from the LVK-internal vetting of a recent trigger), I would do a minimal change for now. Perhaps something as simple as

The HIGH_PROFILE label means that the event has some particular features that call for a quicker and more in-depth vetting by LVK experts.

Then, I don't know what's best/simplest to describe the HIGH_PROFILE label, between making a log message automatically or manually public and updating the user guide. As I'm not aware of such request from external users (that discussion was triggered from the LVK-internal vetting of a recent trigger), I would do a minimal change for now.

In my opinion, it ought to be documented in the User Guide. It's less important to me that there is a log message.

Particularly if it's easy, I certainly support having an internal log message in GraceDB. I'd like to think a little bit more about whether it should be public. Also, if we want to make it public, we'll need to make sure to go through the PnP process.

I do rather wish that the HIGH_PROFILE label wasn't visible at all (I hadn't realized until this discussion started that it was still publicly available), however I hesitate to stop providing that information now. As Max points out, we're not really talking about pulling that information back, so we'll keep it that way (i.e., no change to the visibility of labels at this time).

I think that, since the HIGH_PROFILE label is publicly available, it should be documented in the User Guide. I also propose that the one-liner in the GraceDB documentation that Alex pointed to add a link to the User Guide description, once that description exists. As Leo suggests, we should make a merge request summarizing the criteria for the HIGH_PROFILE label that are defined by https://dcc.ligo.org/LIGO-L2100046.

I do rather wish that the HIGH_PROFILE label wasn't visible at all (I hadn't realized until this discussion started that it was still publicly available)

Unfortunately the decision not to display labels on the superevent public page and on the O4 public events page was made on-the-fly during an ER15 low latency call, so I don't have an associated issue which would document why that was sufficient.

Maybe with the benefit of hindsight, given the (public) documentation of the label's description: Superevent satisfies the condition to convene the full Level 2 RRT ASAP., the label should have been called something like RRT_LEVEL2, or something. Teasing something as HIGH_PROFILE and then not describing it further is bound to make people want to know more.

That being said, I think documenting the criteria in the user guide and then linking from GraceDB would be the best first step.

Though it's not our conscious decision that HIGH_PROFILE is public, it's been public so far, it's silly to suddenly make it private.

It's not like we haven't told external observers what kind of events will automatically get higher scrutiny by RRT experts (see attached, that's from https://wiki.gw-astronomy.org/OpenLVEM/Telecon20230427), that's not a secret. Since we WILL change the condition (to add SSM at least, but we might also change the tight localization condition later), we cannot just reference the presentation from GraceDB documentation. I agree with @leo-singer, it's a good idea to document it in the User Guide.

If people are motivated to add a specific reason for ''HIGH_PROFILE'' automatically to the log message while general principle is in the User Guide, I'm not against that, I'm neutral though it's probably unnecessary except for narrow corner cases. Nothing in the User Guide and something only in log messages sounds like an unnecessary layer of obscurity.

Getting back to the User Guide, I can write a merge request but I'm not sure if it makes sense to do that right now with the High Profile condition we have as of now, knowing that we will soon have to write another MR for SSM. I'm not up to the release schedule of the User Guide and SSM public alert, could somebody give me an idea?

Thank you @keita.kawabe for taking a first look at the text to include.

Since this text will need to go through PnP review, and the hasSSM updates for GWCelery will happen in the next few weeks, I propose that (as you're implying) the text that we prepare for the UserGuide include SSM information. Then, we'll hold off on pushing the updated UserGuide until the SSM search goes live / public.

Since this text will need to go through PnP review, and the hasSSM updates for GWCelery will happen in the next few weeks, I propose that (as you're implying) the text that we prepare for the UserGuide include SSM information.

Please contribute the text for the HIGH_PROFILE label and the SSM searches in two separate MRs. They should be mostly independent, and it will be faster to review both changes if they are two smaller, separate MRs that address independent concerns, rather than one giant MR that addresses multiple concerns.

Incidentally, there is already a MR for the SSM text, although it still needs a lot of work: !274 (merged). I bet that @bhooshan.gadre would appreciate writing and proofreading help with that.

Then, we'll hold off on pushing the updated UserGuide until the SSM search goes live / public.

There is no particular reason to delay any unrelated updates until the SSM text is done. Version numbers are free, after all.

@keita.kawabe - for clarity, I think it would be good to know that external observers can't actually get any extra information about an event by reading this label. Could you confirm that all the elements of the HIGH_PROFILE criterion are currently already public and can be evaluated from information on the superevent page? I.e. an external observer could work out for themselves from the definition on your OpenLVKEM slide, plus the superevent's visible analysis results whether the label is going to be applied?

All the information stated in https://dcc.ligo.org/public/0187/G2300908/004/G2300908-OpenLVKEM_LVK_RRT_ER15_O4-v4.pdf is available in the public page (e.g https://gracedb.ligo.org/superevents/S240830gn/view/).

@thomas-dent, just as @roberto.depietri wrote.

In addition to that, we will make significant SSM S-events High Profile (i.e. SSM FAR is significant and it's preferred, which means that there is no significant non-SSM CBC trigger in the same S-event). That an event is SSM will be published through HasSSM and lack of pastro and through circular text.

There is a manual path to convene Level 2 RRT due to "special science case" (L2100045-RRTv12-flowchart.pdf) though we don't know what that case would be. If that happens, HIGH_PROFILE label will most likely be applied manually to simplify the process of convening all experts. Level 2 discussion will include how we should draft a Circular to explain (or not) the reason why the label was applied, without disclosing confidential information. But I don't think that that level of detail should be described in the User Guide, it's enough to say that we might apply this label in case some unexpected things happen, like I indicated in the presentation I gave to the astronomers (see the screenshot above).

Please contribute the text for the HIGH_PROFILE label and the SSM searches in two separate MRs.

@leo-singer thanks for the suggestion. The text about HIGH_PROFILE will basically be a list of line items, and one of the items will be SSM. Do you want me to

1. make a MR about adding HIGH_PROFILE description to vetting.rst excluding anything about SSM, and
2. make a separate MR to add one line in vetting.rst to say that SSM will get HIGH_PROFILE too?

Sure, that would work.

We'll just have to hold off on adding the information about the interaction between the HIGH_PROFILE label and SSM searches until after !274 (merged) is done. Or, we can ask @bhooshan.gadre to include that minor edit in !274 (merged) itself.

OK, I'll work on the first MR, and will send one line that should be added to vetting.rst to @bhooshan.gadre so he can include it in !274 (merged). Thank you for your guidance!

mentioned in merge request gwcelery!1515 (merged)

I started a MR related to the HIGH-PROFILE label as discussed in the RRT team. gwcelery!1515 (merged) This MR includes following changes

1. HIGH_PROFILE label will be applied if the preferred event is from SSM search.
2. Reason for HIGH_PROFILE label will be logged in gracedb. (Checkout some examples in the MR)
3. HIGH_PROFILE scenario based on tight localization has been revised to also include distance < 2 Gpc

The main problem of documenting HIGH_PROFILE are the following

• we do not explain anywhere in the UserGuide the meaning of the other gracedb LABEL
• The criteria for an event HIGH_PROFILE may change in time.

Thanks @roberto.depietri , as for the first point, I don't think we have to explain all labels, but many feels the need to explain HIGH_PROFILE somewhere, and I think the User Guide is the right place for that. As for the second point, we should update the User Guide as the criteria change.

mentioned in merge request !292 (merged)

I made a merge request here: !292 (merged)

@leo-singer @roberto.depietri how do I proceed from here? Which milestone to set, who should I ask review?

mentioned in commit 74a12157

closed with merge request !292 (merged)