Reworking LVAlert types and contents

changed the description

mentioned in merge request emfollow/gwcelery!91 (merged)

mentioned in issue #7 (closed)

changed the description

mentioned in issue #14 (closed)

changed the description

We have added serialized Labelling objects to LVAlerts for label creation and removal, see #14 (closed). It is not clear whether this will be a long-term solution. I think it makes sense for label creation, but not really for label removal. For example, the API just returns an empty dict for label removal.

A better long-term solution for knowing which server sent an alert might be to add a new field to LVAlerts which just specifies the home page URL (or API root URL, or something) of the server which sent the alert.

changed the description

changed milestone to %Move superevents branch onto production GraceDB

changed the description

Proposal for changes to LVAlert contents

Alert content

Alerts will be JSON-formatted with the following keys and corresponding values:

uid: event graceid or superevent id (string)
alert_type: type of alert (string). There will be several possible options for this field, see below for more information.
data: serialized object which would be returned by the API for the current action (dict with various content). Example: label object for label creation, log object for log creation, signoff object for signoff creation or update, etc.
object: serialized event or superevent object which is relevant to the current alert (dict with various content). May be a duplicate of the "data" field for things like event creation.

The main changes compared to the previous structure are:

The mostly unused "file" field will be removed. This content should generally be available in the "data" field.
The "description" field will be removed. More varied and accurate alert_types should provide essentially the same information in a less verbose manner.
There will be many new alert_types added.
The "labels" field will be removed since it will be included in the "object" field.

Event alerts

Note: the "object" field will always contain the serialized event object for event alerts.

Action	alert_type	data
Event creation	new	serialized event object
Event updated by uploading a new event file	update	serialized event object
Event log creation (with or without file)	log	serialized log object
Label added to event	label_added	serialized label object
Label removed from event	label_removed	{}
VOEvent created for event	voevent	serialized voevent object
EM observation created for event	emobservation	serialized emobservation object
EMBB event log created for event	embb_event_log	serialized EMBB event log object
Event added to superevent	superevent_addition	basic event dict with graceid and link
Event removed from superevent	superevent_removal	{}
Event selected as preferred event for superevent	selected_as_preferred	serialized superevent object
Event removed as preferred event for superevent	removed_as_preferred	serialized superevent object
Operator or advocate signoff creation	signoff_added	serialized signoff object
Operator or advocate signoff update	signoff_updated	serialized signoff object
Operator or advocate signoff deletion	signoff_deleted	serialized signoff object

Superevent alerts

Note: the "object" field will always contain the serialized superevent object for superevent alerts.

Action	alert_type	data
Superevent creation	new	serialized superevent object
Superevent update (preferred event, t_start, t_0, t_end)	update	serialized superevent object
Superevent log creation (with or without file)	log	serialized log object
Label added to superevent	label_added	serialized label object
Label removed from superevent	label_removed	{}
VOEvent created for superevent	voevent	serialized voevent object
EM observation created for superevent	emobservation	serialized emobservation object
Event added to superevent	event_added	basic event dict with graceid and link
Event removed from superevent	event_removed	{}
Operator or advocate signoff creation	signoff_added	serialized signoff object
Operator or advocate signoff update	signoff_updated	serialized signoff object
Operator or advocate signoff deletion	signoff_deleted	serialized signoff object

Notes

The baseline proposal reproduces the API response for each action in the "data" field. In some cases, this might not be desirable. For example, label deletion and event removal from a superevent return an empty response, where we might want the LVAlert to contain the label or event information.

added LVAlert enhancement labels

changed milestone to %Readiness for second OPA test

@tanner.prestegard, these suggestions look great. You're probably way ahead of me here, but it'll be important to update the GraceDb docs which reference how to parse lvalert messages. The 2 places I know of off the top of my head are

Since I originally wrote the latter, I can take a crack at updating it if you'd find it helpful.

Thanks, @reed.essick! You are right that the documentation needs to be updated to reflect the changes. I may take you up on your offer, but I want to wait until things are more fleshed out and I can give a little more thought to how I would like to do the documentation.

Note to self: because the event serializers and the superevent serializers are pretty different, we are going to want to make sure that the end result for common related objects (like labels, logs, etc.) is the same for each category.

I didn't receive any suggestions or comments by the deadline of last Friday, September 14, so we are going ahead with the above proposal. I will make notes of any deviations from the proposal in the comments.

List of things tested manually:

Event creation from web view
Log creation from event and superevent web views
Event and superevent operator signoff creation, update, and deletion from web view
Event and superevent advocate signoff creation, update, and deletion from web view
Event and superevent expose/hide from web view
Confirm superevent as GW from web view
Create EMObservation for event and superevent from web view

Other tests from API:

Label addition and removal to/from event and superevent
Log message creation for event and superevent, with/without file
Advocate signoff creation, update, and deletion for superevent
Expose/hide superevent
VOEvent creation for event and superevent
EMObservation creation for event and superevent
Event and superevent creation
Event replacement with new event file
Superevent update
Adding and removing events to/from superevents

mentioned in commit 6ddc17a4

This is in place and tested on a development machine. The documentation has been updated with information about the alert types and their contents (see https://gracedb-dev1.ligo.org/documentation/lvalert.html).

In addition to the new alert contents, alerts are now issued for the following cases (previously, they were not):

An event created with labels attached
- First, the event is saved and the "new" alert is issued
- Then, each label is added and the corresponding "label_added" is issued
An event is "replaced" with new event file ("update" alert)
An event signoff is deleted ("signoff_deleted" alert)

closed

Action	alert_type	object	file	description
Superevent creation	"new"	superevent	""	"NEW: superevent S180614a"
Superevent update (preferred event, t_start, t_0, t_end)	"update"	log	""	"LOG: Updated superevent parameters: t_0: 1 -> 2, preferred_event: G0001 -> G0002"
Log creation (no file)	"update"	log	""	"LOG: comment"
Log creation (with file)	"update"	log	filename	"UPLOAD: 'filename' comment"
Event added to superevent	"update"	log	""	"LOG: Added event: G0001"
Event removed from superevent	"update"	log	""	"LOG: Removed event: G0001"
Label added to superevent	"label"	Not included in dict	""	"LABEL: H1OK"
Label removed from superevent	"update"	Not included in dict	""	"UPDATE: H1OK removed"
Confirm superevent as GW	"update"	log	""	LOG: Confirmed as a gravitational wave."
Create VOEvent for superevent	"update"	voevent	S180614b-3-Update.xml	"VOEVENT: S180614b-3-Update.xml"
Create EMObservation for superevent	"update"	emobservation	""	"New EMBB observation record for DESGW."
Operator or advocate signoff (initial or update) on superevent	"signoff"	signoff	""	"OK" or "NO"
Operator or advocate signoff deleted	"update"	log	""	"LOG: operator signoff for H1 deleted: H1OK removed and H1OPS reapplied"

Reworking LVAlert types and contents

Current status

Contents of alert JSON packet

Actions for which GraceDB issues alerts

Events

Superevents

Actions for which GraceDB does not issue alerts

Development action items

Child items 0

Activity

Proposal for changes to LVAlert contents

Alert content

Event alerts

Superevent alerts

Notes