Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • alexander.pace/server
  • geoffrey.mo/gracedb-server
  • deep.chatterjee/gracedb-server
  • cody.messick/server
  • sushant.sharma-chaudhary/server
  • michael-coughlin/server
  • daniel.wysocki/gracedb-server
  • roberto.depietri/gracedb
  • philippe.grassia/gracedb
  • tri.nguyen/gracedb
  • jonah-kanner/gracedb
  • brandon.piotrzkowski/gracedb
  • joseph-areeda/gracedb
  • duncanmmacleod/gracedb
  • thomas.downes/gracedb
  • tanner.prestegard/gracedb
  • leo-singer/gracedb
  • computing/gracedb/server
18 results
Show changes
Hide whitespace changes
Inline Side-by-side
Showing
with 1305 additions and 1225 deletions
docs/user_docs/source/dicts/permissions.list 0 → 100644
View file @ 5797d777
[
{
"group": "gw-astronomy:LV-EM:Observers",
"permission": "view_superevent"
},
{
"group": "gw-astronomy:LV-EM:Observers",
"permission": "annotate_superevent"
},
{
"group": "public_users",
"permission": "view_superevent"
}
]
docs/user_docs/source/dicts/signoff.json 0 → 100644
View file @ 5797d777
{
"comment": "Everything looks good",
"instrument": "",
"self": "https://gracedb.ligo.org/api/superevents/TGW800106H/signoffs/ADV/",
"signoff_type": "ADV",
"status": "OK",
"submitter": "albert.einstein@LIGO.ORG"
}
docs/user_docs/source/dicts/superevent.json 0 → 100644
View file @ 5797d777
{
"superevent_id": "S220316r",
"gw_id": null,
"category": "Production",
"created": "2022-03-16 18:30:10 UTC",
"submitter": "albert.einstein@LIGO.ORG",
"preferred_event": "G194542",
"em_type": null,
"t_start": 1331490616.298529,
"t_0": 1331490617.298529,
"t_end": 1331490618.89512,
"gw_events": [
"G194543",
"G194542",
"G194541",
"G194540"
],
"em_events": [],
"far": 9.652186314922026e-07,
"time_coinc_far": null,
"space_coinc_far": null,
"labels": [],
"links": {
"events": "https://gracedb-test.ligo.org/api/superevents/S220316r/events/",
"labels": "https://gracedb-test.ligo.org/api/superevents/S220316r/labels/",
"logs": "https://gracedb-test.ligo.org/api/superevents/S220316r/logs/",
"files": "https://gracedb-test.ligo.org/api/superevents/S220316r/files/",
"self": "https://gracedb-test.ligo.org/api/superevents/S220316r/",
"voevents": "https://gracedb-test.ligo.org/api/superevents/S220316r/voevents/",
"emobservations": "https://gracedb-test.ligo.org/api/superevents/S220316r/emobservations/"
},
"preferred_event_data": {
"submitter": "albert.einstein@LIGO.ORG",
"created": "2022-03-16 18:30:00 UTC",
"group": "CBC",
"graceid": "G194542",
"pipeline": "gstlal",
"gpstime": 1331490617.298529,
"instruments": "H1,L1",
"nevents": 2,
"offline": true,
"search": "AllSky",
"far": 9.652186314922026e-07,
"far_is_upper_limit": false,
"likelihood": 7.102756279364618,
"labels": [
"SKYMAP_READY",
"PASTRO_READY"
],
"extra_attributes": {},
"superevent": "S220316r",
"superevent_neighbours": {
"S220316r": {
"superevent_id": "S220316r",
"gw_events": [
"G194543",
"G194542",
"G194541",
"G194540"
],
"preferred_event": "G194542",
"preferred_event_data": {
"submitter": "albert.einstein@LIGO.ORG",
"created": "2022-03-16 18:30:00 UTC",
"group": "CBC",
"graceid": "G194542",
"pipeline": "gstlal",
"gpstime": 1331490617.298529,
"instruments": "H1,L1",
"nevents": 2,
"offline": true,
"search": "AllSky",
"far": 9.652186314922026e-07,
"far_is_upper_limit": false,
"likelihood": 7.102756279364618,
"labels": [
"SKYMAP_READY",
"PASTRO_READY"
],
"extra_attributes": {}
},
"far": 9.652186314922026e-07,
"t_start": 1331490616.298529,
"t_0": 1331490617.298529,
"t_end": 1331490618.89512
}
},
"links": {
"neighbors": "https://gracedb-test.ligo.org/api/events/G194542/neighbors/",
"log": "https://gracedb-test.ligo.org/api/events/G194542/log/",
"emobservations": "https://gracedb-test.ligo.org/api/events/G194542/emobservation/",
"files": "https://gracedb-test.ligo.org/api/events/G194542/files/",
"labels": "https://gracedb-test.ligo.org/api/events/G194542/labels/",
"self": "https://gracedb-test.ligo.org/api/events/G194542",
"tags": "https://gracedb-test.ligo.org/api/events/G194542/tag/"
}
}
}
docs/user_docs/source/dicts/voevent.json 0 → 100644
View file @ 5797d777
{
"N": 1,
"created": "2018-09-19 18:29:08 UTC",
"file_version": 0,
"filename": "TGW800106H-1-Preliminary.xml",
"issuer": "albert.einstein@LIGO.ORG",
"ivorn": "ivo://gwnet/gcn_sender#TGW800106H-1-Preliminary",
"links": {
"file": "https://gracedb.ligo.org/api/superevents/TGW800106H/files/TGW800106H-1-Preliminary.xml,0",
"self": "https://gracedb.ligo.org/api/superevents/TGW800106H/voevents/1/"
},
"voevent_type": "PR"
}
docs/user_docs/source/general.rst
View file @ 5797d777
......@@ -6,15 +6,15 @@ GraceDB in context
==========================
GraceDB serves as a communications hub and as a database for storing and displaying
information about candidate events:
information about candidate gravitational-wave events and related electromagnetic events:
.. image:: images/annotations_flow_white.png
.. image:: images/data_flow_blue.png
The primary responsibilities of GraceDB are to:
- ingest and store information about events
- ingest and store information about candidate events
- expose that information to users and followup robots
- send alerts about each new piece of information
- send alerts about new information and state changes to users
As such, GraceDB curates content from many sources, but does not itself produce
any original scientific content. The service is mainly geared toward low latency
......@@ -22,44 +22,40 @@ data analysis
and electromagnetic followup, but need not be used exclusively for those
purposes. The diagram above depicts a typical sequence of events:
#. An LVC data analysis pipeline detects an interesting candidate
#. An LVK data analysis pipeline detects an interesting candidate
gravitational wave (GW) event and submits it to GraceDB.
#. GraceDB sends an LVAlert message which notifies LVC followup
#. GraceDB sends an igwn-alert message which notifies LVK followup
robots of the new event.
#. The followup robots perform analyses and report results back to
GraceDB. These results accumulate on the candidate event's page.
#. Eventually, a designated EM approval robot
#. Eventually, a designated `EM approval robot <https://rtd.igwn.org/projects/gwcelery/en/latest/index.html>`__
forwards the event to `GCN/TAN <http://gcn.gsfc.nasa.gov>`__.
#. Astronomers receive a GCN Notice for the new event, perform followup
observations, and report coordinates via GraceDB.
Overview of components
======================
GraceDB consists of the server (`gracedb.ligo.org <https://gracedb.ligo.org>`__)
GraceDB consists of the server (e.g., `gracedb.ligo.org <https://gracedb.ligo.org>`__)
and a set of client tools. Two user interfaces are available: the web interface
for browser access (i.e., the one you are using now), and the
for browser access, and the
`REST <https://en.wikipedia.org/wiki/Representational_state_transfer>`__ API.
These interfaces represent the information in GraceDB in different ways:
the web interface naturally represents information as HTML pages, whereas
the web interface represents information as human-readable HTML pages, whereas
the REST interface delivers JSON-serialized data.
The client tools (available via ``pip``, SL6 or Debian packages, and source
build, see :ref:`installing_the_client`) provide a way to interact via the REST API.
These tools include a Python client class
with methods for all common GraceDB operations.
There is also a ``gracedb`` executable for the command line with much of
the same functionality.
The `ligo-gracedb client package <https://ligo-gracedb.readthedocs.io/>`__ provides a convenient way to interact with the REST API.
This package includes a Python client class with methods for all common GraceDB operations.
There is also a ``gracedb`` executable for the command line with much of the same functionality.
The GraceDB API conforms to the RESTful principles of "uniform interface" and
"resource-oriented architecture," and users experienced with such
interfaces may thus wish to create their own clients. This documentation will
briefly touch on building such clients (see :ref:`coding_against_api`).
"resource-oriented architecture".
Where can I go for help?
==================================
This documentation is not as great as it could be, but we are working on it.
This documentation is not as great as it could be, but
we are working on it. For help with issues not addressed here, please
send mail to uwm-help@ligo.org.
LIGO/Virgo/KAGRA users can join the `GraceDB channel <https://chat.ligo.org/ligo/channels/gracedb>`__ in the collaboration's Mattermost instance or email the Compsoft mailing list for help.
To report a problem, either `post an issue <https://git.ligo.org/lscsoft/gracedb/issues>`__ or email the
`IGWN Computing Helpdesk <mailto:computing-help@ligo.org>`__.
docs/user_docs/source/igwn_alert.rst 0 → 100644
View file @ 5797d777
=========================================
igwn-alert notifications (LVK users only)
=========================================
Introduction
============
igwn-alert is a Kafka-based messaging platform used within the LVK.
This document will describe how GraceDB uses igwn-alert, which topics it manages and publishes to, and the content of igwn-alert messages sent by GraceDB. GraceDB uses `igwn-alert <https://igwn-alert.readthedocs.io/>`__ to send alerts to listeners within the LVK when.
Some helpful resources for installing and configuring igwn-alert are:
- Main `igwn-alert client <https://igwn-alert.readthedocs.io/>`__
- igwn-alert `user guide <https://igwn-alert.readthedocs.io/en/latest/guide.html>`__
- `Tutorial <https://igwn-alert.readthedocs.io/en/latest/guide.html#responding-to-igwn-alert-messages>`__ on setting up igwn-alert and configuring your listener.
igwn-alert and GraceDB
======================
Generally speaking, GraceDB uses igwn-alert to send "push" notifications about different actions that may be taken on the service.
Users can subscribe to different topics (more below) to receive these notifications, filter their content, and optionally trigger follow-up processes, like data quality checks, parameter estimation, and more.
The content of an igwn-alert message is designed to convey actionable information about a state change in GraceDB, including event creation, annotation, and other actions.
.. NOTE::
An igwn-alert message is sent out for *any* new event or annotation that arrives in the GraceDB database.
This means that message volumes may be very high under certain circumstances, and listeners should be constructed to appropriately filter the messages.
igwn-alert topics managed by GraceDB
====================================
By running the ``igwn-alert`` command-line tool, you will receive messages over all **topics** to which you have a listen permission, analogous to a subscription.
There are two types of topics to which GraceDB broadcasts alerts: event topics and superevent topics.
Instance Groups
---------------
The following topic names are universal across each instance of GraceDB
(production, playground, test). However, each instance's topics names are
prepended by the instance name (gracedb, gracedb-playground, gracedb-test) and a
".". For example, the ``cbc_gstlal`` topic for GraceDB Playground is listed in
the subscription interface as:
``gracedb-playground.cbc_gstlal``
And can be evoked in the command line tool by specifying the instance with the
optional ``group`` flag as in the following command:
``igwn-alert -g gracedb-playground listen cbc_gstlal``
Event topics
------------
Event topic names consist of at least two elements::
<group_name>_<pipeline_name>
In other words, the (lower-cased) names of the Group and Pipeline separated by an underscore.
For example, the topic ``cbc_gstlal`` would catch all messages relating to events in the CBC group from the gstlal pipeline.
One can also specify the search name::
<group_name>_<pipeline_name>_<search_name>
which has the effect of narrowing down the messages to only those related to a specific search.
For example, the topic ``cbc_gstlal_allsky`` will contain messages relating to the AllSky search, but not the MDC search.
It is important to note that GraceDB will send an igwn-alert to all topics which match the parameters of the event in question.
For example, the creation of a Burst-cWB-AllSky event will result in messages being sent to the ``burst_cwb_allsky`` topic, as well as the more generic ``burst_cwb`` topic.
This feature allows the user to filter according to search by specifying different igwn-alert processing scripts for different topics.
Superevent topics
-----------------
There are only three superevent topics; one for each category of superevent:
- ``superevent``
- ``test_superevent``
- ``mdc_superevent``
Most users will be interested in the ``superevent`` topic in order to receive igwn-alerts about real GW candidates.
Contents of igwn-alerts sent by GraceDB
=======================================
GraceDB sends igwn-alert messages as a JSON-encoded dictionary.
This dictionary contains the following keys:
- ``alert_type``: short string representing the action which triggered the alert. Examples: ``new``, ``update``, ``label_added``, etc. All alert types are shown in the tables below.
- ``data``: a dictionary representing the relevant object (label, log message, etc.)
- ``object``: a dictionary representing the corresponding "parent" object (i.e., the event or superevent which a log, label, etc. is attached to).
- ``uid``: the unique ID of the relevant event or superevent
Below, we describe the alert contents in more detail.
Examples of the various ``data``/``object`` dictionaries are available in :ref:`models`.
See :ref:`below<example_permissions_list>` for one additional example (list of permissions).
Event alerts
------------
For alerts related to events, the following things are always true:
- ``uid`` is always the event's ``graceid`` (example: G123456).
- ``object`` is always a dictionary corresponding to the event which is affected by the label, log, VOEvent, etc.
The following table shows the ``alert_type`` and ``data`` for different actions:
+-------------------------+---------------------------------+---------------------------------------------------------+
| ``alert_type`` | ``data`` | Occurs when |
+=========================+=================================+=========================================================+
| added_to_superevent | event dictionary | event is added to a superevent |
+-------------------------+---------------------------------+---------------------------------------------------------+
| embb_event_log | EMBB event log dictionary | EM bulletin board event log is created for event |
+-------------------------+---------------------------------+---------------------------------------------------------+
| emobservation | emobservation dictionary | EM observation is created for event |
+-------------------------+---------------------------------+---------------------------------------------------------+
| exposed | list of permission dictionaries | internal-only event is exposed to LV-EM |
+-------------------------+---------------------------------+---------------------------------------------------------+
| hidden | list of permission dictionaries | LV-EM viewable event is marked as internal-only |
+-------------------------+---------------------------------+---------------------------------------------------------+
| label_added | label dictionary | label is added to event |
+-------------------------+---------------------------------+---------------------------------------------------------+
| label_removed | label dictionary | label is removed from event |
+-------------------------+---------------------------------+---------------------------------------------------------+
| log | log dictionary | log message is added to event (may include file upload) |
+-------------------------+---------------------------------+---------------------------------------------------------+
| new | event dictionary | event is created |
+-------------------------+---------------------------------+---------------------------------------------------------+
| removed_as_preferred | event dictionary | event is no longer the preferred event for a superevent |
+-------------------------+---------------------------------+---------------------------------------------------------+
| removed_from_superevent | event dictionary | event is removed from a superevent |
+-------------------------+---------------------------------+---------------------------------------------------------+
| selected_as_preferred | event dictionary | event is selected as preferred event for a superevent |
+-------------------------+---------------------------------+---------------------------------------------------------+
| signoff_created | signoff dictionary | signoff is created for event |
+-------------------------+---------------------------------+---------------------------------------------------------+
| signoff_deleted | signoff dictionary | event signoff is deleted |
+-------------------------+---------------------------------+---------------------------------------------------------+
| signoff_updated | signoff dictionary | event signoff is updated |
+-------------------------+---------------------------------+---------------------------------------------------------+
| update | event dictionary | event is "replaced" by upload of a new event file |
+-------------------------+---------------------------------+---------------------------------------------------------+
| voevent | VOEvent dictionary | VOEvent is created for event |
+-------------------------+---------------------------------+---------------------------------------------------------+
Superevent alerts
-----------------
For alerts related to superevents, the following things are always true:
- ``uid`` is always the superevent's ``superevent_id`` (example: S800106D).
- ``object`` is always a dictionary corresponding to the superevent which is affected by the label, log, VOEvent, etc.
The following table shows the ``alert_type`` and ``data`` for different actions:
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| ``alert_type`` | ``data`` | Occurs when |
+=========================+=================================+=============================================================================================+
| confirmed_as_gw | superevent dictionary | superevent is upgraded to GW status |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| embb_event_log | EMBB event log dictionary | EM bulletin board event log is created for event |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| emobservation | emobservation dictionary | EM observation is created for superevent |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| event_added | superevent dictionary | an event is added to superevent |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| event_removed | superevent dictionary | an event is removed from superevent |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| exposed | list of permission dictionaries | internal-only superevent is exposed to LV-EM and public |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| hidden | list of permission dictionaries | public superevent is marked as internal-only |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| label_added | label dictionary | label is added to superevent |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| label_removed | label dictionary | label is removed from superevent |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| log | log dictionary | log message is added to superevent (may include file upload) |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| new | superevent dictionary | superevent is created |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| signoff_created | signoff dictionary | signoff is created for superevent |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| signoff_deleted | signoff dictionary | superevent signoff is deleted |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| signoff_updated | signoff dictionary | superevent signoff is updated |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| update | superevent dictionary | superevent is updated (``t_start``, ``t_0``, ``t_end``, or ``preferred_event`` are changed) |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
| voevent | VOEvent dictionary | VOEvent is created for superevent |
+-------------------------+---------------------------------+---------------------------------------------------------------------------------------------+
.. _example_permissions_list:
Example: list of permission dictionaries
----------------------------------------
.. literalinclude:: dicts/permissions.list
:language: JSON
docs/user_docs/source/images/annotations_flow_white.pdf deleted 100644 → 0
View file @ b4e766bd
File deleted
docs/user_docs/source/images/annotations_flow_white.png deleted 100644 → 0
View file @ b4e766bd
docs/user_docs/source/images/annotations_flow_white.png

15.1 KiB

docs/user_docs/source/images/data_flow_blue.png 0 → 100644
View file @ 5797d777
docs/user_docs/source/images/data_flow_blue.png

135 KiB

docs/user_docs/source/index.rst
View file @ 5797d777
......@@ -3,7 +3,8 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to GraceDB's documentation!
===================================
GraceDB Documentation
===================================
Contents:
......@@ -11,14 +12,9 @@ Contents:
.. toctree::
:maxdepth: 2
ref_manual
tutorials
Indices and tables
==================
ref_manual
Documentation for the ligo-gracedb client package <https://ligo-gracedb.readthedocs.io/>
LIGO/Virgo Public Alert Guide <https://emfollow.docs.ligo.org/userguide/>
Report a bug (LVK users) <https://git.ligo.org/computing/gracedb/server/-/issues>
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
docs/user_docs/source/labels.rst 0 → 100644
View file @ 5797d777
.. _labels:
=================
Labels in GraceDB
=================
Labels are applied to events and superevents in GraceDB to indicate the state of that object.
Some labels signify that an action is requested, while others signify that a follow-up process has completed and/or what the result of a follow-up process was.
Here is a table showing the currently available labels and their meanings.
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| Label name | Label description |
+=========================+========================================================================================================================================+
| 2022_LENSING_MDC | Event is part of the O4 Lensing MDC. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| ADVNO | EM advocate says event is not okay. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| ADVOK | EM advocate says event is okay. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| ADVREQ | EM advocate signoff requested. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| COMBINEDSKYMAP_READY | Combined skymap is available. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| cWB_r | cWB-AllSky search performed by the cWB-XP |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| cWB_s | cWB-AllSky search performed by the cWB-2G.subthr |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| DQOK | Data quality information is available and does not veto the event. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| DQR_REQUEST | Superevent ready for DQR check. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| DQV | Data quality veto. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| EARLY_WARNING | Event is from an early-warning low-latency pipeline. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| EMBRIGHT_READY | EM Bright information is available. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| EM_COINC | Signifies that a coincidence was found between gravitational-wave candidates and External triggers. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| EM_READY | Indicates data products associated with a Superevent's preferred event are complete |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| EM_SENT | Has been sent to MOU partners. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| EM_Selected | GraceID automatically chosen as the most promising candidate out of a set of entries thought to correspond to the same physical event. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| EM_SelectedConfident | The event has all annotations necessary for an alert and passes the significant alert threshold. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| EM_Superseded | GraceID automatically passed over because another entry was thought to be more promising and to correspond to the same physical event. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| EM_Throttled | GraceID is ignored by automatic processing because the corresponding pipeline submitted too many events too quickly. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| EXT_SKYMAP_READY | External skymap is available |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| GCN_PRELIM_SENT | A preliminary GCN has been sent. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| GRB_OFFLINE | Indicates that offline triggered GRB searches found something coincident with this event. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| GRB_ONLINE | Indicates that online triggered GRB searches found something coincident with this event. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| H1NO | H1 operator says event is not okay. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| H1OK | H1 operator says event is okay. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| H1OPS | H1 operator signoff requested. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| HIGH_PROFILE | Superevent satisfies the condition to convene the full Level 2 RRT ASAP. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| HWINJNO | There were problems with the hardware injection. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| HWINJOK | A hardware injection was successfully performed. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| HWINJREQ | A hardware injection is scheduled. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| INJ | Injection occured near this time. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| L1NO | L1 operator says event is not okay. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| L1OPS | L1 operator signoff requested. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| LENSED_CANDIDATE | Candidate event was uploaded as part of a lensed search. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| LLAMA_COMPLETE | LLAMA has completed annotating the superevent. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| LOW_SIGNIF_LOCKED | The GraceID associated with this event has all annotations required for an alert, passes the low-significance public alert threshold, |
| | and will only be automatically updated when preliminary alerts are sent. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| LOW_SIGNIF_PRELIM_SENT | A low-significance preliminary GCN has been sent. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| LUMIN_GO | Trigger satisfies basic automated checks and should be vetted by humans. Replaced by ADVREQ |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| LUMIN_NO | LUMIN No |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| NOT_GRB | Event is likely not an astrophysical gamma ray burst |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| PASTRO_READY | p_astro is available. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| PE_READY | Parameter estimation results are available |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| RAVEN_ALERT | A RAVEN coincidence event is publishable |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| SIGNIF_LOCKED | The GraceID associated with this event has all annotations required for an alert, passes the significant public alert threshold, |
| | and will only be automatically updated when preliminary alerts are sent. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| SKYMAP_READY | Skymap is available. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| SOG_READY | A coincidence should trigger a speed of gravity measurement. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| SNR_OPTIMIZED | Indicates that the event was SNR-optimized as followup to another uploaded event. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| SUBSOLAR_MASS | Event is from a subsolar mass low-latency pipeline. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| SWIFT_GO | Send notification to SWIFT telescope. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| SWIFT_NO | Do not send notification to SWIFT telescope. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| V1NO | V1 operator says event is not okay. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| V1OK | V1 operator says event is okay. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
| V1OPS | V1 operator signoff requested. |
+-------------------------+----------------------------------------------------------------------------------------------------------------------------------------+
Protected labels
----------------
A few labels are "protected"; i.e., they are only applied by GraceDB itself as part of another process.
These labels **cannot** be applied manually by users and any requests to do so will be rejected by the server.
At present, the following labels are protected as a part of the signoff infrastructure and are only applied by GraceDB after an operator or advocate signoff:
- H1OK, L1OK, V1OK, ADVOK
- H1NO, L1NO, V1NO, ADVNO
docs/user_docs/source/lvalert.rst deleted 100644 → 0
View file @ b4e766bd
=====================================
Integration with LVAlert
=====================================
Introduction
===============================================
GraceDB uses `LVAlert
<https://wiki.ligo.org/DASWG/LVAlert>`__ to send
alerts to listeners within the LVC. The content of the LVAlert message is
designed to convey actionable information about a state change in GraceDB,
whether it involves the creation of a new event, or the updating or labeling of
an existing one.
.. NOTE::
An LVAlert message is sent out for *any* new event or annotation that arrives in the
GraceDB database. This means that
a message volumes may be very high under certain circumstances, and
appropriate filtering is required in order for LVAlert to be useful.
Listening to specific event streams
==============================================
By running ``lvalert_listen``, you will receive messages over all **nodes** to
which you are subscribed. The node names consist of at least two elements::
<group_name>_<pipeline_name>
In other words, the (lower-cased) names of the Group and Pipeline separated by an
underscore. For example, the node ``burst_cwb`` would catch all messages relating to
events in the Burst group from the cWB pipeline. One can also specify the search name::
<group_name>_<pipeline_name>_<search_name>
which has the effect of narrowing down the messages to only those related to a specific
search. For example, the node ``burst_cwb_allsky`` will contain messages relating to the
AllSky search, but not the MDC search. GraceDB tries to send a message to all applicable
nodes. Thus, a message sent to the node ``burst_cwb_allsky`` will *also* be sent to the
node ``burst_cwb``. This property allows the user to filter according to search by
specifying different LVAlert processing scripts for different nodes.
To see the names of all available nodes, simply execute::
lvalert_admin -a username -i
LVAlert message contents
================================================
GraceDB sends messages as a JSON-encoded dictionary. The dictionary contains the
following keys:
- ``uid``: the unique ID (a.k.a. ``graceid``) of the relevant event
- ``alert_type``: ``new``, ``update``, or ``label``
- ``description``: a text description (if applicable)
- ``file``: a URL for the relevant file (if applicable)
- ``object``: a dictionary representing the relevant object
For example, when a new event is created, an LVAlert message is created
with alert type ``new``, and the ``object`` is just the JSON representation of
of the event provided by the REST interface (see :ref:`searching_for_events`).
Below are examples of the possible types of LVAlert message. These were generated
by creating a new event, adding a couple of log messages, creating a Preliminary
VOEvent, and applying the DQV label (in that order).
New event::
{
"alert_type": "new",
"description": "",
"file": "https://gracedb.ligo.org/events/T129911/files/coinc.xml",
"object": {
"created": "2015-06-17 22:10:43 UTC",
"extra_attributes": {
"CoincInspiral": {
"combined_far": 3.772326334623149e-14,
"end_time": 968929613,
"end_time_ns": 817383681,
"false_alarm_rate": 3.654963804145501e-08,
"ifos": "H1,L1",
"mass": 2.621732950210571,
"mchirp": 1.139938473701477,
"minimum_duration": null,
"snr": 19.73621083881572
}
},
"far": 3.772326334623149e-14,
"gpstime": 968929613.8173836,
"graceid": "T129911",
"group": "Test",
"instruments": "H1,L1",
"labels": {},
"likelihood": 8.33784842725385e+44,
"links": {
"embb": "https://gracedb.ligo.org/api/events/T129911/embb/",
"files": "https://gracedb.ligo.org/api/events/T129911/files/",
"labels": "https://gracedb.ligo.org/api/events/T129911/labels/",
"log": "https://gracedb.ligo.org/api/events/T129911/log/",
"neighbors": "https://gracedb.ligo.org/api/events/T129911/neighbors/",
"self": "https://gracedb.ligo.org/api/events/T129911",
"tags": "https://gracedb.ligo.org/api/events/T129911/tag/"
},
"nevents": 2,
"pipeline": "gstlal",
"search": "MDC",
"submitter": "branson.stephens@LIGO.ORG"
},
"uid": "T129911"
}
Log message without file::
{
"alert_type": "update",
"description": "LOG: This is a test.",
"file": "",
"object": {
"N": 4,
"comment": "This is a test.",
"created": "2015-06-17T17:10:43.381117",
"file": null,
"file_version": null,
"filename": "",
"issuer": {
"display_name": "Branson Stephens",
"username": "branson.stephens@LIGO.ORG"
},
"self": "https://gracedb.ligo.org/api/events/T129911/log/4",
"tag_names": [
"analyst_comments"
],
"tags": "https://gracedb.ligo.org/api/events/T129911/log/4/tag/"
},
"uid": "T129911"
}
Log message with a file::
{
"alert_type": "update",
"description": "UPLOAD: bayestar.fits This is a file.",
"file": "bayestar.fits",
"object": {
"N": 6,
"comment": "This is a file.",
"created": "2015-06-17T17:10:43.980188",
"file": "https://gracedb.ligo.org/api/events/T129911/files/bayestar.fits%2C0",
"file_version": 0,
"filename": "bayestar.fits",
"issuer": {
"display_name": "Branson Stephens",
"username": "branson.stephens@LIGO.ORG"
},
"self": "https://gracedb.ligo.org/api/events/T129911/log/6",
"tag_names": [
"sky_loc"
],
"tags": "https://gracedb.ligo.org/api/events/T129911/log/6/tag/"
},
"uid": "T129911"
}
New VOEvent created::
{
"alert_type": "update",
"description": "VOEVENT: T129911-1-Preliminary.xml",
"file": "T129911-1-Preliminary.xml",
"object": {
"N": 1,
"created": "2015-06-17T17:10:44.172876",
"file": "https://gracedb.ligo.org/api/events/T129911/files/T129911-1-Preliminary.xml%2C0",
"file_version": 0,
"filename": "T129911-1-Preliminary.xml",
"issuer": {
"display_name": "Branson Stephens",
"username": "branson.stephens@LIGO.ORG"
},
"ivorn": "ivo://gwnet/gcn_sender#T129911-1-Preliminary",
"self": "https://gracedb.ligo.org/api/events/T129911/voevent/1",
"text": "<voevent text>",
"voevent_type": "PR"
},
"uid": "T129911"
}
New DQV label applied::
{
"alert_type": "label",
"description": "DQV",
"file": "",
"uid": "T129911"
}
.. XXX what kind of alert does a replacement trigger?
Receiving and Parsing LVAlert messages
====================================================
The LVAlert client tools include the ``lvalert_listen`` executable, which can be used to
receive and respond to LVAlert messages::
lvalert_listen -a username -c /path/to/lvalert_config.ini
The ``-c`` (configuration file) option allows you to specify an executable script to be called
each time a message arrives over a particular node. Suppose you are only interested in
events from the burst group, cWB pipeline, and MDC search. Then the
``lvalert_config.ini`` file could look like this::
[burst_cwb_mdc]
executable = /path/to/mdc_event_handler
And the script ``mdc_event_handler`` could be any script that is prepared to receive the
LVAlert message contents through standard input. Here is an example in Python::
#!/usr/bin/env python
import json
from sys import stdin
# Load the LVAlert message contents into a dictionary
streamdata = json.loads(stdin.read())
# Do something with new events having FAR below threshold
alert_type = streamdata['alert_type']
if alert_type == 'new':
# The object is a serialized event. Get the FAR
far = streamdata['object']['far']
if far < 1.e-6:
# Do some interesting processing
pass
Further reading on LVAlert
=====================================================
Further information on using LVAlert can be found on the
`LVAlert Project Page <https://wiki.ligo.org/DASWG/LVAlert>`__
and the `LVAlert Howto <https://wiki.ligo.org/DASWG/LVAlertHowto>`__.
docs/user_docs/source/lvem.rst deleted 100644 → 0
View file @ b4e766bd
=============================
Features for EM Collaboration
=============================
On logging in
=============
A successful login is required in to access GraceDB events and upload
followup information. The login process is the same as for the
`LV-EM wiki <https://gw-astronomy.org/wiki/LV_EM/WebHome>`__: namely,
click "LOGIN" at the upper right and then choose the login method
according to the identity you used for registering for LV-EM membership at
`gw-astronomy.org <https://gw-astronomy.org>`__.
.. NOTE::
Some users may have multiple identities available from the identity providers listed
on the login page. However, only the identity used to register for LV-EM
will work for GraceDB access. For example, even though I have identities from
LIGO, UW-Milwaukee, and Google, only my LIGO login will work for GraceDB since that
is the one I used to register for LV-EM membership. The reason is that
there is no way (at present) to map these different identities to the same
underlying user.
.. _basic_auth_for_lvem:
Scripted access for LV-EM members
============================================
Some processes need to access GraceDB in a *scripted* manner. For example,
an observational group might set up an automated process to listen for LIGO/Virgo GCN
notices and then download the skymaps for further processing
(see the `tutorial <http://nbviewer.ipython.org/github/lpsinger/ligo-virgo-emfollowup-tutorial/blob/master/ligo-virgo-emfollowup-tutorial.ipynb>`__).
As these alerts could come at any time of the day or night, it is not
generally possible for the user to go through the usual login sequence. Traditionally,
GraceDB has handled scripted access with X509 robot certificates or
robot Kerberos keytabs, but these may not be easily accessible to all
LV-EM group members.
Thus, there is an alternative using basic auth (a simple username-and-password
scheme). First, obtain a robotic
access password by navigating to `this page <https://gracedb.ligo.org/options/manage_password>`__
and clicking "Get me a password!" (or by clicking "OPTIONS" on the navigation
menu and then "Password Manager." Each time you click the button, you
will get a new basic auth password, and the old one will be lost. (Note
that these passwords only last for 1 year.) The password is a 20 character
random sequence.
.. NOTE::
This robotic password does not affect the way in which you login to
the GraceDB web interface. It is only for use with the REST interface
as described in the examples below. You will need to continue logging
into the web interface using the identity with which you registered for
LV-EM membership.
Once you've obtained a robotic password, the best way to use it is to create
a ``.netrc`` file containing your username and password (with permissions ``0600``
to make sure that only you can read it). The ``.netrc`` file could look like this::
machine gracedb.ligo.org
login myself@institution.edu
password abc123.....
Place the resulting ``.netrc`` file in your home directory.
Once that's done, you should be able to access the GraceDB REST API
using any tool that supports basic auth.
For example, you can use the GraceDB Python client in much the same
way as described in :ref:`rest_client_basic_usage`, except that the
client class is specially formulated for basic auth::
from ligo.gracedb.rest import GraceDbBasic, HTTPError
service_url = 'https://gracedb.ligo.org/apibasic/'
client = GraceDbBasic(service_url)
try:
r = client.ping()
except HTTPError, e:
print e.message
print "Response code: %d" % r.status
print "Response content: %s" % r.json()
The only real difference is that the ``GraceDbBasic`` client class is used instead
of the ``GraceDb`` class (which assumes that X509 credentials are available).
If you're not comfortable using Python for scripted access to GraceDB, it is
also possible to use ``curl`` to directly make requests to the server with the
same basic auth credentials. Some examples of using curl are available
`here <https://gw-astronomy.org/wiki/LV_EM/TechInfo>`__.
Downloading a skymap
======================
The GraceDB Python client can be used to download
files from Gracedb or add comments, plots, or observation records (see
the next section). Here, we'll
show an example of downloading a skymap. Suppose we know that a particular
GraceDB event (``T125738``) has a skymap file called ``bayestar.fits.gz``.
This file can be retrieved in the following way::
from ligo.gracedb.rest import GraceDbBasic
grace_id = 'T125738' # identifier for the event
filename = 'bayestar.fits.gz' # filename of desired skymap
# Prepend with grace_id for output filename
out_filename = grace_id + '_' + filename
# Instantiate the GraceDB client
service_url = 'https://gracedb.ligo.org/apibasic/'
client = GraceDbBasic(service_url)
# Grab the file from the server and write it
out_file = open(out_filename, "w")
r = client.files(grace_id, filename)
out_file.write(r.read())
out_file.close()
.. _create_emobservation:
Reporting coordinates of followup observations
===============================================
In the following example, the GraceDB Python client is used to create an
observation record consisting of three separate footprints::
# Define the parameters of the observation to be reported
grace_id = 'M158044' # the event's UID
group = 'CRTS' # the MOU group
comment = 'hello my friend' # free text comment
raList = [123.0,124.0,125.0] # RAs of centers (degrees)
decList = [10.0,11.0,13.0] # Dec of centers (degrees)
startTimeList = [ # beginnings of exposures (UTC)
'2015-05-31T12:45:00',
'2015-05-31T12:49:00',
'2015-05-31T12:53:00']
raWidthList = 10.0 # list (or one for all) of widths in RA (degrees)
decWidthList = 10.0 # list (or one for all) of widths in Dec (degrees)
durationList = 20.0 # list (or one for all) of exposure times in sec
# Instantiate the GraceDB client
client = GraceDbBasic()
# Write the EMObservation record to GraceDB
r = client.writeEMObservation(grace_id, group, raList, raWidthList,
decList, decWidthList, startTimeList, durationList, comment)
if r.status == 201: # 201 means 'Created'
print 'Success!'
Note that the start times are always assumed to be in UTC. For users not
familiar with Python, there are several other options available for uploading
observation records:
- by using the webform on each event page (scroll down to the 'EM Observations'
section and click on 'add observation record'). However, this method requires
by-hand data entry.
- by ``curl``-ing directly against the EM observation
resource in the API (`example <https://gw-astronomy.org/wiki/LV_EM/CurlUploadFootprints>`__)
- by coding against the GraceDB REST API
in one's own favorite language. If you choose to go this route, please
consider sending us your script or posting it in the LV-EM wiki Technical Info
page for the benefit of other users. See :ref:`coding_against_api`.
- by email (not yet available, but in the works)
If you discover a mistake in your observation record, the best way to correct
it is to submit a new observation record with corrected values and request that
the old one be deleted. Please send an email to uwm-help@cgca.uwm.edu with
something like "delete GraceDB EMObservation" in the subject line. Tell us
which entry you'd like deleted, and we'll take care of it. In the future, we
are hoping to make these observation records editable by the submitter.
For more on the GraceDB event page and creating EM observation records, see
`this <https://www.youtube.com/watch?v=oIJE4dTISs4>`__ helpful video
by Roy Williams. There is a companion video on the SkymapViewer
`here <https://www.youtube.com/watch?v=ydXUD9KIN98>`__.
docs/user_docs/source/models.rst
View file @ 5797d777
.. _models:
==========================
Data models
Data Models
==========================
What characterizes an event?
......@@ -8,57 +10,259 @@ What characterizes an event?
The different types of events in GraceDB are distinguished by the following parameters:
- ``Group``: the working group responsible for finding the candidate
- values: ``CBC``, ``Burst``, ``External``, ``Test``
- values: ``CBC``, ``Burst``, ``Detchar``, ``External``, ``Test``
- ``Pipeline``: the data analysis software tool used make the detection
- values: ``MBTAOnline``, ``CWB``, ``gstlal``, ``spiir``, ``HardwareInjection``, ``Fermi``, ``Swift``, ``SNEWS``, ``oLIB``
- values: ``CWB2G``, ``spiir``, ``HardwareInjection``, ``X``, ``Q``, ``Omega``, ``Ringdown``, ``Fermi``, ``Swift``, ``CWB``, ``SNEWS``, ``oLIB``, ``pycbc``, ``INTEGRAL``, ``AGILE``, ``gstlal``, ``MLy``, ``MBTAOnline``, ``MBTA``, ``CHIME``, ``PyGRB``, ``aframe``, ``SVOM``, ``IceCube``, ``GWAK``
- ``Search``: the search activity which led to the detection
- values: ``AllSky``, ``AllSkyLong``, ``LowMass``, ``HighMass``, ``GRB``, ``Supernova``, ``MDC``
- values: ``AllSky``, ``LowMass``, ``HighMass``, ``GRB``, ``Supernova``, ``MDC``, ``LowMassSim``, ``AllSkyLong``, ``O2VirgoTest``, ``BBH``, ``IMBH``, ``SubGRB``, ``EarlyWarning``, ``SubGRBTargeted``, ``SSM``, ``FRB``, ``LensingSubthreshold``, ``VTInjection``, ``HEN``
An individual "event stream" is specified by setting the values of these three parameters.
For example, choosing ``Group=CBC``, ``Pipeline=gstlal``, and ``Search=LowMass`` selects the event stream consisting of low-mass inspiral events detected by the gstlal pipeline from the CBC group.
This framework was chosen in order avoid situations where events from different sources would overlap in searches and alerts.
An individual "event stream" is specified by setting the values of these three
parameters. For example, choosing ``Group=CBC, Pipeline=gstlal, and Search=LowMass``
selects the event stream consisting of low-mass inspiral events
detected by the gstlal pipeline from the CBC group. This framework was chosen
in order avoid situations where events from different sources would overlap in searches
and alerts.
.. _base_event_model:
Base event model
====================================
----------------
In addition to the three parameters described above, there are additional
common attributes for all events. These are
In addition to the three parameters described above, there are additional common attributes for all events.
These are:
- ``submitter``: the user who submitted the event
- ``created``: the time at which the event was created
- ``instruments``: the interferometers involved in the detection
- ``far``: the false alarm rate in Hz
- ``gpstime``: the time at which the event occurred (a.k.a. "Event time")
- ``reporting_latency``: defined as the time difference between when an event lands
on GraceDB (``created``) and the reported ``gpstime`` (in seconds)
- ``superevent``: the ``superevent_id`` of the event' parent superevent, if
applicable
- ``superevent_neighbours``: superevents currently in GraceDB whose ``t_0`` is
within a set time window of the event's ``gpstime`` (currently ± 100s). Value
is a dictionary, whose key is the ``superevent_id`` and value is the
:ref:`superevent dictionary<superevent_data_model>`
The base event class was created with GW events in mind, so not all of the fields
will be applicable for any given event. (For example, ``instruments`` and ``far``
do not apply to a Swift GRB event.)
The base event class was created with GW events in mind, so not all of the fields will be applicable for any given event.
For example, ``instruments`` and ``far`` do not apply to a Swift GRB event.
Event subclasses
====================================
----------------
Most events also have pipeline-specific attributes, and these are reflected in event
subclasses. For example, the ``gstlal`` pipeline produces an estimate for the chirp
mass, which is represented in the ``CoincInspiral`` event subclass. The following table
shows the different subclasses with selected attributes:
Most events also have pipeline-specific attributes, and these are reflected in event subclasses.
For example, the ``gstlal`` pipeline produces an estimate for the chirp mass, which is represented in the ``CoincInspiral`` event subclass.
.. raw:: html
.. _annotation_models:
<div id="subclasses_table"></div>
Serialized events
-----------------------------
Event objects are serialized into JSON format in responses from the API and in igwn-alert messages.
Here, we show some examples for the different event subclasses.
.. _annotation_models:
CBC pipelines (gstlal, spiir, PyCBC, MBTAOnline)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. literalinclude:: dicts/event_cbc.json
:language: JSON
:force:
CWB
~~~
.. literalinclude:: dicts/event_cwb.json
:language: JSON
oLIB
~~~~
.. literalinclude:: dicts/event_olib.json
:language: JSON
Machine Learning (MLy, aframe, GWAK)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. literalinclude:: dicts/event_ml.json
:language: JSON
External (GRB)
~~~~~~~~~~~~~~
.. literalinclude:: dicts/event_external_grb.json
:language: JSON
External (Neutrino)
~~~~~~~~~~~~~~~~~~~
.. literalinclude:: dicts/event_external_neutrino.json
:language: JSON
Superevents
===========
What is a superevent?
---------------------
In reality, what we called "events" above might be better characterized as "triggers", since different analysis pipelines may detect the same physical GW event and submit it to GraceDB.
In order to collect the information about a single physical event in one place, reduce the amount of follow-up processing needed, and issue alerts about only one GraceDB entry per physical event, we have created the "superevent" data model.
A `downstream process <https://rtd.igwn.org/projects/gwcelery/en/latest/index.html>`__ separate from GraceDB listens for event creations, analyzes their attributes, and determines how to aggregate events into superevents.
.. _superevent_data_model:
Data model
----------
The main attributes of the superevent data model are:
- ``superevent_id``: a unique date-based ID (Example: ``S180912b``; more information below in :ref:`superevent_date_ids`)
- ``gw_id``: a unique date-based ID only assigned to superevents which are confirmed GWs (Example: ``GW180915BC``; more information below in :ref:`superevent_date_ids`)
- ``category``: superevent category (``Production``, ``Test``, or ``MDC``); more information below in :ref:`superevent_categories`
- ``gw_events``: list of graceids corresponding to Event objects which are part of this superevent and were submitted by GW analysis pipelines
- ``preferred_event``: the ``graceid`` of the superevent's preferred event
- ``preferred_event_data``: the :ref:`event dictionary<base_event_model>` of the
superevent preferred event
- ``em_events``: list of graceids corresponding to Event objects which are part of this superevent and are in the "External" group (i.e., were observed by electromagnetic or neutrino telescopes)
- ``created``: time at which the superevent was created
- ``submitter``: user who created the superevent
- ``t_start``: time corresponding to start of window for grouping events into this superevent
- ``t_end``: time corresponding to end of window for grouping events into this superevent
- ``t_0``: best estimate of time at which the GW event occurred
Serialized superevent
~~~~~~~~~~~~~~~~~~~~~
Here is an example of a superevent which has been serialized into a JSON:
.. literalinclude:: dicts/superevent.json
:language: JSON
.. _superevent_categories:
Categories
----------
There are three categories of superevents:
- ``Production``: "real" superevents which correspond to potential GW events
- ``Test``: used for generic testing superevent creation, annotation, etc.
- ``MDC``: generated as part of the mock data challenge (MDC), which comprises a constant stream of events which are sent to GraceDB for testing by internal and LV-EM users
Each category of superevent may only contain events of the corresponding type; i.e., ``Production`` superevents may only contain production (G, E, and H-type) events, ``Test`` superevents may only contain test (T-type) events, and ``MDC`` superevents may only contain MDC (M-type) events.
Letter suffixes for superevent date-based IDs are also calculated independently for each superevent category (see :ref:`superevent_date_ids`).
.. _superevent_date_ids:
Date-based IDs
--------------
We generate IDs for superevents based on the date of their occurrence, at the time of creation.
These IDs have three parts:
- Prefix
- Six digit (``YYMMDD``) date string
- Letter suffix
A few examples are:
- ``S180920abc``
- ``TS180717b``
- ``GW170817A``
Prefix
~~~~~~
The prefix is determined by the superevent's "status" (is it marked as a "confirmed GW" or not) and its category.
A superevent's category will **never** change, but its status can, resulting in a prefix change.
When this happens, the superevent will be accessible (via URLs, the API, searches, etc.) by both its old and new IDs.
+----------------+--------------------+--------------+
| Category | Not a confirmed GW | Confirmed GW |
+================+====================+==============+
| ``Production`` | S | GW |
+----------------+--------------------+--------------+
| ``Test`` | TS | TGW |
+----------------+--------------------+--------------+
| ``MDC`` | MS | MGW |
+----------------+--------------------+--------------+
Date string
~~~~~~~~~~~
The date string is determined by the superevent's value for ``t_0`` at creation.
This is converted from GPS to UTC time and turned into a ``YYMMDD`` string.
Note that a superevent's ``t_0`` value may change as additional events are added to it, but this will not change the date string in the superevent's ID.
Because these date strings are only 6 digits, they are degenerate over a period of 100 years.
We currently define this period to run from 1980-01-01 00:00:00 UTC to 2079-12-31 23:59:59 UTC.
If GraceDB is still around at that point, we will probably have to switch to four-digit years.
Letter suffix
~~~~~~~~~~~~~
The letter suffix is determined by the chronological ordering of superevents observed on a given date by **creation time in GraceDB**, *not* by the occurence time of the actual GW event.
This is because a unique ID must be determined at creation time and it is not realistic to recalculate it every time a new superevent is created.
The letter suffix just corresponds to the superevent's number within this chronological ordering.
To be explicit: 1 = ``a``, 2 = ``b``, 27 = ``aa``, and so on.
This suffix is calculated **independently** for each category of superevent.
This means that the first ``Production`` superevent for a given date will have a suffix of ``a``, and so will the first ``Test`` superevent for that date.
In this case, ID uniqueness is preserved by the fact that these categories have different prefixes.
When a superevent is confirmed as a GW, a new letter suffix is determined based on how many superevents for the same date have already been confirmed as a GW.
Again, this is dependent on the chronological ordering of the time at which this confirmation action takes place, and not by the occurrence time of the actual GW event.
Uppercase letters are used for suffixes for confirmed GWs.
Example scenario
~~~~~~~~~~~~~~~~
Here's a hypothetical series of superevent creations occurring on March 31, 2018:
- A ``Production`` superevent is created at 12:00:00 UTC with a ``t_0`` corresponding to 11:30:00 UTC
- Another ``Production`` superevent is created at 12:05:00 with a ``t_0`` corresponding to 11:00:00 UTC
- A ``Test`` superevent is created at 12:10:00 with a ``t_0`` corresponding to 12:05:00 UTC
- A third ``Production`` superevent is created at 12:15:00 with a ``t_0`` corresponding to 10:00:00 UTC
The resulting set of superevent IDs will be:
+---------------------+---------------+----------------+---------------+
| Creation time (UTC) | ``t_0`` (UTC) | Category | Superevent ID |
+=====================+===============+================+===============+
| 12:00:00 | 11:30:00 | ``Production`` | ``S180331a`` |
+---------------------+---------------+----------------+---------------+
| 12:05:00 | 11:00:00 | ``Production`` | ``S180331b`` |
+---------------------+---------------+----------------+---------------+
| 12:10:00 | 12:05:00 | ``Test`` | ``TS180331a`` |
+---------------------+---------------+----------------+---------------+
| 12:15:00 | 10:00:00 | ``Production`` | ``S180331c`` |
+---------------------+---------------+----------------+---------------+
Now let's say S180331b is confirmed as a GW at 13:00:00 UTC and S180331a is confirmed as a GW at 13:05:00 UTC.
This would result in the following superevent IDs:
+---------------------+---------------+----------------+---------------+
| Creation time (UTC) | ``t_0`` (UTC) | Category | Superevent ID |
+=====================+===============+================+===============+
| 12:00:00 | 11:30:00 | ``Production`` | ``GW180331B`` |
+---------------------+---------------+----------------+---------------+
| 12:05:00 | 11:00:00 | ``Production`` | ``GW180331A`` |
+---------------------+---------------+----------------+---------------+
| 12:10:00 | 12:05:00 | ``Test`` | ``TS180331a`` |
+---------------------+---------------+----------------+---------------+
| 12:15:00 | 10:00:00 | ``Production`` | ``S180331c`` |
+---------------------+---------------+----------------+---------------+
Note that the "upgraded" superevents would still be accessible through the web interface, API calls, searches, etc. by **either** of their IDs.
Annotations
=======================
*Annotations* are pieces of information about an event that that are added
after the event is created. They are often the results of followup processes,
but are sometimes also provided by the same data analysis pipeline that
initially generated the event. The most common type of annotation is an *event
log message* with the following fields:
*Annotations* are pieces of information about an event or superevent that that are added after the event or superevent is created.
They are often the results of followup processes, but are sometimes also provided by the same data analysis pipeline that initially generated the event.
The most common type of annotation is an *log message* with the following fields:
- ``submitter``: the user who created the log message
- ``created``: the time at which the log message was created
......@@ -66,12 +270,9 @@ log message* with the following fields:
- ``file_version``: the specific version of the file for this message
- ``comment``: the log message text
If the uploaded file is an image, it is displayed along with the comment in the
GraceDB event page. Log messages can also be *tagged* in order to give other
users an idea of the thematic category to which the message belongs. Users can
invent arbitrary tags, but the following set have a special status, as they
affect the display of information in the event page (i.e., the are
*"blessed"*):
If the uploaded file is an image, it is displayed along with the comment in the GraceDB event or superevent page.
Log messages can also be *tagged* in order to give other users an idea of the thematic category to which the message belongs.
Users can invent arbitrary tags, but the following set have a special status, as they affect the display of information in the event page (i.e., they are *"blessed"*):
- ``analyst_comments``: Analyst Comments
- ``em_follow``: EM Followup
......@@ -86,4 +287,40 @@ affect the display of information in the event page (i.e., the are
- ``sig_info``: Significance Info
- ``audio``: Sound Files
Other types of annotations are labels, VOEvent objects, and EM observation records.
Other types of annotations are labels, VOEvent objects, EM observation records, and signoffs.
Serialized annotations
----------------------
Here we show some examples of annotations which have ben serialized into JSON format.
These annotations are basically identical whether they are attached to an event or superevent, and these examples show some of each case.
Log
~~~
.. literalinclude:: dicts/log.json
:language: JSON
Label
~~~~~
.. literalinclude:: dicts/label.json
:language: JSON
Signoff
~~~~~~~
.. literalinclude:: dicts/signoff.json
:language: JSON
Notes:
- ``status``: can be "OK" or "NO"
- ``signoff_type``: can be "ADV" (EM advocate) or "OP" (instrument control room operator)
- ``instrument``: two-letter instrument code, like "H1", "L1", "V1", etc. For advocate signoffs, this is an empty string.
EMObservation
~~~~~~~~~~~~~
.. literalinclude:: dicts/emobservation.json
:language: JSON
VOEvent
~~~~~~~
.. literalinclude:: dicts/voevent.json
:language: JSON
docs/user_docs/source/notifications.rst 0 → 100644
View file @ 5797d777
.. _notifications:
==============================================
Phone and email notifications (LVK users only)
==============================================
Phone and email notifications are issued under various update scenarios to LVK users who have registered for them.
This page will provide some details on the notification sign-up process and the logic involved in issuing these notifications.
Notification signup
===================
LVK users can find the main notifications page by clicking on the "Notifications" tab in the navigation bar.
Here, you will find two headings: contacts and notifications.
Creating a contact
------------------
Click on the "Create new contact" button on the main notifications page to begin the contact creation process.
Once you have reached the contact creation page, choose a phone or email contact by clicking on the corresponding tab.
For phone contacts, you can receive notifications via a text message, voice call, or both.
Fill in the form and submit it to finish the contact creation process.
Verifying a contact
-------------------
Once you have successfully created a contact, you will be returned to the main notifications page.
You should see your newly created contact displayed alongside some buttons.
Your contact is currently unverified, and you cannot receive notifications via this contact until it is verified.
Click on the yellow "Verify" button to the right of the contact to begin the verification process.
Once you have reached the verification page, you can click on the "Request a verification code" button to receive a verification code for your contact.
You will receive the verification code via the following mechanism:
- Email for email contacts
- Voice call for phone contacts who have specified "call"
- Text message for phone contacts who have specified "text" or "both"
Verification codes last for 60 minutes.
If your code expires or if you misplace it, you can request a new verification code.
Once you have your verification code, enter it in the form that is now displayed and hit submit.
If you have entered the code correctly, you will be returned to the main notifications page and your contact will now be marked as "Verified".
Modifying a contact
-------------------
You can edit a contact by clicking on the "Edit" button next to the contact information on the main notifications page.
You are not permitted to modify the contact information, as that would allow the verification process to be circumvented.
You may edit the description for either type of notification, or the phone contact method for phone notifications.
You may also delete a contact at any time, using the "Delete" button next to the contact information on the main notifications page.
If you create a new contact with the same information, it will need to be reverified.
Testing a contact
-----------------
Use the "Test" button next to the contact information on the main notifications page to send a test email, text message, and/or voice call to your contact.
Only verified contacts can be tested.
Creating a notification
-----------------------
A notification is a set of criteria which is linked to a contact or contacts.
It specifies under which conditions the corresponding contacts should receive an notification.
To create a notification, you must have already created and verified a contact.
Click on the "Create new notification" button on the main notifications page.
Once you have reached the notification creation page, decide whether you want to receive notifications about superevents or events (most users will likely want superevents) and click the corresponding tab.
First, you should enter a description of this notification and select a contact or contacts to receive notifications when this notification is triggered (you can select multiple contacts by holding CTRL and clicking).
The rest of the fields in either form correspond to criteria which must be met for the notification to be triggered.
More information about each field will be provided in the following section where the logic for triggering a notification is discussed.
A few notes here:
- The label query field allows a complex filter on the requirement for which labels are/are not applied to an event or superevent. To construct a label query, combine label names with binary AND: ('&' or ',') or binary OR: '|'. They can also be negated with '~' or '-'. For N labels, there must be exactly N-1 binary operators. Parentheses are not allowed. It is suggested to specify at least one label that is not negated, or you may receive more notifications than you would like. Operator precedence is: NOT, AND, OR.
- You can specify either a set of labels or a label query, but not both.
- An event or superevent is considered a neutron star candidate if its source is thought to be a compact binary system where the secondary object has a mass less than 3 M\ :sub:`sun`\ . You can restrict your notifications to only events or superevents which are considered to be a neutron star candidate by checking the corresponding box in the form.
Notification logic
==================
Notifications will be issued in the following situations:
- Event or superevent creation
- Event or superevent update
- Label application to or label removal from an event or superevent
For each type of notification, there is a set of conditions that correspond to a state change which can trigger notifications.
For example: for an update notification, we don't trigger every notification whose FAR threshold is satisfied by the event's FAR, but only those where the event update changes the event's FAR from above threshold to below threshold.
All requirements on other parameters (NS candidate status, label status, etc.) must still be satisfied.
**Notifications are not sent for:**
- **Test or MDC events or superevents**
- **Offline events or for superevents whose preferred events are marked as offline**
Creation notifications
----------------------
When an event or superevent is created, all notifications whose criteria on FAR, NS candidate status, and group, pipeline, and search requirements (events only) are satisfied will be triggered.
Labels are not applied to events or superevents at creation time, so any notifications which have a label requirement will not be triggered.
Update notifications
--------------------
When an event or superevent is updated, all notifications will be triggered whose criteria on FAR, NS candidate status, group, pipeline, and search requirements (events only), and label status are met, **AND**:
- The update changes the event or superevent's FAR from above the notification's FAR threshold to below it, **OR**
- The update changes the event or superevent's NS candidate status from False to True, and the notification requires an NS candidate.
Label added notifications
-------------------------
When a label is applied to event or superevent, all notifications will be triggered whose criteria on FAR, NS candidate status, group, pipeline, and search requirements (events only), and label status, **AND**:
- The label that was just added matches one of the labels required by the notification (i.e., the addition of this label is the final requirement to satisfy the notification's label or label query requirements).
Label removed notifications
---------------------------
Same as "label added" notifications, except that the label that was just removed must be the final requirement to satisfy the notification's label query requirements.
docs/user_docs/source/queries.rst 0 → 100644
View file @ 5797d777
===================================
Querying for events and superevents
===================================
Introduction
===============================================
This section gives an introduction to searches in GraceDB.
Searches can be done from the "Search" and "Latest" pages in the web interface or through the API when using the ``ligo-gracedb`` client package.
In the web interface, use the dropdown menu to set the search type (superevent or event).
When doing a search through the API with the client package, use the ``events()`` and ``superevents()`` methods to query for events and superevents, respectively.
Queries sometimes have a keyword, which may or may not be required.
In general, a query looks like ``keyword: value``.
Multiple attributes can be included in a query (ex: ``key1: val1 key2: val``).
The sections below show different attributes that can be queried on and the corresponding syntax.
Some information on combining queries is provided at the end.
Event queries
=============
**NOTE:** clicking the 'Get neighbors' checkbox will result in an additional neighbors query for each item in the search results, and the neighbors are thus shown in the results table. However, this causes the overall query to take longer, which is why it is un-checked by default.
By instruments
--------------
Order matters, because the instruments are stored in the database as a string.
Examples:
- ``instruments: "H1,L1,V1"``
- ``instruments: "V1,L1"``
By false alarm rate
-------------------
- ``far < 1e-7``
- ``far: 3.6823e-4``
- ``far >= 2e-9``
By event attributes
-------------------
Relational and range queries can be made on selected event attributes.
Examples:
- ``singleinspiral.mchirp >= 0.5 & singleinspiral.eff_distance in 0.0,55``
- ``(si.channel = "DMT-STRAIN" | si.channel = "DMT-PAIN") & si.snr < 5``
- ``mb.snr in 1,3 & mb.central_freq > 1000``
Attributes in the common event object (e.g. ``gpstime``, ``far``, ``instruments``) do not need qualifiers.
Attributes specific to inspiral or burst events, for example, require qualification.
Abbreviations are available: ``si`` for ``singleinspiral``, ``ci`` for coincinspiral and ``mb`` for ``multiburst``. A full mapping of available abbreviations are in the table below:
.. list-table:: Event Subclass Query Abbreviations
:widths: 25 25
:header-rows: 1
* - Event Subclass
- Query Abbreviation
* - ``coincinspiralevent``
- ``ci``, ``coincinspiral``
* - ``grbevent``
- ``grb``
* - ``lalinferenceburstevent``
- ``li``
* - ``mlyburstevent``
- ``ml``
* - ``multiburstevent``
- ``mb``, ``multiburst``
* - ``siminspiralevent``
- ``inj``
* - ``singleinspiral``
- ``si``
Note that by a quirk of the query parser, querying for ``MLyBurstEvent`` attributes
**must** use the ``ml`` abbreviation. In other words, querying for
``mlyburstevent.detection_statistic > 1.0`` will fail, but ``ml.detection_statistic > 1.0``
will succeed.
By GPS time
-----------
Specify an exact GPS time, or a range.
Integers will be assumed to be GPS times, making the ``gpstime`` keyword optional.
Examples:
- ``899999000 .. 999999999``
- ``gpstime: 899999000.0 .. 999999999.9``
By creation time
----------------
Creation time may be indicated by an exact time or a range.
Date/times are in the format ``2009-10-20 13:00:00`` (must be UTC).
If the time is omitted, it is assumed to be ``00:00:00``.
Dates may also consist of certain variants of English-like phrases.
The ``created`` keyword is optional.
Examples:
- ``created: 2009-10-08 .. 2009-12-04 16:00:00``
- ``yesterday..now``
- ``created: 1 week ago .. now``
.. warning::
Due to a bug in GraceDB, it is recommended that you always include the
``created`` keyword, as some queries fail without it.
By graceid
----------
Graceids can be specified either individually, or as a range.
The ``gid`` keyword is optional.
Examples:
- ``gid: G2011``
- ``G2011 .. G3000``
- ``G2011 G2032 G2033``
By group, pipeline, and search
------------------------------
The ``group``, ``pipeline``, and ``search`` keywords are optional.
Names are case-insensitive.
Note that events in the Test group and MDC search will not be shown unless explicitly requested.
Examples:
- ``CBC Burst``
- ``group: Test pipeline: cwb``
- ``Burst cwb search: AllSky``
By label
--------
You may search for events with a particular label or set of labels.
The ``label`` keyword is optional.
Label names can be combined with binary AND: '&' or ',' or binary OR: '|'.
For N labels, there must be exactly N-1 binary operators (parentheses are not allowed).
Additionally, any of the labels in a query string can be negated with '~' or '-'.
Examples:
- ``label: INJ``
- ``EM_READY & ADVOK``
- ``H1OK | L1OK & ~INJ & ~DQV``
See :ref:`labels` for a list of current labels.
By submitter
------------
To specify events from a given submitter, indicate the name of the submitter in **double quotes**.
The ``submitter`` keyword is optional.
While LIGO user names are predictable, most events are submitted through robot accounts and are not as predictable.
This is probably a defect that ought to be remedied.
Examples:
- ``"waveburst"``
- ``submitter: "joss.whedon@ligo.org"``
By superevent status
--------------------
Use the ``in_superevent`` keyword to specify events which are/are not part of any superevent.
Use the ``superevent`` keyword to specify events which are part of a specific superevent.
Use the ``is_preferred_event`` keyword to specify events which are/are not preferred events for any superevent.
Examples:
- ``in_superevent: True``
- ``in_superevent: False``
- ``superevent: S180525c``
- ``is_preferred_event: True``
- ``is_preferred_event: False``
By run identifier
-----------------
Events (and superevents) can be queried by Observation/Engineering/Science run identifier,
which is based on a preset ``gpstime`` range. The ``runid:`` keyword is optional.
Examples and available options are below:
- ``runid: O4``
- ``O3``
- ``O1 O2``
.. list-table:: GraceDB Queryable Run ID's
:widths: 25 25
:header-rows: 1
* - runid
- gpstime/t_0 range
* - ``O4``
- (1422118818, 1443884418), (1396796418, 1422118818), (1368975618, 1389456018)
* - ``O4c``
- (1422118818, 1443884418)
* - ``O4b``
- (1396796418, 1422118818)
* - ``O4a``
- (1368975618, 1389456018)
* - ``ER16``
- (1394982018, 1396796418)
* - ``ER15``
- (1366556418, 1368975618)
* - ``O3``
- (1238166018, 1269363618)
* - ``ER14``
- (1235750418, 1238166018)
* - ``ER13``
- (1228838418, 1229176818)
* - ``O2``
- (1164556817, 1187733618)
* - ``O1``
- (1126623617, 1136649617)
* - ``ER8``
- (1123858817, 1126623617)
* - ``ER7``
- (1117400416, 1118329216)
* - ``ER6``
- (1102089616, 1102863616)
* - ``ER5``
- (1073822416, 1078876816)
* - ``ER4``
- (1057881616, 1061856016)
* - ``ER3``
- (1044136816, 1045785616)
* - ``ER2``
- (1026666016, 1028480416)
* - ``ER1``
- (1011601640, 1013299215)
* - ``ER1test``
- (1010944815, 1011601640)
* - ``S6``
- (931035296, 971622087)
* - ``S6A``
- (931035296, 935798487)
* - ``S6B``
- (937800015, 947260815)
* - ``S6C``
- (949449543, 961545687)
* - ``S6D``
- (956707143, 971622087)
Superevent queries
==================
Many of the queries for superevents are identical to that of events.
Only production superevents are returned by default.
See :ref:`superevent_query_category` for information on specifying Test or MDC superevents.
By id
-----
The keywords ``id`` or ``superevent_id`` are optional.
You can search by a superevent's S-type ID or its GW ID (if it has one).
See :ref:`superevent_date_ids` for more information on superevent IDs.
Examples:
- ``id: S180525a``
- ``superevent_id: S170817b``
- ``GW180428C``
- ``TS181212xz``
.. _superevent_query_category:
By category
-----------
Specify a superevent category (Production, Test, or MDC).
Only production superevents are returned by default.
The keyword ``category`` is optional.
Examples:
- ``Test``
- ``category: MDC``
By GPS time
-----------
Same as for event queries, with keywords ``gpstime`` or ``t_0``.
By other time attributes
------------------------
Queries based on the ``t_start`` and ``t_end`` attributes are also available and require the corresponding keywords.
Examples:
- ``t_start: 899999000``
- ``t_end: 899999000.0 .. 900000000.0``
By preferred event graceids
---------------------------
Specify an event graceid or range of event graceids with keyword ``preferred_event`` to get all superevents with corresponding preferred events.
Examples:
- ``preferred_event: G123456``
- ``preferred_event: G123456 .. G123500``
By event graceids
-----------------
Specify a graceid or range of graceids with keyword ``event`` to get all superevents which contain the corresponding event(s).
Examples:
- ``event: G123456``
- ``event: G123456 .. G123500``
By GW status
------------
Query for superevents which are confirmed as GWs or not with the ``is_gw`` keyword.
Examples:
- ``is_gw: True``
- ``is_gw: False``
By creation time
----------------
Same as for events.
By submitter
------------
Same as for events.
By label
--------
Same as for events.
By run identifier
-----------------
Same as for events.
By public status
----------------
Use the ``is_public`` or ``is_exposed`` keywords.
Examples:
- ``is_public: True``
- ``is_exposed: True``
Or, just add either "public" or "internal" to your query.
By preferred event FAR
----------------------
Examples:
- ``far < 1e-5``
- ``FAR >= 0.01``
- ``far in 1e-7, 2.5e-6``
Combining queries
=================
Queries can be combined by separating them with a space.
This effectively "AND"s them.
Do not use binary operators like '&' or '|' except between labels in a label-based query or in a query on selected event attributes (see examples above).
docs/user_docs/source/ref_manual.rst
View file @ 5797d777
......@@ -12,7 +12,9 @@ Contents:
models
web
rest
lvalert
lvem
auth
queries
labels
igwn_alert
notifications
docs/user_docs/source/responding_to_lvalert.rst deleted 100644 → 0
View file @ b4e766bd
==============================
Responding to LVAlert Messages
==============================
.. sectionauthor:: Reed Essick
This tutorial will show you how to
* register to receive LVAlerts
* subscribe and unsubscribe from pubsub nodes
* instantiate and manage an ``lvalert_listen`` instance
* interact with GraceDB through the Python REST interface in a script that is launched via ``lvalert_listen``
This tutorial assumes that the ``ligo-lvalert`` software package is already installed on
your machine (this is true on all cluster machines).
While we attempt to be pedagogically complete as much as possible, we would
like to stress that the existing documentation and help-strings for the
command-line and Python packages are *very* useful and should be the final
reference if you have any questions.
Registering to receive LVAlert messages
=======================================
LSC-Virgo members can activate accounts by simply completing the form
`here <https://www.lsc-group.phys.uwm.edu/cgi-bin/jabber-acct.cgi>`__.
If you need to create an account that is not attached to your user.name, you
can email uwm-help@cgca.uwm.edu and request an account. Once you have created an
account, you will be able to subscribe the account to different pubsub nodes
and receive lvalert messages.
Subscribing to pubsub nodes
===========================
LVAlert messages are broadcast through pubsub nodes and different messages go
through different nodes. For instance, all gstlal events created in GraceDB are
announced through the pubsub node called::
cbc_gstlal
which includes both highmass and lowmass events. If you instead want to only
receive lowmass events, these are announced through::
cbc_gstlal_lowmass
Importantly, if you subscribe to both ``cbc_gstlal`` and ``cbc_gstlal_lowmass``, you
will receive two alerts for every gstlal lowmass event. The general format of::
group_pipeline[_search]
is followed by all pubsub nodes used to announce events and annotations to
those events in GraceDB.
Fill out the form and follow all instructions to create an account attached to
your "user.name". For the rest of this tutorial, I will refer to the username
as "user.name", but you should replace this with your own account's information.
You'll be prompted for your password after each command.
*Note: To bypass this, create a .netrc file in your home directory and enter your
authentication information*::
machine lvalert.cgca.uwm.edu login user.name password passw0rd
*With this setup, you won't need to include the* ``-a`` *flag for your username,
or enter your password. Your .netrc file should only be accessible by you, so
be sure to do* ``chmod 600 $HOME/.netrc``.
To actually subscribe to a pubsub node, we use ``lvalert_admin``
which allows you to manage your subscriptions. This includes subscribing to new
nodes, unsubscribing from nodes and viewing your current subscriptions. We will
now subscribe your account to ``cbc_gstlal_lowmass``. Run::
lvalert_admin -a user.name --subscribe --node cbc_gstlal_lowmass
You can confirm that your account is successfully subscribed to this node by
running::
lvalert_admin -a user.name --subscriptions
which will list your account's subscriptions. You should see
``cbc_gstlal_lowmass`` in the resulting list. To unsubscribe from a node, use::
lvalert_admin -a user.name --unsubscribe --node cbc_gstlal_lowmass
but for now we'll leave our subscription in place. If you'd like to subscribe
to other nodes, simply repeat the subscription command and replace
``cbc_gstlal_lowmass`` with the name of the node to which you'd like to
subscribe. A complete list of nodes is available by running::
lvalert_admin -a user.name --get-nodes
For this tutorial, let's subscribe to another node to show how things scale. Run::
lvalert_admin -a user.name --subscribe --node cbc_gstlal_highmass
Creating an LVAlert node
========================
Users can create their own LVAlert pubsub nodes as well. Unsurprisingly, this
is also straightforward. Simply run::
lvalert_admin -a user.name --create --node user.name-TestNode
to create a node called ``user.name-TestNode``. Of course, you'll want to change
"user.name" to your account's name. Go ahead and create this node. If you need
to delete it at any time, you can with::
lvalert_admin -a user.name --delete --node user.name-TestNode
but leave it be for the moment. You now have a node owned by your account to
which you can publish alerts. We'll come back to this when we test our set-up.
You will also need to subscribe to this node with::
lvalert_admin -a user.name --subscribe --node user.name-TestNode
Run::
lvalert_admin -a user.name --subscriptions
and make sure you see::
cbc_gstlal_lowmass
cbc_gstlal_highmass
user.name-TestNode
in the output.
Starting and managing an ``lvalert_listen`` instance
====================================================
Now you have an lvalert account and it is subscribed to a few pubsub nodes.
It's time to set up an ``lvalert_listen`` instance which allows your code to
receive and react to announcements broadcast through the pubsub nodes. The
first thing you'll need is a config file. Using your favorite text editor,
create a file called ``myLVAlertListen.ini`` with the following as its contents::
[cbc_gstlal_lowmass]
executable = /bin/true
[cbc_gstlal_highmass]
executable = /bin/false
[user.name-TestNode]
executable = /bin/true
Now run::
lvalert_listen -a user.name -c myLVAlertListen.ini > myLVAlertListen.out &
Congratulations! You've set up an ``lvalert_listen`` instance which reacts to
announcements published to the ``cbc_gstlal_lowmass``, ``cbc_gstlal_highmass`` and
``user.name-TestNode`` nodes.
Here's what's happening: ``lvalert_listen`` hears announcements made to any node to
which your account is subscribed. When an alert is
received, it looks in the config file (loaded into memory) for the associated
section. Importantly, if there is no section in the config file corresponding
to the pubsub node's name (an exact match is required), ``lvalert_listen`` ignores
the announcements from that node even if you are subscribed to it. If it finds
a section, it looks for the "executable" option and attempts to run the
associated value (in this case ``/bin/true``) via Python's subprocess module. The
delegation to ``subprocess.Popen`` does *not* split the value so this must be a
single filename for the executable. If your executable takes in options, we
recommend wrapping it in a simple shell script and specifying the shell script
within ``myLVAlertListen.ini``. We'll get to that in a bit.
In this way, you can have multiple ``lvalert_listen`` instances for a single
account listening to multiple different nodes and doing multiple different
things. Furthermore, if you provide multiple sections in ``myLVAlertListen.ini``
you can react to announcements from different pubsub nodes in different ways, all
within the same ``lvalert_listen`` instance.
Right now your listener (running in the background) isn't doing much. When
``cbc_gstlal_lowmass`` alerts are received, it forks an instance of ``/bin/true`` and
when ``cbc_gstlal_highmass`` alerts are received, it forks an instance of
``/bin/false``. We can improve upon that pretty easily.
Let's start by creating some basic wrapper scripts to print that we've received
alerts. Again, using your favorite text editor, create the file ``lvalert-run_cbc_gstlal_lowmass.sh``
and fill it with::
#!/bin/bash
echo "received an alert about a cbc_gstlal_lowmass event!" >> lvalert_cbc_gstlal_lowmass.out
Similarly, create ``lvalert-run_cbc_gstlal_highmass.sh`` and fill it with::
#!/bin/bash
echo "received an alert about a cbc_gstlal_highmass event!" >> lvalert_cbc_gstlal_highmass.out
Finally, create a file for your test node, ``lvalert-run_user.name-TestNode.sh``,
which contains::
#!/bin/bash
read a
echo "received a test alert: ${a}" >> lvalert_user.name-TestNode.out
Once you've done that, ensure that all three shell scripts are executables (required
by the delegation through ``subprocess.Popen``) with::
chmod +x lvalert-run_cbc_gstlal_lowmass.sh
chmod +x lvalert-run_cbc_gstlal_highmass.sh
chmod +x lvalert-run_user.name-TestNode.sh
and edit myLVAlertListen.ini so it reads::
[cbc_gstlal_lowmass]
executable = ./lvalert-run_cbc_gstlal_lowmass.sh
[cbc_gstlal_highmass]
executable = ./lvalert-run_cbc_gstlal_highmass.sh
[user.name-TestNode]
executable = ./lvalert-run_user.name-TestNode.sh
It is generally a good rule of thumb to provide the full paths to executables
and output files in both ``myLVAlertListen.ini`` as well as these simple shell
scripts. However, for the purpose of this tutorial we'll stick with relative
paths.
Now, because you have modified the ``lvalert_listen.ini`` file, you'll need to
restart your ``lvalert_listen`` instance. Find the PID in the process table, kill
the existing process, and restart the listener using the command from above.
You can also specify a resource name in your call to ``lvalert_listen``
using the ``-r`` flag::
lvalert_listen -a user.name -c myLVAlertListen.ini -r listener1 &
If you don't specify this parameter, a random UUID is generated for the resource name.
The important point to consider is that only one listener can exist for any
(user.name, passw0rd, resource.name) triple *anywhere in the network*.
If you launch a second process with matching values of this triple,
one of the processes is killed automatically (although which process
dies may not be deterministic). Thus, I can kill processes running at CIT by
creating processes at UWM with the same resource name. This can be extremely
dangerous and annoying, so please be careful. If you want to directly specify
resource names for all of your listener processes, you can do something like::
lvalert_listen -a user.name -c myLVAlertListen.ini -r oneInstance &
lvalert_listen -a user.name -c myLVAlertListen.ini -r twoInstance &
This will launch two instances of ``lvalert_listen`` (both using the same
config file) with different resource names (note that this can also be
achieved by not specifying the resource name at all).
They will both react to alerts and fork
processes. If each points to a different config file, I can then get multiple
types of follow-up processes forked for the same announcement through a single
pubsub node.
When alerts are received, you will see a line printed to the associated files.
Note, the scripts for the ``cbc_gstlal`` nodes do not report anything about the
actual alerts received, whereas the script for your test node reads in stdin
(to a variable called "a") and then echo's that into the output file. This is
how ``lvalert_listen`` passes the contents of the alert into the forked subprocess,
via stdin. We'll come back to that later when we interact with GraceDB.
For now, let's test your set-up by publishing a few announcements to your test
pubsub node. Create a file called ``test.txt`` and fill it with some text like::
just a test announcment
Then run::
lvalert_send -a user.name -n user.name-TestNode --file test.txt
This publishes the contents of test.txt as a string to the node
``user.name-TestNode``. If your listener is running in the
background, then you should see a new line in ``lvalert_user.name-TestNode.out``
which reads::
received a test alert: just a test announcement
If you repeat the ``lvalert_send`` command, you should see multiple lines appear,
one for each time you sent an alert.
Note, each time we change the ``lvalert_listen`` config file (``myLVAlertListen.ini``)
we have to restart the listener for the changes to take effect.
However, if the config file points to wrapper script we can modify the contents
of the wrapper script and have the changes take effect immediately for all
future events *without* restarting the ``lvalert_listen`` process. This can be
quite handy, although you should be careful to keep track of what was actually
run when (version controlling the config file and ``lvalert-run_*sh`` scripts is a
good idea).
It is worth stressing that you do *not* have to actually use a wrapper script.
If you have an executable that can be called via subprocess in the same way as
the wrapper script, then you can simply specify that within myLVAlertListen.ini
instead of dealing with wrappers at all. This can reduce the number of files
that have to be managed but because of how ``lvalert_listen`` forks the executable
through subprocess the executable cannot take in any command line options or
arguments.
Now, ``lvalert_listen`` is a fairly robust process and is unlikely to throw errors
or fall over by itself. However, occasionally server-side or local trouble can
cause your listener to die and you will need to restart it.
Several solutions exist, although the preferred option is
`Monit <https://mmonit.com/monit/>`__ which can automatically restart processes and
notify you that it did so.
Reacting to GraceDB
===================
Now that you've got an ``lvalert_listen`` instance running which reacts to a few
different pubsub nodes, let's really dig into the full potential of this
system.
So far, we either haven't used the contents of the alert or have simply printed
them into a file. That's nice, but we can do much better. GraceDB (the main
publisher of alerts) sends JSON (JavaScript Object Notation) strings through
lvalert and there are several convenient tools to parse these in Python.
Similarly, there is an extremely useful RESTful interface to GraceDB
implemented in Python, although command-line executables also exist.
Let's start by mining the JSON string sent by GraceDB for some information.
Create a Python executable ``iReact.py`` and fill it with the following::
#!/usr/bin/python
import json
import sys
alert = json.loads(sys.stdin.read())
print 'uid : ' + alert['uid']
Don't forget to give this executable permissions with::
chmod +x iReact.py
Now, modify your wrapper script for the test node
(``lvalert-run_user.name-TestNode.sh``) so it reads::
#!/bin/bash
./iReact.py >> lvalert_user.name-TestNode.out
When we send messages to the test node, it will now delegate to ``iReact.py``. We
don't have to restart the ``lvalert_listen`` instance because that still points to
``lvalert-run_user.name-TestNode.sh``, which is nice.
Let's go ahead and send a test message in JSON format. Edit ``test.txt`` so it
reads::
{"uid": "G12345"}
and run::
lvalert_send -a user.name --node user.name-TestNode --file test.txt
You should see a new line in ``lvalert_user.name-TestNode.out`` which reads::
uid : G12345
Ta-da! You've now sent, received, parsed, and reacted to a JSON string
through lvalert. This is the key way all follow-up processes listen for events
in GraceDB and react accordingly. Note, the ``sys.stdin.read()`` command will
block until there is something in stdin and this can cause your code to hang if
you don't specify anything. This should not be the case when it is called from
within ``lvalert_listen``, but it can sometimes be annoying when debugging your
follow-up scripts.
Let's do something a bit more concrete with more specific examples of how we
can interface with GraceDB based off lvalert messages.
Open ``iReact.py`` and modify it so it reads::
#!/usr/bin/python
import json
import sys
from ligo.gracedb.rest import GraceDb
alert = json.loads(sys.stdin.read())
print 'uid : ' + alert['uid']
gdb = GraceDb() ### instantiate a GraceDB object which connects to the default server
if alert['alert_type'] == 'new': ### the event was just created and this is the first announcment
gdb.writeLog( alert['uid'], message="user.name heard an alert about this new event!" )
elif alert['alert_type'] == 'update': ### something happened in GraceDB for this event and GraceDB is letting everyone know
gdb.writeLog( alert['uid'], message="user.name heard an alert about an update for this event!" )
Now, if we modify ``test.txt`` to::
{"uid": "G12345", "alert_type": "new", "far": 1e-8}
and send it, ``iReact.py`` will try to write a log entry in GraceDB for event
G12345. It's easy to see that you can filter alerts out (e.g.: only react to
'new' events) and modify your follow-up processes behavior accordingly. To
check that this worked, you'll need to look at the associated GraceDB page,
expand the "full log" section and look for your log message.
**IMPORTANTLY,** I've just made up 'G12345' as an example. If you really want
to test your script, you should choose a test event from GraceDB. A query for
these events is available `here <https://gracedb.ligo.org/events/search/?query=group%3A%20Test>`__.
*NOTE: please do* **NOT** *test your script with important events in GraceDB
like G184098 (the cWB entry for GW150914) or others with low FAR.*
Instead, please use a test event as described above. There are also test instances of
GraceDB available if you'd prefer to not work with the production server right
away. Contact uwm-help@cgca.uwm.edu with a descriptive subject line for more
information.
At this point, you're pretty much ready to go. However, I'll leave you with one
more example for what ``iReact.py`` might look like::
#!/usr/bin/python
import json
import sys
from ligo.gracedb.rest import GraceDb
FarThr = float(sys.argv[1])
alert = json.loads(sys.stdin.read())
print 'uid : '+alert['uid']
gdb = GraceDb() ### instantiate a GraceDB object which connects to the default server
if alert['alert_type'] == 'new': ### the event was just created and this is the first announcment
if alert['far'] < FarThr:
file_obj = open("iReact.txt", "w")
print >> file_obj, "wow! this was a rare event! It had FAR = %.3e < %.3e, which was my threshold"%(alert['far'], FarThr)
file_obj.close()
gdb.writeLog( alert['uid'], message="user.name heard an alert about this new event!", filename="iReact.txt", tagname=["data_quality"] )
Try to figure out exactly what this version does. If you can
understand everything within this script you certainly know enough to get your
follow-up process running! Hint: to get this to run correctly, you'll want to
modify ``lvalert-run_user.name-TestNode.sh`` so it looks like::
#!/bin/bash
./iReact.py 1e-8 >> lvalert_user.name-TestNode.out
docs/user_docs/source/rest.rst
View file @ 5797d777
.. _rest_interface:
==========================
Using the REST interface
==========================
=============
Using the API
=============
.. _installing_the_client:
Installing the client
====================================
API structure
=============
The root of the GraceDB API is at `<https://gracedb.ligo.org/api/>`__.
There are two main components of the API:
The GraceDB client tools should already be installed at the LVC computing clusters.
However, if you want to interact with GraceDB from your own machine, you will need
to install the client tools yourself. The easiest way is to use ``pip`` to install
it from the `Python Package Index <https://pypi.python.org/pypi>`__::
- `Events <https://gracedb.ligo.org/api/events/>`__
- `Superevents <https://gracedb.ligo.org/api/superevents/>`__
pip install ligo-gracedb
These URLs provide a list of all events and superevents which are accessible to you.
(See `here <https://pip.pypa.io/en/latest/installing.html>`__ for instructions in
installing ``pip`` if it is not already available on your machine.) Additionally,
packages for Debian (``.deb``) and Scientific Linux (``.rpm``) are available by
pointing your system to the appropriate repositories as described
`here <https://wiki.ligo.org/DASWG/SoftwareDownloads>`__.
Then the client tools can be installed via::
Information about an individual event or superevent can be accessed by appending its graceid or superevent id to those links.
Examples:
apt-get install python-ligo-gracedb
- ``/api/events/G1234/``
- ``/api/superevents/S140309abc/``
or ::
Annotations to events and superevents are similarly nested:
yum install ligo-gracedb
- ``/api/superevents/S140309abc/logs/`` (list of logs for S140309abc)
- ``/api/superevents/S140309abc/logs/1/`` (log #1 for S140309abc)
- ``/api/superevents/S140309abc/voevents/`` (list of VOEvents for S140309abc)
- ``/api/superevents/S140309abc/files/file1.xml`` (``file.xml``, which is associated with S140309abc)
- and so on.
Alternatively, the client can be built from source. Please note, however, that
the code at the head of the git repository may be volatile or unstable. The
repository is publicly available via HTTPS::
git clone https://git.ligo.org/lscsoft/gracedb-client.git
Web-browsable API
=================
If you visit any of the above links in a web browser, a human-readable version of the API is rendered.
There is a ``links`` key at the API root which links to the main events and superevents API resources.
There are also ``links`` keys in the superevent and event data models, so much of the web API can be traversed using links.
The web-browsable API can be useful to quickly view an object's representation (superevent, event, log message, etc.) as returned by the API, for use in programmatic queries to the API.
Otherwise, a description of the API representation for various objects is available in :ref:`models`.
It is also available through SSH::
git clone git@git.ligo.org:lscsoft/gracedb-client.git
Using the API programmatically
==============================
IGWN maintains a Python package, ligo-gracedb, for simplifying access to the API.
This package's `documentation <https://ligo-gracedb.readthedocs.io/en/latest/>`__ is comprehensive and includes basic installation, configuration, and usage.
For non-Python users, ligo-gracedb also comes with a command-line client.
In order to submit patches, you should fork the repository, implement your changes, and submit a merge request.
Patches should also be documented on the GraceDB Redmine `page <https://bugs.ligo.org/redmine/projects/gracedb>`__.
.. _rest_client_basic_usage:
Authenticating to the API
=========================
As with the GraceDB web interface, unauthenticated access is available to the API, but is restricted to data which have been deemed suitable for public release.
Basic usage of the REST client
====================================
If you are viewing the API through a web browser, then any authentication credentials you previously provided will continue to allow access.
For programmatic access to the API, LIGO/Virgo users can use X.509 certificates generated by ``ligo-proxy-init`` or robot certificates obtained from the LIGO authentication team.
More information on how to use these credentials with ligo-gracedb is available in its `documentation <https://gw.readthedocs.io/projects/ligo-gracedb/en/latest/user_guide.html#credentials>`__.
The documentation in this section will focus on the use of the Python REST client. The
alternative command-line client, ``gracedb`` is a simple wrapper on the REST client
provided for convenience (see below, :ref:`command_line_client`) and is not as
fully featured.
Finally, more details on authentication and authorization in GraceDB are provided in :ref:`auth`.
.. NOTE::
Before using the REST client, credentials for authentication must be available.
Run ligo-proxy-init or, if using a robot certificate, set the environment
variables ``X509_USER_CERT`` and ``X509_USER_KEY`` (for more information, see :ref:`auth`).
.. XXX Probably would be good to actually show people how to set these environment variables.
The REST client is typically used in a script or in the Python interpreter to
accomplish a specific task, such as creating a new event or retrieving information
about events matching a query. The workflow involves importing the client class,
instantiating the client, and then calling the desired method::
from ligo.gracedb.rest import GraceDb, HTTPError
client = GraceDb()
try:
r = client.ping()
except HTTPError, e:
print e.message
print "Response code: %d" % r.status
print "Response content: %s" % r.json()
In the above example, we merely ping the GraceDB server and examine the
response. If there is an error (such as an authentication failure), the
``HTTPError`` exception will be thrown. If not, the response object contains a
status code and a response body in JSON. The ``json`` method on the response
object simply decodes the JSON content. In this particular case, the response
code should be 200 (meaning "OK") and the body contains a large dictionary
of information representing the `API Root resource <https://gracedb.ligo.org/apiweb/>`__.
Most of the examples below will ignore the error handling shown here for the
sake of brevity.
In addition to ``ping``, the most important client methods are:
- ``events`` for accessing a list of events,
- ``files`` for downloading a file or list of files, and ``writeFile`` for uploading,
- ``logs`` for obtaining a list of log entries, and ``writeLog`` to create a new one,
- ``emobservations`` for obtaining a list of EM followup observations, and
``writeEMObservation`` to create a new one,
- ``labels``, ``writeLabel``, and ``removeLabel`` for managing labels,
- ``tags``, ``createTag``, and ``deleteTag`` for managing tags.
Docstrings are available for most of the client methods. To see them, type
``help(client.ping)`` (for example) in the Python interpreter.
.. _searching_for_events:
Searching for events
===================================
Suppose you are working on a script to search for all events matching a
specific query and then to retrieve a piece of information about each event in
the results. For example, the following code retrieves the chirp
mass for each ``gstlal`` event during ER5 with FAR less than 1.0E-4::
from ligo.gracedb.rest import GraceDb
client = GraceDb()
# Retrieve an iterator for events matching a query.
events = client.events('gstlal ER5 far < 1.0e-4')
# For each event in the search results, add the graceid
# and chirp mass to a dictionary.
results = {}
for event in events:
graceid = event['graceid']
mchirp = event['extra_attributes']['CoincInspiral']['mchirp']
results.update({ graceid: mchirp})
Note that the ``events`` method on the client returns an *iterator* on event
dictionaries rather than a list. The chirp mass is an attribute specific to the
inspiral event subclass, hence the different ways of accessing the ``graceid``
and the chirp mass.
But how did I know the structure of the event dictionary so that I could pull
out the chirp mass? The best way is to look at the structure of an example
event in the *browseable* REST API. Here some example events from the different
subclasses to demonstrate the structure of the event dictionaries.
- `Test gstlal MDC <https://gracedb.ligo.org/apiweb/events/T125738>`__ (a CBC event)
- `Test cWB MDC <https://gracedb.ligo.org/apiweb/events/T153811>`__ (a Burst event)
- `External Swift GRB <https://gracedb.ligo.org/apiweb/events/E160846>`__ (a GRB event)
Creating new events
====================================
To create a new event, use the ``createEvent`` method on the client. In this
example, a new ``gstlal`` event is created in the ``Test`` group from a
file on disk::
from ligo.gracedb.rest import GraceDb
client = GraceDb()
event_file = "/path/to/coinc.xml"
r = client.createEvent("Test","gstlal", event_file, search="LowMass")
event_dict = r.json()
graceid = event_dict["graceid"]
The server response includes a JSON representation of the event, and the
event dictionary can thus be obtained as shown. In this example, the event
dictionary is used to get the ``graceid`` of the new event.
.. NOTE::
In order to create events in a group other than ``Test``, the user must
be explicitly authorized. Thus, events in the ``CBC`` or ``Burst`` groups,
for example, can only be created by authorized users. For more information
on authorization, see :ref:`auth`.
Now suppose that a subsequent analysis has updated the values in the original
``coinc.xml`` file. We can *replace* the original event since we know the
``graceid``::
new_event_file = "/path/to/new_coinc.xml"
r = client.replaceEvent(graceid, new_event_file)
This has the effect of updating the values of the various database fields, but
the original version of the event file is kept and can be found in the full
event log.
Annotating events
======================================
As discussed in the :ref:`annotation_models` section, the term refers to pieces of
information added to an event after the time of its creation. Most commonly,
these take the form of event log messages or electromagnetic observation
records (see :ref:`create_emobservation`). The following demonstrates how to
add a log message to an existing event::
from ligo.gracedb.rest import GraceDb
client = GraceDb()
graceid = 'T160779'
message = 'This is a test of the emergency commenting system.'
filename = '/path/to/my_plot.png'
r = client.writeLog(graceid, message, filename, tagname='analyst_comments')
In this example, a plot is uploaded to the log messages of event
``T160779`` with a text comment. The new log message is also tagged with
the ``analyst_comments`` tag, which displays the plot and comment in a
special section for comments from (human) data analysts. The tag name
can actually be anything the user wants, but only a limited list of tags
are *blessed* in the sense that they affect the display of information.
For a reminder of which tags are blessed see :ref:`annotation_models`.
Tags can also be added and removed from existing annotations. See the
docstrings for the client methods ``createTag`` and ``deleteTag``.
Applying labels
====================================
To add a label to an event, the ``graceid`` of the event and the name
of the label must be known::
from ligo.gracedb.rest import GraceDb
client = GraceDb()
graceid = 'T160779'
label_name = 'DQV' # meaning data quality veto
r = client.writeLabel(graceid, label_name)
Care should be taken when applying labels to non-test events, since this
affects the sending of alerts related to potential electromagnetic followup.
The following labels are currently in active use:
* ``INJ``: event results from an injection
* ``DQV``: data quality veto
* ``EM_READY``: approved for EM followup
* ``PE_READY``: parameter estimation results available
* ``H1OPS``, ``L1OPS``: IFO operator signoff requested
* ``H1OK``, ``L1OK``: IFO operator certifies the detector state *okay*
* ``H1NO``, ``L1NO``: detector state *not okay* at event time
* ``ADVREQ``: EM followup advocate signoff requested
* ``ADVOK``: EM followup advocate approves event
* ``ADVNO``: EM followup advocate rejects event
The following labels were added August 2016 in order to add new functionality to
approval_processor:
* ``EM_Throttled``: event ignored due to too many submissions by corresponding pipeline
* ``EM_Selected``: most promising candidate out of set for single physical event
* ``EM_Superseded``: event passed over due to other more-promising candidate for same event.
.. _command_line_client:
Coping with request rate limits
=====================================
GraceDB limits any individual user to no more than 1 event creation request or
10 annotation requests per second. This is to avoid situations where an
automated data analysis pipeline malfunctions and sends huge rates of requests
to GraceDB, crashing the server. If a user's request is *throttled* in this
way, the server returns a response with HTTP status ``429`` and reason ``TOO
MANY REQUESTS``. The recommended wait time is also included in the message, and
is available through the ``retry-after`` when using the GraceDB REST client. If
you have reason to believe that your request may be throttled, you can wrap it
in a ``while`` loop in the following way::
GraceDB limits any individual user to no more than 10 event creation requests or 10 annotation requests per second.
This is to avoid situations where an automated data analysis pipeline malfunctions and sends huge rates of requests to GraceDB, crashing the server.
If a user's request is *throttled* in this way, the server returns a response with HTTP status ``429`` and reason ``TOO MANY REQUESTS``.
The recommended wait time is also included in the message, and is available through the ``retry-after`` when using the ligo-gracedb client.
If you have reason to believe that your request may be throttled, you can wrap it in a ``while`` loop in the following way::
from ligo.gracedb.rest import GraceDb, HTTPError
import time
......@@ -266,69 +77,13 @@ in a ``while`` loop in the following way::
try:
r = gracedb.writeLog(graceid, "Hello, this is a log message.")
success = True
except HTTPError, e:
except HTTPError as e:
try:
rdict = json.loads(e.message)
if 'retry-after' in rdict.keys():
if 'retry-after' in rdict:
time.sleep(int(rdict['retry-after']))
continue
else:
break
except:
break
Using the command-line client
=====================================
The GraceDB command-line client, ``gracedb``, is essentially a thin
wrapper on the Python client. The examples above could be repeated
with the command-line client as follows (assuming ``bash``)::
# Create the new event and store the uid
GRACEID="$(gracedb Test gstlal LowMass /path/to/coinc.xml)"
# Replace the event
gracedb replace $GRACEID /path/to/new_coinc.xml
# Annotate an event with a plot
COMMENT="This is a test of the emergency commenting system."
gracedb --tag-name=analyst_comments upload T160779 /path/to/my_plot.png $COMMENT
# Label an event
gracedb label T160779 DQV
Type ``gracedb -h`` for detailed help with the command-line client.
.. _coding_against_api:
Coding against the GraceDB REST API
=======================================================
Some users may wish to code directly against the GraceDB REST API rather
than use the Python or command-line clients. In order to do this, the user
will need to know which resources are exposed by which URLs, and which HTTP
methods those URLs allow. Fortunately, the
`Django REST Framework <http://www.django-rest-framework.org>`__ (on which
the GraceDB API is built) provides
a convenient *browseable* version of the API which serves as a reference.
The root of the API can be found here:
`https://gracedb.ligo.org/apiweb/ <https://gracedb.ligo.org/apiweb/>`__
A glance at the upper-right hand corner shows that this URL supports only
``OPTIONS`` and ``GET``. The body is a collection of JSON information provided
by the root resource, including ``links``. One of these links points to the
event list resource:
`https://gracedb.ligo.org/apiweb/events/ <https://gracedb.ligo.org/apiweb/events/>`__
which also supports ``POST`` (see the bottom of the page). New events are
created by ``POST``-ing to the event list resource. This results in a new
event with a unique URL. If the parameters of the event change, the event
can be replaced by a ``PUT`` request to that same event URL with the replacement
data in the body. In a similar manner,
new log messages are created by ``POST``-ing to the event log list associated
with a particular event. The data expected by these target URLs is not yet
documented here. However, the source code of the GraceDB Python client
can be consulted for examples.
docs/user_docs/source/tutorials.rst deleted 100644 → 0
View file @ b4e766bd
.. GraceDB documentation tutorials
Tutorials
=========
Contents:
.. toctree::
:maxdepth: 2
responding_to_lvalert