GraceDB Server issueshttps://git.ligo.org/computing/gracedb/server/-/issues2022-09-12T10:34:28Zhttps://git.ligo.org/computing/gracedb/server/-/issues/1Numerical superevent parameters returned as strings2022-09-12T10:34:28ZTanner PrestegardNumerical superevent parameters returned as strings```
{u'created': u'2018-04-06 20:26:52 UTC',
u'em_events': [],
u'gw_events': [u'T0066', u'T0064', u'T0063', u'T0059'],
u'labels': [u'LUMIN_NO', u'EM_READY', u'H1OPS', u'EM_Throttled'],
u'links': {u'emobservations': u'TBD',
u'events...```
{u'created': u'2018-04-06 20:26:52 UTC',
u'em_events': [],
u'gw_events': [u'T0066', u'T0064', u'T0063', u'T0059'],
u'labels': [u'LUMIN_NO', u'EM_READY', u'H1OPS', u'EM_Throttled'],
u'links': {u'emobservations': u'TBD',
u'events': u'https://gracedb-dev1.ligo.org/api/superevents/S0001/events/',
u'files': u'https://gracedb-dev1.ligo.org/api/superevents/S0001/files/',
u'labels': u'https://gracedb-dev1.ligo.org/api/superevents/S0001/labels/',
u'logs': u'https://gracedb-dev1.ligo.org/api/superevents/S0001/logs/',
u'self': u'https://gracedb-dev1.ligo.org/api/superevents/S0001/',
u'voevents': u'https://gracedb-dev1.ligo.org/api/superevents/S0001/voevents/'},
u'preferred_event': u'T0063',
u'submitter': u'tanner.prestegard@LIGO.ORG',
u'superevent_id': u'S0001',
u't_0': u'12.000000',
u't_end': u'23.234354',
u't_start': u'56.000000'}
```https://git.ligo.org/computing/gracedb/server/-/issues/2Superevent queries2022-09-12T10:34:28ZTanner PrestegardSuperevent queriesThis issue is created to get some feedback on what attributes would be most useful for querying for superevents, as well as which attributes should be added to event queries. Here's what I came up with off the top of my head.
### Super...This issue is created to get some feedback on what attributes would be most useful for querying for superevents, as well as which attributes should be added to event queries. Here's what I came up with off the top of my head.
### Superevents
* Search by superevent ID (date-based)
* Superevent submitter
* Ranges of `t_start`, `t_0`, `t_end` (GPS times)
* Whether an alert has been sent out for the superevent
* Whether the superevent is confirmed as a GW
* Preferred event graceid (possibly a range of graceids)
* Event graceids
* Labels
* Do we need the ability to find superevents based on attributes of their preferred events? Ex: find all superevents with a preferred event which was identified by the CWB pipeline. If this seems useful, which attributes?
### Events
* Find all events which are preferred events for superevents
* Find all events which are part of a particular superevent
* Find all events which are part of any superevent (or are NOT part of any superevent)
If there are other filters which would be useful, or if any of these don't seem useful, please comment!https://git.ligo.org/computing/gracedb/server/-/issues/3Date-based IDs for superevents2023-05-11T16:31:37ZTanner PrestegardDate-based IDs for supereventsBased on notes from the March 2018 LVC meeting and discussions with Patrick and Deep, I've developed a strategy for implementing date-based IDs for superevents. An example is given below to demonstrate how this will work, but please ask ...Based on notes from the March 2018 LVC meeting and discussions with Patrick and Deep, I've developed a strategy for implementing date-based IDs for superevents. An example is given below to demonstrate how this will work, but please ask any clarifying questions that are needed since it's a bit confusing to walk through.
## Example
1. A superevent is identified for May 13, 2018. (The date is determined from the initial value of `t_0` assigned to the superevent)
* Initial date-based ID assigned: S180513
2. A second superevent is identified for May 13, 2018.
* Initial date-based ID assigned: S180513b
* A letter suffix is added to the first superevent: S180513 -> S180513a. Hyperlinks and API calls which use either S180513 or S180513a will both point to this superevent, but S180513a will now be displayed as the ID throughout GraceDB.
3. An alert is sent out about S180513b.
* Its prefix is upgraded to "GWA", so its official ID is now GWA180513b. The letter suffix is inherited from its previous ID. Hyperlinks and API calls which use either ID (S180513b or GWA180513b) will both point to this superevent, but GWA180513b will be displayed as the ID throughout GraceDB.
4. GWA180513b is confirmed as a GW.
* Its prefix is upgraded to "GW" and a new letter suffix is determined based on confirmed GWs for this date.
* Resulting ID: GW180513. As before, hyperlinks and API calls using S180513b or GWA180513b will still work, but GW180513 will be generally displayed as the ID.
5. An alert is sent out about S180513a.
* ID changed from S180513a -> GWA180513a, hyperlinks still work for both IDs (and the initial, suffixless ID), etc.
6. GWA180513a is confirmed as a GW.
* ID changed: GWA180513a -> GW180513B. Hyperlinks and API calls still work for all past IDs, etc.
* GW180513 has a letter suffix added: GW180513 -> GW180513A. Hyperlinks and API calls still work for all past IDs, etc.
## Questions
1. Should the intermediate state (GWA prefix) exist or is a two-state arrangement sufficient?
2. If so, should the GWA state have its own letter prefix and not just inherit it from the S state?
3. Is the current strategy for handling the letter suffix for the first superevent on a given date acceptable (i.e., first superevent has no letter suffix until a second superevent is found for that date)? Or should the first superevent always have a letter suffix? Access to the superevent in GraceDB will be unaffected, but there could be confusion in certain cases. Example: offline searches find a second GW on a given date, which adds a letter suffix to the first GW's ID after a paper is already published about the first)
4. Same question as above for the first superevent confirmed as a GW on a given date.
5. Capital or lowercase letter suffixes for confirmed GWs?https://git.ligo.org/computing/gracedb/server/-/issues/4Removing an event from a superevent does not unset it as the preferred event2018-06-13T23:13:13ZTanner PrestegardRemoving an event from a superevent does not unset it as the preferred eventSee original post at https://git.ligo.org/lscsoft/gracedb-client/issues/4.See original post at https://git.ligo.org/lscsoft/gracedb-client/issues/4.https://git.ligo.org/computing/gracedb/server/-/issues/5superevents() query not returning complete set of superevents2018-06-28T12:22:14ZTanner Prestegardsuperevents() query not returning complete set of supereventsSee original post at https://git.ligo.org/lscsoft/gracedb-client/issues/6See original post at https://git.ligo.org/lscsoft/gracedb-client/issues/6https://git.ligo.org/computing/gracedb/server/-/issues/6Superevent lvalert messages must contain valued value for "self" field2018-06-14T15:34:38ZLeo P. SingerSuperevent lvalert messages must contain valued value for "self" fieldSince we multiplex traffic from several different GraceDb servers to the single LVAlert server `lvalert-test.cgca.uwm.edu`, it is necessary for all LVAlert messages to identify the GraceDb server of origin. I have been using the `self` a...Since we multiplex traffic from several different GraceDb servers to the single LVAlert server `lvalert-test.cgca.uwm.edu`, it is necessary for all LVAlert messages to identify the GraceDb server of origin. I have been using the `self` attribute for this purpose. However, `superevent` LVAlert messages set this attribute to `null`. They need to set this attribute to a valid URL. This is a prerequisite of emfollow/gwcelery#21.
```
{
"object": {
"comment": "Set as preferred event for superevent: S180613bw",
"file_version": null,
"tag_names": [
],
"file": null,
"created": "2018-06-13T17:53:37.431668+00:00",
"self": null,
"issuer": "emfollow",
"filename": "",
"tags": null,
"N": 6
},
"uid": "M3404",
"labels": [
],
"alert_type": "update",
"file": "",
"description": "LOG: Set as preferred event for superevent: S180613bw"
}
```
CC @tanner.prestegard.https://git.ligo.org/computing/gracedb/server/-/issues/7Superevent 'new' and 'update' LVAlert messages have inconsistent structure2018-06-25T15:19:38ZLeo P. SingerSuperevent 'new' and 'update' LVAlert messages have inconsistent structureThe `new` and `update` LVAlert messages have very different structures. For the `new` message, the `uid` is the GraceDb ID of the superevent, whereas for the `update` message the `did` is the GraceDb ID of the subevent.
In fact, the upd...The `new` and `update` LVAlert messages have very different structures. For the `new` message, the `uid` is the GraceDb ID of the superevent, whereas for the `update` message the `did` is the GraceDb ID of the subevent.
In fact, the update messages do not even contain the superevent ID in a machine-friendly field; it is only present in the English comment field (e.g. `Set as preferred event for superevent: S180613bw`).
It would make it easier to deal with this LVAlert messages if they shared the same structure: the superevent ID and the preferred event ID should always be found in the same machine-friendly fields.
## New superevent
```json
{
"object": {
"superevent_id": "S180613bw",
"created": "2018-06-13 17:53:37 UTC",
"submitter": "emfollow",
"preferred_event": "M3404",
"t_start": 1212947423.09,
"t_0": 1212947433.09,
"t_end": 1212947443.09,
"gw_events": [
"M3404"
],
"em_events": [
],
"labels": [
],
"links": {
"files": "/apibasic/superevents/S180613bw/files/",
"logs": "/apibasic/superevents/S180613bw/logs/",
"self": "/apibasic/superevents/S180613bw/",
"labels": "/apibasic/superevents/S180613bw/labels/",
"voevents": "/apibasic/superevents/S180613bw/voevents/",
"emobservations": "/apibasic/superevents/S180613bw/emobservations/",
"events": "/apibasic/superevents/S180613bw/events/"
}
},
"uid": "S180613bw",
"labels": [
],
"alert_type": "new",
"file": "",
"description": "NEW: superevent S180613bw"
}
```
## Updated superevent
```json
{
"object": {
"comment": "Set as preferred event for superevent: S180613bw",
"file_version": null,
"tag_names": [
],
"file": null,
"created": "2018-06-13T17:53:37.431668+00:00",
"self": null,
"issuer": "emfollow",
"filename": "",
"tags": null,
"N": 6
},
"uid": "M3404",
"labels": [
],
"alert_type": "update",
"file": "",
"description": "LOG: Set as preferred event for superevent: S180613bw"
}
```
CC @tanner.prestegard.https://git.ligo.org/computing/gracedb/server/-/issues/8Reworking LVAlert types and contents2018-09-21T19:18:37ZTanner PrestegardReworking LVAlert types and contentsThere is a need to add more finely divided categories for LVAlerts and possibly change the type of serialized objects which are included with them. Should more/different information be added as well?
# Current status
## Contents of al...There is a need to add more finely divided categories for LVAlerts and possibly change the type of serialized objects which are included with them. Should more/different information be added as well?
# Current status
## Contents of alert JSON packet
Currently, LVAlert JSON packets include the following keys:
* object: dict representation of a database row; typically an event, superevent, or log object. Note that these should be identical to the representation of these object available from the API.
* description: text description of the action which triggered the alert
* uid: event graceid or superevent id
* alert_type: either "new", "label", "update", or "signoff"
* labels: list of label names attached to the event or superevent
* file: filename string (optional and relative)
## Actions for which GraceDB issues alerts
GraceDB issues LVAlerts when the following actions are taken:
### Events
Constants:
* uid: always the related event's graceid
* labels: always a list of the related event's labels
| Action | alert_type | object | file | description |
| ------ | ---------- | ------ | ---- | ----------- |
| Event creation | "new" | event | Link to initial event file | "" |
| Event log creation (no file) | "update" | log | "" | "LOG: comment" |
| Event log creation (with file) | "update" | log | filename | "UPLOAD: 'filename' comment" |
| Label added to event | "label" | Not included in dict | "" | H1OK |
| Label removed from event | "update" | Not included in dict | "" | "Label H1OK removed" |
| VOEvent created for event | "update" | voevent | G0001-3-Update.xml | "VOEVENT: G0001-3-Update.xml" |
| EM observation created for event | "update" | emobservation | "" | "New EMBB observation record." |
| EMBB event log created for event | "update" | embb event log | "" | "New EMBB event log entry." |
| Event added to superevent | "update" | log | "" | "LOG: Added to superevent S180614b" |
| Event removed from superevent | "update" | log | "" | "LOG: Removed from superevent S180614b" |
| Event selected as preferred event for superevent | "update" | log | "" | "LOG: Set as preferred event for superevent: S180614b" |
| Event removed as preferred event for superevent | "update" | log | "" | "LOG: Removed a preferred event for superevent: S180614b" |
| Operator or advocate signoff (initial or update) on event | "signoff" | signoff | "" | "OK" or "NO" |
### Superevents
Constants:
* uid: always the related superevent's superevent_id
* labels: always a list of the related superevent's labels
| Action | alert_type | object | file | description |
| ------ | ---------- | ------ | ---- | ----------- |
| Superevent creation | "new" | superevent | "" | "NEW: superevent S180614a" |
| Superevent update (preferred event, t_start, t_0, t_end) | "update" | log | "" | "LOG: Updated superevent parameters: t_0: 1 -> 2, preferred_event: G0001 -> G0002" |
| Log creation (no file) | "update" | log | "" | "LOG: comment" |
| Log creation (with file) | "update" | log | filename | "UPLOAD: 'filename' comment" |
| Event added to superevent | "update" | log | "" | "LOG: Added event: G0001" |
| Event removed from superevent | "update" | log | "" | "LOG: Removed event: G0001" |
| Label added to superevent | "label" | Not included in dict | "" | "LABEL: H1OK" |
| Label removed from superevent | "update" | Not included in dict | "" | "UPDATE: H1OK removed" |
| Confirm superevent as GW | "update" | log | "" | LOG: Confirmed as a gravitational wave." |
| Create VOEvent for superevent | "update" | voevent | S180614b-3-Update.xml | "VOEVENT: S180614b-3-Update.xml" |
| Create EMObservation for superevent | "update" | emobservation | "" | "New EMBB observation record for DESGW." |
| Operator or advocate signoff (initial or update) on superevent | "signoff" | signoff | "" | "OK" or "NO" |
| Operator or advocate signoff deleted | "update" | log | "" | "LOG: operator signoff for H1 deleted: H1OK removed and H1OPS reapplied" |
Note that there are no EMBB event logs for superevents since they seem to have fallen out of use (there were only ever 31 of them created and the last one was applied for events in ~2015).
## Actions for which GraceDB does not issue alerts
LVAlerts are *not* issued when the following actions occur:
* Event created with labels attached ("new" alert goes out but no "label" alert; this probably should happen)
* Note that superevents created with labels first issue "new" alerts and then "label" alerts
* Event "replaced" with new event file (should probably send an "update" alert)
* An event signoff is deleted.
* An event log or superevent log has a tag applied or removed (I don't think anyone wants to get this alert)
* A log message is generated to document the above action (I don't think anyone wants to get this alert)
# Development action items
We want to do the following:
* Define some new alert_type categories and what actions they should be used for.
* Add alerts for some of the actions listed above.
* Change what type of serialized object is included with certain alerts
* Add documentation of alert contents and actions which trigger them
We might want to:
* Add entirely new information to alerts (i.e., new dict key(s) and value(s))Readiness for second OPA testhttps://git.ligo.org/computing/gracedb/server/-/issues/9sub-second time strings dropping leading zeros2018-07-03T16:44:01ZPeter Couvaressub-second time strings dropping leading zerosAs Jacob Lange reports on the CBC list, the event time in the event.log on Gracedb is wrong for 170818/G298171. The event time should be 1187058327.0815086 but the event.log says 1187058327.815086. It looks like the leading zero of the ...As Jacob Lange reports on the CBC list, the event time in the event.log on Gracedb is wrong for 170818/G298171. The event time should be 1187058327.0815086 but the event.log says 1187058327.815086. It looks like the leading zero of the sub-second string is being dropped somewhere.https://git.ligo.org/computing/gracedb/server/-/issues/10Some floats cast as strings in LVAlerts and response JSONs2018-06-26T21:46:26ZTanner PrestegardSome floats cast as strings in LVAlerts and response JSONsAs reported in https://git.ligo.org/emfollow/gwcelery/merge_requests/94, there are situations in which certain float variables are being sent as strings in LVAlert JSONs and even in some response JSONs from gracedb-client.
I believe thi...As reported in https://git.ligo.org/emfollow/gwcelery/merge_requests/94, there are situations in which certain float variables are being sent as strings in LVAlert JSONs and even in some response JSONs from gracedb-client.
I believe this is happening because in some of the old code in the events API, the event is converted to a dictionary BEFORE it is saved in the database. Thus, some variables which are extracted from the event file (as strings) are still in memory as strings. If we did the save first, that would cast those variables to the correct type.
I will submit some test events for the active pipelines (gstlal, gstlal-spiir, pycbc, mbtaonline, cwb, lib, swift, fermi, and snews) and check the LVAlert content, as well as the response in gracedb-client.https://git.ligo.org/computing/gracedb/server/-/issues/11use lal routines for GPS time handling2022-08-03T18:03:23ZPeter Couvaresuse lal routines for GPS time handlingAs @kipp.cannon [suggests](https://sympa.ligo.org/wws/arc/cbc/2018-06/msg00357.html), although the specific issue in #9 is fixed, using [LAL](https://git.ligo.org/lscsoft/lalsuite) for gps time handling is probably a more robust solution...As @kipp.cannon [suggests](https://sympa.ligo.org/wws/arc/cbc/2018-06/msg00357.html), although the specific issue in #9 is fixed, using [LAL](https://git.ligo.org/lscsoft/lalsuite) for gps time handling is probably a more robust solution long-term to avoid future problems.https://git.ligo.org/computing/gracedb/server/-/issues/12Broken sky map image links on superevent pages (followup from !8)2018-06-28T17:45:09ZLeo P. SingerBroken sky map image links on superevent pages (followup from !8)Images in superevent pages are showing up with the URL `"undefined"`, so the links and `<img>` tags are broken. See, for example:
https://gracedb-playground.ligo.org/superevents/S180628r/view/Images in superevent pages are showing up with the URL `"undefined"`, so the links and `<img>` tags are broken. See, for example:
https://gracedb-playground.ligo.org/superevents/S180628r/view/https://git.ligo.org/computing/gracedb/server/-/issues/13createVOEvent() no longer returns the VOEvent text2018-06-28T12:21:23ZLeo P. SingercreateVOEvent() no longer returns the VOEvent textOn gracedb.ligo.org, the `createVOEvent()` method returns a JSON data structure that includes a key called `text` whose value is the contents of the VOEvent. This is gone on gracedb-playground.ligo.org.On gracedb.ligo.org, the `createVOEvent()` method returns a JSON data structure that includes a key called `text` whose value is the contents of the VOEvent. This is gone on gracedb-playground.ligo.org.https://git.ligo.org/computing/gracedb/server/-/issues/14LVAlert messages for labels do not have a self link2018-07-11T18:28:43ZLeo P. SingerLVAlert messages for labels do not have a self link@deep.chatterjee found this one:
```
[2018-06-29 13:07:17,894: ERROR/ForkPoolWorker-1] gwcelery.tasks.lvalert.listen[5f526064-5919-425a-b17c-029b20cb5b72]: LVAlert message does not contain an API URL: {'labels': ['SKYMAP_READY'], 'alert...@deep.chatterjee found this one:
```
[2018-06-29 13:07:17,894: ERROR/ForkPoolWorker-1] gwcelery.tasks.lvalert.listen[5f526064-5919-425a-b17c-029b20cb5b72]: LVAlert message does not contain an API URL: {'labels': ['SKYMAP_READY'], 'alert_type': 'label', 'uid': 'S180629cb', 'file': '', 'description': 'LABEL: SKYMAP_READY added'}
```
@tanner.prestegard, would you please take a look?https://git.ligo.org/computing/gracedb/server/-/issues/15Rework permissions structure2018-09-21T19:22:21ZTanner PrestegardRework permissions structureWe need to define a permissions structure for controlling superevent actions, as well as upgrade the old one for events. Here is a proposal for how to define this structure going forward.
**Update** (6 Sept 2018): this issue is no long...We need to define a permissions structure for controlling superevent actions, as well as upgrade the old one for events. Here is a proposal for how to define this structure going forward.
**Update** (6 Sept 2018): this issue is no longer going to cover redoing the events permission structure; it will only focus on creating the superevents permission structure.
# Creation
| Action | Allowed users | Comments |
| ------ | ------------- | -------- |
| create event | Specific LVK users only | Currently allowed for pipeline accounts and specific users; remove individual users? |
| create test event | All LVK users | |
| create superevent | emfollow/superevent manager | |
| create test superevent | All LVK users | |
| create MDC superevent | emfollow/superevent manager | |
# Updates
| Action | Allowed users | Comments |
| ------ | ------------- | -------- |
| update/replace event | Only the user who originally submitted the event | Or should it be anyone who is allowed to submit events for the given pipeline? |
| update/replace test event | All LVK users | |
| update superevent | emfollow/superevent manager | Applies to production and MDC superevents |
| update test superevent | All LVK users | |
| add/remove event from superevent | emfollow/superevent manager | Production and MDC |
| add/remove event from test superevent | All LVK users | |
| confirm superevent as gw | Some special people | Should emfollow/superevent manager be on this list? |
| confirm test superevent as gw | All LVK members | |
| confirm mdc superevent as gw | emfollow/superevent manager | |
# Annotations
| Action | Allowed users | Comments |
| ------ | ------------- | -------- |
| add log message/file | All LVK (all events/superevents); LV-EM (only exposed events/superevents) | Not allowed for public users (?) |
| tag log message/file | All LVK (all event/superevent logs) | Not allowed for LV-EM or public |
| untag log message/file | All LVK (all event/superevent logs) | Not allowed for LV-EM or public |
| add label | Specific LVK members can add specific labels | Needs some thought and a finalized list of labels to define this |
| remove label | Specific LVK members can remove the same specific labels | |
| create voevent | emfollow | Are others needed? |
| create emobservation | All LVK (all events/superevents) and all LV-EM (all exposed events/superevents) | This is a little weird because as far as I know, only LV-EM people should be uploading EM observations |
| add/update/remove operator signoff | LVK members in control rooms | Control room groups controlled by IP address |
| add/update/remove advocate signoff | LVK members in em_advocates group | |
# Viewing
| Action | Allowed users | Comments |
| ------ | ------------- | -------- |
| view event | All LVK (all events/superevents); LV-EM/Public (exposed events/superevents? Or just superevents?) | Applies equally to production and test events. Note to self: need to consider event subtypes and permissions on those as well |
| view logs | All LVK (all event/superevent logs); LV-EM/Public (exposed logs only) | Logs will be exposed via a tag ('lv-em' or 'public'); files associated with exposed logs will also be exposed | |
| view voevents | All LVK (all voevents); not sure about LV-EM or public | Currently, all VOEvents are viewable to anyone who can view the event|
| view emobservations | All LVK (all emobservations); not sure about LV-EM or public | Currently, all EMObservations are viewable to anyone who can view the event |
# Main questions
1. Who can add/remove which labels? Or should all LVK users be able to add/remove all labels?
2. Who can expose/hide events and superevents?
3. Who can expose/hide logs with the 'lv-em' and 'public' tags?
4. Does exposing a superevent to external users mean that we should expose all of the individual events as well?
5. Who can confirm superevents as GWs?Move superevents branch onto production GraceDBhttps://git.ligo.org/computing/gracedb/server/-/issues/16Refurbish events API2022-08-03T18:04:27ZTanner PrestegardRefurbish events APIThe events API needs to be redone for a few reasons:
1. Incomplete validation and error handling
2. Difficult to implement permissions - redoing this would make #15 much easier
3. Many redundancies and inefficiencies
4. Doesn't make...The events API needs to be redone for a few reasons:
1. Incomplete validation and error handling
2. Difficult to implement permissions - redoing this would make #15 much easier
3. Many redundancies and inefficiencies
4. Doesn't make use of the builtin features in django-rest-framework
One possible difficulty is that some changes might require corresponding client changes, so we might run into yet another case where we have another server-client incompatibility.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/17Unit tests2022-08-03T18:05:38ZTanner PrestegardUnit testsThe unit tests are really lacking and are absolutely needed. Especially for authentication and permissions.The unit tests are really lacking and are absolutely needed. Especially for authentication and permissions.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/18Change pipeline names for LIB and gstlal-spiir2018-09-19T15:13:53ZTanner PrestegardChange pipeline names for LIB and gstlal-spiirRequest from Reed Essick on January 20, 2017 (from https://bugs.ligo.org/redmine/issues/5043)
> In all our publications and internal documents, the pipeline is referred to as "oLIB." Therefore, it is the name "oLIB" that should be assoc...Request from Reed Essick on January 20, 2017 (from https://bugs.ligo.org/redmine/issues/5043)
> In all our publications and internal documents, the pipeline is referred to as "oLIB." Therefore, it is the name "oLIB" that should be associated with significance estimates and be allowed to create events in GraceDb. The name "LIB" commonly refers to the parameter estimation follow-up done for all burst events. While oLIB uses the LIB pipeline, they are nonetheless separate concepts. LIB PE results do not contain significance estimates in the same way that LALInference PE results do not.
>
> To minimize/mitigate possible confusion (which appears to be common), we propose changing the pipeline name in GraceDb. However, this will have many repercussions which we should carefully consider. An incomplete list includes
>
> * the need to update all historical events in GraceDb to follow the new nomenclature
> * the need to update all LVAlert nodes so they follow the new nomenclature
> * the need to update all follow-up processes so they listen to the new LVAlert nodes
>
> This change will almost certainly have to wait until a break between observing runs.
Request from Qi Chu on July 9, 2018 (from https://bugs.ligo.org/redmine/issues/6191)
> Dear Tanner,
>
> I would like to request the name for the SPIIR pipeline on the gracedb to be changed to "spiir" in order to remove the possible confusion that triggers from this pipeline are dependent on the that of gstlal pipeline. The connection with the gstlal pipeline is that the code of this pipeline is packed in the gstlal dev packages.
>
> Cheers!Readiness for second OPA testhttps://git.ligo.org/computing/gracedb/server/-/issues/19Improve server stability and performance2022-08-03T18:10:24ZTanner PrestegardImprove server stability and performanceStarted on October 12, 2017, copied from redmine (https://bugs.ligo.org/redmine/issues/5946)
We have a generic goal of attempting to improve GraceDB's stability and performance. Ideally, it should be able to handle a significant load (l...Started on October 12, 2017, copied from redmine (https://bugs.ligo.org/redmine/issues/5946)
We have a generic goal of attempting to improve GraceDB's stability and performance. Ideally, it should be able to handle a significant load (lots of automated processes triggering and querying after an event is identified) and provide reasonably fast performance (page loading, API queries, etc.). But we really need a more precise definition of what we want out of the server. A specific issue that we would like to rectify is the gateway timeout issue - it's been reduced, but not removed.
Some ideas of things we can do:
Significant profiling and rewriting of code - reduce memory footprint and number of database queries. We should use select_related and prefetch_related wherever we can.
Improve web UI performance - web pages shouldn't take as long to load, should cache files, etc.
Switch to PostgreSQL
Use gUnicorn with Apache as a reverse proxy - allows us to eliminate mod_wsgi plugin and hopefully boost performance
Possible issues:
We don't have a standard way of measuring performance. Note: unit tests might help with that.
We don't have a good way to imitate the production environment for load testing.O4 Prephttps://git.ligo.org/computing/gracedb/server/-/issues/20Flip between images that have the same tag2024-01-30T16:51:24ZTanner PrestegardFlip between images that have the same tagStarted in 2014. Copied from redmine (https://bugs.ligo.org/redmine/issues/1156)
> Hi Branson,
>
> I have two feature requests for the lightboxes that are displayed in GraceDb for uploaded images. See <https://gracedb.ligo.org/events/...Started in 2014. Copied from redmine (https://bugs.ligo.org/redmine/issues/1156)
> Hi Branson,
>
> I have two feature requests for the lightboxes that are displayed in GraceDb for uploaded images. See <https://gracedb.ligo.org/events/view/T81925>.
>
> The first request is for the pop-up to have an opaque white background, to improve readability. The transparent backgrounds make it difficult to read the text on the png sky maps.
>
> The second request is to add the ability to flip between images that belong to the same tag or group. This event has two sky maps, one from BAYESTAR and one from the full MCMC. It would be very handy if it was possible to flip between them when inside the popup.
>
> Thank you!
> Leohttps://git.ligo.org/computing/gracedb/server/-/issues/21Introduce type-ahead- or tab-completion-like features to the GraceDB search2022-08-03T18:14:45ZTanner PrestegardIntroduce type-ahead- or tab-completion-like features to the GraceDB searchStarted on May 9, 2014 by Branson. Copied from redmine (https://bugs.ligo.org/redmine/issues/1337)
From a conversation with Fan, Erik, and Patrick on May 8, 2014.
Fan suggested a type-ahead feature, as in Google search. You start typin...Started on May 9, 2014 by Branson. Copied from redmine (https://bugs.ligo.org/redmine/issues/1337)
From a conversation with Fan, Erik, and Patrick on May 8, 2014.
Fan suggested a type-ahead feature, as in Google search. You start typing, and the event list is narrowed down as you go, before your very eyes.
I pointed out that this might be difficult, as we can't load huge numbers of events into a datastore in order to facilitate this.
Patrick suggested that even keyword completion feature would be really useful. If you start typing 'Te...', GraceDB could fill in 'Test' by looking at her lexicon of keywords.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/22Overhaul of search feature2022-08-03T18:16:59ZTanner PrestegardOverhaul of search featureStarted on April 15, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5432)
The search feature really needs to be redone. There are several requests for new features (#1337, #2175, #3543, #5052) and the code (gracedb/quer...Started on April 15, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5432)
The search feature really needs to be redone. There are several requests for new features (#1337, #2175, #3543, #5052) and the code (gracedb/query.py) is really clunky. There is also a serious lack of consistency regarding when logical operators, quotes, keywords, etc. can/should be used.
Ideas from Patrick:
define a "language" for the search and STICK TO IT. Can get ideas from Google, other search syntaxes.
get feedback from users on any commonly used searches (primarily by automated systems) in order to make sure they don't break with the update (may have to break them, we'll see)
could be similar to natural language processing
expand search capabilities beyond what we have now, including the ability to search by mass, other parameters
improve overall architecture
think about design, understand uses, make a ~1 page write-up describing your planBackloghttps://git.ligo.org/computing/gracedb/server/-/issues/23GraceDB search suggestions2022-08-03T18:17:49ZTanner PrestegardGraceDB search suggestionsFrom Brian O'Reilly, started on January 24, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5052)
The search help does not reflect the actual functionality of the search. It may be that the search functions as expected a...From Brian O'Reilly, started on January 24, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5052)
The search help does not reflect the actual functionality of the search. It may be that the search functions as expected and one cal only use relational operators in restricted ways, but this isn't clear. Also the layout of the search results returned from the "Search" tab is different from what you see if you do a search from the "Latest" tab.
For example, "far<1e-6 ~INJ" works but the help seems to indicate that the syntax should be "far<1e-6 & ~INJ"
"H1OK | L1OK & ~INJ & ~DQV" works but there doesn't seem to be any way to add a FAR cut, e.g. "far<1e-6", to this
query.
You can see all results from a particular pipeline removing injections and adding a FAR cut: "cwb far<1e-7 ~inj", but from the help one expects the syntax to be "cwb & far<1e-7 & ~inj"
It would be nice to be able to remove a given pipeline from the search results. "~cwb" for example does not work.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/24Search for hardware injection by id H208670 misinterpreted2022-08-03T18:18:19ZTanner PrestegardSearch for hardware injection by id H208670 misinterpretedStarted on February 2, 2016 by Branson. Copied from redmine (https://bugs.ligo.org/redmine/issues/3543)
The query parser strips off the 'H2' and interprets it as an instrument search. The remaining '08670' is interpreted as an integer, ...Started on February 2, 2016 by Branson. Copied from redmine (https://bugs.ligo.org/redmine/issues/3543)
The query parser strips off the 'H2' and interprets it as an instrument search. The remaining '08670' is interpreted as an integer, and therefore a gpstime query.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/25Allow negative searches2022-08-03T18:19:12ZTanner PrestegardAllow negative searchesStarted on June 3, 2015 by Branson. Copied from redmine (https://bugs.ligo.org/redmine/issues/2175)
> On Jun 3, 2015, at 10:42 AM, Salvatore Vitale <salvatore.vitale@ligo.mit.edu> wrote:
>
> Hi Branson,
>
> Is there a way to perform a...Started on June 3, 2015 by Branson. Copied from redmine (https://bugs.ligo.org/redmine/issues/2175)
> On Jun 3, 2015, at 10:42 AM, Salvatore Vitale <salvatore.vitale@ligo.mit.edu> wrote:
>
> Hi Branson,
>
> Is there a way to perform a negative query on graceDB. E.g. I tried
>
> search: !MDC pipeline: cwb
>
> but that doesn't work. I tried a few other syntaxes (not, !=, <>) but none seems to work
>
> Thanks,
> salvoBackloghttps://git.ligo.org/computing/gracedb/server/-/issues/26Visualization issues with EMBB2019-04-01T14:32:47ZTanner PrestegardVisualization issues with EMBBStarted on April 21, 2016. Copied from redmine (https://bugs.ligo.org/redmine/issues/4053)
I (Alex) received a report from Peter Shawan (pshawhan@umd.edu) regarding visualization issues with the EM Bulletin Board:
> I see what you mean...Started on April 21, 2016. Copied from redmine (https://bugs.ligo.org/redmine/issues/4053)
I (Alex) received a report from Peter Shawan (pshawhan@umd.edu) regarding visualization issues with the EM Bulletin Board:
> I see what you mean about Skymap Viewer: it seems to display a different set of tiles when I select the GOTO box; whatever the previously displayed set was, I think. For instance, if I view the most recent (top) skymap from https://gracedb.ligo.org/events/view/G211117 and Show Bulletin Board, then de-select GOTO, de-select the last LOFAR-TKSP, and then select GOTO, it displays the tiles from that LOFAR-TKSP set. That's not the only problem, though. The selection box for the last EWE set actually displays the tiles from the middle LOFAR-TKSP set. My hunch is that this is a bug activated by the EWE entry with N_regions=0, but that is just a hunch.
I was able to reproduce the problem by following the steps below:
* Alternating the last two checkboxes in the bulletin board appear to turn on/off the same visualization layer. The color-coding for the layers is definitely incorrect, though I don't know (at first glance) which dataset is being rendered incorrectly.
* As of this posting, I have not observed the bug in other events so I need to determine the commonality in other events.
* Also, users have reported in the same email thread that visualization is slow to render.
I have uploaded the email thread to this ticket and will update it as new reports come in.[Re__GOTO_team_observations_with_SWASP_in_GraceDB.rtf](/uploads/9a517fab2354420f746f362a694883dc/Re__GOTO_team_observations_with_SWASP_in_GraceDB.rtf)https://git.ligo.org/computing/gracedb/server/-/issues/27Cannot load gracedb in iframe2022-08-03T18:19:37ZTanner PrestegardCannot load gracedb in iframeCreated by Chad Hanna on September 30, 2016. Copied from redmine ()
There is probably a good reason for this, but I see:
Refused to display 'https://gracedb.ligo.org/latest/?query=gstlal' in a frame because an ancestor violates the fol...Created by Chad Hanna on September 30, 2016. Copied from redmine ()
There is probably a good reason for this, but I see:
Refused to display 'https://gracedb.ligo.org/latest/?query=gstlal' in a frame because an ancestor violates the following Content Security Policy directive: "frame-ancestors 'none'" in one of my local pages. I would like to embed gracedb with a fixed set of queries into another site. If it is possible to change the Security Policy directive to allow this, that would be swell.https://git.ligo.org/computing/gracedb/server/-/issues/28Images attached to GraceDb pages are not cached2022-03-22T18:22:35ZTanner PrestegardImages attached to GraceDb pages are not cachedCreated January 6, 2017 by Leo Singer. Copied from redmine (https://bugs.ligo.org/redmine/issues/4984)
I noticed that none of the images attached to a GraceDb are cached. Since they are versioned and have unique and permanent filenames,...Created January 6, 2017 by Leo Singer. Copied from redmine (https://bugs.ligo.org/redmine/issues/4984)
I noticed that none of the images attached to a GraceDb are cached. Since they are versioned and have unique and permanent filenames, there is no reason not to let the user's browser cache them. This would reduce bandwidth by a lot for frequently viewed events. For example, the event page for GW170104 pulls down almost 40 MB of resources! Most of those should be cached.https://git.ligo.org/computing/gracedb/server/-/issues/29GraceDb.writeLog raises BadStatusLine error (when GraceDb is overwhelmed?)2019-04-16T22:01:13ZTanner PrestegardGraceDb.writeLog raises BadStatusLine error (when GraceDb is overwhelmed?)Created by Leo Singer on January 17, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5018)
I can very reliably get ligo.gracedb.rest.GraceDb.writeLog() to raise a BadStatusLine error by submitting several test events and...Created by Leo Singer on January 17, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5018)
I can very reliably get ligo.gracedb.rest.GraceDb.writeLog() to raise a BadStatusLine error by submitting several test events and running BAYESTAR on all of them simultaneously. At least one of the jobs will usually die with the traceback below.
This may be a random error, or it may be related to the high rate of GraceDb log message uploads that this entails. Note that in order to get a useful traceback you will first have to apply Merge Request !6. I suggest pointing the rest and cli interfaces to the gracedb-test server so as not to overwhelm the production server when running this code.
```
$ for x in {1..5}; do ((graceid=$(gracedb Test MBTAOnline AllSky coinc.xml); gracedb upload -t psd $graceid psd.xml.gz && bayestar_localize_lvalert $graceid)&); done
...
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib64/python2.7/threading.py", line 811, in __bootstrap_inner
self.run()
File "/usr/lib64/python2.7/threading.py", line 764, in run
self.__target(*self.__args, **self.__kwargs)
File "/mnt/qfs3/lsinger/local/lib/python2.7/site-packages/ligo/gracedb/logging.py", line 70, in _run
self._gracedb.writeLog(self._graceid, text)
File "/mnt/qfs3/lsinger/local/lib/python2.7/site-packages/ligo/gracedb/rest.py", line 759, in writeLog
'displayName': displayName}, files=files)
File "/mnt/qfs3/lsinger/local/lib/python2.7/site-packages/ligo/gracedb/rest.py", line 347, in post
return self.post_or_put("POST", *args, **kwargs)
File "/mnt/qfs3/lsinger/local/lib/python2.7/site-packages/ligo/gracedb/rest.py", line 372, in post_or_put
return self.request(method, url, body, headers)
File "/mnt/qfs3/lsinger/local/lib/python2.7/site-packages/ligo/gracedb/rest.py", line 480, in request
return GsiRest.request(self, method, *args, **kwargs)
File "/mnt/qfs3/lsinger/local/lib/python2.7/site-packages/ligo/gracedb/rest.py", line 316, in request
response = self.get_response(conn)
File "/mnt/qfs3/lsinger/local/lib/python2.7/site-packages/ligo/gracedb/rest.py", line 265, in get_response
return conn.getresponse()
File "/usr/lib64/python2.7/httplib.py", line 1089, in getresponse
response.begin()
File "/usr/lib64/python2.7/httplib.py", line 444, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python2.7/httplib.py", line 408, in _read_status
raise BadStatusLine(line)
BadStatusLine: ''
```
I'm guessing that this is a server-side issue and the django log will be more informative.https://git.ligo.org/computing/gracedb/server/-/issues/30Filenames truncated on upload2022-08-03T18:22:34ZTanner PrestegardFilenames truncated on uploadAdded by Chad Hanna on February 28, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5236)
Hi,
I am not exactly sure where things are going wrong, but I have found that I cannot upload a long filename without it being tr...Added by Chad Hanna on February 28, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5236)
Hi,
I am not exactly sure where things are going wrong, but I have found that I cannot upload a long filename without it being truncated. See e.g.,
https://gracedb.ligo.org/events/view/G275744
in the log entry where it tries to point to:
https://gracedb.ligo.org/apiweb/events/G275744/files/H1L1V1-GSTLAL_INSPIRAL_PLOTSUMMARY_ALL_LLOID_COMBINED_02_mchirp_acc_frac_scatter_SpinTaylorT4threePo,1
which doesn't exist.https://git.ligo.org/computing/gracedb/server/-/issues/31Intermitter server gateway timeouts2018-08-20T14:29:09ZTanner PrestegardIntermitter server gateway timeoutsCreated on April 5, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5418)
Several follow-up processes (approval processor, event supervisor, probably others) and one search pipeline (cWB) have reported receiving 504 gat...Created on April 5, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5418)
Several follow-up processes (approval processor, event supervisor, probably others) and one search pipeline (cWB) have reported receiving 504 gateway timeout errors when attempting to write log messages to GraceDB. Peter S reports that it happens for approval processor about once per day. It seems as though the log is still written, but the correct response is not sent, as the connection hangs for 2 minutes, then terminates.
The server also accumulates lingering threads owned by the wsgi_daemon user over time. It's not clear if these two issues are related.
I upgraded the production server to mod_wsgi 4.5.11 on 28 Mar 2017 in the hopes that it would take care of the lingering threads (tests on the development servers indicated that it cleared up lingering threads caused by overloading the server with log write processes), but it hasn't.https://git.ligo.org/computing/gracedb/server/-/issues/32Clean up reports page2022-08-03T18:30:42ZTanner PrestegardClean up reports pageCreated by Patrick Brady on July 22, 2015. Copied from redmine (https://bugs.ligo.org/redmine/issues/2313)
The GraceDB reports pages needs to be cleaned up. This means removing some things, moving some things to other places, and/or pro...Created by Patrick Brady on July 22, 2015. Copied from redmine (https://bugs.ligo.org/redmine/issues/2313)
The GraceDB reports pages needs to be cleaned up. This means removing some things, moving some things to other places, and/or providing access to some of the information via a simple query to GraceDB.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/33New strategy for managing documentation2022-08-03T18:31:42ZTanner PrestegardNew strategy for managing documentationCreated on February 27, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6108)
Currently, GraceDB provides Sphinx-built user and admin documentation via the web interface. This is served by Apache via an extremely clunky ...Created on February 27, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6108)
Currently, GraceDB provides Sphinx-built user and admin documentation via the web interface. This is served by Apache via an extremely clunky mechanism which requires duplicates of the server's CSS and JS files in the documentation folders for use by Sphinx.
Proposing a new way of managing documentation:
1. Divide user documentation into two parts:
1. FAQ/Help page written in Django views/templates and hosted on GraceDB. Describes GraceDB and how to use the web interface.
2. readthedocs page for gracedb-client
2. Admin docs move to a git repository (preferably something private like gracedb/admin-tools or gracedb/scripts). Could just be part of the repo's wiki.
This is obviously not a high priority, but something that could be done at some point to alleviate the headaches associated with the current structure.https://git.ligo.org/computing/gracedb/server/-/issues/34Some improvements to the performance summary2022-08-03T18:32:07ZTanner PrestegardSome improvements to the performance summaryStarted by Branson, August 18, 2015. Copied from redmine (https://bugs.ligo.org/redmine/issues/2382)
From Erik:
> Hi again,
>
> The performance summary is nice. You might want to qualify "status" with "status", or whatever is the appr...Started by Branson, August 18, 2015. Copied from redmine (https://bugs.ligo.org/redmine/issues/2382)
From Erik:
> Hi again,
>
> The performance summary is nice. You might want to qualify "status" with "status", or whatever is the appropriate word for that... Maybe once there, it will be helpful to have the actual string corresponding the the "status". BTW, any reason for picking the 3-day stride for these statistics? If such stride is a parameter to the script that populates that page, you might want to keep a few strides run in parallel... say, a day, week, since the beginning of the run?
>
> Thanks,
> --Erikhttps://git.ligo.org/computing/gracedb/server/-/issues/35Updated help page2022-08-03T18:33:10ZTanner PrestegardUpdated help pageGraceDB could really use a "help" page with instructions for LVC and LV-EM people who are having difficulty with their account. Ideally, we should instruct them to look at the Shibboleth session page and get information from there, send ...GraceDB could really use a "help" page with instructions for LVC and LV-EM people who are having difficulty with their account. Ideally, we should instruct them to look at the Shibboleth session page and get information from there, send it to us if needed, etc.
There is some kind of shibboleth page (I think managed by Puppet) which should be updated too.
Finally, the instructions for LV-EM users on gw-astronomy are way out of date and should be updated. There is some question of whether it is OK to tell users how to sign up for LV-EM:observers or if they or their PI should be required to "know" that they have to talk to the EM follow chairs in order to get access.https://git.ligo.org/computing/gracedb/server/-/issues/36Create a logo for GraceDB2022-03-22T18:20:47ZTanner PrestegardCreate a logo for GraceDBCreate by Branson on October 24, 2014. Copied from redmine (https://bugs.ligo.org/redmine/issues/1659)
Then expose it at an open URL and add it to the SP metadata. This is considered good form.Create by Branson on October 24, 2014. Copied from redmine (https://bugs.ligo.org/redmine/issues/1659)
Then expose it at an open URL and add it to the SP metadata. This is considered good form.https://git.ligo.org/computing/gracedb/server/-/issues/37Update front-end package manager configuration2022-08-03T18:33:50ZTanner PrestegardUpdate front-end package manager configurationCreated on December 14, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/6053)
We currently use `bower` for managing a small set of front-end CSS and JS packages. We should fix this by creating a package.json or bower.jso...Created on December 14, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/6053)
We currently use `bower` for managing a small set of front-end CSS and JS packages. We should fix this by creating a package.json or bower.json file in the server code repository so this is self-contained. There should also be some instructions (at least on Gitlab) for how to set up the repository, including running bower to install the packages.
We may also need to move to something other than bower. I get the following message when installing bower:
```
root@gracedb-test:~# npm install -g bower
npm WARN deprecated bower@1.8.2: ...psst! Your project can stop working at any moment because its dependencies can change. Prevent this by migrating to Yarn: https://bower.io/blog/2017/how-to-migrate-away-from-bower/
/usr/bin/bower -> /usr/lib/node_modules/bower/bin/bower
+ bower@1.8.2
updated 1 package in 3.776s
```
We can maybe move to yarn? Need to look into this more, see the above link.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/38Round latency values in 'Latest' display2022-08-03T18:35:58ZTanner PrestegardRound latency values in 'Latest' displayCreated by Peter Shawhan on March 30, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5407)
The 'Latest' query output includes latencies in seconds with six decimal places of precision. The decimal part is not even meani...Created by Peter Shawhan on March 30, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5407)
The 'Latest' query output includes latencies in seconds with six decimal places of precision. The decimal part is not even meaningful, because the insertion time is only recorded with a precision of one second. So I recommend changing the 'Latest' output format to give latencies rounded to integer seconds.https://git.ligo.org/computing/gracedb/server/-/issues/39Add latency to GraceDB event pages2019-12-18T03:35:40ZTanner PrestegardAdd latency to GraceDB event pagesCreated by Cody Messick on April 18, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6134)
The latency is currently only listed in the "Latest" section, it's a piece of information that people often inquire about, so it ...Created by Cody Messick on April 18, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6134)
The latency is currently only listed in the "Latest" section, it's a piece of information that people often inquire about, so it would be really nice to have it in the actual event page somewhere.Alexander PaceAlexander Pacehttps://git.ligo.org/computing/gracedb/server/-/issues/40Include fractional seconds in all formats of event time2022-08-03T18:36:23ZTanner PrestegardInclude fractional seconds in all formats of event timeCreated by Peter Shawhan on January 13, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5012)
When the Event Time (near the top of the page) is given as GPS, it includes fractional seconds. However, the other formats (LL...Created by Peter Shawhan on January 13, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5012)
When the Event Time (near the top of the page) is given as GPS, it includes fractional seconds. However, the other formats (LLO Local, LHO Local, UTC, Virgo Local) are all truncated to be an integer second. (Actually it's not clear at a glance whether GraceDB is rounding or truncating, but I think it is truncating.) It would be helpful to include the fractional part of the second in those formats too.https://git.ligo.org/computing/gracedb/server/-/issues/41Event page date/time selection2022-08-03T18:38:45ZTanner PrestegardEvent page date/time selectionCreated by Stephen Privitera on August 25, 2015. Copied from redmine (https://bugs.ligo.org/redmine/issues/2419)
This is just a design suggestion. There should be one and only one place where the date format can be chosen (i.e. GPS vs. ...Created by Stephen Privitera on August 25, 2015. Copied from redmine (https://bugs.ligo.org/redmine/issues/2419)
This is just a design suggestion. There should be one and only one place where the date format can be chosen (i.e. GPS vs. LLO time vs. UTC ...). At the moment, the default display for me is GPS in the "Basic Info" and LLO local time everywhere else. It would be nice if by default these were the same, and if I decide I don't like whatever the default display is, saying so in once should be enough!https://git.ligo.org/computing/gracedb/server/-/issues/42Solidify event file formats with pipelines2022-08-03T18:40:03ZTanner PrestegardSolidify event file formats with pipelinesCreated on September 28, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5917)
Based on the discussion that Alex and I had today, we decided we should try to clarify how each pipeline will be submitting event files. E.g....Created on September 28, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5917)
Based on the discussion that Alex and I had today, we decided we should try to clarify how each pipeline will be submitting event files. E.g., file type, file format, required parameters, optional parameters, etc. Ideally, we could get a short document from each pipeline on this so that we have a written record to go off of. This is not a top priority but would be quite useful going into O3.https://git.ligo.org/computing/gracedb/server/-/issues/43Re-work web interface2022-08-03T18:41:01ZTanner PrestegardRe-work web interfaceCreated on July 15, 2015 by Branson. Copied from redmine (https://bugs.ligo.org/redmine/issues/2279).
The web interface is pretty out of date and clunky in several way. In general, we should:
* Make the theme better
* Add better mobil...Created on July 15, 2015 by Branson. Copied from redmine (https://bugs.ligo.org/redmine/issues/2279).
The web interface is pretty out of date and clunky in several way. In general, we should:
* Make the theme better
* Add better mobile compatibility
* Improve page loading time/caching
* Improve javascript code which manages event pageshttps://git.ligo.org/computing/gracedb/server/-/issues/44Unify how file versions are handled2022-08-03T18:42:31ZTanner PrestegardUnify how file versions are handledSome resources return files without a version even if a version was requested, and vice versa. This should be standardized so that it is the same everywhere!Some resources return files without a version even if a version was requested, and vice versa. This should be standardized so that it is the same everywhere!Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/45"Docker-ize" the GraceDB server code2018-12-17T19:45:50ZTanner Prestegard"Docker-ize" the GraceDB server codeWe'd like to update the GraceDB server code and its dependencies so that it can be turned into a Docker image and easily deployed. This will involve:
* Changing the server code to play nicely with this construction
* Creating a docker i...We'd like to update the GraceDB server code and its dependencies so that it can be turned into a Docker image and easily deployed. This will involve:
* Changing the server code to play nicely with this construction
* Creating a docker image for the server code alone, with gunicorn running as the entrypoint
* Creating an apache/shibboleth image - Scott K suggests to package these together and then use supervisord or systemd as the entrypoint
* Creating a MySQL image
* Making a docker-compose image, file, whatever it's called to get all of the containers to start and talk to each other.
Questions:
* How to manage logs so that they are preserved if the container goes down and appended to when it starts up again?
* How to manage database in the same way?
* Cron jobs included as part of one of the above images?
* How to include current "bin" repository?
I started working on this a few months ago; the branch is stale but can be easily brought up-to-date: https://git.ligo.org/lscsoft/gracedb/tree/dockerhttps://git.ligo.org/computing/gracedb/server/-/issues/47Adding KAGRA events to GraceDB2022-03-31T15:39:21ZTanner PrestegardAdding KAGRA events to GraceDBCreated by Alex on January 28, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5071)
The purpose of this ticket is to track the future development of the uploading and sharing protocol for events from KAGRA. The below po...Created by Alex on January 28, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5071)
The purpose of this ticket is to track the future development of the uploading and sharing protocol for events from KAGRA. The below points came as a result of a face-to-face conversation at the University of Tokyo on January 28, 2017. Open issues include:
1. ~~How to compartmentalize KAGRA events from LIGO events? Options include (but are not limited to):
Standing up a separate KAGRA GraceDB instance. KAGRA will use separate authentication to restrict access, but access to coincident LIGO events would be unavailable.~~
~~Using the existing GraceDB infrastructure, but restricting access to events to users with a separate KAGRA authentication, e.g., only KAGRA users can have access to events uploaded by KAGRA.~~
2. ~~Event data-exchange between LIGO and KAGRA users. Scenarios include events that are determined to be significant due to LIGO-KAGRA coincidence; what data about the event would be available to LIGO/KAGRA members?~~
3. ~~If and how to restrict KAGRA uploads in the age of public LIGO data? This is an open issue for VIRGO events as well.~~
4. Event data format. What will be the format of events uploaded to GraceDB? Would there need to be modification to GraceDB's data parser to accept KAGRA events?
Please add any relevant parties to this conversation, as needed. Relevant watchers for the ticket should be:
* Nobuyuki Kanda <kanda@sci.osaka-cu.ac.jp>
* Hideyuki Tagoshi <tagoshi@sci.osaka-cu.ac.jp>
---
**Updating this ticket, September 15 2021:**
As of today, KAGRA members have:
1. Access to GraceDB via X509 certificates (https://git.ligo.org/computing/helpdesk/-/issues/506)
2. Access to GraceDB via Shibboleth (https://git.ligo.org/lscsoft/gracedb/-/issues/186)
To my best understanding of the MOU and how it's implemented now, KAGRA members have equal upload and access privileges as LSC members. So the previous discussion regarding separation of GraceDB instances and restricting data access seems to be moot. That leaves the event data format.
This is just a matter of getting an example event upload that has entries for KAGRA's contribution. So, part of the `instruments` column, an additional `sngl_inspiral` table, etc. Once pipelines have an example event upload ("Sample Event?" in the table below), then I can upload and fix GraceDB's upload parser. The things I need to test are:
- Does the event file get ingested into GraceDB without error?
- Are the various event properties and and table entries parsed and input into the database? Are the KAGRA-relevent columns ingested? For example, does it recognize a `K1` instrument column; is KAGRA's `sngl_inspiral` table in the db?
- Are the tables legible and formatted correctly on the event's landing page? Is the data visible?
- Is the KAGRA data returned as part of the LVAlert and event `HTTPResponse` packet?
I started the table below to track the progress.
| Pipeline | Sample Event? | Upload Correctly? | Parse Correctly? | View Correctly? | LVAlert Contents | Link |
| --- | --- | --- | --- | --- | --- | --- |
|`CWB` | :x: | :x: | :x: | :x: | :x: | |
|`gstlal` | :white_check_mark: | :white_check_mark: | :white_check_mark:| :white_check_mark: | :white_check_mark: | [G153205](https://gracedb-test.ligo.org/events/G153205/view/) |
|`MBTAOnline` | :x: | :x: | :x: | :x: | :x: | |
|`oLIB` | :x: | :x: | :x: | :x: | :x: | |
|`pycbc` | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: | [G153215](https://gracedb-test.ligo.org/events/G153215/view/) |
|`spiir` | :x: | :x: | :x: | :x: | :x: | |
Ahead of O4, I will also need a list of KAGRA members who will need to upload new events, and to what pipelines.O4 Infrastructure ImprovementsAlexander PaceAlexander Pacehttps://git.ligo.org/computing/gracedb/server/-/issues/48Add "nickname" field for events2019-04-22T18:23:47ZTanner PrestegardAdd "nickname" field for eventsCreated by Reed Essick on August 9, 2016. Copied from redmine (https://bugs.ligo.org/redmine/issues/4553)
There has been some discussion about creating a "nickname" field in GraceDB to group different GraceIDs associated with the same p...Created by Reed Essick on August 9, 2016. Copied from redmine (https://bugs.ligo.org/redmine/issues/4553)
There has been some discussion about creating a "nickname" field in GraceDB to group different GraceIDs associated with the same physical event. This field could be very useful moving forward and would be incorporated into the automatic event selection performed within approval_processor. Nonetheless, this is not absolutely necessary on a short timescale.
I believe the plan was to create another column in the database's tables, called 'nickname', and to populate it with things like "GW150914" or "LVT151012a", etc.https://git.ligo.org/computing/gracedb/server/-/issues/49Add CSRF protection2022-08-03T18:43:49ZTanner PrestegardAdd CSRF protectionCreated by Alex on April 18, 2016. Copied from redmine (https://bugs.ligo.org/redmine/issues/4038)
There has been interest expressed in implementing cross-site request forgery (CSRF) protection on GraceDB:
https://docs.djangoproject.co...Created by Alex on April 18, 2016. Copied from redmine (https://bugs.ligo.org/redmine/issues/4038)
There has been interest expressed in implementing cross-site request forgery (CSRF) protection on GraceDB:
https://docs.djangoproject.com/ja/1.9/ref/csrf/
This isn't a bug or an urgent feature request; I'm just documenting this for later.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/50MySQL strict mode2019-04-22T18:22:09ZTanner PrestegardMySQL strict modeCreated November 16, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5985)
After upgrading to Django 1.11, you get the following warning when doing migrations:
```
?: (mysql.W002) MySQL Strict Mode is not set for databa...Created November 16, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5985)
After upgrading to Django 1.11, you get the following warning when doing migrations:
```
?: (mysql.W002) MySQL Strict Mode is not set for database connection 'default'
HINT: MySQL's Strict Mode fixes many data integrity problems in MySQL, such as data truncation upon insertion, by escalating warnings into errors. It is strongly recommended you activate it. See: https://docs.djangoproject.com/en/1.11/ref/databases/#mysql-sql-mode
```
The link suggests adding one of the following to your DATABASES OPTIONS:
```
'init_command': "SET sql_mode='STRICT_TRANS_TABLES'"
'init_command': "SET sql_mode='STRICT_ALL_TABLES'"
```
Both of these can lead to issues with the MyISAM engine (see https://www.noelherrick.com/blog/mysql-strict_all_tables-vs-strict_trans_tables). I think we're going to ignore this warning until we potentially update to InnoDB.https://git.ligo.org/computing/gracedb/server/-/issues/51Change table engine for database2022-08-03T18:44:32ZTanner PrestegardChange table engine for databaseCreated on November 16, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5984)
Starting this issue just to have a place to record some research, notes, etc. on our database configuration. Currently, we use MySQL (effectiv...Created on November 16, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5984)
Starting this issue just to have a place to record some research, notes, etc. on our database configuration. Currently, we use MySQL (effectively; moving to MariaDB shortly, in Debian 9) and we use MyISAM as a table engine. My understanding is that MyISAM is antiquated and really only around for legacy support. Note that MariaDB offers an updated alternative to MyISAM called Aria. But most things I've read seem to indicate that InnoDB is highly preferred and is a good general-purpose engine. The main issues I see with MyISAM is that it doesn't have transactions (I would think that is very important) and it technically doesn't have foreign keys, but Django seems to handle that sufficiently well on its own.
However, I note that Branson specifically switched from InnoDB to MyISAM in the following commit from 2014:
```
commit 2d4b3c483a4e40af8ff3ef3336bd9272b28ef01a
Author: Branson Stephens <branson.stephens@ligo.org>
Date: Sat Sep 27 16:10:37 2014 -0500
Committed changes to avoid new tables using INNODB and old ones using MYISAM. This leads to fk constraint errors.
```
So do we really want to go to InnoDB? I ran a few tests and noted that just running the initial migrations for a new 'gracedb' database took ~ 2 mins with InnoDB, compared to 10 secs with MyISAM. So speed might be an issue (or maybe there are just some knobs to tweak which would help). I don't think this is a top priority so I'm just jotting down some notes here for future reference. Note that these are based on brief research and are definitely not exhaustive.
InnoDB benefits:
* ACID transactions
* Row-level locking (only table-level with MyISAM)
* Foreign key constraints (technically don't exist in MyISAM)
* Automatic crash recovery
* Table compression (read/write)
* Spatial data types and indexes
MyISAM benefits:
* Fast COUNT(*) (when WHERE, GROUP BY, or JOIN is not used)
* Smaller disk footprint
* Very high table compression (read only)
If we end up wanting to switch to InnoDB, we need to do two things:
* Remove the line in settings/base.py which forces new tables to use MyISAM ('init_command': "SET storage_engine=MyISAM")
* Convert all tables from MyISAM to InnoDB:
```
SELECT CONCAT('ALTER TABLE ', TABLE_SCHEMA, '.', TABLE_NAME,' ENGINE=InnoDB;') FROM Information_schema.TABLES WHERE TABLE_SCHEMA = 'gracedb' AND ENGINE = 'MyISAM' AND TABLE_TYPE = 'BASE TABLE';
```
Note that this SQL command hasn't been tested and definitely should be!https://git.ligo.org/computing/gracedb/server/-/issues/52Change database backend from MySQL (MariaDB) to PostgresQL2022-02-01T18:44:18ZTanner PrestegardChange database backend from MySQL (MariaDB) to PostgresQLCreated on November 16, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5986)
PostgreSQL seems to the the preferred RDBMS for much of the Django community. A comparison here: https://www.digitalocean.com/community/tutori...Created on November 16, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5986)
PostgreSQL seems to the the preferred RDBMS for much of the Django community. A comparison here: https://www.digitalocean.com/community/tutorials/sqlite-vs-mysql-vs-postgresql-a-comparison-of-relational-database-management-systems.
Making the switch would likely be complicated. But if we can make the argument that it would significantly improve performance (speed and/or reliability), it may be worth it.
One point of note: the abstract base `AutoIncrementModel` would need to be reworked since it executes raw MySQL queries, which likely wouldn't be compatible.O4 Infrastructure ImprovementsDuncan MeacherDuncan Meacherhttps://git.ligo.org/computing/gracedb/server/-/issues/53Enable users to edit EMObservation records2022-08-03T18:45:00ZTanner PrestegardEnable users to edit EMObservation recordsCreated by Branson on September 30, 2015. Copied from redmine (https://bugs.ligo.org/redmine/issues/2604)
From Giulia Stratta:
> Also, it might be useful to edit the [observation record] entries after submission,
> in case of typos/mi...Created by Branson on September 30, 2015. Copied from redmine (https://bugs.ligo.org/redmine/issues/2604)
From Giulia Stratta:
> Also, it might be useful to edit the [observation record] entries after submission,
> in case of typos/mistakes.https://git.ligo.org/computing/gracedb/server/-/issues/54Managing state in GraceDB2019-04-22T18:24:30ZTanner PrestegardManaging state in GraceDBCreated on June 28, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6180)
There has been some discussion of having a state table in GraceDB. Presently, we are thinking of creating some new labels in order to define certa...Created on June 28, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6180)
There has been some discussion of having a state table in GraceDB. Presently, we are thinking of creating some new labels in order to define certain states instead. Today, I added
```
DQOK green Data quality information is available and does not veto the event.
PASTRO_READY green p_astro is available.
EMBRIGHT_READY green EM Bright information is available.
SKYMAP_READY green Skymap is available.
GCN_PRELIM_SENT black A preliminary GCN has been sent.
```
I think we will clean up some of the old labels as well at some point, and update descriptions. Additional labels will probably be added as well.
There was some discussion of adding more information to a label, like tying a particular state to the files related to that state (ex: SKYMAP_READY would be tied to a list of skymap files). Would require some significant changes to the schema, doesn't sound fun..https://git.ligo.org/computing/gracedb/server/-/issues/55List of skymaps and preferred skymap2022-08-03T18:46:59ZTanner PrestegardList of skymaps and preferred skymapCreated on June 28, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6177)
There was some discussion at the DASWG face-to-face about having a list of skymaps and a preferred skymap for superevents. Not sure if this is goi...Created on June 28, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6177)
There was some discussion at the DASWG face-to-face about having a list of skymaps and a preferred skymap for superevents. Not sure if this is going to be pursued or not, so just creating this issue to track it.
We would probably have to create a skymap model and link it to the superevent like we have for events and superevents. Might need a special 'upload_skymap' function in gracedb-client, as well as a special endpoint in the API.https://git.ligo.org/computing/gracedb/server/-/issues/56New file format for PyCBC2019-04-22T18:24:06ZTanner PrestegardNew file format for PyCBCCreated on July 3, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6186)
Discussion with Alex Nitz and Tito Dal Canton on June 22, 2018:
PyCBC wants to move away from .xml files to something like JSON or hdf5. They also...Created on July 3, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6186)
Discussion with Alex Nitz and Tito Dal Canton on June 22, 2018:
PyCBC wants to move away from .xml files to something like JSON or hdf5. They also want to be able to do everything in a single upload, including PSDs, metadata, etc. (not sure why, or if we should do that). It sounds like hdf5 would be preferred.
They also want some way to upload dynamic attributes from their analysis. This sounds nearly impossible with a relational database with a fixed schema. One option I thought of was to add two columns like "dynamic_field_name", "dynamic_field_value" whose contents could change. But we would still have to specify the field type (char, float, etc.) which would restrict things quite a bit. I promised to do some research into this.
Doing this work would be broken up into two parts:
* Determining current requirements for pycbc uploads, translating them to the pycbc group, and determining the structure of the new file format
* Writing translator code for the new file. We should do something like what has been done for CWB (but much nicer) where there is a "translator" class which reads the data and assigns event attributes. Alex thought that they may be able to help with this part if it would be useful for us and would help speed things up.
Questions:
* Should they still be allowed to submit .xml files? Or will we just do hdf5 going forward?https://git.ligo.org/computing/gracedb/server/-/issues/57Rework of authentication and authorization framework2018-12-17T19:44:16ZTanner PrestegardRework of authentication and authorization frameworkWe currently rely on Apache to do a lot of the authorization and authentication in GraceDB. We should get the information from Apache, log the user in, then handle everything else in Django. This will be important for having public acces...We currently rely on Apache to do a lot of the authorization and authentication in GraceDB. We should get the information from Apache, log the user in, then handle everything else in Django. This will be important for having public access for O3. I've started work on this in the following branch: https://git.ligo.org/lscsoft/gracedb/tree/auth_updatehttps://git.ligo.org/computing/gracedb/server/-/issues/58Add backup IdP to GraceDB2018-09-06T15:30:27ZTanner PrestegardAdd backup IdP to GraceDBThe issues with login.ligo.org last week clearly demonstrate the need to allow authentication through GraceDB with the backup IdPs. I played around with it a bit but was not able to get it working in production, due to the fact that we ...The issues with login.ligo.org last week clearly demonstrate the need to allow authentication through GraceDB with the backup IdPs. I played around with it a bit but was not able to get it working in production, due to the fact that we don't pull the base LIGO metadata (we get it from InCommon).https://git.ligo.org/computing/gracedb/server/-/issues/59Password expiration reminders for basic auth tokens2022-08-03T18:48:03ZTanner PrestegardPassword expiration reminders for basic auth tokensCreated January 4, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/4976)
We'd like to implement a system where reminder e-mails are sent out when your basic auth token is about to expire.
Thoughts:
1. How long before exp...Created January 4, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/4976)
We'd like to implement a system where reminder e-mails are sent out when your basic auth token is about to expire.
Thoughts:
1. How long before expiration date?
2. How many reminders?
3. Should update password management page with details. Emphasize 1 year expiration date.
How to implement: daily cron job that checks for any passwords expiring within the next X days?https://git.ligo.org/computing/gracedb/server/-/issues/60Allow group and search fields in creating "triggers"2019-03-06T18:48:39ZTanner PrestegardAllow group and search fields in creating "triggers"Created by Chad Hanna on February 26, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5214)
At present, when creating triggers you can only choose pipelines and labels:
https://gracedb.ligo.org/options/trigger/create
I...Created by Chad Hanna on February 26, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5214)
At present, when creating triggers you can only choose pipelines and labels:
https://gracedb.ligo.org/options/trigger/create
I would like to be able to choose groups and searches too, which would then allow the full control of other gracedb searches.Rework of phone and email alertshttps://git.ligo.org/computing/gracedb/server/-/issues/61Allow more flexible notifications2019-03-06T18:48:29ZTanner PrestegardAllow more flexible notificationsCreated August 18, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5699)
We should allow notifications to be created for just a far threshold (all pipelines) or just a label query (all pipelines) without having to actual...Created August 18, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5699)
We should allow notifications to be created for just a far threshold (all pipelines) or just a label query (all pipelines) without having to actually select all of the pipelines.Rework of phone and email alertshttps://git.ligo.org/computing/gracedb/server/-/issues/62Make google users more clearly identified in GraceDB2022-08-03T18:48:42ZTanner PrestegardMake google users more clearly identified in GraceDBCreated on August 16, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5693)
Email from Jim Ennis, 15 Aug 2017:
> My identity is 106501611151735591651@GOOGLE.COM. Not that I respect fame and fortune for this, but that str...Created on August 16, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5693)
Email from Jim Ennis, 15 Aug 2017:
> My identity is 106501611151735591651@GOOGLE.COM. Not that I respect fame and fortune for this, but that strikes me as rather... anonymous.
We could check what information google puts into the session (maybe talk to Mike M. about this) and see if we can more clearly identify people.
Might be hard to do this retroactively.. Could maybe put something in the middleware which adds info if it's not already saved in the database.https://git.ligo.org/computing/gracedb/server/-/issues/63Fix the way instruments are stored for events2022-08-03T18:49:25ZTanner PrestegardFix the way instruments are stored for eventsCreated August 16, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5694)
Instruments are currently associated with events by a string like "H1,L1" or "H1,L1,V1". This is an ineffective way of doing it and prevents effici...Created August 16, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5694)
Instruments are currently associated with events by a string like "H1,L1" or "H1,L1,V1". This is an ineffective way of doing it and prevents efficient instrument-based queries.
We should create an instruments model and just have a many-to-many relationship with events (may need to create a go-between like "labelling").
I think there is also an 'ifos' variable: we should resolve the redundancy issue if that's the case.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/65Don't reapply labels that already exist2022-08-03T18:59:03ZTanner PrestegardDon't reapply labels that already existCreated on August 25, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5714)
Labels that already exist on an event should not be able to be applied. Annoying when ADVOK is applied multiple times and you get multiple phone...Created on August 25, 2017. Copied from redmine (https://bugs.ligo.org/redmine/issues/5714)
Labels that already exist on an event should not be able to be applied. Annoying when ADVOK is applied multiple times and you get multiple phone/text alerts.
Points to consider:
* Should an XMPP alert be sent out if, for example, the advocate signoff label was already ADVOK, but the comment is just changed? Should it still be an "alert for label" or just an update?
* We definitely shouldn't send out any alerts for "normal" labels like INJ, DQV, etc. if they're reapplied.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/66Load testing2022-08-03T18:59:50ZTanner PrestegardLoad testingCreated on February 5, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6087)
Starting a ticket to define a procedure for load testing. General overview:
* Collect/design some utilities for load testing and monitoring th...Created on February 5, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6087)
Starting a ticket to define a procedure for load testing. General overview:
* Collect/design some utilities for load testing and monitoring the server
* Define how we will evaluate the performance of the server
* Discuss procedure with CGCA admins and EM follow-up group and iterateO4 Prephttps://git.ligo.org/computing/gracedb/server/-/issues/67Add new required attributes for events2019-03-04T19:36:46ZTanner PrestegardAdd new required attributes for eventsCreated on June 26, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6169)
From the emfollow design document, section 4.1.3:
> Each trigger generated by a search should be submitted to GraceDb. In addition to event infor...Created on June 26, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6169)
From the emfollow design document, section 4.1.3:
> Each trigger generated by a search should be submitted to GraceDb. In addition to event information specific to each search, such searches will specify
>
> * the tag/version of code used by the search,
> * the version of calibrated h(t) processed,
> * which low-latency DQ flags were applied,
> * whether the alert corresponds to a hardware injection or not,
> * whether the alert was generated online or offline,
> * the instruments from which h(t) was actually used to estimate the trigger’s significance
> * an estimate of the False Alarm Rate normalized to the livetime for the set of instruments specified (i.e.: joint livetime for a LHO-LLO search).
> If searches do not provide all this information at event-creation time, GraceDb should reject the new event and should not create a new entry in the database.
> Some of these already exist (offline flag, hardware injection (should be specified by applying INJ label), instruments, FAR), but we may need to enforce that they are set (and set with something reasonable). At first glance, we will have to add the following fields to the Event model:
>
> * code_version: CharField
> * hoft_verion: CharField
> * dq_flags: CharField (comma-separated string?) Depends on how these will be uploaded, I don't think we want to add a whole table of DQ flags to GraceDBReadiness for second OPA testhttps://git.ligo.org/computing/gracedb/server/-/issues/68GraceDB instance for lockloss events2022-08-03T19:00:09ZTanner PrestegardGraceDB instance for lockloss eventsCreated on July 3, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6187)
Jamie Rollins is interested in setting up a gracedb instance for tracking lockloss events. Here's a transcript of an email:
> On Thu, Jun 14 2018,...Created on July 3, 2018. Copied from redmine (https://bugs.ligo.org/redmine/issues/6187)
Jamie Rollins is interested in setting up a gracedb instance for tracking lockloss events. Here's a transcript of an email:
> On Thu, Jun 14 2018, Tanner Prestegard <tanner.prestegard@ligo.org> wrote:
>
> > Here's a bit of a brain dump of information and some code links. The actual code is a bit confusing and pretty messy, so I don't know how helpful it will be. In general, for an event, there is a base Event class which has attributes that are present for all events, and then a "sub-event" class which has a different type and additional attributes, depending on the pipeline that submitted it.
>
> Hey, Tanner, thank you very much for this info. This is great.
>
> > Here's a link to the base event class: https://git.ligo.org/lscsoft/gracedb/blob/master/gracedb/events/models.py#L86-147. Each line which assigns something like "Field" or "ForeignKey" is a column in the database for the Event table.
>
> > For LIB events, here's a link to the "sub-event" class: https://git.ligo.org/lscsoft/gracedb/blob/master/gracedb/events/models.py#L890-902.
>
> > Here's an oLIB event json: https://gracedb.ligo.org/apiweb/events/G299534/files/1187734908.99-0.0.json,0. Some additional information, like data analysis group, pipeline, etc. is provided from the method for creating events in gracedb-client.
>
> This looks good. Submitting a json with the existing gracedb client
> would be great.
>
> > Maybe we can create a new "sub-event" type for lockloss events? Off the top of my head, that seems like it might be the easiest solution to leverage the current infrastructure.
>
> Yeah this seems like a reasonable place to start. In the long run it
> might be nice to be able to specify the schemas in a config file, rather
> than being baked in, so that you don't have to patch upstream for
> different schemas.
>
> > Hopefully this is somewhat helpful. If you can come up with some initial ideas for the following, that would be helpful:
>
> * Which attributes a lockloss event should include
> * What type (float, int, char, datetime, foreign key to another object, etc.) each attribute should be
> Here's a brain dump of all the stuff I can think of right now that we
> would want to at least initially track for lock loss events:
>
> * initial event time (float (assuming GPS))
> * refined event time (float (assuming GPS))
> * lock loss state index (int)
> * lock loss state string (str)
> * previous state index (int)
> * previous state string (str)
> In addition to these it would be really great to add arbitrary tags to
> an event that would then be searchable (i.e. query "tag=foo tag=bar").
>
> I'm sure we'll realize that there are other attributes we want to track
> down the line, but we may be able to mostly cover that kind of thing
> with tags for a while.
>
> > *Whether the full picture will include more class definitions than just that of lockloss events
> I'm pretty sure that if we had a database for detector related stuff
> then we would find other things we would want to track with it.
> Glitches are the big thing I can think of now. But at least initially I
> don't want to hamstring getting a lock loss database running by adding
> additional complexity right now. We can add a new event schema down the
> line if need be, yes?
>
> > We'll have to iterate some, but this would be a good start. Take your time on this because I have no time at all to spend on this before the OPA test in 2 weeks.
>
> Yeah this is really great Tanner. Thank you very much. If we can get a
> prototype thing running for us to test that would be truly awesome.
> Please let me know what exactly I can do to help, like contributing
> code.
>
> jamie.https://git.ligo.org/computing/gracedb/server/-/issues/69IntegrityError for control room middleware2019-04-22T18:28:07ZTanner PrestegardIntegrityError for control room middlewareThe middleware that adds/removes users from the control room group is throwing IntegrityErrors suddenly. I just noticed it today when trying to test web signoffs on gracedb-dev2, and later, I got notifications when a user was trying to ...The middleware that adds/removes users from the control room group is throwing IntegrityErrors suddenly. I just noticed it today when trying to test web signoffs on gracedb-dev2, and later, I got notifications when a user was trying to GET a file on gracedb-playground. I noted that the user's REMOTE_ADDR corresponded to the H1 control room.
Potentially, this could be fixed by the new auth system which is in development on the auth_update branch, but we would have to test it extensively to be sure. I don't think there is a strong need to fix it before we merge that branch into master, since it seems to occur so rarely and moving to the new branch should happen in the near future.[error_email.log](/uploads/b8b2ceb7c413139a272ac24828cd410b/error_email.log)https://git.ligo.org/computing/gracedb/server/-/issues/70Critical unit tests2018-09-25T15:13:14ZTanner PrestegardCritical unit testsWe need unit tests for checking permissions for superevents. We have to check all API "actions", views, etc. and all web views.We need unit tests for checking permissions for superevents. We have to check all API "actions", views, etc. and all web views.Move superevents branch onto production GraceDBhttps://git.ligo.org/computing/gracedb/server/-/issues/71Rework labels2019-01-31T16:27:59ZTanner PrestegardRework labelsThere is a desire to add several new labels and remove some of the old ones. Will we completely remove some of them? Should some of them remain in place for historical purposes, but be prevented from being added in the future? Unclear...There is a desire to add several new labels and remove some of the old ones. Will we completely remove some of them? Should some of them remain in place for historical purposes, but be prevented from being added in the future? Unclear.
This issue is developing and being documented in the following Google Sheet: https://docs.google.com/spreadsheets/d/1E5LHIZDW9gEZZpUa-BsyfW7W62SGNbtk-ej4FDDwwyA/edit#gid=0Readiness for second OPA testhttps://git.ligo.org/computing/gracedb/server/-/issues/72Policy for alerts2018-10-05T17:01:56ZTanner PrestegardPolicy for alertsWe should define a policy for alerts and post it at least on a page on GraceDB, possibly in the DCC as well. The main questions to be answered by this policy are:
1. Should any alerts (phone, email, LVAlert) be sent out outside of obse...We should define a policy for alerts and post it at least on a page on GraceDB, possibly in the DCC as well. The main questions to be answered by this policy are:
1. Should any alerts (phone, email, LVAlert) be sent out outside of observing runs?
2. How should we handle existing alerts when beginning a new observing run?
There is potential for an issue to occur due to people leaving the collaboration, but still having alerts set up. But as long as the LDAP is kept up-to-date, that shouldn't be a problem, since we already have a nightly cron job which deletes all Contacts and Triggers for users who are no longer in the LVC.https://git.ligo.org/computing/gracedb/server/-/issues/73Access, authorization, and permissions2019-03-04T19:36:00ZTanner PrestegardAccess, authorization, and permissionsThis issue will be used to track ongoing questions about how to manage the following things in GraceDB:
* External (LV-EM and public) access
* Information shown to external users
* Permissions assigned to usersThis issue will be used to track ongoing questions about how to manage the following things in GraceDB:
* External (LV-EM and public) access
* Information shown to external users
* Permissions assigned to usershttps://git.ligo.org/computing/gracedb/server/-/issues/74Versioning of VOEvent information2022-08-03T19:01:01ZTanner PrestegardVersioning of VOEvent informationOn the low-latency call today, there was discussion about what information should be used to build a VOEvent and whether we can do some sort of versioning of this information, presumably so that VOEvents can be built using "past" informa...On the low-latency call today, there was discussion about what information should be used to build a VOEvent and whether we can do some sort of versioning of this information, presumably so that VOEvents can be built using "past" information. This issue attempts to track the discussion and come up with a list of solutions.
The problem at hand was framed as:
* How do we store the information about what content should be used in building a VOEvent?
* How do we populate that content when building a VOEvent?
* How do we access that content?
Another way of framing it is:
* All content from a superevent which is used to build a VOEvent should be versioned/tracked.https://git.ligo.org/computing/gracedb/server/-/issues/75Moving old detections to superevents and making them public2022-08-03T19:01:51ZTanner PrestegardMoving old detections to superevents and making them publicWe want to make old events like GW170817 into superevents and available publicly.
The steps will be:
1. Aggregate the events into superevents
2. Copy over relevant content (logs, files, etc.)
* Metadata: do we need log submitters, t...We want to make old events like GW170817 into superevents and available publicly.
The steps will be:
1. Aggregate the events into superevents
2. Copy over relevant content (logs, files, etc.)
* Metadata: do we need log submitters, timestamps, etc. to match that of the original event? Patrick: No
3. Add new information like samples files which are included for these events on GWOSC.
4. Internal vetting and approval.
5. Public release
Steps 1 and 2 will likely be handled by superevent_manager/gwcelery.
Questions are:
* What kind of timescale are we working on?
* Catalog paper and engineering run are expected near the end of November. Can we get this done on a similar timescale?
* Maybe - need to talk to Tom about AWS/public implementation planO4 CBC Improvementshttps://git.ligo.org/computing/gracedb/server/-/issues/76Change datatype of Retraction field in VOEvent to int2018-12-03T21:04:06ZLeo P. SingerChange datatype of Retraction field in VOEvent to intAll of the other flag-like params in VOEvents have `dataType="int"` and `value="0"` or `value="1"`. The `Retraction` param is the exception. Please change the `Retraction` param to match.All of the other flag-like params in VOEvents have `dataType="int"` and `value="0"` or `value="1"`. The `Retraction` param is the exception. Please change the `Retraction` param to match.Update VOEvent schema/contentshttps://git.ligo.org/computing/gracedb/server/-/issues/77Remove "Vetted" param2018-12-11T16:56:50ZLeo P. SingerRemove "Vetted" paramOur VOEvents contain the following param:
```
<Param name="Vetted" dataType="int" value="0" ucd="meta.number" unit="">
<Description>
Indicates whether this candidate has undergone basic vetting by humans
</Description>
</Param>
```
How...Our VOEvents contain the following param:
```
<Param name="Vetted" dataType="int" value="0" ucd="meta.number" unit="">
<Description>
Indicates whether this candidate has undergone basic vetting by humans
</Description>
</Param>
```
However, this is redundant because we *define* preliminary alerts as alerts that have not undergone human vetting whereas initial, update, and retraction alerts have been vetted.
I suggest removing this flag. If we want to preserve the human-readable comment, then it can go inside `<Why><Description></Description></Why>`.Update VOEvent schema/contentshttps://git.ligo.org/computing/gracedb/server/-/issues/78Add Packet_Type param to VOEvents2018-12-06T18:48:39ZLeo P. SingerAdd Packet_Type param to VOEventsThere is one parameter that we must add to our VOEvents for support of existing GCN VOEvent clients:
<Param name="Packet_Type" dataType="int" value="150">
<Description>The Notice Type number is assigned/used within GCN, eg t...There is one parameter that we must add to our VOEvents for support of existing GCN VOEvent clients:
<Param name="Packet_Type" dataType="int" value="150">
<Description>The Notice Type number is assigned/used within GCN, eg type=150 is an LVC_PRELIMINARY notice</Description>
</Param>
The allowed values are:
LVC_PRELIMINARY=150
LVC_INITIAL=151
LVC_UPDATE=152Update VOEvent schema/contentshttps://git.ligo.org/computing/gracedb/server/-/issues/79Add verification to phone and email alerts2019-03-06T18:48:03ZTanner PrestegardAdd verification to phone and email alertsCurrently, there is no verification of phone or email contact information when signing up for an alert. This could allow accidental information leakage. We should do the following:
1. Add a "verified" boolean column to the Contact mod...Currently, there is no verification of phone or email contact information when signing up for an alert. This could allow accidental information leakage. We should do the following:
1. Add a "verified" boolean column to the Contact model that is False by default
2. Users can't test their contacts until they are verified (grey out test link, add a check in the contact test view)
3. When a user creates a contact, we should generate a random string (length TBD), attach it to the instance, and send it to their email or phone.
4. Add a contact verification page (linked from options page) where they can enter the string
5. Alerts should check that the contacts are verified
6. Unit tests for all of this!Rework of phone and email alertshttps://git.ligo.org/computing/gracedb/server/-/issues/82Bad encoding of file download URLs2022-08-03T19:03:05ZTanner PrestegardBad encoding of file download URLsLinks on event detail pages (in the log messages) (probably for superevents, too) with special characters in them like '#' don't work properly. They are OK on the event file list page (probably because the `url` template tag encodes the...Links on event detail pages (in the log messages) (probably for superevents, too) with special characters in them like '#' don't work properly. They are OK on the event file list page (probably because the `url` template tag encodes them properly).
We can use probably use `encodeURLComponent()` in Javascript to fix the issue in the log messages. We should check on the file lists for superevents too. And we will have to patch the client to encode the URL properly in the `files()` method.https://git.ligo.org/computing/gracedb/server/-/issues/83Clean up old X509 certificates2018-12-17T19:43:07ZTanner PrestegardClean up old X509 certificatesThere are a number of old and clearly unused X509 certificates in the database which should be cleared out.
I am also not sure if the 'refresh_users_from_ldap' script clears out old certificates or not. Should check on that.There are a number of old and clearly unused X509 certificates in the database which should be cleared out.
I am also not sure if the 'refresh_users_from_ldap' script clears out old certificates or not. Should check on that.https://git.ligo.org/computing/gracedb/server/-/issues/84Unit tests for event web urls2018-11-19T17:23:38ZTanner PrestegardUnit tests for event web urlsAdd unit tests to check access/authorization for web URLs in the events app.
Things which need to be tested:
* [x] Event detail pages
* [x] Event file list pages
* [x] Event file download pages
* [x] Event creation page
* [x] Event neig...Add unit tests to check access/authorization for web URLs in the events app.
Things which need to be tested:
* [x] Event detail pages
* [x] Event file list pages
* [x] Event file download pages
* [x] Event creation page
* [x] Event neighbor pages
* [x] ~~Event VOEvent creation pages~~ **(deleted)**
* [x] Modify t_90 pages
* [x] Modify permissions pages
* [x] Modify signoff pages
* [x] Log entry creation page
* [x] Log entry tag page
* [x] ~~Process EMBB event log page~~ **(deleted)**
* [x] Process EMObservation pagePublic-facing GraceDB2018-11-19https://git.ligo.org/computing/gracedb/server/-/issues/85Clean up access options for userprofile pages2018-11-15T20:22:44ZTanner PrestegardClean up access options for userprofile pagesCheck on access for userprofile pages for unauthenticated users, may want to create unit testsCheck on access for userprofile pages for unauthenticated users, may want to create unit testsPublic-facing GraceDBhttps://git.ligo.org/computing/gracedb/server/-/issues/86Unit tests for new auth middleware and backends2018-11-26T18:24:55ZTanner PrestegardUnit tests for new auth middleware and backendsNeed unit tests for new auth middleware and backends to make sure they are working as expected. Should check both main site and API components.Need unit tests for new auth middleware and backends to make sure they are working as expected. Should check both main site and API components.Public-facing GraceDB2018-11-16https://git.ligo.org/computing/gracedb/server/-/issues/87Session management2019-09-12T19:08:32ZTanner PrestegardSession managementNeed a page where admins can go to and see all sessions and kill them, if needed.
Should also add some content to the user profile options page where users can see their active sessions and kill them, if they want.
Finally, need a cron...Need a page where admins can go to and see all sessions and kill them, if needed.
Should also add some content to the user profile options page where users can see their active sessions and kill them, if they want.
Finally, need a cron job for deleting old sessionsPublic-facing GraceDB2018-11-30https://git.ligo.org/computing/gracedb/server/-/issues/88Fully implement public "switch"2022-08-03T19:03:28ZTanner PrestegardFully implement public "switch"Fully implement the settings variable which will allow or not allow unauthenticated access. Should also have unit tests which test this capability.Fully implement the settings variable which will allow or not allow unauthenticated access. Should also have unit tests which test this capability.Backlog2018-11-30https://git.ligo.org/computing/gracedb/server/-/issues/89Make documentation accessible2018-11-19T20:18:19ZTanner PrestegardMake documentation accessibleCurrently, the documentation is built by Sphinx and served by apache. We need to serve it separately so that we can easily handle authentication for it.
Might be possible to just serve the files via X-Sendfile and write some simple views.Currently, the documentation is built by Sphinx and served by apache. We need to serve it separately so that we can easily handle authentication for it.
Might be possible to just serve the files via X-Sendfile and write some simple views.Public-facing GraceDB2018-11-30https://git.ligo.org/computing/gracedb/server/-/issues/90Remove log and emobservation creation boxes from public event/superevent pages2018-11-19T21:31:15ZTanner PrestegardRemove log and emobservation creation boxes from public event/superevent pagesPublic users can't create logs or emobservations, so those forms should not be shown.Public users can't create logs or emobservations, so those forms should not be shown.Public-facing GraceDBhttps://git.ligo.org/computing/gracedb/server/-/issues/91New lvem_view function2022-08-03T19:05:11ZTanner PrestegardNew lvem_view functionThe old "lvem_view" which was done with a shibboleth rewrite of the LSC group membership in to the LVEM group membership hasn't been working for a long time. We can probably rewrite this in Django and keep shibboleth out of the loop.
I...The old "lvem_view" which was done with a shibboleth rewrite of the LSC group membership in to the LVEM group membership hasn't been working for a long time. We can probably rewrite this in Django and keep shibboleth out of the loop.
Idea: some middleware (probably at the bottom of the stack) which removes all group memberships from the request, puts the membership info in their session, and adds them to the LV-EM observers (and probably LV-EM) groups. Then on the response cycle, it re-adds the appropriate groups and removes the LV-EM membership(s).
It could be activated by some kind of pop-up banner type thing in the web view. Users should be required to be in the LVC group to see it and activate it.https://git.ligo.org/computing/gracedb/server/-/issues/92Test public views with different browsers2018-11-27T17:32:51ZTanner PrestegardTest public views with different browsersNeed to test public views (especially superevent detail pages) with different browsers (Chrome, Firefox, Safari, etc.). Can test in Windows too.Need to test public views (especially superevent detail pages) with different browsers (Chrome, Firefox, Safari, etc.). Can test in Windows too.Public-facing GraceDBhttps://git.ligo.org/computing/gracedb/server/-/issues/93Test event API access2018-11-27T16:59:27ZTanner PrestegardTest event API accessWe need to do some testing of how access is handled for the events API. But I am not sure that we have time for developing a complete set of unit tests at present. And, since we actually don't want to allow *any* public access to the e...We need to do some testing of how access is handled for the events API. But I am not sure that we have time for developing a complete set of unit tests at present. And, since we actually don't want to allow *any* public access to the events API, that helps simplify things.
For now, I think we should just add the `IsAuthenticated` permission and write a few simple unit tests which loop over the events API urls and make sure that unauthenticated attempts to get to them fail.Public-facing GraceDBhttps://git.ligo.org/computing/gracedb/server/-/issues/94Update superevent pages2018-11-30T17:20:16ZTanner PrestegardUpdate superevent pagesWe need to make some changes to what information is shown on various pages (search results, superevent detail page, etc.), both for authenticated users and for public users. We also need to make changes for how superevents are serialize...We need to make some changes to what information is shown on various pages (search results, superevent detail page, etc.), both for authenticated users and for public users. We also need to make changes for how superevents are serialized for public users. Probably want to make a new serializer and select based on user authentication status.
More information to come.Public-facing GraceDBhttps://git.ligo.org/computing/gracedb/server/-/issues/95Clean up sky map URLs in VOEvents for unauthenticated access2018-12-03T21:02:26ZLeo P. SingerClean up sky map URLs in VOEvents for unauthenticated accessReduce the number of different URLs that are given in the sky map section to two: one for the FITS file and one for the PNG file. Rename to `skymap_fits` and `skymap_png`.
```
<Group type="GW_SKYMAP" name="bayestar">
...Reduce the number of different URLs that are given in the sky map section to two: one for the FITS file and one for the PNG file. Rename to `skymap_fits` and `skymap_png`.
```
<Group type="GW_SKYMAP" name="bayestar">
<Param name="skymap_fits" dataType="string" value="..." ucd="meta.ref.url" unit="">
<Description>Sky Map FITS</Description>
</Param>
<Param name="skymap_png" dataType="string" value="..." ucd="meta.ref.url" unit="">
<Description>Sky Map image</Description>
</Param>
</Group>
```Update VOEvent schema/contentshttps://git.ligo.org/computing/gracedb/server/-/issues/96Rearrange p_astro and em_bright fields to match formatting in user guide2018-12-05T15:47:26ZLeo P. SingerRearrange p_astro and em_bright fields to match formatting in user guideThey should be gathered into Groups, their names should be adjusted, and the descriptions should be updated.
```
<Group type="Classification">
<Description>
Source classification: binary neutron star ...They should be gathered into Groups, their names should be adjusted, and the descriptions should be updated.
```
<Group type="Classification">
<Description>
Source classification: binary neutron star (BNS), neutron star-black hole (NSBH), binary black hole (BBH), or terrestrial (noise)
</Description>
<Param name="BNS" dataType="float" value="0.95" ucd="stat.probability" unit="">
<Description>Probability that the source is a binary neutron star merger</Description>
</Param>
<Param name="NSBH" dataType="float" value="0.01" ucd="stat.probability" unit="">
<Description>Probability that the source is a neutron star - black hole merger</Description>
</Param>
<Param name="BBH" dataType="float" value="0.03" ucd="stat.probability" unit="">
<Description>Probability that the source is a binary black hole merger</Description>
</Param>
<Param name="Terrestrial" dataType="float" value="0.01" ucd="stat.probability" unit="">
<Description>Probability that the source is terrestrial (i.e., a background noise fluctuation or a glitch)</Description>
</Param>
</Group>
<Group type="Properties">
<Description>
Qualitative properties of the source, conditioned on the assumption that the signal is an astrophysical compact binary merger
</Description>
<Param name="HasNS" dataType="float" value="0.95" ucd="stat.probability" unit="">
<Description>Probability that at least one object in the binary has a mass that is less than 3 solar masses</Description>
</Param>
<Param name="HasRemnant" dataType="float" value="0.91" ucd="stat.probability" unit="">
<Description>Probability that a nonzero mass was ejected outside the central remnant object</Description>
</Param>
</Group>
```Update VOEvent schema/contentshttps://git.ligo.org/computing/gracedb/server/-/issues/97Rolling deletion of Test events and superevents2022-08-03T19:04:41ZTanner PrestegardRolling deletion of Test events and supereventsTest events currently make up 46% of the events in the database and take up 25% of the storage. We should not be preserving Test events indefinitely (on principle) and it could help speed things up a bit. I would like to establish a ni...Test events currently make up 46% of the events in the database and take up 25% of the storage. We should not be preserving Test events indefinitely (on principle) and it could help speed things up a bit. I would like to establish a nightly cron job that deletes events and superevents older than 3 months or so.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/98Reimplement code presently implemented using pylal2018-12-12T12:17:46ZThomas DownesReimplement code presently implemented using pylal`pylal` has been deprecated for some time and is no long available in debian/RPM-packaged form. To ensure that existing systems don't break `apt-get install python-pylal` will install `lalapps` but it doesn't actually contain `pylal`.
P...`pylal` has been deprecated for some time and is no long available in debian/RPM-packaged form. To ensure that existing systems don't break `apt-get install python-pylal` will install `lalapps` but it doesn't actually contain `pylal`.
Please re-implement [this code](https://git.ligo.org/lscsoft/gracedb/blob/master/gracedb/events/serialize.py#L151) in contemporary form.https://git.ligo.org/computing/gracedb/server/-/issues/99API endpoints for getting user details2019-02-21T05:24:50ZTanner PrestegardAPI endpoints for getting user detailsI'd like to add two new API endpoints:
* One for a quick "whoami" check that just responds with the username
* Another with more details, like their group memberships, first/last name, permissions, etc.I'd like to add two new API endpoints:
* One for a quick "whoami" check that just responds with the username
* Another with more details, like their group memberships, first/last name, permissions, etc.https://git.ligo.org/computing/gracedb/server/-/issues/100event_added and event_removed lvalerts should contain event packet instead of...2022-08-03T19:05:40ZDeep Chatterjeedeep.chatterjee@ligo.orgevent_added and event_removed lvalerts should contain event packet instead of superevent dataBased on the docs [here](https://gracedb.ligo.org/documentation/lvalert.html#superevent-alerts) the alert `data` for `event_added` and `event_removed` is set to the superevent dictionary. It makes more sense to have the added event dicti...Based on the docs [here](https://gracedb.ligo.org/documentation/lvalert.html#superevent-alerts) the alert `data` for `event_added` and `event_removed` is set to the superevent dictionary. It makes more sense to have the added event dictionary in instead.https://git.ligo.org/computing/gracedb/server/-/issues/101Browser not rendering .xml.gz files correctly2022-08-03T19:06:39ZTanner PrestegardBrowser not rendering .xml.gz files correctlyThe content-type and encoding are set correctly in Django (application/xml and gzip), but those response headers do not seem to be making it back to the browser. Maybe there is some issue with them being passed through the Apache revers...The content-type and encoding are set correctly in Django (application/xml and gzip), but those response headers do not seem to be making it back to the browser. Maybe there is some issue with them being passed through the Apache reverse proxy?
Can debug some with Chrome developer tools and with Apache logs.
See an example here: ![encoding_error](/uploads/7d88026a3ecd9db97786b9c6d014fe09/encoding_error.png)https://git.ligo.org/computing/gracedb/server/-/issues/102Certificate challenge2019-02-21T05:23:08ZTanner PrestegardCertificate challengeUsers who have certificates in their browsers will receive a certificate challenge when they go to anything under `/api/`. The certificate should *not* be required, but it's pretty annoying and most users won't know to just hit cancel (...Users who have certificates in their browsers will receive a certificate challenge when they go to anything under `/api/`. The certificate should *not* be required, but it's pretty annoying and most users won't know to just hit cancel (if they try to submit an invalid certificate, they will not be allowed access).
This is also happening on the main web view pages because the javascript makes calls to the API to get some information.https://git.ligo.org/computing/gracedb/server/-/issues/103Control room group issues2019-04-22T18:28:07ZTanner PrestegardControl room group issuesUsers seem to be getting left in the control room groups when the response cycle through the middleware should be removing them.Users seem to be getting left in the control room groups when the response cycle through the middleware should be removing them.https://git.ligo.org/computing/gracedb/server/-/issues/104Non-internal VOEvents should have the 'public' tag set2019-01-31T16:27:30ZLeo P. SingerNon-internal VOEvents should have the 'public' tag setNon-internal VOEvents should be publicly visible in GraceDb. I guess that the easiest way to make that happen is to set the `public` tag.Non-internal VOEvents should be publicly visible in GraceDb. I guess that the easiest way to make that happen is to set the `public` tag.