GraceDB Server issueshttps://git.ligo.org/computing/gracedb/server/-/issues2023-04-06T14:38:43Zhttps://git.ligo.org/computing/gracedb/server/-/issues/272Update VOEvent packet name to contain IGWN instead of LVC2023-04-06T14:38:43ZDeep Chatterjeedeep.chatterjee@ligo.orgUpdate VOEvent packet name to contain IGWN instead of LVCProposed change "LVC" -> "IGWN". Related issue on py-gcn: https://github.com/nasa-gcn/pygcn/issues/37
This involves updating packet names here: https://git.ligo.org/computing/gracedb/server/-/blob/f9045632831da7afd736081c05afa44bd0faa3...Proposed change "LVC" -> "IGWN". Related issue on py-gcn: https://github.com/nasa-gcn/pygcn/issues/37
This involves updating packet names here: https://git.ligo.org/computing/gracedb/server/-/blob/f9045632831da7afd736081c05afa44bd0faa3e2/gracedb/annotations/voevent_utils.py#L36-42
Updating the IVORN here: https://git.ligo.org/computing/gracedb/server/-/blob/f9045632831da7afd736081c05afa44bd0faa3e2/config/settings/base.py#L235https://git.ligo.org/computing/gracedb/server/-/issues/270remove fluence from cWB alerts for the start of O42023-04-27T17:37:23ZErik Katsavounidisremove fluence from cWB alerts for the start of O4The issue of how and if to communicate fluence/hrss information for bursts public alerts was brought up over email and on the LL telecon of 20230405 ( https://wiki.ligo.org/Bursts/EMFollow/Telecon20230405 ). The consensus reached during ...The issue of how and if to communicate fluence/hrss information for bursts public alerts was brought up over email and on the LL telecon of 20230405 ( https://wiki.ligo.org/Bursts/EMFollow/Telecon20230405 ). The consensus reached during the group telecon is not to communicate that for cWB (or any burst) alerts for the beginning of O4.
The immediate action required is to remove the corresponding lines in the VOEvent building code: https://git.ligo.org/computing/gracedb/server/-/blob/master/gracedb/annotations/voevent_utils.py#L597-634 .
At this moment, the EMfollow userguide requires no action as it correctly makes no mention of the fluence as a quantity that we distribute as part of the Burst public alerts. Likewise, there is no action currently needed by any of the burst pipelines.
The burst group should engage in deciding how and if the fluence/hrss information may become part of the burst public alerts in future updates of the O4 alert generation system.O4 Debugging and Improvementshttps://git.ligo.org/computing/gracedb/server/-/issues/269Request for ' SNR-OPTIMIZED ' label2023-04-06T14:55:42ZMax TrevorRequest for ' SNR-OPTIMIZED ' labelPyCBC would like to add a label ` SNR-OPTIMIZED ` to events uploaded by PyCBC's snr-optimizer. Currently the only way to identify such events is through a comment in the event log, see for example https://gracedb-playground.ligo.org/even...PyCBC would like to add a label ` SNR-OPTIMIZED ` to events uploaded by PyCBC's snr-optimizer. Currently the only way to identify such events is through a comment in the event log, see for example https://gracedb-playground.ligo.org/events/G907237/view/ . Having this label would be useful to easily find and identify these events as a distinct type of upload from PyCBC.
The description should be 'Indicates that the event was uploaded by the PyCBC SNR optimizer as followup to another PyCBC event.'O4 Debugging and Improvementshttps://git.ligo.org/computing/gracedb/server/-/issues/268Public and private facing information in per-pipeline preferred event table2023-05-12T17:42:44ZRyan MageePublic and private facing information in per-pipeline preferred event table## Description of feature request
The per-pipeline preferred event table has had lots of discussion, but should be more or less final. One of the open [actions](https://git.ligo.org/cbc/action_items/-/issues/40#note_663222) from the LVK ...## Description of feature request
The per-pipeline preferred event table has had lots of discussion, but should be more or less final. One of the open [actions](https://git.ligo.org/cbc/action_items/-/issues/40#note_663222) from the LVK was to fix the information exposed internally and externally. The information currently visible. Can you confirm that the information below will already be exposed in the appropriate public/private facing tables? If not, please consider this a feature request.
Internal table:
Anything goes, keep SNRs in there but maybe add the extra information below as well.
External table:
FAR, pastro, source classification and EMbright information (and gpstime I suppose @viola.sordini).
## Use cases
Helping astronomers parse alerts.Critical Path O4 Developmenthttps://git.ligo.org/computing/gracedb/server/-/issues/267Revamp public alert page for O42023-12-04T15:55:13ZAlexander PaceRevamp public alert page for O4I got an email forwarded to me from leo singer from someone who asked him why the page https://gracedb.ligo.org/superevents/public/O4/ wasn't working... which was a reminder to me that the Public Alerts page needs to be updated ahead of ...I got an email forwarded to me from leo singer from someone who asked him why the page https://gracedb.ligo.org/superevents/public/O4/ wasn't working... which was a reminder to me that the Public Alerts page needs to be updated ahead of O4.
This is what I'm thinking: `/superevents/public/O3/` (the current page) and `/superevents/public/O4/` will both forward to a single page, `/superevents/public/`.
On that page, there's two tables that are identical in content to the current public table, except one is for O4 (at the top) and one is for O3 (underneath). The HTML for the table is already in place so it doesn't need to be duplicated, but the querysets being fed into the table need to be different.
So what needs to happen is:
- [ ] [urls.py](https://git.ligo.org/computing/gracedb/server/-/blob/8f85c42dcb699248e382397406eb5f6992870c3e/gracedb/superevents/urls.py#L38-40) should be updated to forward URLs to the new page at `/superevents/public/`
- [ ] [views.py](https://git.ligo.org/computing/gracedb/server/-/blob/8f85c42dcb699248e382397406eb5f6992870c3e/gracedb/superevents/views.py#L183-257) should be modified so that the `context` dictionary has two keys: (`O3:`, `O4:`) with the table contents as their values. Basically that means the `self.object_list` (which is a queryset of public superevents) in `get_context_data` needs to be filtered again for each observation run. Fortunately there is already the `RUN_MAP` dictionary ([here](https://git.ligo.org/computing/gracedb/server/-/blob/8f85c42dcb699248e382397406eb5f6992870c3e/gracedb/search/constants.py#L19-55)) that has gpstimes to filter `t_0` on. So, one filter for O3, and then defining a new variable like `O4_START` and then filtering from `O4_START` to now. There may be a better way of doing that. Side question: if someone tries to load the page before O4, and in this scenario, `O4_START` > now... is that going to break the query? Maybe? Then just catch the error with an `except:` and then return an empty dict, easy peasy.
- [ ] [public_alerts.html](https://git.ligo.org/computing/gracedb/server/-/blob/8f85c42dcb699248e382397406eb5f6992870c3e/gracedb/templates/superevents/public_alerts.html) should be modified to accept the new `context` dictionary and render two tables. The table format should be cleaned up too to make it more consistent with the rest of the site, but that's largely the discretion of the dev.
There may be a few other sundry items in there to clean up that I forgot about.Critical Path O4 Developmenthttps://git.ligo.org/computing/gracedb/server/-/issues/264Enable sorting based on column headers in the neighbors table on event pages2023-03-28T19:06:46ZRyan MageeEnable sorting based on column headers in the neighbors table on event pages## Description of feature request
Enable sorting of events in the neighbors table on the event page by any of the columns. Specifically, I would find this useful for tracking latencies and comparing FARs and SNRs via the web interface. ...## Description of feature request
Enable sorting of events in the neighbors table on the event page by any of the columns. Specifically, I would find this useful for tracking latencies and comparing FARs and SNRs via the web interface. Just having this as a "click header to sort / toggle" would be useful.
## Use cases
I would find this extremely useful in examining and troubleshooting latencies in the early warning analysis (e.g. events like https://gracedb-playground.ligo.org/events/G940563/view/). It would be really nice to track how these events evolve.
## Benefits
Easy diagnostic via the web interface.
## Drawbacks
No.
## Suggested solutionsO4 Debugging and Improvementshttps://git.ligo.org/computing/gracedb/server/-/issues/260[Errno 111] Connection refused2023-03-08T15:21:44ZAlexander Pace[Errno 111] Connection refusedI somehow made some change to `gracedb-dev1` triggers a client error that `gstlal` has seen on occasion on `gracedb-playground`. And I'm seeing it with the redhat and conda distributions of the client code. Example:
```
[~]$ gracedb -s ...I somehow made some change to `gracedb-dev1` triggers a client error that `gstlal` has seen on occasion on `gracedb-playground`. And I'm seeing it with the redhat and conda distributions of the client code. Example:
```
[~]$ gracedb -s https://gracedb-playground.ligo.org/api/ ping
Response from https://gracedb-playground.ligo.org/api/: 200 OK
[~]$ gracedb -s https://gracedb-dev1.ligo.org/api/ ping
Error: HTTPSConnectionPool(host='gracedb-dev1.ligo.org', port=443): Max retries exceeded with url: /api/ (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f4991fd3a20>: Failed to establish a new connection: [Errno 111] Connection refused',))
[~]$ conda activate igwn-py39
(igwn-py39) [~]$ gracedb -s https://gracedb-dev1.ligo.org/api/ ping
Error: HTTPSConnectionPool(host='gracedb-dev1.ligo.org', port=443): Max retries exceeded with url: /api/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7faa599e8070>: Failed to establish a new connection: [Errno 111] Connection refused'))
```
I'm not seeing an error in the gunicorn logs, so i think it might be being booted at apache before it even reaches the gracedb app. I have verbose apache logging enabled on dev1, so maybe this will elucidate what might be happening in the cloud.O4 Debugging and Improvementshttps://git.ligo.org/computing/gracedb/server/-/issues/259Add HIGH_PROFILE label for RRT purposes2023-04-11T23:07:49ZKeita KawabeAdd HIGH_PROFILE label for RRT purposesPlease add `HIGH_PROFILE` label for RRT purposes.
High Profile event candidates are the ones that require ASAP dicussion of the full Level 2 RRT as defined in [L2100046](https://dcc.ligo.org/L2100046) and graphically explained in the wo...Please add `HIGH_PROFILE` label for RRT purposes.
High Profile event candidates are the ones that require ASAP dicussion of the full Level 2 RRT as defined in [L2100046](https://dcc.ligo.org/L2100046) and graphically explained in the workflow chart [L2100045](https://dcc.ligo.org/L2100045).
We plan to supply an automated logic to apply `HIGH_PROFILE` label to High Profile superevents on gwcelery side.
All Level 2 RRT members on duty will have to subscribe to `HIGH_PROFILE` alert via twillio. This will make it simpler to convene the full Level 2 RRT discussion for majority of the cases.Critical Path O4 Developmenthttps://git.ligo.org/computing/gracedb/server/-/issues/258Server Error: ParseResults([ParseResults([13, 57], {})], {'miltime': [13, 57]})2024-03-13T21:11:48ZAlexander PaceServer Error: ParseResults([ParseResults([13, 57], {})], {'miltime': [13, 57]})A few of these popped up when a user was trying to make a superevent query:
```
Internal Server Error: /api/superevents/
KeyError at /api/superevents/
ParseResults([ParseResults([13, 57], {})], {'miltime': [13, 57]})
Request Method: G...A few of these popped up when a user was trying to make a superevent query:
```
Internal Server Error: /api/superevents/
KeyError at /api/superevents/
ParseResults([ParseResults([13, 57], {})], {'miltime': [13, 57]})
Request Method: GET
Request URL: http://gracedb-playground.ligo.org/api/superevents/?query=created%3A+1357152000.000000+..+1360608000.000000
Django Version: 3.2.16
Python Executable: /usr/bin/python3
Python Version: 3.7.3
Python Path: ['/usr/local/bin', '/usr/local/lib/python3.7/dist-packages', '/app/gracedb_project', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/lib/python3/dist-packages', '/app/gracedb_project', '/app/gracedb_project/gracedb', '/usr/local/lib/python3.7/dist-packages/IPython/extensions']
Server time: Tue, 28 Feb 2023 19:13:49 +0000
Installed Applications:
[...]
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.7/dist-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/rest_framework/viewsets.py", line 125, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/usr/local/lib/python3.7/dist-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/usr/local/lib/python3.7/dist-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/usr/local/lib/python3.7/dist-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/rest_framework/mixins.py", line 38, in list
queryset = self.filter_queryset(self.get_queryset())
File "/usr/local/lib/python3.7/dist-packages/rest_framework/generics.py", line 150, in filter_queryset
queryset = backend().filter_queryset(self.request, queryset, self)
File "/app/gracedb_project/gracedb/api/v1/superevents/filters.py", line 38, in filter_queryset
filter_params = parseSupereventQuery(query)
File "/app/gracedb_project/gracedb/search/query/superevents.py", line 293, in parseSupereventQuery
matches = (stringStart + OneOrMore(q) + stringEnd).parseString(s).asList()
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 1131, in parse_string
loc, tokens = self._parse(instring, 0)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 817, in _parseNoCache
loc, tokens = self.parseImpl(instring, pre_loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 3886, in parseImpl
loc, exprtokens = e._parse(instring, loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 817, in _parseNoCache
loc, tokens = self.parseImpl(instring, pre_loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 4790, in parseImpl
loc, tokens = self_expr_parse(instring, loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 817, in _parseNoCache
loc, tokens = self.parseImpl(instring, pre_loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 3999, in parseImpl
loc2, toks = expr1._parse(instring, loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 817, in _parseNoCache
loc, tokens = self.parseImpl(instring, pre_loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 3886, in parseImpl
loc, exprtokens = e._parse(instring, loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 817, in _parseNoCache
loc, tokens = self.parseImpl(instring, pre_loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 3999, in parseImpl
loc2, toks = expr1._parse(instring, loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 817, in _parseNoCache
loc, tokens = self.parseImpl(instring, pre_loc, doActions)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 4117, in parseImpl
doActions,
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 856, in _parseNoCache
tokens = fn(instring, tokens_start, ret_tokens)
File "/usr/local/lib/python3.7/dist-packages/pyparsing/core.py", line 291, in wrapper
ret = func(*args[limit:])
File "/app/gracedb_project/gracedb/events/nltime.py", line 82, in convertToAbsTime
}[timeOfDayStr]
Exception Type: KeyError at /api/superevents/
Exception Value: ParseResults([ParseResults([13, 57], {})], {'miltime': [13, 57]})
Request information:
USER: ryan.magee@ligo.org
GET:
query = 'created: 1357152000.000000 .. 1360608000.000000'
POST: No POST data
FILES: No FILES data
```
might have something to do with the number of decimal points...? not sure yetO4 Debugging and Improvementshttps://git.ligo.org/computing/gracedb/server/-/issues/257add "SNR" field for MLy2023-02-21T19:58:47ZAlexander Paceadd "SNR" field for MLyAdd an `SNR` field for `MLy` events as requested by @vasileios.skliris over Mattermost. Example data files are already being uploaded to playground ([example](https://gracedb-playground.ligo.org/events/G870115/files/T_1359304922.5_HLV.js...Add an `SNR` field for `MLy` events as requested by @vasileios.skliris over Mattermost. Example data files are already being uploaded to playground ([example](https://gracedb-playground.ligo.org/events/G870115/files/T_1359304922.5_HLV.json)).
After the model change and migration, then `translator.py` needs to get edited to read in the field from the json upload, then change the view template so it shows up in the page table, and then `view_utils.py` so it shows up in the httpresponse and igwn-alert.O4 Debugging and Improvementshttps://git.ligo.org/computing/gracedb/server/-/issues/254"Cannot resolve keyword..." query error2023-07-17T15:17:28ZAlexander Pace"Cannot resolve keyword..." query errorA user this afternoon repeatedly attempted to query events using `gracedb-client` for (what I assume was supposed to be) `FAR > 1e-20 gpstime: 899999000.0 .. 999999999.9`, but accidentally (?) put the `gpstime: ....` query in the `orderb...A user this afternoon repeatedly attempted to query events using `gracedb-client` for (what I assume was supposed to be) `FAR > 1e-20 gpstime: 899999000.0 .. 999999999.9`, but accidentally (?) put the `gpstime: ....` query in the `orderby=` keyword argument. I'll paste the traceback at the end of this post.
Turns out if i put any value in the `orderby=` field that doesn't correspond to a valid db column, then it throws up an error. We should add some error trapping to catch that and return a 400 error.
```
Internal Server Error: /api/events/
FieldError at /api/events/
Cannot resolve keyword 'gpstime: 899999000.0 .. 999999999.9' into field. Choices are: coincinspiralevent, created, embbeventlog, emobservation, eventlog, far, gpstime, graceid, grbevent, group, group_id, id, instruments, labelling, labels, lalinferenceburstevent, likelihood, mlyburstevent, multiburstevent, nevents, offline, perms, pipeline, pipeline_id, pipeline_preferred, pipeline_preferred_id, reporting_latency, search, search_id, signoff, siminspiralevent, singleinspiral, submitter, submitter_id, superevent, superevent_id, superevent_preferred_for, voevent
Request Method: GET
Request URL: http://gracedb-dev1.ligo.org/api/events/?query=FAR+%3E+1e-20&sort=gpstime%3A+899999000.0+..+999999999.9
Django Version: 3.2.16
Python Executable: /usr/bin/python3
Python Version: 3.7.3
Python Path: ['/usr/local/bin', '/usr/local/lib/python3.7/dist-packages', '/app/gracedb_project', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/lib/python3/dist-packages', '/app/gracedb_project', '/app/gracedb_project/gracedb', '/usr/local/lib/python3.7/dist-packages/IPython/extensions']
Server time: Wed, 15 Feb 2023 21:07:00 +0000
Installed Applications:
['django.contrib.auth',
'django.contrib.admin',
'django_ses',
'django.contrib.contenttypes',
'user_sessions',
'django.contrib.sites',
'django.contrib.staticfiles',
'django.contrib.messages',
'alerts',
'annotations',
'api',
'core',
'events',
'ligoauth',
'search',
'superevents',
'rest_framework',
'guardian',
'django_twilio',
'django_extensions',
'django.contrib.sessions',
'computedfields',
'django_postgres_vacuum']
Installed Middleware:
['django.middleware.cache.UpdateCacheMiddleware',
'core.middleware.maintenance.MaintenanceModeMiddleware',
'events.middleware.PerformanceMiddleware',
'core.middleware.accept.AcceptMiddleware',
'core.middleware.api.ClientVersionMiddleware',
'core.middleware.api.CliExceptionMiddleware',
'django.middleware.common.CommonMiddleware',
'core.middleware.proxy.XForwardedForMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'user_sessions.middleware.SessionMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'ligoauth.middleware.ShibbolethWebAuthMiddleware',
'ligoauth.middleware.ControlRoomMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware']
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.7/dist-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/usr/local/lib/python3.7/dist-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/usr/local/lib/python3.7/dist-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/usr/local/lib/python3.7/dist-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/app/gracedb_project/gracedb/api/v1/events/views.py", line 402, in get
events = events.order_by(sort).select_subclasses()
File "/usr/local/lib/python3.7/dist-packages/django/db/models/query.py", line 1149, in order_by
obj.query.add_ordering(*field_names)
File "/usr/local/lib/python3.7/dist-packages/django/db/models/sql/query.py", line 2016, in add_ordering
self.names_to_path(item.split(LOOKUP_SEP), self.model._meta)
File "/usr/local/lib/python3.7/dist-packages/django/db/models/sql/query.py", line 1563, in names_to_path
"Choices are: %s" % (name, ", ".join(available)))
Exception Type: FieldError at /api/events/
Exception Value: Cannot resolve keyword 'gpstime: 899999000.0 .. 999999999.9' into field. Choices are: coincinspiralevent, created, embbeventlog, emobservation, eventlog, far, gpstime, graceid, grbevent, group, group_id, id, instruments, labelling, labels, lalinferenceburstevent, likelihood, mlyburstevent, multiburstevent, nevents, offline, perms, pipeline, pipeline_id, pipeline_preferred, pipeline_preferred_id, reporting_latency, search, search_id, signoff, siminspiralevent, singleinspiral, submitter, submitter_id, superevent, superevent_id, superevent_preferred_for, voevent
Request information:
USER: lorena.magana-zertuche@ligo.org
GET:
query = 'FAR > 1e-20'
sort = 'gpstime: 899999000.0 .. 999999999.9'
POST: No POST data
FILES: No FILES data
```
Note I changed gracedb--> gracedb-dev1 to prevent error emails if someone clicks the link.
**Edit:** in the client code, the kwarg is `orderby=`, but when the form is submitted, it's `sort=` and that's the variable on the server-side.O4 Debugging and Improvementshttps://git.ligo.org/computing/gracedb/server/-/issues/252Twilio SMS undelivered2023-04-27T17:40:09ZAlexander PaceTwilio SMS undeliveredI tried to test SMS from gracedb-dev1 and never got the message. I thought that maybe it was broken, so i looked in the twilio logs and saw this:
![Screen_Shot_2023-02-10_at_11.23.42_AM](/uploads/efd33be3bc6a7d7cd0b5dce933a58dcf/Screen_...I tried to test SMS from gracedb-dev1 and never got the message. I thought that maybe it was broken, so i looked in the twilio logs and saw this:
![Screen_Shot_2023-02-10_at_11.23.42_AM](/uploads/efd33be3bc6a7d7cd0b5dce933a58dcf/Screen_Shot_2023-02-10_at_11.23.42_AM.png)
"Toll-free number has not been verified. Toll-free verification required."
I'm assuming this is related to the new phone number we got @peter-shawhan @daniel.wysocki?Critical Path O4 Developmenthttps://git.ligo.org/computing/gracedb/server/-/issues/251TwilioRestException2023-04-08T21:48:20ZAlexander PaceTwilioRestExceptionThere were three errors this morning, which i'm pretty sure was triggered by someone pushing the "test phone" button on production gracedb. The phone number in the traceback below is redacted for privacy.
```
Internal Server Error: /ale...There were three errors this morning, which i'm pretty sure was triggered by someone pushing the "test phone" button on production gracedb. The phone number in the traceback below is redacted for privacy.
```
Internal Server Error: /alerts/contact/1050/request-code/
TwilioRestException at /alerts/contact/1050/request-code/
HTTP 400 error: Unable to create record: Permission to send an SMS has not been enabled for the region indicated by the 'To' number: +81XXXXXXXXXX.
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
File "/usr/local/lib/python3.7/dist-packages/django/core/handlers/base.py", line 181, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.7/dist-packages/django/views/generic/base.py", line 70, in view
return self.dispatch(request, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/django/utils/decorators.py", line 43, in _wrapper
return bound_method(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/django/views/generic/base.py", line 98, in dispatch
return handler(request, *args, **kwargs)
File "/app/gracedb_project/gracedb/alerts/views.py", line 357, in get
self.object.send_verification_code()
File "/app/gracedb_project/gracedb/alerts/models.py", line 133, in send_verification_code
body=msg)
File "/usr/local/lib/python3.7/dist-packages/twilio/rest/api/v2010/account/message/__init__.py", line 92, in create
data=data,
File "/usr/local/lib/python3.7/dist-packages/twilio/base/version.py", line 209, in create
raise self.exception(method, uri, response, 'Unable to create record')
Exception Type: TwilioRestException at /alerts/contact/1050/request-code/
Exception Value: HTTP 400 error: Unable to create record: Permission to send an SMS has not been enabled for the region indicated by the 'To' number: +81XXXXXXXXXX.
Request information:
USER: hisaaki.shinkai@shibbi.pki.itc.u-tokyo.ac.jp
```
Based on the username and the country code, maybe something's up with Japanese SMS? Or non-US numbers? This seems like something that needs to be looked up in Twilio and not our implementation.
btw if the `USER:` indicator were missing for any reason, you can look up a user's contact info by querying the database in the django shell (`python3 manage.py shell`):
```
In [1]: from alerts.models import Notification, Contact
In [2]: Contact.objects.filter(phone='+81XXXXXXXXXX')
Out[2]: <QuerySet [<Contact: hisaaki.shinkai@shibbi.pki.itc.u-tokyo.ac.jp: Phone and text>]>
```
Or maybe all the users who have japanese phone numbers:
```
In [5]: Contact.objects.filter(phone__contains='+81')
Out[5]: <QuerySet [<Contact: hisaaki.shinkai@shibbi.pki.itc.u-tokyo.ac.jp: Phone and text>, <Contact: koh.ueno@ligo.org: test>, <Contact: minori.shikauchi@ligo.org: cellphone>, <Contact: leo.tsukada@ligo.org: phone>, <Contact: leo.tsukada@ligo.org: SMS>]>
```Critical Path O4 Developmenthttps://git.ligo.org/computing/gracedb/server/-/issues/248server 2.17.1 can't handle tokens2023-01-19T14:22:07ZDuncan Macleodduncan.macleod@ligo.orgserver 2.17.1 can't handle tokensIt seems that the current gracedb-playground is rejected my SciToken:
<details open>
<summary>gracedb production instance</summary>
```console
$ python -c "from ligo.gracedb.cli.client import main; main()" -s https://gracedb.ligo.org/a...It seems that the current gracedb-playground is rejected my SciToken:
<details open>
<summary>gracedb production instance</summary>
```console
$ python -c "from ligo.gracedb.cli.client import main; main()" -s https://gracedb.ligo.org/api/ credentials server
{
"username": "duncan.macleod@LIGO.ORG",
"first_name": "Duncan",
"last_name": "Macleod",
"email": "duncan.macleod@ligo.org",
"is_internal_user": true
}
```
</details>
<details open>
<summary>gracedb playground</summary>
```console
$ python -c "from ligo.gracedb.cli.client import main; main()" -s https://gracedb-playground.ligo.org/
api/ credentials server
{
"username": "AnonymousUser"
}
```
</details>
<details>
<summary>software info</summary>
```console
$ conda list gracedb
# packages in environment at /cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/envs/igwn-testing:
#
# Name Version Build Channel
ligo-gracedb 2.10.0 pyhd8ed1ab_0 conda-forge
```
</details>
</details>https://git.ligo.org/computing/gracedb/server/-/issues/247Different information in pipeline_preferred_events dictionary2024-03-13T21:17:01ZCody MessickDifferent information in pipeline_preferred_events dictionaryWe need more information in the `pipeline_preferred_events` dictionary, specifically the SNR from each ifo for CBC events. What would make our life in gwcelery easiest is if you could add the `extra_attributes` field that's populated in ...We need more information in the `pipeline_preferred_events` dictionary, specifically the SNR from each ifo for CBC events. What would make our life in gwcelery easiest is if you could add the `extra_attributes` field that's populated in the `preferred_event_data` field for each preferred pipeline event, that way we could just reuse the same code path when picking pipeline preferred events as we do when picking globally preferred events.
I also don't think we need the `likelihood` field, as not all pipelines use likelihoods.Critical Path O4 Developmenthttps://git.ligo.org/computing/gracedb/server/-/issues/246Append "Z" to ISO 8601 datetimes strings in VOEvents2022-12-03T01:47:45ZLeo P. SingerAppend "Z" to ISO 8601 datetimes strings in VOEventsSee emfollow/gwcelery!1007, emfollow/userguide!156.See emfollow/gwcelery!1007, emfollow/userguide!156.https://git.ligo.org/computing/gracedb/server/-/issues/245Update token issuer/scope to new IGWN issuer2023-04-11T20:10:09ZDuncan Macleodduncan.macleod@ligo.orgUpdate token issuer/scope to new IGWN issuerThe GraceDB server needs to be updated to accept tokens from the new issuer:
- issuer: `https://cilogon.org/igwn`
- scopes: `gracedb.read`
Given how early in the scitokens roll-out we are, it's probably not necessary to also accept old...The GraceDB server needs to be updated to accept tokens from the new issuer:
- issuer: `https://cilogon.org/igwn`
- scopes: `gracedb.read`
Given how early in the scitokens roll-out we are, it's probably not necessary to also accept older tokens, we can just bump users onto the new ones.Critical Path O4 Developmenthttps://git.ligo.org/computing/gracedb/server/-/issues/243HasMassGap properties in VO and LVK collaboration author.2023-02-08T15:52:25ZRoberto DePietriHasMassGap properties in VO and LVK collaboration author.VO events no longer have the MasGapp classification, but they now have a HasMassGapp property. Also, the VO should be marked as LVK collaboration, .The main file involved with the change is 'gracedb/annotations/voevent_utils.py'. Changes...VO events no longer have the MasGapp classification, but they now have a HasMassGapp property. Also, the VO should be marked as LVK collaboration, .The main file involved with the change is 'gracedb/annotations/voevent_utils.py'. Changes are drafted in the merge request in the fork [1].
Related issues/merge requests:
1. Draft implementation https://git.ligo.org/roberto.depietri/gracedb/-/merge_requests/1
1. https://git.ligo.org/emfollow/userguide/-/issues/285
1. https://git.ligo.org/emfollow/userguide/-/merge_requests/141https://git.ligo.org/computing/gracedb/server/-/issues/241Upgrade pyparsing to 3.0.0 or above2023-02-08T15:59:30ZDaniel WysockiUpgrade pyparsing to 3.0.0 or aboveGraceDB is stuck on `pyparsing==2.3.0` due to an API change. There are some new features in `3.0.0` which would be useful, especially for visualization purposes, so we should upgrade.GraceDB is stuck on `pyparsing==2.3.0` due to an API change. There are some new features in `3.0.0` which would be useful, especially for visualization purposes, so we should upgrade.Daniel WysockiDaniel Wysockihttps://git.ligo.org/computing/gracedb/server/-/issues/238race condition when creating new symlinks2022-10-06T13:51:14ZAlexander Pacerace condition when creating new symlinksThere have been a couple of scenarios when RAVEN (@brandon.piotrzkowski) has tried to simultaneously upload multiple files of the same filename (i.e., `coincidence_far.json`) at the same time, and while the code made it past the part in ...There have been a couple of scenarios when RAVEN (@brandon.piotrzkowski) has tried to simultaneously upload multiple files of the same filename (i.e., `coincidence_far.json`) at the same time, and while the code made it past the part in the [code](https://git.ligo.org/computing/gracedb/server/-/blob/master/gracedb/core/vfile.py#L185) where it removes the old symlink, it would `OSError` (traceback at the end of this issue) when trying to [create](https://git.ligo.org/computing/gracedb/server/-/blob/master/gracedb/core/vfile.py#L191) a new one because another process beat it to it.
This commit: https://git.ligo.org/computing/gracedb/server/-/commit/3daa20fd8e5736d7176cffee39235a507fd88423 fixes it by just catching the exception and moving on. The versioned file still gets created in the logs and is still available, but there's no real scenario in which we should implement the logic to prioritize one symlink over the other in this case.
```
...
...
File "/app/gracedb_project/gracedb/superevents/utils.py", line 248, in create_log
raise e
File "/app/gracedb_project/gracedb/superevents/utils.py", line 244, in create_log
event_or_superevent.datadir, data_file)
File "/app/gracedb_project/gracedb/core/vfile.py", line 271, in create_versioned_file
fdest.close()
File "/app/gracedb_project/gracedb/core/vfile.py", line 224, in close
self._repoint_symlink()
File "/app/gracedb_project/gracedb/core/vfile.py", line 191, in _repoint_symlink
os.symlink(name, self.fullname)
Exception Type: FileExistsError at /api/superevents/MS221004az/logs/
Exception Value: [Errno 17] File exists: 'coincidence_far.json,7' -> '/app/gracedb_project/../db_data/5f/8/0bea0146287984087bb980b9a0d5840958463/coincidence_far.json'
Request information:
USER: emfollow
GET: No GET data
POST:
tagname = 'ext_coinc'
comment = "RAVEN: Computed coincident FAR(s) in Hz with external trigger <a href='https://gracedb-test.ligo.org/events/M315290'>M315290</a>"
```