GraceDB Server issueshttps://git.ligo.org/computing/gracedb/server/-/issues2019-10-28T17:00:42Zhttps://git.ligo.org/computing/gracedb/server/-/issues/180Collect better usage statistics2019-10-28T17:00:42ZAlexander PaceCollect better usage statisticsI had a request from Chad if it was reasonable to collect better usage statistics from gracedb and lvalert. This seems totally doable, but would just require some parsing/archiving from the gunicorn access logs. The archiving schema is c...I had a request from Chad if it was reasonable to collect better usage statistics from gracedb and lvalert. This seems totally doable, but would just require some parsing/archiving from the gunicorn access logs. The archiving schema is changing with the new deployment, but it shouldn't be that hard to implement after docker-swarm comes up and running.
Note that this ticket has some overlap with #176, so maybe this can be a "two-birds" thing once I get it running.Alexander PaceAlexander Pacehttps://git.ligo.org/computing/gracedb/server/-/issues/181Inconsistent URLs for logs, tags, and voevents, for events and superevents2019-11-15T00:56:19ZLeo P. SingerInconsistent URLs for logs, tags, and voevents, for events and supereventsEvents and superevents have inconsistent URLs for tags and logs. This is confusing and reduces the opportunities for code reuse in client code.
Events use `log`, `tag`, and `voevent`, **singular**:
* `https://gracedb-test.ligo.org/api/...Events and superevents have inconsistent URLs for tags and logs. This is confusing and reduces the opportunities for code reuse in client code.
Events use `log`, `tag`, and `voevent`, **singular**:
* `https://gracedb-test.ligo.org/api/events/{graceid}/log/`
* `https://gracedb-test.ligo.org/api/events/{graceid}/log/{N}/tag/`
* `https://gracedb-test.ligo.org/api/events/{graceid}/voevent/`
Whereas superevents use `logs`, `tags`, and `voevents`, **plural**:
* `https://gracedb-test.ligo.org/api/superevents/{superevent_id}/logs/{N}`
* `https://gracedb-test.ligo.org/api/superevents/{superevent_id}/logs/{N}/tags/`
* `"https://gracedb-test.ligo.org/api/superevents/{superevent_id}/voevents/`https://git.ligo.org/computing/gracedb/server/-/issues/182Ongoing problem of missing notifications from GraceDB2019-11-15T19:24:58ZLeo P. SingerOngoing problem of missing notifications from GraceDBSee original issue, [emfollow/O3break#32](https://git.ligo.org/emfollow/o3break/issues/32).See original issue, [emfollow/O3break#32](https://git.ligo.org/emfollow/o3break/issues/32).https://git.ligo.org/computing/gracedb/server/-/issues/183Pause button for the notifications in https://gracedb.ligo.org/alerts/2019-11-17T07:11:23ZNicolas ArnaudPause button for the notifications in https://gracedb.ligo.org/alerts/## Description of feature request
It would be great if there were a "Pause" button in the "Notifications" section of https://gracedb.ligo.org/alerts/ in addition to "Edit" and "Delete". Currently to deactivate a notification one has to d...## Description of feature request
It would be great if there were a "Pause" button in the "Notifications" section of https://gracedb.ligo.org/alerts/ in addition to "Edit" and "Delete". Currently to deactivate a notification one has to delete it. Setting it to "pause" would allow to keep it deactivated for a while. That would ease the re-enabling of that notification at a later time.
## Use cases
Debug/test/engineering run notifications that one would like to only keep active for a given time period but that could be activated again at a later time.
People on rota may want to have particular notifications enabled only when they are on duty.
## Benefits
No need to set notification(s) again, with the risk to make typos.
Have examples of notifications that worked in the past and could be reused, either as such or slightly modified.
## Drawbacks
People could click the "Pause" button by mistake or forget it had been set for a given notification. So clicking on "Pause" should trigger an "Are you sure?" popup and deactivated notifications should be clearly identified on the Alerts GraceDB page: different font/color/etc.
## Suggested solutions
Condition blocks to be added to the existing code!?https://git.ligo.org/computing/gracedb/server/-/issues/187likelihood value is truncated for cwb events2020-02-06T02:19:08ZAlexander Pacelikelihood value is truncated for cwb eventsI got an email this morning from Edoardo Milotti:
```
Hi Alex,
I noticed a small problem in the events uploaded by cWB on the playground. The field that is
identified as “likelihood” in the text files (a floating point number) is used...I got an email this morning from Edoardo Milotti:
```
Hi Alex,
I noticed a small problem in the events uploaded by cWB on the playground. The field that is
identified as “likelihood” in the text files (a floating point number) is used by GraceDB
to calculate the “SNR” field in the event page: SNR = sort(likelihood). It seems that the
underlying code treats “likelihood” as an integer and the SNR result differs from the expected
one, can you please check? A quick check shows that the same problem exists for real events in GraceDB.
Thank you.
Best, Edoardo
```
I dug into it on production and playground, and it would appear that he's (kind of) right. Likelihood is still a float value, but it's being truncated when the event file is read in.
For example, [this event](https://gracedb-playground.ligo.org/events/G240121/view/) on playground. The uploaded file has the following value for likelihood:
```
...
...
likelihood: 1.503621e+02
...
...
```
But in the database, it shows it as:
```
In [1]: from events.models import Event
In [2]: a = Event.getByGraceid('G240121')
In [3]: a
Out[3]: <MultiBurstEvent: G240121>
In [4]: a.likelihood
Out[4]: 150.0
```
The result of this is that the snr that is reported on the event page is slightly incorrect from the expected value, since GraceDB computes snr=sqrt(likelihood) for CWB events ([code is here](https://git.ligo.org/lscsoft/gracedb/blob/master/gracedb/events/translator.py#L473)).
Also, as far as I can tell, this behavior has been in place since before even O1. For instance, here's the first cwb event of O1: https://gracedb.ligo.org/events/G185420/view/
I've spot checked other values for cwb and coincinspiral events, and the data looks right but I have to inspect it more thoroughly.Alexander PaceAlexander Pacehttps://git.ligo.org/computing/gracedb/server/-/issues/188cWB skymap not found on the "public alerts" GraceDB page2020-01-15T12:13:58ZNicolas ArnaudcWB skymap not found on the "public alerts" GraceDB pageThe root cause of this is probably outside GraceDB and I apologize if this has already been discussed elsewhere -- or is being addressed at the moment. For the recent cWB public alert, https://gracedb.ligo.org/superevents/public/O3/ disp...The root cause of this is probably outside GraceDB and I apologize if this has already been discussed elsewhere -- or is being addressed at the moment. For the recent cWB public alert, https://gracedb.ligo.org/superevents/public/O3/ displays
> S200114f (...) No public skymap image found.
likely because the Bayestar sky maps are labelled "cWB" instead of "bayestar": https://gracedb.ligo.org/apiweb/superevents/S200114f/files/cWB.fits.gz. While fixing the naming convention for the cWB skymaps (that may be on purpose), a test could be added to GraceDB to look for cWB.fits.gz if bayestar.fits.gz is not found.https://git.ligo.org/computing/gracedb/server/-/issues/189Improve caching in django2020-06-17T02:45:49ZAlexander PaceImprove caching in djangoThe settings exist (https://git.ligo.org/lscsoft/gracedb/blob/master/config/settings/base.py#L242) in gracedb's configuration for memcached caching, which is a quick and easy way to cache webpage views. This will be particularly helpful ...The settings exist (https://git.ligo.org/lscsoft/gracedb/blob/master/config/settings/base.py#L242) in gracedb's configuration for memcached caching, which is a quick and easy way to cache webpage views. This will be particularly helpful when new events come in and hundreds of users start hitting the website. Even some modest caching (I don't know what that means yet? 10 seconds? 30 seconds?) would greatly reduce server load and prevent django from making db queries and rendering new templates every time someone goes to the website or hits "reload".
A couple of issues:
* The configuration is set up for `memcached` caching in memory, but the `memcached` daemon isn't actually running or installed on any of the development machines or AWS containers. I installed it manually on `gracedb-dev2` and it started memcaching almost immediately. Almost.
* The `MIDDLEWARE` section of `config/settings/base.py` needs to be edited to look like this:
```
# List of middleware classes to use.
MIDDLEWARE = [
'core.middleware.maintenance.MaintenanceModeMiddleware',
'events.middleware.PerformanceMiddleware',
'core.middleware.accept.AcceptMiddleware',
'core.middleware.api.ClientVersionMiddleware',
'core.middleware.api.CliExceptionMiddleware',
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
'core.middleware.proxy.XForwardedForMiddleware',
'user_sessions.middleware.SessionMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'ligoauth.middleware.ShibbolethWebAuthMiddleware',
'ligoauth.middleware.ControlRoomMiddleware',
]
```
Note that the order apparently matters.
One other complication: it occurred to me that we're running a docker swarm of nodes, and yeah, each one has plenty of memory. However, they won't be able to access each others' local memory cache. Hmmm. I can run some tests and monitor the memory and caching on each node, but it doesn't seem efficient.
Last thing:
Amazon offers something called memcached "Elasticache", which appears to be a shared memory cache for different nodes:
* https://aws.amazon.com/elasticache/memcached/
It seems to be what we're looking for. Also this requires a new django backend:
* https://pypi.org/project/django-elasticache/
So I'm guessing the process is going to look like:
1) Log into AWS and find out how to make a new elasticache partition for each one of the different tiers. This can probably be automated with ansible, but at first I'll just click through the web interface like a caveman.
2) Modify `requirements.txt` to install `django-elasticache`.
3) Modify `config/settings/container/base.py` to include the elasticache stuff under `CACHES`. The address is going to be different, but that can be automated with a deployment environment variable in the docker swarm deployment yml.
4) Modify `MIDDLEWARE` to include the django-elasticache middleware. I'm not sure what this will look like exactly, but it should probably model the block that I pasted up there.
Other useful links:
https://docs.djangoproject.com/en/3.0/topics/cache/
https://www.tutorialspoint.com/django/django_caching.htm
https://devcenter.heroku.com/articles/django-memcacheAlexander PaceAlexander Pacehttps://git.ligo.org/computing/gracedb/server/-/issues/191apache improvements2020-01-29T17:52:33ZAlexander Paceapache improvementsMaking notes here about ways to optimize connections via apache. I was reading through some sources about apache worker concurrency (since CPU usage seems to be a bottleneck? at least that's an operative theory), here's an informative on...Making notes here about ways to optimize connections via apache. I was reading through some sources about apache worker concurrency (since CPU usage seems to be a bottleneck? at least that's an operative theory), here's an informative one:
https://serverfault.com/questions/775855/how-to-configure-apache-workers-for-maximum-concurrency
So I need to find out which modules are being loaded and what settings are being used. Here's the setup for `dev2` and `playground` (AWS).
`gracedb-dev2`:
```
root@gracedb-dev2:/etc/apache2# apachectl -M | grep mpm
mpm_worker_module (shared)
```
```
root@gracedb-dev2:/etc/apache2# cat mods-enabled/worker.conf
<IfModule mpm_worker_module>
ServerLimit 25
StartServers 2
ThreadLimit 64
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
ListenBacklog 511
</IfModule>
```
`gracedb-playground`:
```
root@17b0c88a4c4f:/etc/apache2# apachectl -M | grep mpm
mpm_event_module (shared)
```
```
root@17b0c88a4c4f:/etc/apache2# cat mods-enabled/mpm_event.conf
# event MPM
# StartServers: initial number of server processes to start
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestWorkers: maximum number of worker threads
# MaxConnectionsPerChild: maximum number of requests a server process serves
<IfModule mpm_event_module>
StartServers 2
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
MaxRequestWorkers 150
MaxConnectionsPerChild 0
</IfModule>
# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
```
So that's a start at least. They look to be the same (I'm assuming the default...), but i have some knobs to tweak potentially. I'll look into doing some testing with [ab](https://httpd.apache.org/docs/2.4/programs/ab.html) and [siege](https://www.joedog.org/siege-home/) and see what i can come up with.https://git.ligo.org/computing/gracedb/server/-/issues/192Two suggestions for the "neighbors" table on G event pages2020-02-26T21:53:15ZTito Dal CantonTwo suggestions for the "neighbors" table on G event pagesThe "neighbors" table would be much more useful with the following features:
* A column showing the network SNR for CBC events
* The ability to click on a column to sort the table according to that columnThe "neighbors" table would be much more useful with the following features:
* A column showing the network SNR for CBC events
* The ability to click on a column to sort the table according to that columnhttps://git.ligo.org/computing/gracedb/server/-/issues/198Comments on GraceDB's Visual Interface2020-09-15T09:52:26ZAlexander PaceComments on GraceDB's Visual InterfaceHi All:
I'm opening up the forum to comments regarding GraceDB's visual interface. There are probably many items that I haven't caught yet, so I would appreciate any constructive feedback about anything you have encountered in the past ...Hi All:
I'm opening up the forum to comments regarding GraceDB's visual interface. There are probably many items that I haven't caught yet, so I would appreciate any constructive feedback about anything you have encountered in the past two months since I pushed the interface to [Playground](https://gracedb-playground.ligo.org/) two months ago.
Mostly I'm looking for:
* Lack of functionality that was present in the old site that isn't obviously available.
* Obviously broken visual interface elements (links and screenshots would be appreciated).
* Constructive feedback regarding the interface. I'm looking for comments along the lines of "visual element X should display Y information" or "it would be more clear if this element showed XYZ". Comments like "doesn't look good" do not give me much to work with and are largely subjective.
This ticket should be long-lasting, but I'm going to close the "official" comment period in one week (on June 25) so I can push changes into production with the next version of the [server code](https://git.ligo.org/lscsoft/gracedb/tree/gracedb-2.10.0). I'm not expecting it to be perfect, but I'd rather get an 85% done product out and then polish things as they come up since we're not in observation.
Thanks.https://git.ligo.org/computing/gracedb/server/-/issues/200Proposals and comments for curated event pages2021-09-15T16:11:46ZAlexander PaceProposals and comments for curated event pagesAs the public-facing GWTC-* (curated) event pages are developed, I'll be taking feedback on this ticket before going live. Further background on the curation process can be found here: https://dcc.ligo.org/LIGO-T2000569.
In particular,...As the public-facing GWTC-* (curated) event pages are developed, I'll be taking feedback on this ticket before going live. Further background on the curation process can be found here: https://dcc.ligo.org/LIGO-T2000569.
In particular, I'll be looking for feedback as to how to properly distill all the information that's already in GraceDB in such a way to be digestible and clear for readers outside of the analyst community.
The related OpenProject charge can be found here: https://cbcprojects.ligo.org/projects/gracedb-event-curation/Alexander PaceAlexander Pace2020-11-24https://git.ligo.org/computing/gracedb/server/-/issues/201Update O3 public alerts page to point to published catalog2020-11-23T16:55:28ZJonah KannerUpdate O3 public alerts page to point to published catalog@alexander.pace
Here's a suggestion from Beverly Berger to add a pointer to GWTC-2 on the O3 public alerts page.
-jonah
Hi, Jonah.
I had a thought about how to add information on the final resolution of alert triggers to https://grac...@alexander.pace
Here's a suggestion from Beverly Berger to add a pointer to GWTC-2 on the O3 public alerts page.
-jonah
Hi, Jonah.
I had a thought about how to add information on the final resolution of alert triggers to https://gracedb.ligo.org/superevents/public/O3/. I think it would be sufficient to say something like see (relevant table of events in GWOSC) and (section in the catalog paper where the reasons for the final set of events are given).
Beverlyhttps://git.ligo.org/computing/gracedb/server/-/issues/204Uploading results of targeted GRB/FRB followup searches2023-07-31T15:01:40ZTito Dal CantonUploading results of targeted GRB/FRB followup searchesIn at least two occasions, people have recently requested PyGRB candidates to be uploaded to GraceDB for detchar and PE followup. One difficulty with this is that PyGRB, being a targeted followup search of an existing transient, reports ...In at least two occasions, people have recently requested PyGRB candidates to be uploaded to GraceDB for detchar and PE followup. One difficulty with this is that PyGRB, being a targeted followup search of an existing transient, reports a p-value (false-alarm probability) associated with data around that particular transient, instead of a false-alarm rate as commonly understood in GraceDB land. PyGRB is also a coherent search, which may create some more impedance mismatch in terms of LIGOLW tables. A similar issue would also arise for any X-pipeline candidate (also from targeted GRB/FRB followup searches), although we have not had any particularly interesting X-pipeline candidates yet.
@alexander.pace suggested opening this issue, but I am not entirely sure if this is a PyGRB/X-pipeline problem, or a GraceDB problem, or maybe more of a schema problem. If we had an interesting continuous-wave or stochastic candidate, for example, would people want to see that in GraceDB as well, and what would the solution be?
Tagging @derek.davis, @francesco-pannarale and @ian-harry.https://git.ligo.org/computing/gracedb/server/-/issues/206Proposals and comments for public release of data products2022-08-03T18:46:59ZAlexander PaceProposals and comments for public release of data productsTicket to track discussion regarding O4 release of low-latency data products.
Please refer to the minutes from day-one of the low-latency virtual face-to-face on September 15, 2021 [here](https://docs.google.com/document/d/1L0hLw1A3H20...Ticket to track discussion regarding O4 release of low-latency data products.
Please refer to the minutes from day-one of the low-latency virtual face-to-face on September 15, 2021 [here](https://docs.google.com/document/d/1L0hLw1A3H20Xphjj4vhuN7aLWd2roTmpwDav9PUzNXM/edit#).
In particular, the way public exposure of data products works currently is:
- Only information on a _superevent_'s page is made public. The includes the web front-end and queries from the API.
- G-_event_ pages are not made public. The result of the F2F discussion on 2021/09/15 seemed to indicate that it should remain that way.
- _Superevent_ properties (such as FAR) are inherited from the superevent's preferred event. For reference: a superevent's preferred event is a one-to-one relationship with an event in the database. See here: (https://git.ligo.org/lscsoft/gracedb/-/blob/master/gracedb/superevents/models.py#L92). As such, when a superevent defines its FAR is just a relationship with its preferred event, for example: https://git.ligo.org/lscsoft/gracedb/-/blob/master/gracedb/superevents/models.py#L431
- Data products, such as skymaps, p_astro and such, are uploaded for a superevent and given a couple of tags, such as `skyloc`, which gives the file its own section on a superevent landing page:
![Screen_Shot_2021-09-15_at_12.57.48_PM](/uploads/b6001318e20cb5d5f97973b5a40b632e/Screen_Shot_2021-09-15_at_12.57.48_PM.png)
- Adding a `public` tag will, understandably, make that file (log message) public by having the Django template generator filter based on that public tag (example: https://git.ligo.org/lscsoft/gracedb/-/blob/master/gracedb/templates/superevents/preferred_event_info_table_public.html)
- Similarly, there is a filter of the `public` tag, as well as an authentication check that takes place that selectively returns information via API calls.
- A superevent is a super-set of G-events. This is defined by a ForeignKey database relation (here: https://git.ligo.org/lscsoft/gracedb/-/blob/master/gracedb/events/models.py#L189). This relationship can be created or destroyed by `superevent_manager`s. There's no internal logic deciding this relationship.
Okay. So that's how it works right now. What I propose to _guide the discussion_ is this:
1. What information or products do you want from what events to be made public? Are these event properties, such as FAR and SNR, are there skymaps or other data products?
2. Are these G-event properties coming from G-events are are not already part of a superevent's event relationship? In other words, outside events that are not part of a superevent's event list? If the superevent and event are already linked, then from a technical standpoint, the superevent's landing page and API call can return the information to the public. This is a policy limitation, not a technical limitation. I do not dictate policy.
3. For data products such as skymaps, omega scans, and whatnot, the files are attached to log message objects in the database. Currently, the log message objects are linked via a one-to-one relationship in the database to either a superevent or a g-event.
Elaborating on the last point: concerning data products and files. What I propose from a technical standpoint would be expanding the log-message (and by extension, file product) relationship to include a many-to-many relationship. What this would mean is, a log message and file would be created for a g-event. That's relationship number one. A superevent manager process would determine if that product should be displayed on the superevent page. A new log message is created for the superevent, and that log message would have a linked relationship with the g-event's log message. The set of tags that are usually applied to a superevent's log message (`skyloc`, `public`, etc) are applied to the inherited log message, then boom, it's public.
I think this approach is a way to start and guide the discussion. I believe it has the advantage of:
1. Allowing superevent managers to selectively choose what information and what products are inherited from what g-events.
2. The mechanism for making data products public is unchanged.
3. Adding data products to a superevent (not just public products) is reduced to one API call instead of the convoluted data-transfer bonanza that took place in O3. This is analogous to the "server-side copy" that was discussed in O3b but never brought to fruition.
4. The existing superevent pages, log messages, public views, and event relationships will remain unchanged. Unless a superevent manager changes them.
I see this as a sound technical approach, but I am open to other suggestions. From a policy standpoint, it is up to @erik-katsavounidis and @shaon.ghosh to determine what information from what events and at what time should be linked to a superevent.O4 CBC Improvementshttps://git.ligo.org/computing/gracedb/server/-/issues/207GraceDB considerations before Oct. 2021 MDC2021-10-05T01:37:24ZAlexander PaceGraceDB considerations before Oct. 2021 MDCI would like to be on the same page and tie up some loose ends before October 2021's MDC event. Specifically, please confirm that pipelines and GWCelery will be using the `gracedb-playground` and `lvalert-playground` infrastructure.
Ad...I would like to be on the same page and tie up some loose ends before October 2021's MDC event. Specifically, please confirm that pipelines and GWCelery will be using the `gracedb-playground` and `lvalert-playground` infrastructure.
Additionally, could pipeline head please confirm which `search`es they will be using for event uploads? I will then confirm that the appropriate LVAert nodes are in place to send messages?
Thanks. I'll add more to this ticket as they come up.
`CWB`, `gstlal`, `MBTAOnline`, `oLIB`, `pycbc`, `spiir`https://git.ligo.org/computing/gracedb/server/-/issues/208Request to change sname format2021-11-01T15:17:31ZGregory Ashtongregory.ashton@ligo.orgRequest to change sname format## Description of feature request
Change the sname format from `SYYMMDDabc` to `SYYMMDD_HHMMSSabc`
## Use cases
## Benefits
<!-- Describe the benefits of adding this feature -->
For anyone relating events from papers to GraceDB, th...## Description of feature request
Change the sname format from `SYYMMDDabc` to `SYYMMDD_HHMMSSabc`
## Use cases
## Benefits
<!-- Describe the benefits of adding this feature -->
For anyone relating events from papers to GraceDB, this removes ambiguity and the need to remember which sname maps to which GW name. Or, at least it reduces the number of cases (I think to zero with good confidence, though there could be edge cases).
## Drawbacks
<!--
Are there any drawbacks to adding this feature?
Can you think of any ways in which this will negatively affect the service for any set of users?
-->
snames become longer and "uglier". However, the choice for GW names to be long and ugly has already been made. So this just provides better consistency.
## Suggested solutions
<!-- Do you have any ideas for how to implement this feature? -->
For O4 onwards only. Though, it could also be applied retroactively to GW events not in GraceDB already.Alexander PaceAlexander Pacehttps://git.ligo.org/computing/gracedb/server/-/issues/209Re-enable X-Pipeline2023-02-22T15:36:48ZAlexander PaceRe-enable X-Pipeline**Background:** `X-Pipeline` or just `X` has been floating around in GraceDB since way before I've been on this project, but has [never uploaded an event](https://gracedb.ligo.org/search/?query=X&query_type=E&results_format=S). I was att...**Background:** `X-Pipeline` or just `X` has been floating around in GraceDB since way before I've been on this project, but has [never uploaded an event](https://gracedb.ligo.org/search/?query=X&query_type=E&results_format=S). I was attempting to clean up event logic back in mid 2020, and so I added `X`, `Q`, and `Omega` pipelines to a list of pipelines that were being phased out, and [returned a warning message](https://git.ligo.org/lscsoft/gracedb/-/blob/ca14eb8e8eb0c4111ecac38d6e879472fea1b111/gracedb/api/v1/events/views.py#L524-L525) if a user attempted to upload an event to that pipeline. Not that it would have worked anyway, because the [logic](https://git.ligo.org/lscsoft/gracedb/-/blob/master/gracedb/events/view_logic.py#L66-L67) to ingest X-pipeline event files had never actually been implemented and likely would have returned an error.
I received a request over [mattermost](https://chat.ligo.org/ligo/pl/t1zgpxrm8fdz8qk4xrk4jiueby) to revive the pipeline.
Before proceeding with this, I need from @amber-stuver:
1) An example event upload. I don't know the output file format (xml? json?) or what fields that are in the file should be stored in the database. I can look it over as a first step to compare to other event types that are in GraceDB, but I need the file first and foremost. It can be attached to this ticket.
2) What kind of search type is it? Right now, GraceDB ingests events from `CoincInspiral` searches, `GRB` searches, `Multiburst` searches, etc. If `X` fits into one of those categories, storing event data and constructing the view and `REST` response is simpler. But this will make more sense when I get the example upload.
3) Who is going to be uploading and populating the pipeline? If it's individual users, I need just your `@LIGO.org` email address. Or, if there is a robot account that is uploading, please apply for a cert from https://robots.ligo.org/ and then I'll add it as an uploader.
Once I get the example upload and the other information that I need, the steps I need to do are:
1) Remove `X` from the depreciated pipelines list.
2) Add logic to `view_logic.py` to read in `X` events.
3) Determine views for `X` events, add it to settings.
4) Add uploader permissions for the pipeline
5) Test event uploads, ingestion into the database, and webpage views.
6) Make appropriate LVAlert topics for `X-pipeline`
Then I'll have to push a server code change and deploy it.
Drop any questions or sample files you have here and I'll get back to you.Alexander PaceAlexander Pacehttps://git.ligo.org/computing/gracedb/server/-/issues/210Reducing queries by packing igwn-alert2022-03-18T01:03:58ZAlexander PaceReducing queries by packing igwn-alertAs discussed on low-latency call, December 8 2021. The purpose of this ticket is to solicit feedback and suggestions for what information to include in `igwn-alert` packets with the goal of reducing costly queries to GraceDB.
Relevant p...As discussed on low-latency call, December 8 2021. The purpose of this ticket is to solicit feedback and suggestions for what information to include in `igwn-alert` packets with the goal of reducing costly queries to GraceDB.
Relevant past commits:
* https://git.ligo.org/lscsoft/gracedb/-/commit/e52b0c2ea248efbbb221ed51c58d55a3e5c4a3de
* https://git.ligo.org/lscsoft/gracedb/-/commit/2402e914dd6afd28035f6d06086bd6519f8018a9
Current MR's:
* https://git.ligo.org/lscsoft/gracedb/-/merge_requests/52/O4 Infrastructure ImprovementsAlexander PaceAlexander Pacehttps://git.ligo.org/computing/gracedb/server/-/issues/211Add SciTokens support2023-07-17T15:07:39ZDuncan Macleodduncan.macleod@ligo.orgAdd SciTokens supportThe GraceDB server needs to accept a SciToken as an authorisation option.The GraceDB server needs to accept a SciToken as an authorisation option.O4 AdvanceDuncan MeacherDuncan Meacherhttps://git.ligo.org/computing/gracedb/server/-/issues/212Metadata for triggers and candidate events2022-03-22T13:38:22ZAlexander PaceMetadata for triggers and candidate eventsThe [charge](https://dcc.ligo.org/LIGO-T2100502) for O4 data product management states:
> Metadata: We define metadata as a set of lightweight data products or links to data products
associated with a given trigger. For example, lightwe...The [charge](https://dcc.ligo.org/LIGO-T2100502) for O4 data product management states:
> Metadata: We define metadata as a set of lightweight data products or links to data products
associated with a given trigger. For example, lightweight data products such as the FAR and SNR or
paths links to parameter estimation posteriors.
I think this can be accomplished with a new table that's linked with a 1:1 foreign key to a g-event. Also from the charge, a proposed format the metadata is below. Inserted as a screenshot since copy/paste from pdf totally gnarled up the formatting.
![Screen_Shot_2022-02-28_at_9.29.27_PM](/uploads/92a3e044fab88bcb533741c5f9604d15/Screen_Shot_2022-02-28_at_9.29.27_PM.png)
I think a first cut would involve taking advantage of postgres' [json datatype](https://www.postgresql.org/docs/9.4/datatype-json.html).
Querying for metadata would have to be implemented too.O4 CBC Improvements