GraceDB Server issueshttps://git.ligo.org/computing/gracedb/server/-/issues2019-07-09T23:22:30Zhttps://git.ligo.org/computing/gracedb/server/-/issues/162Show component masses in Superevent preview2019-07-09T23:22:30ZNicola De Lillonicola.delillo@ligo.orgShow component masses in Superevent preview## Description of feature request
It is a web interface change. It would require to add both in the LATEST both in the SEARCH page, the column "MASS1" and "MASS2" (or simply M1 M2) for CBC events/superevent raws showed in both lists. I t...## Description of feature request
It is a web interface change. It would require to add both in the LATEST both in the SEARCH page, the column "MASS1" and "MASS2" (or simply M1 M2) for CBC events/superevent raws showed in both lists. I think the values should come from the preferred event analyses and then update every time the analysts upload a new parameter estimation result that they consider more appropriate.
## Use cases
Evertime looking at latest or DB
## Benefits
It is useful to spot quickly interesting and appealing events for any scientist in the field that are scrolling the DB.
## Drawbacks
NOt any in mind at the moment
## Suggested solutions
It could simply be one more column in the raws shoed both in LATEST and in the SEARCH lists.https://git.ligo.org/computing/gracedb/server/-/issues/161User's favourites or Followed Events list2019-07-09T22:07:10ZNicola De Lillonicola.delillo@ligo.orgUser's favourites or Followed Events list## Description of feature request
Insert a tabin between the already present tabs LATEST and ALERTS. This tab could be called "FOLLOWED EVENTS" (Not PREFERRED, it would confuse people with "preferred events" in case of a super-events).Th...## Description of feature request
Insert a tabin between the already present tabs LATEST and ALERTS. This tab could be called "FOLLOWED EVENTS" (Not PREFERRED, it would confuse people with "preferred events" in case of a super-events).The tab would link to the page "FOLLOWED EVENTS" which would appear really as it looks now the LATEST page. The difference is that the entry showed in "FOLLOWED EVENTS" page will show only the events or supervents flagged as "FOLLOWED" by the user.
Events should be flag-able either from the SEARCH page or from the LATEST page.
## Use cases
1) internal use for LIGO: It is useful for experts ROTA or advocates or PE-Rotaers that can easily keep track of the events they got assigned.
2) In general any scientist interested in a particular event can follow it. i.e.: if I am interested in tracking the analysis for only Binary Neutron stars system, I would flag as FOLLOW only that events.
## Benefits
I think it helps really tracking the events a scientist want to follow.
## Drawbacks
Not really any drawbacks at the moment as far as I can see. Apart that one has to redesign the toolbar adding the "FOLLOWED EVENTS" tab.
## Suggested solutions
Here attached a figure that shows how to display (the example is for LATEST page, BUt consider please doing that also for the SEARCH page) the boxes to flag in the FOLLOW column. All the flagged events (either a flag or dot or a filling color) will go in the FOLLOWED EVENT page. Of course rememer to add a 'X' symbol in the FOLLOWED EVENT page if one wants to remove that event.
![LATEST_examp](/uploads/4b8a8e328fff42644b539cc6ee63d061/LATEST_examp.png)https://git.ligo.org/computing/gracedb/server/-/issues/160gracedb playground querying issues2019-07-10T20:36:08ZChad Hannagracedb playground querying issuesHi,
I don't know how to summarize this into disjoint PRs, so here it all is.
If I execute this query through the web interface:
https://gracedb-playground.ligo.org/search/?query=gstlal+1245542026.0+..+1246542026.0&query_type=E&resul...Hi,
I don't know how to summarize this into disjoint PRs, so here it all is.
If I execute this query through the web interface:
https://gracedb-playground.ligo.org/search/?query=gstlal+1245542026.0+..+1246542026.0&query_type=E&results_format=S
I get 474 events. If I execute what I think is the same query from the command line:
```
[gstlalcbc@cbc mario_review]$ gracedb search events "gstlal 1245542026.0 .. 1246542026.0" | wc
125 1123 15396
```
I get 125 events
I don't know which one is "right" but I assume the web interface. Fine. So How do I get the information out? Well... I tried downloading the XML file and I get:
```
Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /search/.
Reason: Error reading from remote server
```
Last resort: I tried to make a table big enough that I could just use cut and paste from the web page and I get:
`Error, 400: Too many rows! Please try loading a smaller number.`
So... what do I do exactly?https://git.ligo.org/computing/gracedb/server/-/issues/159Old database tables2019-09-11T14:33:24ZTanner PrestegardOld database tablesThere are a number of tables in the production database which appear to be completely unused by the Django application:
* coinc_definer
* coinc_event
* coinc_event_map
* coinc_inspiral
* events_approval
* experiment
* experiment_map
* e...There are a number of tables in the production database which appear to be completely unused by the Django application:
* coinc_definer
* coinc_event
* coinc_event_map
* coinc_inspiral
* events_approval
* experiment
* experiment_map
* experiment_summary
* ligolwids
* multi_burst
* process
* process_params
* search_summary
* search_summvars
* sngl_inspiral
* south_migrationhistory
* time_slide
Unsure about the `auth_message` table: maybe used for messages (?). Doubtful.
These tables should be briefly checked for the contents and likely just deleted from the MariaDB interface with `DROP TABLE tbl_name;`https://git.ligo.org/computing/gracedb/server/-/issues/158Gracedb should parse FITS files2019-06-07T15:04:34ZJonah KannerGracedb should parse FITS files## Description of feature request
Add some parsing of FITS files included in VOEvents into gracedb. This could be done at the time a voevent is created, or later by a cronjob. There are several outcomes that people have suggested for ...## Description of feature request
Add some parsing of FITS files included in VOEvents into gracedb. This could be done at the time a voevent is created, or later by a cronjob. There are several outcomes that people have suggested for this:
* Create PNG of 2-D skymaps
* Create PNG of 3-D skymaps
* Create PNG of p-astro info
* Ingest distance info into schema of each voevent
## Use cases
* Could display PNG of skymaps with predictable and reliable filename
* Could enhance views of gracedb events to show these PNGs in a structured way
* Could include distance info in tables listing sets of events, so that people could sort or filter based on this property
## Benefits
Currently, the PNG files are not uploaded with predictable names, and so are not useful. This would allow PNG files to be more reliable and useful in views. Adding distance info to schema would allow people to search and sort by distance.
## Drawbacks
* Distance information includes 2 numbers, so represents a significant change to the schema for VOEvents
* If this ingestion is done at the time a new VOEvent is created, could slow down the process
of sending out GCNs, which is a serious drawback. For this reason, may be better to create the VOEvent in the database without this, and then 'back-fill' with a cronjob or similar.
## Suggested solutions
If it were me, I would:
* Add a distance and distance error field to the VOEvent schema. These fields would be 'null' at the time of VoEvent creation, and would not appear in the VoEvent XML files (as now).
* Set up a cronjob that runs every 10 minutes or so, and does the following:
* Gets a list of all VOEvents where the distance column is 'null'
* For each of these, generate PNG images of skymaps, and save to disk in a predictable location
* Read out distance information from FITS header, and ingest into database.https://git.ligo.org/computing/gracedb/server/-/issues/155Create a public friendly view of O3 public superevents2019-07-30T17:56:18ZJonah KannerCreate a public friendly view of O3 public supereventsThe goal is to create a view of public O3 superevents that will be relatively easy for people outside the collaboration (including astronomers and knowledgeable journalists) to understand and use. Drafts pages are available at these loc...The goal is to create a view of public O3 superevents that will be relatively easy for people outside the collaboration (including astronomers and knowledgeable journalists) to understand and use. Drafts pages are available at these locations:
* https://gracedb-dev2.ligo.org/superevents/public/O3/
Current features include:
* Display all O3 public superevents
* Include link to "detail" view for each event
* Display p_astro values
* Display link to GCN circular archive
* Display both a human readable date and precise GPS time
* Include links for Omega scans visible only to LVC members
* Display publicly visible analyst comments
* Clearly mark retracted events (currently in red)
* Include count of total number of O3 detection candidates
* Responsive design, with columns that can toggle on-and-off to fit mobile devices
* Sortable columns
Comments would be most useful if included here by June 3.https://git.ligo.org/computing/gracedb/server/-/issues/154GRB Parameters not parsed correctly2019-05-16T18:34:02ZBrandon PiotrzkowskiGRB Parameters not parsed correctly## Description of problem
An issue in Sentry has popped where trigger_duration for Fermi GRBs is missing:
https://sentry.io/organizations/ligo-caltech/issues/998630013/?environment=production&query=is%3Aunresolved
Also for subthreshold ...## Description of problem
An issue in Sentry has popped where trigger_duration for Fermi GRBs is missing:
https://sentry.io/organizations/ligo-caltech/issues/998630013/?environment=production&query=is%3Aunresolved
Also for subthreshold Fermi GRBS trigger_duration and trigger_id is missing
example: https://gracedb-playground.ligo.org/events/E22136/view/
## Suggested solutions
--> We need to add more fields within `populateGrbEventFromVOEventFile` to handle these different eventshttps://git.ligo.org/computing/gracedb/server/-/issues/153502 Bad Gateway errors causing issues with graceDB uploads2023-04-12T20:17:20ZPatrick Godwin502 Bad Gateway errors causing issues with graceDB uploadsI ran into this problem both using the python gracedb client and the CLI, for example:
```
gracedb upload of ranking_data.xml.gz for ID G332165 failed on attempt 1/5: 502: Bad Gateway
gracedb upload of ranking_data.xml.gz for ID G332165...I ran into this problem both using the python gracedb client and the CLI, for example:
```
gracedb upload of ranking_data.xml.gz for ID G332165 failed on attempt 1/5: 502: Bad Gateway
gracedb upload of ranking_data.xml.gz for ID G332165 failed on attempt 2/5: 502: Bad Gateway
gracedb upload of ranking_data.xml.gz for ID G332165 failed on attempt 3/5: 502: Bad Gateway
gracedb upload of ranking_data.xml.gz for ID G332165 failed on attempt 4/5: 502: Bad Gateway
gracedb upload of ranking_data.xml.gz for ID G332165 failed on attempt 5/5: 502: Bad Gateway
gracedb upload of ranking_data.xml.gz for ID G332165 failed
```
This is causing issues with ranking data to be uploaded, which causes circulars to not be sent out since the p_astro calculation needs the ranking data to be available.https://git.ligo.org/computing/gracedb/server/-/issues/152Mutually exclusive labels2022-08-04T01:44:26ZStuart AndersonMutually exclusive labelsAs discussed in https://git.ligo.org/emfollow/gwcelery/merge_requests/500 consider making some sets of GraceDB labels mutually exclusive so that clients (e.g., gwcelery) can change which one is set in a single transaction, e.g., setting ...As discussed in https://git.ligo.org/emfollow/gwcelery/merge_requests/500 consider making some sets of GraceDB labels mutually exclusive so that clients (e.g., gwcelery) can change which one is set in a single transaction, e.g., setting DQOK would automatically unset DQV.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/151Unable to download event queries through GraceDB in LIGOLW format. Proxy error.2022-08-04T01:43:51ZRyan MageeUnable to download event queries through GraceDB in LIGOLW format. Proxy error.## Description of problem
<!--
Describe in detail what you are trying to do and what the result is.
Exact timestamps, error tracebacks, and screenshots (if applicable) are very helpful.
-->
I've performed an event query on gracedb (and ...## Description of problem
<!--
Describe in detail what you are trying to do and what the result is.
Exact timestamps, error tracebacks, and screenshots (if applicable) are very helpful.
-->
I've performed an event query on gracedb (and gracedb-playground). I obtain fewer than 1000 events. When I attempt to download the resulting LIGOLW file, I get the message:
Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /search/.
Reason: Error reading from remote server
## Expected behavior
<!-- What do you expect to happen instead? -->
I expect the file to download.
## Steps to reproduce
<!-- Step-by-step procedure for reproducing the issue -->
Perform an event query with the following parameters:
pipeline: gstlal created: 2019-04-01 .. 2019-05-09
As of right now, that shows 839 events for me.
## Context/environment
<!--
Describe the environment you are working in:
* If using the ligo-gracedb client package, which version?
* Your operating system
* Your browser (web interface issues only)
* If you are experiencing this problem while working on a LIGO or Virgo computing cluster, which cluster are you using?
-->
Performed on the gracedb web-interface. I am logged in to both gracedb and gracedb-playground and can see the events I am trying to download.
## Suggested solutions
<!-- Any ideas for how to resolve this problem? -->https://git.ligo.org/computing/gracedb/server/-/issues/150Event query by superevent does not work ("expected string or buffer")2019-05-16T17:12:55ZLeo P. SingerEvent query by superevent does not work ("expected string or buffer")See screenshot.
![Screen_Shot_2019-05-09_at_11.23.47](/uploads/2038a583b14381c56cc20b85c149d296/Screen_Shot_2019-05-09_at_11.23.47.png)See screenshot.
![Screen_Shot_2019-05-09_at_11.23.47](/uploads/2038a583b14381c56cc20b85c149d296/Screen_Shot_2019-05-09_at_11.23.47.png)https://git.ligo.org/computing/gracedb/server/-/issues/149Skymap format check in buildVOEvent module2019-05-15T14:12:28ZGiuseppe GrecoSkymap format check in buildVOEvent moduleSuggestion to add a check during the VOtable creation about the skymap format in order to confirm that the skymap is a valid healpix format. The check could be add in the `buildVOEvent` module in the `construct_voevent_file` method
http...Suggestion to add a check during the VOtable creation about the skymap format in order to confirm that the skymap is a valid healpix format. The check could be add in the `buildVOEvent` module in the `construct_voevent_file` method
https://git.ligo.org/lscsoft/gracedb/blob/master/gracedb/superevents/buildVOEvent.pyhttps://git.ligo.org/computing/gracedb/server/-/issues/148Visually distinguish private vs public information2022-08-04T01:43:26ZStuart AndersonVisually distinguish private vs public informationConsider adding an option to visually distinguish private vs public information for privileged users while they are logged in, e.g., different background color or a water mark overlay. Note, if that is too visually distracting there coul...Consider adding an option to visually distinguish private vs public information for privileged users while they are logged in, e.g., different background color or a water mark overlay. Note, if that is too visually distracting there could be a toggle button, e.g., "highlight public" or "highlight private", to enable an inline comparison of public vs private information.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/147Intermittent 502 'Bad Gateway' error2022-03-22T18:16:03ZTanner PrestegardIntermittent 502 'Bad Gateway' errorFrom @patrick\-brady:
> Hi Tanner:
>
> I've started going through the sentry logs for gwcelery. I did find a a number of cases of the 502 bad gateway error. This does not appear to be coming from any single api request. Here are links to...From @patrick\-brady:
> Hi Tanner:
>
> I've started going through the sentry logs for gwcelery. I did find a a number of cases of the 502 bad gateway error. This does not appear to be coming from any single api request. Here are links to 4 different sentry messages:
>
> https://sentry.io/organizations/ligo-caltech/issues/994229585/events/ac120124b2aa41e6aa5280fceafb403b/
>
> https://sentry.io/organizations/ligo-caltech/issues/993796824/events/c337068a03f246f1bbbd8bc4149a6256/
>
> https://sentry.io/organizations/ligo-caltech/issues/998916934/events/567af335c4f248d7bb1cfd78a33017e8/
>
> https://sentry.io/organizations/ligo-caltech/issues/992306270/events/bb4f0fd48cde40719383c24174fd1189/
>
> I spent about half an hour looking around for information related to 502 errors on AWS and found the following: https://blog.transposit.com/the-mysterious-case-of-the-bad-gateway-502-b9f370207d87
> This person tracked their errors to use of dropwizard (https://www.dropwizard.io/1.3.5/docs/). I have no idea if that is part of the stack use on ALB for gracedb, but it's what it did that may give hints on addressing the 502 errors.
> I know this is not your expertise and that Tom has extremely limited time left, but I thought I'd put this to you and see if you can make anything of it.
>
> Cheers,
> Patrickhttps://git.ligo.org/computing/gracedb/server/-/issues/146Update documentation2019-08-30T13:24:40ZTanner PrestegardUpdate documentationThe server code documentation is old and bad in some places. Specifically, we should rework the outdated "using the rest API section" and mostly point to the client code documentation. There are also some search documentation updates ne...The server code documentation is old and bad in some places. Specifically, we should rework the outdated "using the rest API section" and mostly point to the client code documentation. There are also some search documentation updates needed.https://git.ligo.org/computing/gracedb/server/-/issues/145Upgrade to new version of python-glue2022-03-22T18:13:25ZTanner PrestegardUpgrade to new version of python-glueGlue was updated a few months ago and there are expected to be some breaking changes. This is mostly involved in the processing of event file in GraceDB. We have pinned to version 1.60 for now, but need to upgrade sooner or later.
We w...Glue was updated a few months ago and there are expected to be some breaking changes. This is mostly involved in the processing of event file in GraceDB. We have pinned to version 1.60 for now, but need to upgrade sooner or later.
We will likely need two have multiple "translators" for pipelines which upload in this format, and will need to somehow detect which format they are using to select the translator properly, in order to have a smooth transition.Alexander PaceAlexander Pacehttps://git.ligo.org/computing/gracedb/server/-/issues/144Remove LV-EM access2022-03-22T18:19:00ZTanner PrestegardRemove LV-EM accessWe don't have MOUs anymore from O1/O2, so the privileged access based on this should be removed. We should do it in a smart way so that the functionality remains and can be expanded to additional groups, as we may have MOUs with a few se...We don't have MOUs anymore from O1/O2, so the privileged access based on this should be removed. We should do it in a smart way so that the functionality remains and can be expanded to additional groups, as we may have MOUs with a few select groups in the future.https://git.ligo.org/computing/gracedb/server/-/issues/143Upgrade to Python 3 and Django 2.22020-07-15T18:48:09ZTanner PrestegardUpgrade to Python 3 and Django 2.2We will have to get to Python 3 by the end of the year and there's no reason to not get there early. We should also upgrade to the latest LTS release of Django (2.2) while we're at it.We will have to get to Python 3 by the end of the year and there's no reason to not get there early. We should also upgrade to the latest LTS release of Django (2.2) while we're at it.https://git.ligo.org/computing/gracedb/server/-/issues/141Sentry instance for AWS2019-05-16T17:15:46ZTanner PrestegardSentry instance for AWSWe need a better way of handling errors in the AWS gracedb instance. The issues with superevent directory creation for S190412m generated about 4000 error emails to each of the admins and disrupted my email traffic for days.We need a better way of handling errors in the AWS gracedb instance. The issues with superevent directory creation for S190412m generated about 4000 error emails to each of the admins and disrupted my email traffic for days.https://git.ligo.org/computing/gracedb/server/-/issues/140Add VOEvent columns2019-07-11T13:22:52ZTanner PrestegardAdd VOEvent columnsThere are a variety of VOEvent attributes which are required from the user to create a VOEvent and are included in the resulting files, but are **not** saved in the VOEvent database table.
We should
a) Fix this by adding the columns
b)...There are a variety of VOEvent attributes which are required from the user to create a VOEvent and are included in the resulting files, but are **not** saved in the VOEvent database table.
We should
a) Fix this by adding the columns
b) Try to retroactively update VOEvents from their corresponding files