GraceDB Server issueshttps://git.ligo.org/computing/gracedb/server/-/issues2019-09-18T15:28:42Zhttps://git.ligo.org/computing/gracedb/server/-/issues/171Bad file versioning for event replacement2019-09-18T15:28:42ZTanner PrestegardBad file versioning for event replacementWhen replacing an event, there a few problems with file versioning:
* The file version is *not* passed from the API view (`api.v1.events.EventList.put`) to `events.translator.handle_uploaded_data` (see [here](https://git.ligo.org/lscsof...When replacing an event, there a few problems with file versioning:
* The file version is *not* passed from the API view (`api.v1.events.EventList.put`) to `events.translator.handle_uploaded_data` (see [here](https://git.ligo.org/lscsoft/gracedb/blob/master/gracedb/api/v1/events/views.py#L622)). This isn't a problem **only** if the new event file has a different filename.
* `handle_uploaded_data` assumes a file version of 0 for both the new event file **and** the generated files (`event.log`, `coinc.xml`), which is just plain wrong - version 0 of these files was already generated when the event was initially created.https://git.ligo.org/computing/gracedb/server/-/issues/169firefox fails to download S180814bv bayestar.fits.gz, but OK with bayestar.fi...2019-08-15T19:31:16ZKeita Kawabefirefox fails to download S180814bv bayestar.fits.gz, but OK with bayestar.fits.gz,1I cannot imagine that astronomers are using firefox for followup purposes so this might not be an urgent issue, but I'd appreciate if somebody could investigate.
During RRT for S190814bv, I reported that the size of bayestar.fits.gz VS ...I cannot imagine that astronomers are using firefox for followup purposes so this might not be an urgent issue, but I'd appreciate if somebody could investigate.
During RRT for S190814bv, I reported that the size of bayestar.fits.gz VS bayestar.fits.gz,1 were totally different, and Geoffrey Mo concurred.
After one day the problem still persists for me, but it seems like firefox just fails to download the whole bayestar.fits.gz (it can download bayestar.fits.gz,1).
wget, curl and chrome all worked fine.
The symptom is that using firefox,
https://gracedb.ligo.org/api/superevents/S190814bv/files/bayestar.fits.gz
gives me a smaller file than it should be.
When I restart firefox and download again the file size changes a bit, but it's always ~710000 bytes when the files downloaded by curl, wget or chrome are all identical and are 4563601 bytes.
Combinations of restarting firefox, clearing browser cache, making a new user profile, going into private browser mode, deleting ./mozilla folder and using different computers (one osx, one linux) didn't help.
In the example below, 'Bayestar (1).fits.gz' was downloaded by chrome, everything else was downloaded using firefox. I can read bayestar.fits.gz,1 and 'bayestar (1).fits.gz' using astropy.
```
(base) ~/Downloads$ ls -l bayestar*
-rw-r--r--@ 1 keita.kawabe staff 4563601 Aug 15 11:03 bayestar (1).fits.gz
-rw-rw-rw-@ 1 keita.kawabe staff 711742 Aug 15 11:02 bayestar.fits(1).gz
-rw-rw-rw-@ 1 keita.kawabe staff 711742 Aug 15 11:04 bayestar.fits(2).gz
-rw-rw-rw-@ 1 keita.kawabe staff 711870 Aug 15 11:05 bayestar.fits(3).gz
-rw-rw-rw-@ 1 keita.kawabe staff 711772 Aug 15 11:07 bayestar.fits(4).gz
-rw-rw-rw-@ 1 keita.kawabe staff 711698 Aug 15 11:29 bayestar.fits(5).gz
-rw-rw-rw-@ 1 keita.kawabe staff 711698 Aug 15 11:30 bayestar.fits(6).gz
-rw-rw-rw-@ 1 keita.kawabe staff 711771 Aug 15 11:31 bayestar.fits(7).gz
-rw-rw-rw-@ 1 keita.kawabe staff 711731 Aug 15 08:50 bayestar.fits.gz
-rw-rw-rw-@ 1 keita.kawabe staff 4563601 Aug 15 08:51 bayestar.fits.gz,1
(base) ~/Downloads$ diff bayestar.fits.gz,1 bayestar\ \(1\).fits.gz
(base) ~/Downloads$
```https://git.ligo.org/computing/gracedb/server/-/issues/166Some notifications missing for S190718y2019-07-23T00:46:37ZNicolas ArnaudSome notifications missing for S190718yHi @tanner.prestegard
I made a quick poll this morning during the weekly Virgo DetChar meeting about the receiving of S190718y notifications. There were about 15 people attending that meeting. The results are
* Almost nobody received a...Hi @tanner.prestegard
I made a quick poll this morning during the weekly Virgo DetChar meeting about the receiving of S190718y notifications. There were about 15 people attending that meeting. The results are
* Almost nobody received an e-mail notification.
* About half of the people did not receive the call/text notification they had subscribed to.
Thanks in advance for investigating that issue -- I don't think we had this problem earlier in the run.
Nicolashttps://git.ligo.org/computing/gracedb/server/-/issues/165Add source frame masses and redshift2019-07-15T14:47:51ZPatrick BradyAdd source frame masses and redshift## Description of feature request
<!--
Describe your feature request!
Is it a web interface change? Some underlying feature? An API resource?
The more detail you can provide, the better.
-->
Albert Lazzarini asked: Is it possible to add ...## Description of feature request
<!--
Describe your feature request!
Is it a web interface change? Some underlying feature? An API resource?
The more detail you can provide, the better.
-->
Albert Lazzarini asked: Is it possible to add information to the internal graceDB pages? The additional information that would be helpful is to clearly indicate that the masses listed are in the DETECTOR frame (see screen shot. Also, using the Standard Model, the redshift Z associated with the luminosity distance could also be provided.
## Use cases
<!-- List some specific cases where this feature will be useful -->
Would be useful in both superevents and regular events
## Benefits
<!-- Describe the benefits of adding this feature -->
Would make it easier to know source frame masses now that we are seeing mergers to cosmological distances.
## Drawbacks
<!--
Are there any drawbacks to adding this feature?
Can you think of any ways in which this will negatively affect the service for any set of users?
-->
Needs reasonable parameter estimation.
## Suggested solutions
<!-- Do you have any ideas for how to implement this feature? -->https://git.ligo.org/computing/gracedb/server/-/issues/163What's on? - Most viewed / trending event page2019-07-10T01:54:43ZNicola De Lillonicola.delillo@ligo.orgWhat's on? - Most viewed / trending event page## Description of feature request
Add a page (with a tab to reach that in the tab-bar) that shows a list of every gravitational wave events detected, but sorted by visualisation (i.e. every time a user logs to the event page). The page c...## Description of feature request
Add a page (with a tab to reach that in the tab-bar) that shows a list of every gravitational wave events detected, but sorted by visualisation (i.e. every time a user logs to the event page). The page could look like a list with simple and clean entries that might be of interested of less regular visitors:
SUPEREVENT_ID PREFERRED_EVENT DATE_TIME FAR MASS1 MASS2 VISUALIZED
Of course one must implement a visualisation counter to produce the VISUALIZED Column
It would be also nice if this page can have a switch button at the top of the the list, between "MOST VISUALIZED" and "TRENDING" event. While "MOST VISUALIZED" would refer to the above list, "TRENDING" list would show the most visualized event at the moment (might be in the last 1 week or 1 mont, two months or 6 months - even better if one can select this for option to shape the list).
The tab name to redirect to this page could be any. At the moment I can suggest "What's on" , "Hall of fame", "Most viewed". But further research and ideas could come later.
## Use cases
Not everybody want to follow or receive all of the alerts. The DB could appear overwhelming for people outside the collaboration of the GW field. Some scientists (both outside and inside the gravitational wave field) might only be interested casually in what is going on with detections. This way they can now follow which one were the most interested events ever or which ones get all the hype in some specific period.
## Benefits
It would keep hyped people inside and outside the gravitational wave field and make the GraceDB more user-friendly and appealing for casual visitors.
## Drawbacks
Not anyone at the moment I can think of
## Acknowledgement
Thank you to Jennifer Wright, trending event idea was from her.https://git.ligo.org/computing/gracedb/server/-/issues/162Show component masses in Superevent preview2019-07-09T23:22:30ZNicola De Lillonicola.delillo@ligo.orgShow component masses in Superevent preview## Description of feature request
It is a web interface change. It would require to add both in the LATEST both in the SEARCH page, the column "MASS1" and "MASS2" (or simply M1 M2) for CBC events/superevent raws showed in both lists. I t...## Description of feature request
It is a web interface change. It would require to add both in the LATEST both in the SEARCH page, the column "MASS1" and "MASS2" (or simply M1 M2) for CBC events/superevent raws showed in both lists. I think the values should come from the preferred event analyses and then update every time the analysts upload a new parameter estimation result that they consider more appropriate.
## Use cases
Evertime looking at latest or DB
## Benefits
It is useful to spot quickly interesting and appealing events for any scientist in the field that are scrolling the DB.
## Drawbacks
NOt any in mind at the moment
## Suggested solutions
It could simply be one more column in the raws shoed both in LATEST and in the SEARCH lists.https://git.ligo.org/computing/gracedb/server/-/issues/161User's favourites or Followed Events list2019-07-09T22:07:10ZNicola De Lillonicola.delillo@ligo.orgUser's favourites or Followed Events list## Description of feature request
Insert a tabin between the already present tabs LATEST and ALERTS. This tab could be called "FOLLOWED EVENTS" (Not PREFERRED, it would confuse people with "preferred events" in case of a super-events).Th...## Description of feature request
Insert a tabin between the already present tabs LATEST and ALERTS. This tab could be called "FOLLOWED EVENTS" (Not PREFERRED, it would confuse people with "preferred events" in case of a super-events).The tab would link to the page "FOLLOWED EVENTS" which would appear really as it looks now the LATEST page. The difference is that the entry showed in "FOLLOWED EVENTS" page will show only the events or supervents flagged as "FOLLOWED" by the user.
Events should be flag-able either from the SEARCH page or from the LATEST page.
## Use cases
1) internal use for LIGO: It is useful for experts ROTA or advocates or PE-Rotaers that can easily keep track of the events they got assigned.
2) In general any scientist interested in a particular event can follow it. i.e.: if I am interested in tracking the analysis for only Binary Neutron stars system, I would flag as FOLLOW only that events.
## Benefits
I think it helps really tracking the events a scientist want to follow.
## Drawbacks
Not really any drawbacks at the moment as far as I can see. Apart that one has to redesign the toolbar adding the "FOLLOWED EVENTS" tab.
## Suggested solutions
Here attached a figure that shows how to display (the example is for LATEST page, BUt consider please doing that also for the SEARCH page) the boxes to flag in the FOLLOW column. All the flagged events (either a flag or dot or a filling color) will go in the FOLLOWED EVENT page. Of course rememer to add a 'X' symbol in the FOLLOWED EVENT page if one wants to remove that event.
![LATEST_examp](/uploads/4b8a8e328fff42644b539cc6ee63d061/LATEST_examp.png)https://git.ligo.org/computing/gracedb/server/-/issues/158Gracedb should parse FITS files2019-06-07T15:04:34ZJonah KannerGracedb should parse FITS files## Description of feature request
Add some parsing of FITS files included in VOEvents into gracedb. This could be done at the time a voevent is created, or later by a cronjob. There are several outcomes that people have suggested for ...## Description of feature request
Add some parsing of FITS files included in VOEvents into gracedb. This could be done at the time a voevent is created, or later by a cronjob. There are several outcomes that people have suggested for this:
* Create PNG of 2-D skymaps
* Create PNG of 3-D skymaps
* Create PNG of p-astro info
* Ingest distance info into schema of each voevent
## Use cases
* Could display PNG of skymaps with predictable and reliable filename
* Could enhance views of gracedb events to show these PNGs in a structured way
* Could include distance info in tables listing sets of events, so that people could sort or filter based on this property
## Benefits
Currently, the PNG files are not uploaded with predictable names, and so are not useful. This would allow PNG files to be more reliable and useful in views. Adding distance info to schema would allow people to search and sort by distance.
## Drawbacks
* Distance information includes 2 numbers, so represents a significant change to the schema for VOEvents
* If this ingestion is done at the time a new VOEvent is created, could slow down the process
of sending out GCNs, which is a serious drawback. For this reason, may be better to create the VOEvent in the database without this, and then 'back-fill' with a cronjob or similar.
## Suggested solutions
If it were me, I would:
* Add a distance and distance error field to the VOEvent schema. These fields would be 'null' at the time of VoEvent creation, and would not appear in the VoEvent XML files (as now).
* Set up a cronjob that runs every 10 minutes or so, and does the following:
* Gets a list of all VOEvents where the distance column is 'null'
* For each of these, generate PNG images of skymaps, and save to disk in a predictable location
* Read out distance information from FITS header, and ingest into database.https://git.ligo.org/computing/gracedb/server/-/issues/152Mutually exclusive labels2022-08-04T01:44:26ZStuart AndersonMutually exclusive labelsAs discussed in https://git.ligo.org/emfollow/gwcelery/merge_requests/500 consider making some sets of GraceDB labels mutually exclusive so that clients (e.g., gwcelery) can change which one is set in a single transaction, e.g., setting ...As discussed in https://git.ligo.org/emfollow/gwcelery/merge_requests/500 consider making some sets of GraceDB labels mutually exclusive so that clients (e.g., gwcelery) can change which one is set in a single transaction, e.g., setting DQOK would automatically unset DQV.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/148Visually distinguish private vs public information2022-08-04T01:43:26ZStuart AndersonVisually distinguish private vs public informationConsider adding an option to visually distinguish private vs public information for privileged users while they are logged in, e.g., different background color or a water mark overlay. Note, if that is too visually distracting there coul...Consider adding an option to visually distinguish private vs public information for privileged users while they are logged in, e.g., different background color or a water mark overlay. Note, if that is too visually distracting there could be a toggle button, e.g., "highlight public" or "highlight private", to enable an inline comparison of public vs private information.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/136Failed to receive phone call for S190408an2022-08-04T01:40:28ZBrian O'ReillyFailed to receive phone call for S190408anI received a text message for this event but my phone did not ring. I have my alert set for call and text based on the ADVREQ label.
My number is in the US, 225 area code.I received a text message for this event but my phone did not ring. I have my alert set for call and text based on the ADVREQ label.
My number is in the US, 225 area code.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/132tooltip time conversion2022-08-04T01:31:07ZStuart Andersontooltip time conversionConsider having the tooltip window convert GPS to UTC and vice-a-versa, i.e., if a user hovers their cursor over a table element that contains a GPS time have the pop-up window show that same time in UTC (and vice-a-versa).Consider having the tooltip window convert GPS to UTC and vice-a-versa, i.e., if a user hovers their cursor over a table element that contains a GPS time have the pop-up window show that same time in UTC (and vice-a-versa).Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/128Require FAR to be non-negative2022-08-04T01:28:20ZTanner PrestegardRequire FAR to be non-negativePipelines submitting with negative FAR is probably a sign that something is wrong. It is also triggering alerts with FAR thresholds since a negative FAR is obviously less than any reasonable FAR threshold.Pipelines submitting with negative FAR is probably a sign that something is wrong. It is also triggering alerts with FAR thresholds since a negative FAR is obviously less than any reasonable FAR threshold.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/122Open alerts should be public in GraceDb2022-08-04T01:25:41ZLeo P. SingerOpen alerts should be public in GraceDbIt's useful to be able to download old VOEvents. We should make this possible for anonymous GraceDb users. Please add the `public` tag to any VOEvent that is created with `internal=False`.It's useful to be able to download old VOEvents. We should make this possible for anonymous GraceDb users. Please add the `public` tag to any VOEvent that is created with `internal=False`.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/116Add log filtering by tag2022-08-04T01:24:41ZTanner PrestegardAdd log filtering by tagAdd a filter to the logs API where only logs with certain tags applied will be retrieved. Would need a client update as well.Add a filter to the logs API where only logs with certain tags applied will be retrieved. Would need a client update as well.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/112Filenames can't contain commas2022-08-04T01:22:48ZTanner PrestegardFilenames can't contain commasBecause the versioning method adds `,#` to the end and we rely on splitting on a comma. So we need to check this on all models with filenames and everywhere files can be uploaded (event creation/replacement specifically). It might be a...Because the versioning method adds `,#` to the end and we rely on splitting on a comma. So we need to check this on all models with filenames and everywhere files can be uploaded (event creation/replacement specifically). It might be automatically handled in the superevents API, but I'm not sure.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/111Handling "Page not found (404)"2022-08-03T19:08:55ZStuart AndersonHandling "Page not found (404)"Currently an attempt to access a deep link, e.g., https://gracedb-playground.ligo.org/superevents/S181203f/view/, without an active login session returns 404 "Page not found" and the message "No Superevent matches the given query."
Plea...Currently an attempt to access a deep link, e.g., https://gracedb-playground.ligo.org/superevents/S181203f/view/, without an active login session returns 404 "Page not found" and the message "No Superevent matches the given query."
Please consider enhancing the 404 page to conditionally indicate (if there is no active Shibboleth session) that authorized users should first try logging in, and provide a login hyperlink to do so.
For bonus points, see if there is an easy way for users that select the login link (and successfully authenticate) to automatically have their browsers reload the originally requested page.
Note, for users with a valid Shibboleth session and still land at a URL with no valid page, please also consider changing messages like No Superevent matches the given query." to "No Superevent matches the given query or you are not authorizes to view it."Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/97Rolling deletion of Test events and superevents2022-08-03T19:04:41ZTanner PrestegardRolling deletion of Test events and supereventsTest events currently make up 46% of the events in the database and take up 25% of the storage. We should not be preserving Test events indefinitely (on principle) and it could help speed things up a bit. I would like to establish a ni...Test events currently make up 46% of the events in the database and take up 25% of the storage. We should not be preserving Test events indefinitely (on principle) and it could help speed things up a bit. I would like to establish a nightly cron job that deletes events and superevents older than 3 months or so.Backloghttps://git.ligo.org/computing/gracedb/server/-/issues/88Fully implement public "switch"2022-08-03T19:03:28ZTanner PrestegardFully implement public "switch"Fully implement the settings variable which will allow or not allow unauthenticated access. Should also have unit tests which test this capability.Fully implement the settings variable which will allow or not allow unauthenticated access. Should also have unit tests which test this capability.Backlog2018-11-30https://git.ligo.org/computing/gracedb/server/-/issues/75Moving old detections to superevents and making them public2022-08-03T19:01:51ZTanner PrestegardMoving old detections to superevents and making them publicWe want to make old events like GW170817 into superevents and available publicly.
The steps will be:
1. Aggregate the events into superevents
2. Copy over relevant content (logs, files, etc.)
* Metadata: do we need log submitters, t...We want to make old events like GW170817 into superevents and available publicly.
The steps will be:
1. Aggregate the events into superevents
2. Copy over relevant content (logs, files, etc.)
* Metadata: do we need log submitters, timestamps, etc. to match that of the original event? Patrick: No
3. Add new information like samples files which are included for these events on GWOSC.
4. Internal vetting and approval.
5. Public release
Steps 1 and 2 will likely be handled by superevent_manager/gwcelery.
Questions are:
* What kind of timescale are we working on?
* Catalog paper and engineering run are expected near the end of November. Can we get this done on a similar timescale?
* Maybe - need to talk to Tom about AWS/public implementation planO4 CBC Improvements