Include psd as part of coinc.xml upload
Description of feature request
Include a psd as part of the initial event upload (in the
The goal would be to reduce the overall time to issue an alert. A few notes:
- @kipp.cannon specifically did not include the psd with the initial upload in order to reduce the overall size and thus reduce the latency of the initial trigger.
- @leo-singer noted that the psd is needed to make a skymap, which is needed to to send a GCN.
The advantage would be to reduce the number of transactions with GraceDB, potentially improving reliability and latency. Of course I would want to test this. For reference in this ticket, I am looking at this event. The respective sizes of the
psd.xml files are:
-rw-r--r--@ 1 alexander.pace staff 152K Sep 30 09:26 psd.xml.gz -rw-r--r--@ 1 alexander.pace staff 53K Sep 30 09:25 coinc.xml
What could we be missing here? I'm brain-dumping questions here:
- Where does this psd information get stored?
- Should there be a new database entry to hold psds?
- Are there size limitations to entries in MySQL/ Amazon Relational Database? Are there different rules for text vs binary?
- What would database performance and reliability be when GstLAL is hitting it with 100kb database entries across 100 different events at the same time.
- Should there be another entry in the
coinc.xmlfile along the lines of
has_psd? This would:
- Give GraceDB's input parser a head's up to read and parse a psd and add it to the database. Absence of this flag would tell GraceDB not to read/ingest the psd, and retain backwards compatibility for older events.
- Have this flag be part of the event info that gets sent out via LVAlert for follow-up processes to check for.
- Should there be a size cap for these? I'm worried about the database ballooning out of control in the long run. Let's have this discussion.
Also pinging @chad-hanna