Include psd as part of coinc.xml upload
Description of feature request
Include a psd as part of the initial event upload (in the coinc.xml
upload).
Use cases
The goal would be to reduce the overall time to issue an alert. A few notes:
- @kipp.cannon specifically did not include the psd with the initial upload in order to reduce the overall size and thus reduce the latency of the initial trigger.
- @leo-singer noted that the psd is needed to make a skymap, which is needed to to send a GCN.
Benefits
The advantage would be to reduce the number of transactions with GraceDB, potentially improving reliability and latency. Of course I would want to test this. For reference in this ticket, I am looking at this event. The respective sizes of the coinc.xml
and psd.xml
files are:
-rw-r--r--@ 1 alexander.pace staff 152K Sep 30 09:26 psd.xml.gz
-rw-r--r--@ 1 alexander.pace staff 53K Sep 30 09:25 coinc.xml
Drawbacks
What could we be missing here? I'm brain-dumping questions here:
- Where does this psd information get stored?
- Should there be a new database entry to hold psds?
- Are there size limitations to entries in MySQL/ Amazon Relational Database? Are there different rules for text vs binary?
- What would database performance and reliability be when GstLAL is hitting it with 100kb database entries across 100 different events at the same time.
Suggested solutions
Another brain-dump:
- Should there be another entry in the
coinc.xml
file along the lines ofhas_psd
? This would: - Give GraceDB's input parser a head's up to read and parse a psd and add it to the database. Absence of this flag would tell GraceDB not to read/ingest the psd, and retain backwards compatibility for older events.
- Have this flag be part of the event info that gets sent out via LVAlert for follow-up processes to check for.
- Should there be a size cap for these? I'm worried about the database ballooning out of control in the long run. Let's have this discussion.
Also pinging @chad-hanna