Implement server-side copying and linking.
Description of feature request
Currently the workflow for adding files (such as skymaps) from
G events to superevents involves downloading the file from GraceDB, and then re-uploading it to the superevent. This workflow evolved out of the inability make items attached to G-events public. Short of drastically changing the server permissions infrastructure, it would be a lot easier to make the copy on the server-side. This might involve symlinking to the existing file, since they're the same anyway? I'll dig into that.
These files are copied by the orchestrator from the preferred event into the superevent before the preliminary GCN is sent out.
This reduces the network traffic back-and-forth between GWCelery and GraceDB. It also reduces the overall number of API calls to GraceDB.
I have to be vigilant about respecting GraceDB's versioning system. I also wonder how, if I implement symlinking whether or not the access controls will be respected. GWCelery needs to decide on a filename scheme (like including the gid in filenames in the case where identical files are copied from different events).
After discussing with the group, I think the implementation should look something like:
gracedb.migrateFiles(origin_id, destination_id, file_list) * origin_id (string) - the event or superevent ID of the file origin * desination_id (string)- the event or superevent ID of the file destination * origin_filename (tuple or list of tuples) - tuple of strings where the format is (orgin_file_name.ext,version , destination_file_name.ext)
Other things to keep in mind:
there should be logging on the side of the destination event that says the filename (with version), and the origin event.
there should be logging on the origin event like, "file XXXX copied to SXXXXX"
This should use GraceDB's existing file ingestion mechanism so the current versioning
This is (obviously) going to require a corresponding change to the API. Also make sure that LVAlerts are sent out for file copies. There should be an LVAlert sent out for the sending/receiving events to the corresponding (event/superevent) nodes.