gwrucio issueshttps://git.ligo.org/james-clark/gwrucio/-/issues2018-12-15T07:32:33Zhttps://git.ligo.org/james-clark/gwrucio/-/issues/6Allow file uploads at registration time2018-12-15T07:32:33ZJames Alexander Clark PhDAllow file uploads at registration timeThe replication/archiving topology of:
1. Register data at observatory
1. Rule 1: replicate data from observatory to caltech archive
1. Rule 2: Replicate archived data to other sites
Is somewhat slow and the "archived" data only exis...The replication/archiving topology of:
1. Register data at observatory
1. Rule 1: replicate data from observatory to caltech archive
1. Rule 2: Replicate archived data to other sites
Is somewhat slow and the "archived" data only exists at caltech for as long as the rule that placed it there exists.
Instead, we should modify `DatasetInjector()` in `rucio_data.py` to optionally use the rucio upload client when the files are attached to a dataset. That allows direct uploads (without FTS) using the protocol on the target RSE and creates an independent dataset at the RSE which will survive expiration of rules associated with it.
This should be a few hour's work.Optimal Transfer TopologyJames Alexander Clark PhDJames Alexander Clark PhD2018-12-21