Allow file uploads at registration time
The replication/archiving topology of:
- Register data at observatory
- Rule 1: replicate data from observatory to caltech archive
- Rule 2: Replicate archived data to other sites
Is somewhat slow and the "archived" data only exists at caltech for as long as the rule that placed it there exists.
Instead, we should modify DatasetInjector()
in rucio_data.py
to optionally use the rucio upload client when the files are attached to a dataset. That allows direct uploads (without FTS) using the protocol on the target RSE and creates an independent dataset at the RSE which will survive expiration of rules associated with it.
This should be a few hour's work.