|
# Initial Setup For Replication To CNAF
|
|
|
|
This is a historical record to describe an initial setup used to get urgently
|
|
|
|
needed O3 h(t) frame files to CNAF.
|
|
|
|
|
|
|
|
This topic covers:
|
|
|
|
1. Setup CNAF RSE
|
|
|
|
1. Bulk/offline registration
|
|
|
|
1. Live/online registration
|
|
|
|
1. Dataset replication (via a replication rule)
|
|
|
|
|
|
|
|
## Motivation
|
|
|
|
The CNAF compute site has historically been unable to receive data via LDR.
|
|
|
|
They are also seeing performance issues with CVMFS when reading frame files
|
|
|
|
at a rate/volume necessary for continuous-wave analyses. CNAF have an urgent
|
|
|
|
need for immeidate O3 h(t) file transfers.
|
|
|
|
|
|
|
|
# End-point Configuration
|
|
# End-point Configuration
|
|
At CNAF we have a choice of protocols:
|
|
At CNAF we have a choice of protocols:
|
|
* `gsiftp`: familiar globus ftp transfers. Ultimately fails due to a checksum bug (see below).
|
|
* `gsiftp`: familiar globus ftp transfers. Ultimately fails due to a checksum bug (see below).
|
... | @@ -91,25 +75,3 @@ delegated to FTS must be created with e.g.: |
... | @@ -91,25 +75,3 @@ delegated to FTS must be created with e.g.: |
|
/usr/bin/voms-proxy-init -voms virgo:/virgo/virgo -hours 24
|
|
/usr/bin/voms-proxy-init -voms virgo:/virgo/virgo -hours 24
|
|
```
|
|
```
|
|
|
|
|
|
# Data registration
|
|
|
|
To begin this exercise, I will run everything under my own user account
|
|
|
|
(`jclark`). We should then transition to using the `rucio` user account
|
|
|
|
and relevant robot certificates.
|
|
|
|
|
|
|
|
Two ways to register data:
|
|
|
|
* offline: select a fixed range of times, datatype (e.g., `H-H1_HOFT_C00`),
|
|
|
|
register all frames in those times
|
|
|
|
* offline: select a start time and datatype, register frames as they arrive.
|
|
|
|
|
|
|
|
First, we need to set up a `diskcache` daemon.
|
|
|
|
|
|
|
|
## DiskCache
|
|
|
|
Create a configuration file
|
|
|
|
|
|
|
|
## Offline registration
|
|
|
|
|
|
|
|
|
|
|
|
## Online registration
|
|
|
|
|
|
|
|
# Data replication
|
|
|
|
|
|
|