Changes
Page history
removed duplication
authored
May 03, 2019
by
James Clark
Show whitespace changes
Inline
Side-by-side
O3/CNAF/3-registration.md
View page @
ee975bb8
# Initial Setup For Replication To CNAF
This is a historical record to describe an initial setup used to get urgently
needed O3 h(t) frame files to CNAF.
This topic covers:
1.
Setup CNAF RSE
1.
Bulk/offline registration
1.
Live/online registration
1.
Dataset replication (via a replication rule)
## Motivation
The CNAF compute site has historically been unable to receive data via LDR.
They are also seeing performance issues with CVMFS when reading frame files
at a rate/volume necessary for continuous-wave analyses. CNAF have an urgent
need for immeidate O3 h(t) file transfers.
# End-point Configuration
At CNAF we have a choice of protocols:
*
`gsiftp`
: familiar globus ftp transfers. Ultimately fails due to a checksum bug (see below).
*
`srm`
: succesful.
Note that rucio is completely agnostic about the protocol used at each site:
there is no problem transferring from a gsiftp end-point to an srm end-point,
for example.
## Local CNAF configuration
## Create, configure CNAF RSE
Create RSE:
```
rucio-admin rse add CNAF
```
Add
`gsiftp`
protocol:
```
rucio-admin rse add-protocol\
--prefix /storage/gpfs_virgo4/Runs/home/rucio/evaluation \
--hostname gridftp-plain-virgo.cr.cnaf.infn.it\
--domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy": 1}}' \
--scheme gsiftp \
--port 2811 \
--impl rucio.rse.protocols.gfal.Default \
CNAF
```
Add
`srm`
protocol:
```
rucio-admin rse add-protocol\
--prefix /virgoplain/rucio \
--hostname storm-fe-archive.cr.cnaf.infn.it \
--domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy": 1}}' \
--scheme srm \
--port 8444 \
--web-service-path '/srm/managerv2?SFN=' \
--impl 'rucio.rse.protocols.gfal.Default' \
CNAF
```
Set disk quota (artificially high for now, just need this to be populated):
```
rucio-admin -a root account set-limits root CNAF 1000000000000
```
Set FTS server to use with this end-point:
```
rucio-admin rse set-attribute --rse CNAF --key fts --value https://fts3-pilot.cern.ch:8446
rucio-admin rse set-attribute --rse CNAF --key fts_testing --value https://fts3-pilot.cern.ch:8446
```
Add some metadata which has proved useful for other tests:
```
rucio-admin rse set-attribute --rse CNAF --key site --value Bologna
rucio-admin rse set-attribute --rse CNAF --key ALL --value 1
rucio-admin rse set-attribute --rse CNAF --key LIGO_LAB --value 0
rucio-admin rse set-attribute --rse CNAF --key HDFS --value 0
rucio-admin rse set-attribute --rse CNAF --key ARCHIVE --value 0
```
## Register rucio server in Virgo VOMS
CNAF is an EGI site, meaning the rucio host certificate must be registered with
the Virgo VOMS:
1.
Import rucio server host certificate to browser
1.
Join Virgo @
[
http://www.italiangrid.it/users/join_vo
](
http://www.italiangrid.it/users/join_vo
)
This also means the x509 proxy certificate generated on the server and
delegated to FTS must be created with e.g.:
```
/usr/bin/voms-proxy-init -voms virgo:/virgo/virgo -hours 24
```
# Data registration
To begin this exercise, I will run everything under my own user account
...
...
@@ -111,5 +19,3 @@ Create a configuration file
## Online registration
# Data replication