O3/CNAF/1-overview.md

+# Initial Setup For Replication To CNAF
+This is a historical record to describe an initial setup used to get urgently
+needed O3 h(t) frame files to CNAF.
+
+This topic covers:
+ 1. Setup CNAF RSE
+ 1. Bulk/offline registration
+ 1. Live/online registration
+ 1. Dataset replication (via a replication rule)
+
+## Motivation
+The CNAF compute site has historically been unable to receive data via LDR.
+They are also seeing performance issues with CVMFS when reading frame files
+at a rate/volume necessary for continuous-wave analyses.  CNAF have an urgent
+need for immeidate O3 h(t) file transfers.
+
+# End-point Configuration
+At CNAF we have a choice of protocols:
+ * `gsiftp`: familiar globus ftp transfers.  Ultimately fails due to a checksum bug (see below).
+ * `srm`: succesful.
+
+Note that rucio is completely agnostic about the protocol used at each site:
+there is no problem transferring from a gsiftp end-point to an srm end-point,
+for example.
+
+## Local CNAF configuration
+
+## Create, configure CNAF RSE
+
+Create RSE:
+```
+rucio-admin rse add CNAF
+```
+
+Add `gsiftp` protocol:
+```
+rucio-admin rse add-protocol\
+    --prefix /storage/gpfs_virgo4/Runs/home/rucio/evaluation \
+    --hostname gridftp-plain-virgo.cr.cnaf.infn.it\
+    --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy": 1}}' \
+    --scheme gsiftp \
+    --port 2811 \
+    --impl rucio.rse.protocols.gfal.Default \
+    CNAF
+```
+
+Add `srm` protocol:
+```
+rucio-admin rse add-protocol\
+    --prefix /virgoplain/rucio  \
+    --hostname storm-fe-archive.cr.cnaf.infn.it \
+    --domain-json '{"wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy": 1}}' \
+    --scheme srm \
+    --port 8444 \
+    --web-service-path '/srm/managerv2?SFN=' \
+    --impl 'rucio.rse.protocols.gfal.Default' \
+    CNAF
+```
+
+Set disk quota (artificially high for now, just need this to be populated):
+```
+rucio-admin -a root account set-limits root CNAF 1000000000000
+```
+
+Set FTS server to use with this end-point:
+```
+rucio-admin rse set-attribute --rse CNAF --key fts --value https://fts3-pilot.cern.ch:8446
+rucio-admin rse set-attribute --rse CNAF --key fts_testing --value https://fts3-pilot.cern.ch:8446
+```
+
+Add some metadata which has proved useful for other tests:
+```
+rucio-admin rse set-attribute --rse CNAF --key site --value Bologna
+rucio-admin rse set-attribute --rse CNAF --key ALL --value 1
+rucio-admin rse set-attribute --rse CNAF --key LIGO_LAB --value 0
+rucio-admin rse set-attribute --rse CNAF --key HDFS --value 0
+rucio-admin rse set-attribute --rse CNAF --key ARCHIVE --value 0
+```
+
+
+
+## Register rucio server in Virgo VOMS
+CNAF is an EGI site, meaning the rucio host certificate must be registered with
+the Virgo VOMS:
+ 1. Import rucio server host certificate to browser
+ 1. Join Virgo @ [http://www.italiangrid.it/users/join_vo](http://www.italiangrid.it/users/join_vo)
+
+This also means the x509 proxy certificate generated on the server and
+delegated to FTS must be created with e.g.:
+```
+/usr/bin/voms-proxy-init -voms virgo:/virgo/virgo -hours 24
+```
+
+# Data registration
+To begin this exercise, I will run everything under my own user account
+(`jclark`).  We should then transition to using the `rucio` user account
+and relevant robot certificates.
+
+Two ways to register data:
+ * offline: select a fixed range of times, datatype (e.g., `H-H1_HOFT_C00`),
+ 	 register all frames in those times
+ * offline: select a start time and datatype, register frames as they arrive.
+
+First, we need to set up a `diskcache` daemon.
+
+## DiskCache
+Create a configuration file
+
+## Offline registration
+
+
+## Online registration
+
+# Data replication
+