Commit 3d9a9362 authored by James Clark's avatar James Clark
Browse files

Added help descriptions for daemons as reference

parent 5473d025
Pipeline #96429 passed with stages
in 1 minute and 40 seconds
Rucio Daemons
Full list of daemons (`link <>`):
The Abacus-Account daemon is responsible for updating account usages. It
checks if there are new entries in the UpdatedAccountCounter table and updates
the account counters in the AccountCounter table by adding or substrating the
amount and size of files and recalculating the quotas.
Abacus collection replica is a daemon to update the state and counters of
collection replicas.
The Abacus-RSE daemon is responsible for updating RSE usages. It checks if
there are new entries in the UpdatedRSECounter table and updates the RSE
counter in the RSECounter table by adding or substrating the amount of files
and the size.
The Atropos Daemon is responsible for the deletion of rules with expired eol_at
(end the life at), according to the Lifetime Model. Once the rule is deleted,
the replicas covered by the rule will not be protected anymore and eventually,
will be deleted by another daemon.
The auditor daemon is the one responsable for the detection of inconsistencies
on storage, i.e.: dark data discovery.
Automatix is a daemon used to inject random generated files to an RSE. It is
used to continuosly check that an RSE is reachable and operating as spected.
The BB8 daemon is responsible for rebalancing data between RSEs.
The C3PO daemon is responsible for dynamic data placement.
Rucio Cache Consumer is a daemon to add/delete cache replicas to Rucio catalog.
Conveyor is a group of daemons to manage file transfers. The conveyor-finisher
is the resposible to update Rucio internal state after the transfer has
Conveyor is a daemon to manage file transfers. The conveyor-poller is in charge
of poll the transfer tool to check if the submitted transfers are already done.
Conveyor is a daemon to manage file transfers. The conveyor-receiver is similar
to conveyor-poller, but instead of poll the transfer tool, conveyor-receiver
gets knowledge about the state of submitted transfers by listening to the
transfer tool messages.
The conveyor-stager is responsible for issuing staging requests to the tape system.
The Conveyor-Submitter daemon is responsible for managing non-tape file
transfers. It prepares transfer jobs and submits them to the transfertool.
The Conveyor-Throttler daemon is responsible for managing the internal queue of
transfer requests. Depending of transfer limits and current and waiting
transfers, it decides whether a transfer should be put in the queue or not.
The Dark-Reaper daemon is responsible for the deletion of quarantined replicas.
This daemon is responsible to make file list dumps. The rucio-auditor daemon
use these dumps to discover dark data and check Rucio database consistency.
The Hermes daemon is responsible for delivering messages via STOMP to a
messagebroker and via SMTP as email.
The Judge-Cleaner daemon is responsible for cleaning expired replication rules.
It deletes rules by checking if the ‘expired_at’ date property is older than
the current timestamp. If the rule is expired, it will first remove one lock
for the replica and parent datasets if the DID belongs to any. Then it will set
a tombstone to the replica to mark it as deletable if there are no rules
protecting the replica. After these steps, the rule gets deleted.
The Judge-Evaluator daemon is responsible for execution and reevaluation of
replication rules. First it checks if there are DIDs that have changed content
e.g. attached or deattched DIDs.In case of a new attachment, the replication
rule for the dataset has to be applied to the attached DID, too. If the
attached DID has already a replica on a RSE that satisfies the RSE expression
of the rule, then the lock counter of that replia gets increased. If it does
not have any replica satisfying the rule, then a new replica has to be created.
In case of a new dettachment, the replica has to be removed or the lock counter
of the replica has to be decreased, depending on which RSE the replica exist.
If the DID is a dataset, its properties like size and length also get updated
and also an entry is saved to mark a change for possible collection replicas
which have to be updated by another daemon.
Judge-Injector is a daemon to asynchronously inject replication rules.
The Judge-Repairer daemon is responsible for the repair of stuck replication rules.
Kronos is a daemon that consume tracer massages and update the replica accessed
time accordingly.
The light-reaper is responsible of deletion of temporary files
The role of the daemon is get as input the list of PFNs declared bad and to
classify them into 2 categories: the temporary available ones, and the ones
that have a real problem and that will need to be recovered by the necromancer.
This special type of minos daemon lists the expired TEMPORARY_UNAVAILABLE
replicas and puts them back into AVAILABLE state.
The Necromancer daemon is responsible for managing bad replicas. If a replica
that got declared bad has other replicas, it will try to recover it by
requesting a new transfer. If there are no replicas anymore, then the file gets
marked as lost.
The Reaper daemon is responsible for replica deletion. It deletes them by
checking if there are replicas that are not locked and have a tombstone to
indicate that they can be deleted.
Replica-Recoverer is a daemon that declares suspicious replicas that are
available on other RSE as bad. Consequently, automatic replica recovery is
triggered via necromancer daemon, which creates a rule for such bad replicas.
The Transmogrifier daemon is responsible for the creation of replication rules
for DIDs matching a subscription.
The Undertaker deamon is responsible for managing expired DIDs. It deletes
DIDs, but not replicas by checking if there are DIDs where the ‘expired_at’
date property is older than the current timestamp.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment