|
|
# GUARDCTRL systemd daemon supervision
|
|
|
# GUARDCTRL systemd process supervision
|
|
|
|
|
|
Guardian comes with `guardctrl`, a tool for controlling and managing guardian node processes. It is a wrapper around [systemd](https://www.freedesktop.org/wiki/Software/systemd/), which is the built-in init and service supervision system standard on all major linux distributions. systemd handles stopping, starting, and logging of the guardian daemons. `guardctrl` is essentially a convenient wrapper around systemctl and journalctl to allow specifying nodes by name, as opposed to by their underlying systemd service names which are slightly more cumbersome to work with for the average user.
|
|
|
|
... | ... | @@ -21,7 +21,7 @@ $ sudo apt install guardctrl |
|
|
```
|
|
|
The `guardctrl` package depends on `guardian` package, so you'll automatically get them both. guardctrl will install the command line interface, as well as all the needed systemd service unit files.
|
|
|
|
|
|
### create and configure guardctrl user
|
|
|
### creating and configuring the guardctrl user
|
|
|
|
|
|
`guardctrl` expects to be using the `systemd --user` instance of the invoking user. This means that `guardctrl` should always be invoked as the same user so that processes are managed in a unified way. The `guardctrl` interface knows it's running as the correct user by the presence of the `~/.guardctrl-home` file. If this file is not present, guardctrl will assume it's running remotely and will try to ssh to GUARDCTRL_USER@GUARDCTRL_HOST to issue the command.
|
|
|
|
... | ... | @@ -74,7 +74,7 @@ Because we want caRepeater to be running system-wide before starting any of the |
|
|
Wants=caRepeater.service
|
|
|
After=caRepeater.service
|
|
|
```
|
|
|
NOTE: the above assumes the existence of a `caRepeater.service`, so verify that it's there already, or create it if it doesn't exist:
|
|
|
NOTE: the above assumes the existence of a `caRepeater.service`. If you don't already have one, here's an example service description:
|
|
|
```shell
|
|
|
# /etc/systemd/system/caRepeater.service
|
|
|
[Unit]
|
... | ... | @@ -90,7 +90,7 @@ User=nobody |
|
|
WantedBy=multi-user.target
|
|
|
```
|
|
|
|
|
|
### configure journald for persistent logs
|
|
|
### configuring journald for persistent logs
|
|
|
|
|
|
For the LIGO sites, we want to store logs from all guardian processes in perpetuity. To this end, the journald system logger needs to be configured for "persistent" storage. This is done by setting `Storage=persistent` in `/etc/systemd/journald.conf` (included below are some other variables for increasing the log rate limit, and for increasing the disk storage limits for the logs):
|
|
|
```
|
... | ... | @@ -106,7 +106,7 @@ Reload the journald config after these changes are made: |
|
|
$ sudo systemctl force-reload systemd-journald
|
|
|
```
|
|
|
|
|
|
### specify local environment
|
|
|
### specifying local environment
|
|
|
|
|
|
The `guardian@.service` expects an `/etc/guardian/local-env` environment file to exist, for providing any needed environment variables to the supervised guardian processes. Here's an example of the file for H1 at LHO:
|
|
|
```shell
|
... | ... | @@ -149,7 +149,7 @@ If for some reason you need to pass special environment variables to `guardctrl` |
|
|
|
|
|
#### local guardian user access
|
|
|
|
|
|
Occasionally it might be necessary to access the guardian user directly. If passwordless SSH access has been enabled, this will need to be done with something like su. However, su and sudo do not provide access to the user dbus session needed to interact with `systemctl --user`. The systemd-container package includes the `machinectl` interface whose `shell` command allows for a clean user environment with all dbus interfaces available:
|
|
|
Occasionally it might be necessary to access the guardian user directly. If passwordless SSH access has been enabled as described above, you won't be able to access a guardian user terminal via ssh directly, and you'll need to use e.g. su or sudo from root. However, su and sudo do not provide access to the user dbus session needed to interact with `systemctl --user`. The systemd-container package includes the `machinectl` interface whose `shell` command allows for a clean user environment with all dbus interfaces available:
|
|
|
```shell
|
|
|
root@h1guardian1:~# machinectl shell guardian@ /bin/bash
|
|
|
guardian@h1guardian1:~$ systemctl --user status
|
... | ... | |