| ... | @@ -150,15 +150,53 @@ deb [signed-by=/usr/share/keyrings/cdssoft-unstable-archive-keyring.gpg] http:// |
... | @@ -150,15 +150,53 @@ deb [signed-by=/usr/share/keyrings/cdssoft-unstable-archive-keyring.gpg] http:// |
|
|
* nds2-io-node
|
|
* nds2-io-node
|
|
|
* daqd-stream-server
|
|
* daqd-stream-server
|
|
|
|
|
|
|
|
|
### System Users ###
|
|
|
|
|
|
|
|
A number of system users are created by the puppet config to run each system as different users.
|
|
|
|
|
|
|
|
* daqd_data
|
|
|
|
* ndsmetadata
|
|
|
|
* ndsio
|
|
|
|
* memcache
|
|
|
|
* ndsproxy
|
|
|
|
* ndsacl
|
|
|
|
|
|
|
## Puppet Configuration ##
|
|
## Puppet Configuration ##
|
|
|
|
|
|
|
|
A local puppet policy has been created to configure the nds-* systems. It is put in /root/puppet/modules/cds_data_stream. It is then applied with 'puppet apply --modulepath /root/puppet/modules /root/puppet/modules/cds_data_stream/nodes/nodes.pp --verbose'.
|
|
A local puppet policy has been created to configure the nds-* systems. It is put in /root/puppet/modules/cds_data_stream. It is then applied with 'puppet apply --modulepath /root/puppet/modules /root/puppet/modules/cds_data_stream/nodes/nodes.pp --verbose'. This puppet has been written as an example, and is expected to be adopted or superseded by the LDAS admins and integrated into the main LDAS puppet systems.
|
|
|
|
|
|
|
|
Very little to no manual work should be needed to setup a node. Mainly installing the keytab (proxy system only) and creating empty files in /var/lib/ndsmetadata/channel-lists (metadata server only, could be automated).
|
|
Very little to no manual work should be needed to setup a node. Mainly installing the keytab (proxy system only) and creating empty files in /var/lib/ndsmetadata/channel-lists (metadata server only, could be automated).
|
|
|
|
|
|
|
|
Typicall work has been to commit the puppet changes to git, pull the puppet onto daq1.dcs, then use a script to push the puppet files out to the nds-* boxes. Then run a script to apply the puppet to each of the nds-* systems.
|
|
Typically work has been to commit the puppet changes to git, pull the puppet onto daq1.dcs, then use a script to push the puppet files out to the nds-* boxes. Then run a script to apply the puppet to each of the nds-* systems.
|
|
|
|
|
|
|
|
Scripts:
|
|
Scripts:
|
|
|
* /root/local_puppet.sh
|
|
* /root/local_puppet.sh
|
|
|
* /root/pp.sh (push puppet)
|
|
* /root/pp.sh (push puppet)
|
|
|
* /root/rc.sh (run commands on each of the nds-* systems)
|
|
* /root/rc.sh (run commands on each of the nds-* systems)
|
|
|
|
|
|
|
|
## Firewall ##
|
|
|
|
|
|
|
|
A shorewall config is installed which limits:
|
|
|
|
|
|
|
|
* ssh to internal cluster interfaces
|
|
|
|
* memcached to nds-* and daq1
|
|
|
|
|
|
|
|
## Trouble shooting ##
|
|
|
|
|
|
|
|
Ensure that:
|
|
|
|
* data is flowing from CDS. It should be about 66 MB/s.
|
|
|
|
* disk cache is up on nds.dcs
|
|
|
|
* nds.dcs is up (the system needs the channel list updates)
|
|
|
|
* the frame directories are mounted on each system
|
|
|
|
|
|
|
|
The above descriptions of the services should provide help in localizing issues. However a full restart could be done as follows.
|
|
|
|
|
|
|
|
* Stop every nds service on each box.
|
|
|
|
* rm /dev/shm/daqd_stream
|
|
|
|
* start the daqd-stream-server process on each box
|
|
|
|
* on nds-1
|
|
|
|
* start nds-proxy
|
|
|
|
* start nds-metadata-server
|
|
|
|
* On all nodes
|
|
|
|
* start nds2-io-node
|
|
|
|
* start memcached |
|
|
|
\ No newline at end of file |
