Skip to content
Snippets Groups Projects

Update emfollow-test to Rocky Linux 8

    • Closed Issue created by Leo P. Singer @leo-singer

    Among RHEL and its derivatives, version 8 and later support user Systemd units. Update emfollow-test to RHEL 8 or one of its derivatives that is a candidate for the next LDAS CIT reference operating system.

    In the process, we should capture a list of any special RPMs that we need in addition to the normal set of packages for LDAS CIT hosts, and include that list in the GWCelery docs.

    See also #360 (closed), !343 (closed).

    Edited
    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items 0

    • No child items are currently assigned. Use child items to break down this issue into smaller parts.

    Activity

    • Leo P. Singer changed milestone to %O4

      changed milestone to %O4

    • Leo P. Singer added Deployment label

      added Deployment label

    • Leo P. Singer assigned to @stuart.anderson, @philippe.grassia, and @juan-barayoga

      assigned to @stuart.anderson, @philippe.grassia, and @juan-barayoga

    • Leo P. Singer changed the description

      changed the description

    • Leo P. Singer mentioned in merge request !343 (closed)

      mentioned in merge request !343 (closed)

      • Stuart Anderson
        Stuart Anderson @stuart.anderson ·
        Maintainer

        @leo-singer are you requesting that we start by destroying the current emfollow-test and replace it with an RL8 image? Or should we leave emfollow-test as is for now and stand up a new temporary emfollow-rl8 first?

      • Leo P. Singer
        Leo P. Singer @leo-singer ·
        Author Owner

        It doesn't matter that much, because the only development activity that we had planned to do in the near term on emfollow-test is the user systemd work, which won't work on SL7. As long as you keep up a backup, it may be easiest to destroy emfollow-test and create a new RL8 image in its place, because that way we can provide you with near-instant feedback using our existing continuous deployment setup.

      • Stuart Anderson
        Stuart Anderson @stuart.anderson ·
        Maintainer

        OK, we will keep a backup of emfollow-test and recreate that as a new clean RL8 image.

      • Please register or sign in to reply
    • Philippe Grassia
      Philippe Grassia @philippe.grassia ·
      Developer

      the only I found so far is rocky8-zfsnfs it will require significant rework to fit eemfollow's profile.

    • Cody Messick mentioned in issue #360 (closed)

      mentioned in issue #360 (closed)

      • Philippe Grassia
        Philippe Grassia @philippe.grassia ·
        Developer

        got my kickstart fixed to support RL8.5 (now that zfs is properly packaged for 8.5) adapted a copy of the dedicated puppet manifest used by SL7 emfollow to support RL8

        • remove mgt of yum repos
        • add or adapt configuration file templates from SL7 to RL8

        first successful run of puppet agent -t but still A LOT of (non fatal) errors on the way

        [root@emfollow8-test ~]# puppet agent -t 
Info: Using environment 'devel'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for emfollow8-test.cluster.ldas.cit
Info: Applying configuration version '1638580751'
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install devtoolset-8' returned 1: Error: Unable to find a match: devtoolset-8
Error: /Stage[main]/Profile::Software::Individual/Package[devtoolset-8]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install devtoolset-8' returned 1: Error: Unable to find a match: devtoolset-8
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install devtoolset-10' returned 1: Error: Unable to find a match: devtoolset-10
Error: /Stage[main]/Profile::Software::Individual/Package[devtoolset-10]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install devtoolset-10' returned 1: Error: Unable to find a match: devtoolset-10
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install iperf' returned 1: Error: Unable to find a match: iperf
Error: /Stage[main]/Profile::Software::Individual/Package[iperf]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install iperf' returned 1: Error: Unable to find a match: iperf
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install ldg-client' returned 1: Error: Unable to find a match: ldg-client
Error: /Stage[main]/Profile::Software::Individual/Package[ldg-client]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install ldg-client' returned 1: Error: Unable to find a match: ldg-client
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install lscsoft-all' returned 1: Error: Unable to find a match: lscsoft-all
Error: /Stage[main]/Profile::Software::Individual/Package[lscsoft-all]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install lscsoft-all' returned 1: Error: Unable to find a match: lscsoft-all
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install mpich-3.0' returned 1: Error: Unable to find a match: mpich-3.0
Error: /Stage[main]/Profile::Software::Individual/Package[mpich-3.0]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install mpich-3.0' returned 1: Error: Unable to find a match: mpich-3.0
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install osg-wn-client' returned 1: Error: Unable to find a match: osg-wn-client
Error: /Stage[main]/Profile::Software::Individual/Package[osg-wn-client]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install osg-wn-client' returned 1: Error: Unable to find a match: osg-wn-client
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python-pandas' returned 1: Error: Unable to find a match: python-pandas
Error: /Stage[main]/Profile::Software::Individual/Package[python-pandas]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python-pandas' returned 1: Error: Unable to find a match: python-pandas
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python2-avahi' returned 1: Error: Unable to find a match: python2-avahi
Error: /Stage[main]/Profile::Software::Individual/Package[python2-avahi]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python2-avahi' returned 1: Error: Unable to find a match: python2-avahi
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install nagios-plugins-all' returned 1: Error: 
 Problem: package nagios-plugins-all-2.3.3-5.el8.x86_64 requires nagios-plugins-disk_smb, but none of the providers can be installed
  - package nagios-plugins-disk_smb-2.3.3-5.el8.x86_64 requires perl(utf8::all), but none of the providers can be installed
  - conflicting requests
  - nothing provides perl(Import::Into) needed by perl-utf8-all-0.024-7.el8.noarch
  - nothing provides perl(PerlIO::utf8_strict) needed by perl-utf8-all-0.024-7.el8.noarch
Error: /Stage[main]/Profile::Service::Nagios::Base/Package[nagios-plugins-all]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install nagios-plugins-all' returned 1: Error: 
 Problem: package nagios-plugins-all-2.3.3-5.el8.x86_64 requires nagios-plugins-disk_smb, but none of the providers can be installed
  - package nagios-plugins-disk_smb-2.3.3-5.el8.x86_64 requires perl(utf8::all), but none of the providers can be installed
  - conflicting requests
  - nothing provides perl(Import::Into) needed by perl-utf8-all-0.024-7.el8.noarch
  - nothing provides perl(PerlIO::utf8_strict) needed by perl-utf8-all-0.024-7.el8.noarch
Error: /Stage[main]/Profile::Service::Avahi/File[/etc/avahi/avahi-daemon.conf]: Could not evaluate: Could not retrieve information from environment devel source(s) puppet:///modules/profile/RedHat8/etc/avahi/avahi-daemon.conf
Notice: /Stage[main]/Profile::Service::Avahi/Service[avahi-daemon]: Dependency File[/etc/avahi/avahi-daemon.conf] has failures: true
Warning: /Stage[main]/Profile::Service::Avahi/Service[avahi-daemon]: Skipping because of failed dependencies
Error: /Stage[main]/Profile::Service::Logind/File[/etc/systemd/logind.conf]: Could not evaluate: Could not retrieve information from environment devel source(s) puppet:///modules/profile/RedHat8/etc/systemd/logind.conf
Notice: /Stage[main]/Profile::Service::Logind/Service[systemd-logind]: Dependency File[/etc/systemd/logind.conf] has failures: true
Warning: /Stage[main]/Profile::Service::Logind/Service[systemd-logind]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Logind/Exec[logind-daemon-reload]: Skipping because of failed dependencies
Error: /Stage[main]/Profile::Kdump/File[/etc/kdump.conf]: Could not evaluate: Could not retrieve information from environment devel source(s) puppet:///modules/profile/RedHat8/etc/kdump.conf
Notice: /Stage[main]/Profile::Kdump/Service[kdump]: Dependency File[/etc/kdump.conf] has failures: true
Warning: /Stage[main]/Profile::Kdump/Service[kdump]: Skipping because of failed dependencies
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install osg-oasis' returned 1: Error: Unable to find a match: osg-oasis
Error: /Stage[main]/Profile::Service::Cvmfs::Base/Package[osg-oasis]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install osg-oasis' returned 1: Error: Unable to find a match: osg-oasis
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/cvmfs/default.conf20211203-167297-1yazy2n.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/cvmfs/base.pp, line: 17)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/cvmfs/default.conf20211203-167297-1yazy2n.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/cvmfs/base.pp, line: 17)
Wrapped exception:
No such file or directory - A directory component in /etc/cvmfs/default.conf20211203-167297-1yazy2n.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Cvmfs::Base/File[/etc/cvmfs/default.conf]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/cvmfs/default.conf20211203-167297-1yazy2n.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/cvmfs/base.pp, line: 17)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/cvmfs/default.local20211203-167297-1mio2r6.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/cvmfs/base.pp, line: 17)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/cvmfs/default.local20211203-167297-1mio2r6.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/cvmfs/base.pp, line: 17)
Wrapped exception:
No such file or directory - A directory component in /etc/cvmfs/default.local20211203-167297-1mio2r6.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Cvmfs::Base/File[/etc/cvmfs/default.local]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/cvmfs/default.local20211203-167297-1mio2r6.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/cvmfs/base.pp, line: 17)
Notice: /Stage[main]/Profile::Service::Cvmfs::Base/Exec[wipecache]: Dependency File[/etc/cvmfs/default.conf] has failures: true
Notice: /Stage[main]/Profile::Service::Cvmfs::Base/Exec[wipecache]: Dependency File[/etc/cvmfs/default.local] has failures: true
Warning: /Stage[main]/Profile::Service::Cvmfs::Base/Exec[wipecache]: Skipping because of failed dependencies
Error: /Stage[main]/Profile::Access/File[/etc/security/access.conf]: Could not evaluate: Could not retrieve information from environment devel source(s) puppet:///modules/profile/common/etc/security/access.conf-emfollow8-test
Notice: /Stage[main]/Profile::Access/Exec[access_authconfig]: Dependency File[/etc/security/access.conf] has failures: true
Warning: /Stage[main]/Profile::Access/Exec[access_authconfig]: Skipping because of failed dependencies
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install pam_krb5' returned 1: Error: Unable to find a match: pam_krb5
Error: /Stage[main]/Profile::Kerberos::Base/Package[pam_krb5]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install pam_krb5' returned 1: Error: Unable to find a match: pam_krb5
Error: /Stage[main]/Profile::Kerberos::Keytab/File[/etc/krb5.keytab]: Could not evaluate: Could not retrieve information from environment devel source(s) puppet:///modules/profile/common/etc/keytab.emfollow8-test
Notice: /Stage[main]/Profile::Kerberos::Base/Exec[enablekrb5]: Dependency Package[pam_krb5] has failures: true
Notice: /Stage[main]/Profile::Kerberos::Base/Exec[enablekrb5]: Dependency File[/etc/krb5.keytab] has failures: true
Warning: /Stage[main]/Profile::Kerberos::Base/Exec[enablekrb5]: Skipping because of failed dependencies
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=ceph* install ceph-common' returned 1: Error: Unknown repo: 'ceph*'
Error: /Stage[main]/Profile::Ceph::Client/Package[ceph-common]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=ceph* install ceph-common' returned 1: Error: Unknown repo: 'ceph*'
Notice: /Stage[main]/Profile::Ceph::Client/File[/etc/ceph/ceph.conf]: Dependency Package[ceph-common] has failures: true
Warning: /Stage[main]/Profile::Ceph::Client/File[/etc/ceph/ceph.conf]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ceph::Client/File[/etc/ceph/ceph.client.dcs.keyring]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ceph::Client/Mount[/ceph]: Skipping because of failed dependencies
Error: /Stage[main]/Profile::Resolver::Network/File[/etc/sysconfig/network]: Could not evaluate: Could not retrieve information from environment devel source(s) puppet:///modules/profile/RedHat8/etc/sysconfig/network
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install munin-node' returned 1: Error: 
 Problem: package munin-node-2.0.66-1.el8.noarch requires perl(Net::SNMP), but none of the providers can be installed
  - conflicting requests
  - nothing provides perl(Digest::SHA1) >= 1.02 needed by perl-Net-SNMP-6.0.1-25.el8.1.noarch
Error: /Stage[main]/Profile::Service::Munin::Client/Package[munin-node]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install munin-node' returned 1: Error: 
 Problem: package munin-node-2.0.66-1.el8.noarch requires perl(Net::SNMP), but none of the providers can be installed
  - conflicting requests
  - nothing provides perl(Digest::SHA1) >= 1.02 needed by perl-Net-SNMP-6.0.1-25.el8.1.noarch
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/munin/munin-node.conf20211203-167297-1dq0uj2.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/munin/client.pp, line: 17)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/munin/munin-node.conf20211203-167297-1dq0uj2.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/munin/client.pp, line: 17)
Wrapped exception:
No such file or directory - A directory component in /etc/munin/munin-node.conf20211203-167297-1dq0uj2.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Munin::Client/File[/etc/munin/munin-node.conf]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/munin/munin-node.conf20211203-167297-1dq0uj2.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/munin/client.pp, line: 17)
Notice: /Stage[main]/Profile::Service::Munin::Client/Service[munin-node]: Dependency File[/etc/munin/munin-node.conf] has failures: true
Warning: /Stage[main]/Profile::Service::Munin::Client/Service[munin-node]: Skipping because of failed dependencies
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install llkafka-nodeconfig-cit' returned 1: Error: Unable to find a match: llkafka-nodeconfig-cit
Error: /Stage[main]/Profile::Service::Kafka::Common/Package[llkafka-nodeconfig-cit]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install llkafka-nodeconfig-cit' returned 1: Error: Unable to find a match: llkafka-nodeconfig-cit
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python2-llkafka' returned 1: Error: Unable to find a match: python2-llkafka
Error: /Stage[main]/Profile::Service::Kafka::Common/Package[python2-llkafka]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python2-llkafka' returned 1: Error: Unable to find a match: python2-llkafka
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python-cjson' returned 1: Error: Unable to find a match: python-cjson
Error: /Stage[main]/Profile::Service::Kafka::Common/Package[python-cjson]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python-cjson' returned 1: Error: Unable to find a match: python-cjson
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python36-lal' returned 1: Error: Unable to find a match: python36-lal
Error: /Stage[main]/Profile::Service::Kafka::Common/Package[python36-lal]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python36-lal' returned 1: Error: Unable to find a match: python36-lal
Error: Systemd start for kafka2lsmp-O3E2ERep1-H1L1V1K1-alwayson.service failed!
journalctl log for kafka2lsmp-O3E2ERep1-H1L1V1K1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:25 PST. --
-- No entries --

Error: /Stage[main]/Profile::Service::Kafka::Live_replay/Service[kafka2lsmp-O3E2ERep1-H1L1V1K1-alwayson.service]/ensure: change from 'stopped' to 'running' failed: Systemd start for kafka2lsmp-O3E2ERep1-H1L1V1K1-alwayson.service failed!
journalctl log for kafka2lsmp-O3E2ERep1-H1L1V1K1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:25 PST. --
-- No entries --

Error: Systemd start for kafka2lsmp-O3E2ERep2-H1L1V1K1-alwayson.service failed!
journalctl log for kafka2lsmp-O3E2ERep2-H1L1V1K1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:25 PST. --
-- No entries --

Error: /Stage[main]/Profile::Service::Kafka::Live_replay/Service[kafka2lsmp-O3E2ERep2-H1L1V1K1-alwayson.service]/ensure: change from 'stopped' to 'running' failed: Systemd start for kafka2lsmp-O3E2ERep2-H1L1V1K1-alwayson.service failed!
journalctl log for kafka2lsmp-O3E2ERep2-H1L1V1K1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:25 PST. --
-- No entries --

Error: Systemd start for kafka2lsmp-O3ReplayMDC-H1L1V1-alwayson.service failed!
journalctl log for kafka2lsmp-O3ReplayMDC-H1L1V1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:26 PST. --
-- No entries --

Error: /Stage[main]/Profile::Service::Kafka::Live_replay/Service[kafka2lsmp-O3ReplayMDC-H1L1V1-alwayson.service]/ensure: change from 'stopped' to 'running' failed: Systemd start for kafka2lsmp-O3ReplayMDC-H1L1V1-alwayson.service failed!
journalctl log for kafka2lsmp-O3ReplayMDC-H1L1V1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:26 PST. --
-- No entries --

Error: Systemd start for kafka2lsmp-O4GaussianMDC-H1L1V1-alwayson.service failed!
journalctl log for kafka2lsmp-O4GaussianMDC-H1L1V1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:26 PST. --
-- No entries --

Error: /Stage[main]/Profile::Service::Kafka::Live_replay/Service[kafka2lsmp-O4GaussianMDC-H1L1V1-alwayson.service]/ensure: change from 'stopped' to 'running' failed: Systemd start for kafka2lsmp-O4GaussianMDC-H1L1V1-alwayson.service failed!
journalctl log for kafka2lsmp-O4GaussianMDC-H1L1V1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:26 PST. --
-- No entries --

Error: Systemd start for kafka2lsmp-live-H1L1V1-alwayson.service failed!
journalctl log for kafka2lsmp-live-H1L1V1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:26 PST. --
-- No entries --

Error: /Stage[main]/Profile::Service::Kafka::Live_replay/Service[kafka2lsmp-live-H1L1V1-alwayson.service]/ensure: change from 'stopped' to 'running' failed: Systemd start for kafka2lsmp-live-H1L1V1-alwayson.service failed!
journalctl log for kafka2lsmp-live-H1L1V1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:26 PST. --
-- No entries --

Error: Systemd start for kafka2lsmp-live-K1-alwayson.service failed!
journalctl log for kafka2lsmp-live-K1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:26 PST. --
-- No entries --

Error: /Stage[main]/Profile::Service::Kafka::Live_replay/Service[kafka2lsmp-live-K1-alwayson.service]/ensure: change from 'stopped' to 'running' failed: Systemd start for kafka2lsmp-live-K1-alwayson.service failed!
journalctl log for kafka2lsmp-live-K1-alwayson.service:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:20:26 PST. --
-- No entries --

Notice: /Stage[main]/Profile::Service::Kafka::Live_replay/Exec[restart-live_replay]: Dependency Service[kafka2lsmp-O3E2ERep1-H1L1V1K1-alwayson.service] has failures: true
Notice: /Stage[main]/Profile::Service::Kafka::Live_replay/Exec[restart-live_replay]: Dependency Service[kafka2lsmp-O3E2ERep2-H1L1V1K1-alwayson.service] has failures: true
Notice: /Stage[main]/Profile::Service::Kafka::Live_replay/Exec[restart-live_replay]: Dependency Service[kafka2lsmp-O3ReplayMDC-H1L1V1-alwayson.service] has failures: true
Notice: /Stage[main]/Profile::Service::Kafka::Live_replay/Exec[restart-live_replay]: Dependency Service[kafka2lsmp-O4GaussianMDC-H1L1V1-alwayson.service] has failures: true
Notice: /Stage[main]/Profile::Service::Kafka::Live_replay/Exec[restart-live_replay]: Dependency Service[kafka2lsmp-live-H1L1V1-alwayson.service] has failures: true
Notice: /Stage[main]/Profile::Service::Kafka::Live_replay/Exec[restart-live_replay]: Dependency Service[kafka2lsmp-live-K1-alwayson.service] has failures: true
Warning: /Stage[main]/Profile::Service::Kafka::Live_replay/Exec[restart-live_replay]: Skipping because of failed dependencies
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install llldd' returned 1: Error: Unable to find a match: llldd
Error: /Stage[main]/Profile::Service::Llldd::Common/Package[llldd]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install llldd' returned 1: Error: Unable to find a match: llldd
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install llldd-config-cit' returned 1: Error: Unable to find a match: llldd-config-cit
Error: /Stage[main]/Profile::Service::Llldd::Common/Package[llldd-config-cit]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install llldd-config-cit' returned 1: Error: Unable to find a match: llldd-config-cit
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install llldd-systemd' returned 1: Error: Unable to find a match: llldd-systemd
Error: /Stage[main]/Profile::Service::Llldd::Common/Package[llldd-systemd]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install llldd-systemd' returned 1: Error: Unable to find a match: llldd-systemd
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install ligo-grid-mapfile-manager' returned 1: Error: Unable to find a match: ligo-grid-mapfile-manager
Error: /Stage[main]/Profile::Service::Lgmm/Package[ligo-grid-mapfile-manager]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install ligo-grid-mapfile-manager' returned 1: Error: Unable to find a match: ligo-grid-mapfile-manager
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python-ldap' returned 1: Error: Unable to find a match: python-ldap
Error: /Stage[main]/Profile::Service::Lgmm/Package[python-ldap]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install python-ldap' returned 1: Error: Unable to find a match: python-ldap
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/grid-security/whitelist20211203-167297-8qw3fc.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/lgmm.pp, line: 21)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/grid-security/whitelist20211203-167297-8qw3fc.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/lgmm.pp, line: 21)
Wrapped exception:
No such file or directory - A directory component in /etc/grid-security/whitelist20211203-167297-8qw3fc.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Lgmm/File[/etc/grid-security/whitelist]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/grid-security/whitelist20211203-167297-8qw3fc.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/lgmm.pp, line: 21)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/lgmm/lgmm_config.py20211203-167297-fzrbvi.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/lgmm.pp, line: 21)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/lgmm/lgmm_config.py20211203-167297-fzrbvi.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/lgmm.pp, line: 21)
Wrapped exception:
No such file or directory - A directory component in /etc/lgmm/lgmm_config.py20211203-167297-fzrbvi.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Lgmm/File[/etc/lgmm/lgmm_config.py]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/lgmm/lgmm_config.py20211203-167297-fzrbvi.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/lgmm.pp, line: 21)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/lgmm/lgmm_config_kagra.py20211203-167297-15n4vmu.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/lgmm.pp, line: 21)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/lgmm/lgmm_config_kagra.py20211203-167297-15n4vmu.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/lgmm.pp, line: 21)
Wrapped exception:
No such file or directory - A directory component in /etc/lgmm/lgmm_config_kagra.py20211203-167297-15n4vmu.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Lgmm/File[/etc/lgmm/lgmm_config_kagra.py]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/lgmm/lgmm_config_kagra.py20211203-167297-15n4vmu.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/lgmm.pp, line: 21)
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install hadoop-client' returned 1: Error: Unable to find a match: hadoop-client
Error: /Stage[main]/Profile::Service::Hdfs::Client/Package[hadoop-client]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install hadoop-client' returned 1: Error: Unable to find a match: hadoop-client
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y install hadoop-hdfs-fuse' returned 1: Error: Unable to find a match: hadoop-hdfs-fuse
Error: /Stage[main]/Profile::Service::Hdfs::Client/Package[hadoop-hdfs-fuse]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y install hadoop-hdfs-fuse' returned 1: Error: Unable to find a match: hadoop-hdfs-fuse
Error: Could not find user hdfs
Info: Unknown failure using insync_values? on type: File[/hdfs] / property: owner to compare values ["hdfs"] and 0
Error: /Stage[main]/Profile::Service::Hdfs::Client/File[/hdfs]/owner: change from 'root' to 'hdfs' failed: Could not find user hdfs
Error: Could not find group hdfs
Info: Unknown failure using insync_values? on type: File[/hdfs] / property: group to compare values ["hdfs"] and 0
Error: /Stage[main]/Profile::Service::Hdfs::Client/File[/hdfs]/group: change from 'root' to 'hdfs' failed: Could not find group hdfs
Error: Could not set 'present' on ensure: No such file or directory - A directory component in /etc/hadoop/conf.empty/core-site.xml20211203-167297-pq0psi.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/hdfs/client.pp, line: 10)
Error: Could not set 'present' on ensure: No such file or directory - A directory component in /etc/hadoop/conf.empty/core-site.xml20211203-167297-pq0psi.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/hdfs/client.pp, line: 10)
Wrapped exception:
No such file or directory - A directory component in /etc/hadoop/conf.empty/core-site.xml20211203-167297-pq0psi.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Hdfs::Client/File[/etc/hadoop/conf.empty/core-site.xml]/ensure: change from 'absent' to 'present' failed: Could not set 'present' on ensure: No such file or directory - A directory component in /etc/hadoop/conf.empty/core-site.xml20211203-167297-pq0psi.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/hdfs/client.pp, line: 10)
Notice: /Stage[main]/Profile::Service::Monit/Service[monit]/ensure: ensure changed 'stopped' to 'running' (corrective)
Info: /Stage[main]/Profile::Service::Monit/Service[monit]: Unscheduling refresh on Service[monit]
Notice: /Stage[main]/Profile::Service::Hdfs::Client/Mount[/hdfs]: Dependency File[/hdfs] has failures: true
Warning: /Stage[main]/Profile::Service::Hdfs::Client/Mount[/hdfs]: Skipping because of failed dependencies
Notice: /Stage[main]/Profile::Software::Group/Yum::Group[perl-runtime]/Exec[yum-groupinstall-perl-runtime]/returns: Last metadata expiration check: 1:34:04 ago on Fri Dec  3 15:46:43 2021.
Notice: /Stage[main]/Profile::Software::Group/Yum::Group[perl-runtime]/Exec[yum-groupinstall-perl-runtime]/returns: Module or Group 'perl-runtime' is not available.
Notice: /Stage[main]/Profile::Software::Group/Yum::Group[perl-runtime]/Exec[yum-groupinstall-perl-runtime]/returns: Error: Nothing to do.
Error: 'yum -y groupinstall 'perl-runtime'' returned 1 instead of one of [0]
Error: /Stage[main]/Profile::Software::Group/Yum::Group[perl-runtime]/Exec[yum-groupinstall-perl-runtime]/returns: change from 'notrun' to ['0'] failed: 'yum -y groupinstall 'perl-runtime'' returned 1 instead of one of [0] (corrective)
Error: /Stage[main]/Profile::Cgroup::Priv/Profile::Cgroup::Mod[atd]/File[/etc/pam.d/atd]: Could not evaluate: Could not retrieve information from environment devel source(s) puppet:///modules/profile/RedHat8/etc/pam.d/atd
Error: /Stage[main]/Profile::Cgroup::Priv/Profile::Cgroup::Mod[crond]/File[/etc/pam.d/crond]: Could not evaluate: Could not retrieve information from environment devel source(s) puppet:///modules/profile/RedHat8/etc/pam.d/crond
Error: /Stage[main]/Profile::Cgroup::Priv/Profile::Cgroup::Mod[postlogin-ac]/File[/etc/pam.d/postlogin-ac]: Could not evaluate: Could not retrieve information from environment devel source(s) puppet:///modules/profile/RedHat8/etc/pam.d/postlogin-ac
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /opt/puppetlabs/facter/facts.d/datadir20211203-167297-wz1vkz.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/hdfs/client.pp, line: 10)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /opt/puppetlabs/facter/facts.d/datadir20211203-167297-wz1vkz.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/hdfs/client.pp, line: 10)
Wrapped exception:
No such file or directory - A directory component in /opt/puppetlabs/facter/facts.d/datadir20211203-167297-wz1vkz.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Hdfs::Client/File[/opt/puppetlabs/facter/facts.d/datadir]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /opt/puppetlabs/facter/facts.d/datadir20211203-167297-wz1vkz.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/hdfs/client.pp, line: 10)
Notice: /Stage[main]/Profile::Service::Hdfs::Client/File[/etc/hadoop/conf.empty/hdfs-site.xml]: Dependency File[/opt/puppetlabs/facter/facts.d/datadir] has failures: true
Warning: /Stage[main]/Profile::Service::Hdfs::Client/File[/etc/hadoop/conf.empty/hdfs-site.xml]: Skipping because of failed dependencies
Error: /Stage[main]/Profile::Startup::Base/File[/opt/dcs/etc/profile.d/conda.csh]: Could not evaluate: Could not retrieve information from environment devel source(s) file:///cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/etc/profile.d/conda.csh
Error: /Stage[main]/Profile::Startup::Base/File[/opt/dcs/etc/profile.d/conda.sh]: Could not evaluate: Could not retrieve information from environment devel source(s) file:///cvmfs/oasis.opensciencegrid.org/ligo/sw/conda/etc/profile.d/conda.sh
Error: /Stage[main]/Profile::Service::Openssh::Base/File[/etc/ssh/ssh_config]: Could not evaluate: Could not retrieve information from environment devel source(s) puppet:///modules/profile/RedHat8/etc/ssh/ssh_config
Notice: /Stage[main]/Profile::Service::Lgmm/File[/opt/dcs/sbin/active_users]: Dependency Package[python-ldap] has failures: true
Warning: /Stage[main]/Profile::Service::Lgmm/File[/opt/dcs/sbin/active_users]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Lgmm/File[/opt/dcs/sbin/lgmm]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Lgmm/File[/etc/cron.hourly/lgmm]: Skipping because of failed dependencies
Error: Systemd start for sshd failed!
journalctl log for sshd:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:21:19 PST. --
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Scheduled restart job, restart counter is at 2.
Dec 03 17:16:24 emfollow8-test systemd[1]: Stopped OpenSSH server daemon.
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Consumed 9ms CPU time
Dec 03 17:16:24 emfollow8-test systemd[1]: Starting OpenSSH server daemon...
Dec 03 17:16:24 emfollow8-test sshd[167283]: /etc/ssh/sshd_config line 167: Directive 'IgnoreRhosts' is not allowed within a Match block
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Main process exited, code=exited, status=255/n/a
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Failed with result 'exit-code'.
Dec 03 17:16:24 emfollow8-test systemd[1]: Failed to start OpenSSH server daemon.
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Consumed 8ms CPU time
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Scheduled restart job, restart counter is at 3.
Dec 03 17:17:06 emfollow8-test systemd[1]: Stopped OpenSSH server daemon.
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Consumed 8ms CPU time
Dec 03 17:17:06 emfollow8-test systemd[1]: Starting OpenSSH server daemon...
Dec 03 17:17:06 emfollow8-test sshd[167294]: /etc/ssh/sshd_config line 167: Directive 'IgnoreRhosts' is not allowed within a Match block
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Main process exited, code=exited, status=255/n/a
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Failed with result 'exit-code'.
Dec 03 17:17:06 emfollow8-test systemd[1]: Failed to start OpenSSH server daemon.
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Consumed 8ms CPU time
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Scheduled restart job, restart counter is at 4.
Dec 03 17:17:48 emfollow8-test systemd[1]: Stopped OpenSSH server daemon.
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Consumed 8ms CPU time
Dec 03 17:17:48 emfollow8-test systemd[1]: Starting OpenSSH server daemon...
Dec 03 17:17:48 emfollow8-test sshd[167392]: /etc/ssh/sshd_config line 167: Directive 'IgnoreRhosts' is not allowed within a Match block
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Main process exited, code=exited, status=255/n/a
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Failed with result 'exit-code'.
Dec 03 17:17:48 emfollow8-test systemd[1]: Failed to start OpenSSH server daemon.
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Consumed 7ms CPU time
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Scheduled restart job, restart counter is at 5.
Dec 03 17:18:30 emfollow8-test systemd[1]: Stopped OpenSSH server daemon.
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Consumed 7ms CPU time
Dec 03 17:18:30 emfollow8-test systemd[1]: Starting OpenSSH server daemon...
Dec 03 17:18:30 emfollow8-test sshd[167401]: /etc/ssh/sshd_config line 167: Directive 'IgnoreRhosts' is not allowed within a Match block
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Main process exited, code=exited, status=255/n/a
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Failed with result 'exit-code'.
Dec 03 17:18:30 emfollow8-test systemd[1]: Failed to start OpenSSH server daemon.
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Consumed 9ms CPU time
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Scheduled restart job, restart counter is at 6.
Dec 03 17:19:12 emfollow8-test systemd[1]: Stopped OpenSSH server daemon.
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Consumed 9ms CPU time
Dec 03 17:19:12 emfollow8-test systemd[1]: Starting OpenSSH server daemon...
Dec 03 17:19:12 emfollow8-test sshd[167404]: /etc/ssh/sshd_config line 167: Directive 'IgnoreRhosts' is not allowed within a Match block
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Main process exited, code=exited, status=255/n/a
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Failed with result 'exit-code'.
Dec 03 17:19:12 emfollow8-test systemd[1]: Failed to start OpenSSH server daemon.
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Consumed 10ms CPU time

Error: /Stage[main]/Profile::Service::Openssh::Base/Service[sshd]/ensure: change from 'stopped' to 'running' failed: Systemd start for sshd failed!
journalctl log for sshd:
-- Logs begin at Fri 2021-12-03 15:04:36 PST, end at Fri 2021-12-03 17:21:19 PST. --
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Scheduled restart job, restart counter is at 2.
Dec 03 17:16:24 emfollow8-test systemd[1]: Stopped OpenSSH server daemon.
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Consumed 9ms CPU time
Dec 03 17:16:24 emfollow8-test systemd[1]: Starting OpenSSH server daemon...
Dec 03 17:16:24 emfollow8-test sshd[167283]: /etc/ssh/sshd_config line 167: Directive 'IgnoreRhosts' is not allowed within a Match block
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Main process exited, code=exited, status=255/n/a
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Failed with result 'exit-code'.
Dec 03 17:16:24 emfollow8-test systemd[1]: Failed to start OpenSSH server daemon.
Dec 03 17:16:24 emfollow8-test systemd[1]: sshd.service: Consumed 8ms CPU time
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Scheduled restart job, restart counter is at 3.
Dec 03 17:17:06 emfollow8-test systemd[1]: Stopped OpenSSH server daemon.
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Consumed 8ms CPU time
Dec 03 17:17:06 emfollow8-test systemd[1]: Starting OpenSSH server daemon...
Dec 03 17:17:06 emfollow8-test sshd[167294]: /etc/ssh/sshd_config line 167: Directive 'IgnoreRhosts' is not allowed within a Match block
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Main process exited, code=exited, status=255/n/a
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Failed with result 'exit-code'.
Dec 03 17:17:06 emfollow8-test systemd[1]: Failed to start OpenSSH server daemon.
Dec 03 17:17:06 emfollow8-test systemd[1]: sshd.service: Consumed 8ms CPU time
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Scheduled restart job, restart counter is at 4.
Dec 03 17:17:48 emfollow8-test systemd[1]: Stopped OpenSSH server daemon.
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Consumed 8ms CPU time
Dec 03 17:17:48 emfollow8-test systemd[1]: Starting OpenSSH server daemon...
Dec 03 17:17:48 emfollow8-test sshd[167392]: /etc/ssh/sshd_config line 167: Directive 'IgnoreRhosts' is not allowed within a Match block
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Main process exited, code=exited, status=255/n/a
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Failed with result 'exit-code'.
Dec 03 17:17:48 emfollow8-test systemd[1]: Failed to start OpenSSH server daemon.
Dec 03 17:17:48 emfollow8-test systemd[1]: sshd.service: Consumed 7ms CPU time
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Scheduled restart job, restart counter is at 5.
Dec 03 17:18:30 emfollow8-test systemd[1]: Stopped OpenSSH server daemon.
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Consumed 7ms CPU time
Dec 03 17:18:30 emfollow8-test systemd[1]: Starting OpenSSH server daemon...
Dec 03 17:18:30 emfollow8-test sshd[167401]: /etc/ssh/sshd_config line 167: Directive 'IgnoreRhosts' is not allowed within a Match block
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Main process exited, code=exited, status=255/n/a
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Failed with result 'exit-code'.
Dec 03 17:18:30 emfollow8-test systemd[1]: Failed to start OpenSSH server daemon.
Dec 03 17:18:30 emfollow8-test systemd[1]: sshd.service: Consumed 9ms CPU time
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Service RestartSec=42s expired, scheduling restart.
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Scheduled restart job, restart counter is at 6.
Dec 03 17:19:12 emfollow8-test systemd[1]: Stopped OpenSSH server daemon.
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Consumed 9ms CPU time
Dec 03 17:19:12 emfollow8-test systemd[1]: Starting OpenSSH server daemon...
Dec 03 17:19:12 emfollow8-test sshd[167404]: /etc/ssh/sshd_config line 167: Directive 'IgnoreRhosts' is not allowed within a Match block
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Main process exited, code=exited, status=255/n/a
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Failed with result 'exit-code'.
Dec 03 17:19:12 emfollow8-test systemd[1]: Failed to start OpenSSH server daemon.
Dec 03 17:19:12 emfollow8-test systemd[1]: sshd.service: Consumed 10ms CPU time
 (corrective)
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install condor' returned 1: Error: Unknown repo: 'condor-stable'
Error: /Stage[main]/Profile::Service::Condor::Base/Package[condor]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install condor' returned 1: Error: Unknown repo: 'condor-stable'
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install condor-classads' returned 1: Error: Unknown repo: 'condor-stable'
Error: /Stage[main]/Profile::Service::Condor::Base/Package[condor-classads]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install condor-classads' returned 1: Error: Unknown repo: 'condor-stable'
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install condor-externals' returned 1: Error: Unknown repo: 'condor-stable'
Error: /Stage[main]/Profile::Service::Condor::Base/Package[condor-externals]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install condor-externals' returned 1: Error: Unknown repo: 'condor-stable'
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install condor-procd' returned 1: Error: Unknown repo: 'condor-stable'
Error: /Stage[main]/Profile::Service::Condor::Base/Package[condor-procd]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install condor-procd' returned 1: Error: Unknown repo: 'condor-stable'
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install condor-credmon-oauth' returned 1: Error: Unknown repo: 'condor-stable'
Error: /Stage[main]/Profile::Service::Condor::Base/Package[condor-credmon-oauth]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install condor-credmon-oauth' returned 1: Error: Unknown repo: 'condor-stable'
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install python2-condor' returned 1: Error: Unknown repo: 'condor-stable'
Error: /Stage[main]/Profile::Service::Condor::Base/Package[python2-condor]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install python2-condor' returned 1: Error: Unknown repo: 'condor-stable'
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install python2-scitokens' returned 1: Error: Unknown repo: 'condor-stable'
Error: /Stage[main]/Profile::Service::Condor::Base/Package[python2-scitokens]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install python2-scitokens' returned 1: Error: Unknown repo: 'condor-stable'
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install python3-condor' returned 1: Error: Unknown repo: 'condor-stable'
Error: /Stage[main]/Profile::Service::Condor::Base/Package[python3-condor]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install python3-condor' returned 1: Error: Unknown repo: 'condor-stable'
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install python3-scitokens' returned 1: Error: Unknown repo: 'condor-stable'
Error: /Stage[main]/Profile::Service::Condor::Base/Package[python3-scitokens]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install python3-scitokens' returned 1: Error: Unknown repo: 'condor-stable'
Error: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install singularity' returned 1: Error: Unknown repo: 'condor-stable'
Error: /Stage[main]/Profile::Service::Condor::Base/Package[singularity]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/dnf -d 0 -e 1 -y --enablerepo=condor-stable install singularity' returned 1: Error: Unknown repo: 'condor-stable'
Error: Could not enable condor: 
Error: /Stage[main]/Profile::Service::Condor::Base/Service[condor]/enable: change from 'false' to 'true' failed: Could not enable condor: 
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/condor/condor_cred20211203-167297-29evfk.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/condor/base.pp, line: 19)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/condor/condor_cred20211203-167297-29evfk.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/condor/base.pp, line: 19)
Wrapped exception:
No such file or directory - A directory component in /etc/condor/condor_cred20211203-167297-29evfk.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/condor_cred]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/condor/condor_cred20211203-167297-29evfk.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/condor/base.pp, line: 19)
Error: Cannot create /etc/condor/config.d; parent directory /etc/condor does not exist
Error: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d]/ensure: change from 'absent' to 'directory' failed: Cannot create /etc/condor/config.d; parent directory /etc/condor does not exist
Notice: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d/00-htcondor-9.0.config]: Dependency File[/etc/condor/config.d] has failures: true
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d/00-htcondor-9.0.config]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d/00-htcondor-9.0.config.rpmsave]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d/00-htcondor-9.0.config.rpmnew]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d/00-ldas]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d/10-cgroup]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d/10-security]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d/40-oauth-credmon.conf]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d/40-oauth-credmon.conf.rpmnew]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/config.d/40-oauth-credmon.conf.rpmsave]: Skipping because of failed dependencies
Error: Cannot create /etc/condor/modules; parent directory /etc/condor does not exist
Error: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/modules]/ensure: change from 'absent' to 'directory' failed: Cannot create /etc/condor/modules; parent directory /etc/condor does not exist
Notice: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/modules/cpuinfo]: Dependency File[/etc/condor/modules] has failures: true
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/modules/cpuinfo]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/modules/facter]: Skipping because of failed dependencies
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/condor/scitokens-private.pem20211203-167297-17pqvqa.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/condor/base.pp, line: 19)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/condor/scitokens-private.pem20211203-167297-17pqvqa.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/condor/base.pp, line: 19)
Wrapped exception:
No such file or directory - A directory component in /etc/condor/scitokens-private.pem20211203-167297-17pqvqa.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Condor::Base/File[/etc/condor/scitokens-private.pem]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/condor/scitokens-private.pem20211203-167297-17pqvqa.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/condor/base.pp, line: 19)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/singularity/singularity.conf20211203-167297-15slw3l.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/condor/base.pp, line: 19)
Error: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/singularity/singularity.conf20211203-167297-15slw3l.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/condor/base.pp, line: 19)
Wrapped exception:
No such file or directory - A directory component in /etc/singularity/singularity.conf20211203-167297-15slw3l.lock does not exist or is a dangling symbolic link
Error: /Stage[main]/Profile::Service::Condor::Base/File[/etc/singularity/singularity.conf]/ensure: change from 'absent' to 'file' failed: Could not set 'file' on ensure: No such file or directory - A directory component in /etc/singularity/singularity.conf20211203-167297-15slw3l.lock does not exist or is a dangling symbolic link (file: /etc/puppetlabs/code/environments/devel/modules/profile/manifests/service/condor/base.pp, line: 19)
Error: Could not find user condor
Info: Unknown failure using insync_values? on type: File[/local/condor] / property: owner to compare values ["condor"] and 0
Error: /Stage[main]/Profile::Service::Condor::Base/File[/local/condor]/owner: change from 'root' to 'condor' failed: Could not find user condor
Error: Could not find group condor
Info: Unknown failure using insync_values? on type: File[/local/condor] / property: group to compare values ["condor"] and 0
Error: /Stage[main]/Profile::Service::Condor::Base/File[/local/condor]/group: change from 'root' to 'condor' failed: Could not find group condor
Notice: /Stage[main]/Profile::Service::Condor::Base/File[/local/condor/execute]: Dependency File[/local/condor] has failures: true
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/local/condor/execute]: Skipping because of failed dependencies
Error: Could not find user condor
Info: Unknown failure using insync_values? on type: File[/var/lib/condor] / property: owner to compare values ["condor"] and 0
Error: /Stage[main]/Profile::Service::Condor::Base/File[/var/lib/condor]/owner: change from 'root' to 'condor' failed: Could not find user condor
Error: Could not find group condor
Info: Unknown failure using insync_values? on type: File[/var/lib/condor] / property: group to compare values ["condor"] and 0
Error: /Stage[main]/Profile::Service::Condor::Base/File[/var/lib/condor]/group: change from 'root' to 'condor' failed: Could not find group condor
Notice: /Stage[main]/Profile::Service::Condor::Base/File[/var/lib/condor/checkpoint]: Dependency File[/var/lib/condor] has failures: true
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/var/lib/condor/checkpoint]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/var/lib/condor/execute]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Base/File[/var/lib/condor/spool]: Skipping because of failed dependencies
Error: Could not find user condor
Info: Unknown failure using insync_values? on type: File[/var/log/condor] / property: owner to compare values ["condor"] and 0
Error: /Stage[main]/Profile::Service::Condor::Base/File[/var/log/condor]/owner: change from 'root' to 'condor' failed: Could not find user condor
Error: Could not find group condor
Info: Unknown failure using insync_values? on type: File[/var/log/condor] / property: group to compare values ["condor"] and 0
Error: /Stage[main]/Profile::Service::Condor::Base/File[/var/log/condor]/group: change from 'root' to 'condor' failed: Could not find group condor
Error: Could not find user condor
Info: Unknown failure using insync_values? on type: File[/var/run/condor] / property: owner to compare values ["condor"] and 0
Error: /Stage[main]/Profile::Service::Condor::Base/File[/var/run/condor]/owner: change from 'root' to 'condor' failed: Could not find user condor
Error: Could not find group condor
Info: Unknown failure using insync_values? on type: File[/var/run/condor] / property: group to compare values ["condor"] and 0
Error: /Stage[main]/Profile::Service::Condor::Base/File[/var/run/condor]/group: change from 'root' to 'condor' failed: Could not find group condor
Error: Cannot create /etc/condor/accounting; parent directory /etc/condor does not exist
Error: /Stage[main]/Profile::Service::Condor::Schedd::Addons/File[/etc/condor/accounting]/ensure: change from 'absent' to 'directory' failed: Cannot create /etc/condor/accounting; parent directory /etc/condor does not exist
Notice: /Stage[main]/Profile::Service::Condor::Schedd::Addons/File[/etc/condor/accounting/valid_tags]: Dependency File[/etc/condor/accounting] has failures: true
Warning: /Stage[main]/Profile::Service::Condor::Schedd::Addons/File[/etc/condor/accounting/valid_tags]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Schedd::Addons/File[/etc/condor/accounting/valid_users]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Schedd::Addons/File[/etc/condor/config.d/30-scratch-mount]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Schedd::Addons/File[/etc/condor/config.d/50-transfer-limits.config]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Schedd::Addons/File[/etc/condor/config.d/50-transfer-limits]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Schedd::Addons/File[/etc/condor/config.d/99-memory]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Schedd::Addons/File[/etc/condor/config.d/99-transform]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Schedd::Addons/Exec[valid_tags]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Schedd::Addons/Exec[valid_users]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Condor::Schedd::Generic/Profile::Service::Condor::Schedd::Base[generic]/File[/etc/condor/config.d/02-scheduler]: Skipping because of failed dependencies
Error: Could not find user condor
Info: Unknown failure using insync_values? on type: File[/etc/systemd/system/condor.service.d/memory.conf] / property: owner to compare values ["condor"] and 0
Error: /Stage[main]/Profile::Service::Condor::Schedd::Generic/Profile::Service::Condor::Schedd::Base[generic]/File[/etc/systemd/system/condor.service.d/memory.conf]/owner: change from 'root' to 'condor' failed: Could not find user condor
Error: Could not find group condor
Info: Unknown failure using insync_values? on type: File[/etc/systemd/system/condor.service.d/memory.conf] / property: group to compare values ["condor"] and 0
Error: /Stage[main]/Profile::Service::Condor::Schedd::Generic/Profile::Service::Condor::Schedd::Base[generic]/File[/etc/systemd/system/condor.service.d/memory.conf]/group: change from 'root' to 'condor' failed: Could not find group condor
Notice: /Stage[main]/Profile::Service::Condor::Base/Exec[condor_reconfig_schedd]: Dependency File[/etc/systemd/system/condor.service.d/memory.conf] has failures: true
Warning: /Stage[main]/Profile::Service::Condor::Base/Exec[condor_reconfig_schedd]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Service::Crond::Base/Service[crond]: Skipping because of failed dependencies
Info: Stage[main]: Unscheduling all events on Stage[main]
Notice: Applied catalog in 84.00 seconds
      • Juan Barayoga
        Juan Barayoga @juan-barayoga ·
        Developer

        Hi @philippe.grassia, I created a data/RedHat/8.yaml file which is a duplicate of data/RedHat.yaml. Update this file to possibly knock some of the warnings off.

      • Please register or sign in to reply
    • Philippe Grassia
      Philippe Grassia @philippe.grassia ·
      Developer

      that is going to help but first I need to fix the include profile::yum::repo::something or other or rather make sure the proper repo templates are populated

    • Philippe Grassia
      Philippe Grassia @philippe.grassia ·
      Developer

      emfollow-test was replaced by a rockylinux 8.5 instance. It is still WIP but it should be sufficiently advanced and stable to allow gwcelery work on it in parallel.

    • Philippe Grassia
      Philippe Grassia @philippe.grassia ·
      Developer

      nearing completion, nagios was fixed, firewalld being tested

    • Stuart Anderson
      Stuart Anderson @stuart.anderson ·
      Maintainer

      @leo-singer I am closing this as done as I think you and @deep.chatterjee where happy enough with the emfollow-test upgrade. However, please feel free to open another ticket regarding ssh-kerberos.

      @philippe.grassia your call on where to track the separate emfollow-playground upgrade.

    • Stuart Anderson closed

      closed

    Please register or sign in to reply