SystemD error during slow CPanel Update
Hi,
I'm an end user with Root access using an Inmotion Hosting Cloud VPS (48 core, 6GB RAM, SSD, PHP7.4, Redis, Zend Opcache, BBC1, CENTOS 7.9 virtuozzo).
WHM CPanel Update to 92.0.3 ran from WHM seemed very slow, taking about 20-30mins. I saw the following errors whilst it seems to wait a long time, I've copied the relevant lines below each error message. Same errors when forcing a CPanel update afterwards to make sure everything is okay. SSH login seems to be slow especially waiting for the root command line to appear, but may be coincidental
Systemctl keeps showing as degraded. Running systemctl daemon-reload usually times out, daemon-reexec works quite quickly though. Current Time: Monday, 30-Nov-2020 23:55:56 GMTRestart Time: Monday, 30-Nov-2020 23:16:27 GMTParent Server Config. Generation: 2Parent Server MPM Generation: 1Server uptime: 39 minutes 29 seconds Server load: 0.78 1.04 1.20Total accesses: 2800 - Total Traffic: 13.0 MB - Total Duration: 534765CPU Usage: u2.59 s45.84 cu4.79 cs5.18 - 2.47% CPU load1.18 requests/sec - 5.6 kB/second - 4876 B/request - 190.988 ms/request1 requests currently being processed, 124 idle workers I'd like to see the errors go away and normal service resumed, if possible! Many thanks.
Use of uninitialized value in string eq at /usr/local/cpanel/Cpanel/RestartSrv/Systemd.pm line 138.
return 0 if $info->{'LoadState'} eq 'not-found';
Use of uninitialized value in string ne at /usr/local/cpanel/Cpanel/Init/Enable/Systemd.pm line 128.
return 0 if $info->{'LoadState'} ne 'loaded';Systemctl keeps showing as degraded. Running systemctl daemon-reload usually times out, daemon-reexec works quite quickly though. Current Time: Monday, 30-Nov-2020 23:55:56 GMTRestart Time: Monday, 30-Nov-2020 23:16:27 GMTParent Server Config. Generation: 2Parent Server MPM Generation: 1Server uptime: 39 minutes 29 seconds Server load: 0.78 1.04 1.20Total accesses: 2800 - Total Traffic: 13.0 MB - Total Duration: 534765CPU Usage: u2.59 s45.84 cu4.79 cs5.18 - 2.47% CPU load1.18 requests/sec - 5.6 kB/second - 4876 B/request - 190.988 ms/request1 requests currently being processed, 124 idle workers I'd like to see the errors go away and normal service resumed, if possible! Many thanks.
-
Hey there! The issues you're experiencing seem like they are at a lower level than the cPanel software, as we don't control systemd. Do you see anything out of the ordinary in /var/log/messages from around the time you experienced the "systemctl daemon-reload" timeouts? 0 -
Hi, Thank you. I just tried daemon-reload again and it timed out, the log shows these messages: Dec 2 09:46:17 vps12345 systemd: Reloading. Dec 2 09:46:24 vps12345 systemd: [/usr/lib/systemd/system/netdata.service:72] Unknown lvalue 'ProtectControlGroups' in section 'Service' Dec 2 09:46:24 vps12345 systemd: [/etc/systemd/system/redis2.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit' Dec 2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis7.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit' Dec 2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis4.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit' Dec 2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit' Dec 2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis5.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit' Dec 2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis6.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit' Dec 2 09:46:25 vps12345 systemd: [/etc/systemd/system/redis3.service:5] Unknown lvalue 'StartLimitIntervalSec' in section 'Unit' Dec 2 09:46:25 vps12345 systemd: [/usr/lib/systemd/system/vzfifo.service:19] Support for option SysVStartPriority= has been removed and it is ignored
systemctl reenable redis.service Removed symlink /etc/systemd/system/multi-user.target.wants/redis.service. Created symlink from /etc/systemd/system/multi-user.target.wants/redis.service to /etc/systemd/system/redis.service. Failed to execute operation: Connection timed out root@vps12345 [~]# systemctl reenable netdata.service Removed symlink /etc/systemd/system/multi-user.target.wants/netdata.service. Created symlink from /etc/systemd/system/multi-user.target.wants/netdata.service to /usr/lib/systemd/system/netdata.service. Failed to execute operation: Connection timed out root@vps12345 [~]# rpm -q systemd systemd-219-78.el7_9.2.x86_64
Thanks.0 -
Thanks for the push in the right direction! I commented out the lines concerned, and after rebooting the server normal service is resumed. I'm not sure where they came from but it seems my version of SystemD is not up to date, so it didn't recognise the commands. I did install Loggly last week so perhaps it was responsible. There are no available updates for my server so I guess it's best to wait until perhaps CentOS 7.9 supplies a later version. I read some advice stating it wasn't advisable to update it manually. Thanks again for the pointer. 0 -
I'm glad things are working well! Since that isn't a tool that's related to cPanel I can't so for sure on the *why* behind the issue or what the cause may have been, but at least we were able to make some good progress! 0 -
Absolutely, I just did a forced CPanel update and completed in 2 mins, not the 20 odd mins like yesterday. :) 0 -
Much better :D 0
Please sign in to leave a comment.
Comments
7 comments