WHM Service Status page shows all services down
-
Hi @H.R. Holmer I actually think this issue may be different than what you're referencing. Besides, that there isn't one such process that obtains the data on the page, it's facilitated by multiple though there are several users which could be related at /var/cpanel/userhomes To confirm can you please run the following and let me know the output: cat /usr/local/cpanel/version
ps faux |grep tailwatch[d]
tail -100 /usr/local/cpanel/logs/tailwatchd_log
I realize it seems unrelated but if what I think is occurring is it's related:/scripts/restartsrv_eximstats --check
Thanks!0 -
Thanks for the reply, @cPanelLauren ... here it is plus OS info and anything that might be a bit different in the setup [QUOTE]cat /usr/local/cpanel/version
11.70.0.51 (fairly typical config other than differences shown below) [QUOTE]ps faux |grep tailwatch[d]
root 86214 0.0 0.0 63868 14828 ? S 04:27 0:08 tailwatchd [QUOTE]/scripts/restartsrv_eximstats --check
The 'tailwatchd' service passed the check: tailwatchd (tailwatchd) is running as root with PID 86214 (systemd+/proc check method). System and other possibly relevant info:- CentOS 7.5-flavored CloudLinux (3.10.0-714.10.2.lve1.5.17.1.el7 which is the newest kernel)
- CageFS not used un-installed (it left behind a mess, but I'm fairly sure it doesn't have any links interfering)
- EA4-provisioned Apache 2.4 (CL packages) EA-PHP56,71,72 only, no other PHP provisioned
- Only (CL's) LSAPI handler provisioned, no PHP-FPM handlers of any kind are provisioned (makes me wonder what cP uses for its PHP-FPM service for cPanel Daemons although I have it disabled -- not the issue here, checked that)
- All CL "hardened" ALT-PHP versions removed and CL's PHP "selector" disabled -- only cP's PHP "selector" deployed
- No actual human clients are on this server - only me - ergo, unlikely to be something "exotic" running that I'm unaware of
[146747] [2018-06-05 00:06:48 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [146753] [2018-06-05 00:06:50 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [146988] [2018-06-05 00:07:10 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [146988] [2018-06-05 00:07:10 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [146988] [2018-06-05 00:07:10 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1528171200 - 1528174800 [146988] [2018-06-05 00:07:10 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [146988] [2018-06-05 00:07:10 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [146988] [2018-06-05 00:07:10 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [146993] [2018-06-05 00:07:13 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [147238] [2018-06-05 00:07:33 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [147238] [2018-06-05 00:07:33 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [147238] [2018-06-05 00:07:33 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1528171200 - 1528174800 [147238] [2018-06-05 00:07:33 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [147238] [2018-06-05 00:07:33 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [147238] [2018-06-05 00:07:33 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [147244] [2018-06-05 00:07:38 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [149041] [2018-06-05 00:09:03 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [149041] [2018-06-05 00:09:03 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [149041] [2018-06-05 00:09:04 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1528171200 - 1528174800 [149041] [2018-06-05 00:09:04 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [149041] [2018-06-05 00:09:04 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [149041] [2018-06-05 00:09:04 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [149049] [2018-06-05 00:09:09 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [2654] [2018-06-09 19:06:57 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [2654] [2018-06-09 19:06:57 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [2654] [2018-06-09 19:06:57 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1528585200 - 1528588800 [2654] [2018-06-09 19:06:57 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [2654] [2018-06-09 19:06:57 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [2654] [2018-06-09 19:06:57 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [3395] [2018-06-09 19:07:02 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [212693] [2018-06-09 23:55:29 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [212693] [2018-06-09 23:55:29 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [212693] [2018-06-09 23:55:29 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1528599600 - 1528603200 [212693] [2018-06-09 23:55:29 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [212693] [2018-06-09 23:55:29 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [212693] [2018-06-09 23:55:29 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [212700] [2018-06-09 23:55:34 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [285620] [2018-06-10 00:45:52 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [285620] [2018-06-10 00:45:52 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [285620] [2018-06-10 00:45:52 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1528603200 - 1528606800 [285620] [2018-06-10 00:45:52 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [285620] [2018-06-10 00:45:52 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [285620] [2018-06-10 00:45:52 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [285631] [2018-06-10 00:45:57 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [354678] [2018-06-16 00:07:09 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [354678] [2018-06-16 00:07:09 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [354678] [2018-06-16 00:07:09 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1529121600 - 1529125200 [354678] [2018-06-16 00:07:09 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [354678] [2018-06-16 00:07:09 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [354678] [2018-06-16 00:07:09 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [354683] [2018-06-16 00:07:14 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [2028125] [2018-06-21 00:06:45 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [2028125] [2018-06-21 00:06:45 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [2028125] [2018-06-21 00:06:45 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1529553600 - 1529557200 [2028125] [2018-06-21 00:06:45 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [2028125] [2018-06-21 00:06:45 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [2028125] [2018-06-21 00:06:45 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [2028130] [2018-06-21 00:06:47 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [2028187] [2018-06-21 00:07:07 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [2028187] [2018-06-21 00:07:07 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [2028187] [2018-06-21 00:07:07 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1529553600 - 1529557200 [2028187] [2018-06-21 00:07:07 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [2028187] [2018-06-21 00:07:07 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [2028187] [2018-06-21 00:07:07 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [2028192] [2018-06-21 00:07:09 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [2028270] [2018-06-21 00:07:29 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [2028270] [2018-06-21 00:07:29 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [2028270] [2018-06-21 00:07:29 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1529553600 - 1529557200 [2028270] [2018-06-21 00:07:29 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [2028270] [2018-06-21 00:07:29 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [2028270] [2018-06-21 00:07:29 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [2028275] [2018-06-21 00:07:34 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [2029189] [2018-06-21 00:08:26 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [2029189] [2018-06-21 00:08:26 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [2029189] [2018-06-21 00:08:26 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1529553600 - 1529557200 [2029189] [2018-06-21 00:08:26 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [2029189] [2018-06-21 00:08:26 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [2029189] [2018-06-21 00:08:26 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [2029194] [2018-06-21 00:08:31 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [2275238] [2018-06-21 17:49:17 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [2275238] [2018-06-21 17:49:17 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [2275238] [2018-06-21 17:49:18 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1529614800 - 1529618400 [2275238] [2018-06-21 17:49:18 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [2275238] [2018-06-21 17:49:18 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [2275238] [2018-06-21 17:49:18 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [2275243] [2018-06-21 17:49:23 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [2616] [2018-06-22 02:05:05 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [2616] [2018-06-22 02:05:05 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [2616] [2018-06-22 02:05:05 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1529647200 - 1529650800 [2616] [2018-06-22 02:05:05 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [2616] [2018-06-22 02:05:05 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [2616] [2018-06-22 02:05:05 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [3213] [2018-06-22 02:05:10 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [86209] [2018-06-22 04:27:08 -0400] [Cpanel::TailWatch] [INFO] Opened /usr/local/cpanel/logs/tailwatchd_log in append mode [86209] [2018-06-22 04:27:08 -0400] [Cpanel::TailWatch] [INFO] inotify enabled. watch file is /var/cpanel/.tailwatchd_inotify_alarm_trick [86209] [2018-06-22 04:27:08 -0400] [Cpanel::TailWatch::Eximstats] Loading email sending limits from 1529654400 - 1529658000 [86209] [2018-06-22 04:27:08 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::JailManager' is not enabled. [86209] [2018-06-22 04:27:08 -0400] [Cpanel::TailWatch] APNSPush not enabled because it requires an Apple certificate and key file to be installed. [86209] [2018-06-22 04:27:08 -0400] [Cpanel::TailWatch] The tailwatchd driver 'Cpanel::TailWatch::ModSecLog' is not enabled. [86214] [2018-06-22 04:27:13 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting.
Yes, ModSecurity is not being run and intentionally un-provisioned. (Creates issues with an already complicated interaction of mod_rewrites, redirects, other .htaccess rules, and symlinks needed for CGI scripts run on multiple cP account websites to share a common data-set, and do a few other tricks... (plus it just feels like more bloat). But I doubt it's related to this issue anyway. Thanks again for the reply here and allowing this thread. I'm still "moderated" and this is the first thing that's been allowed up...LOL. But I do see this same issue in hundreds of search results so I'm hopeful whatever happens to be causing my case is common to many when we publicly resolve it here it will help others. :) Post Script: I think you are right about my issue being different than the one I all related services, but still have the same issue with Service Status as per screenshot following: 530030 -
Hi @H.R. Holmer Thank you so much for the detailed information. Unfortunately, this is proving to be a bit different than what I thought it was originally as well. The one thing I am curious about is why the following message keeps popping up in the tailwatchd logs: [Cpanel::TailWatch] [INFO] Waiting for server to finish booting.
If you compare the timestamp to the reboot times of the server do they match? Basically, I want to know if this error is being shown due to an error indicating that startup didn't complete successfully or if they are occurring because the server did indeed reboot at those times. Also, can you provide the last 100 lines of the chkservd logs:tail -1000 /var/log/chkservd.log
0 -
Hi @cPanelLauren ... Thanks for sticking with us to sort this out. Timestamps for reboots are actual reboots. Yes, interestingly at least one of those reboots was due to seeing that "you must reboot to update" message in WHM which I should not be seeing with cPanel using CL with KernelCare. I have followed this issue and cP says it fixed this in v.68 (after not quite in v.66) but we had been suffering from it almost constantly, past v.68 CL told me they are still seeing that issue frequently and they're not sure cP's fix in v.68 resolved it. I realize that's the opinion of one person at CL in support, and they might share some blame in the problem, a bit, so I'm not casting aspersions here -- just pointing out that I think it's still a possible issue -- not sure if it's related to ours because... Yes, recently we did specify the kernel for the OS start using the correct grub2 command to set the 0 kernel as default, and we have indeed fully updated the kernel -- but that only fixed (removed) the message in WHM "you need to reboot" We shall see how long it stays "fixed" ... I'm hopeful, but our updating the kernel, and verifying running matches boot, has not helped with the issue of Service Status continuing to show all down. :-( I appreciate you continuing to work with me on this. Here are the most recent lines of checkservd logs using the command you supplied (but 100 lines, not 1000 lines): # tail -100 /var/log/chkservd.log tail -100 /var/log/chkservd.log Service Check Finished Service Check Started Loading services .....apache_php_fpm....clamd....cpanel_php_fpm....cpanellogd....cpdavd....cphulkd....cpsrvd....crond....dnsadmin....exim....ftpd....httpd....imap....ipaliases....lfd....lmtp....mailman....mysql....named....nscd....p0f....pop....queueprocd....rsyslogd....spamd....sshd..Done [2018-05-27 04:14:49 -0400] Disk check .... / (/) [20.93%] ... /backup (/backup) [18.98%] ... /boot (/boot) [73.41%] ... /tmp (/tmp) [5.79%] ... /var/tmp (/var/tmp) [5.79%] ... {status:ok} ... Done [2018-05-27 04:14:49 -0400] OOM check ....Done [2018-05-27 04:14:49 -0400] Service check .... queueprocd [[check command:+][socket connect:N/A]]... sshd [[check command:+][socket connect:N/A]]... spamd [[check command:+][socket connect:N/A]]... rsyslogd [[check command:+][socket connect:N/A]]... pop [[check command:+][socket connect:+]]... p0f [[check command:+][socket connect:N/A]]... nscd [[check command:+][socket connect:N/A]]... named [[check command:+][socket connect:N/A]]... mysql [[check command:+][socket connect:N/A]]... mailman [[check command:+][socket connect:N/A]]... lmtp [[check command:+][socket connect:+]]... lfd [[check command:+][socket connect:N/A]]... ipaliases [[check command:+][socket connect:N/A]]... imap [[socket_service_auth:1][check command:+][socket connect:+]]... httpd [[check command:N/A][socket connect:+]]... ftpd [[socket_service_auth:1][check command:+][socket connect:+]]... exim [[check command:+][socket connect:+]]... dnsadmin [[http_service_auth:1][check command:+][socket connect:+]]... crond [[check command:+][socket connect:N/A]]... cpsrvd [[http_service_auth:1][check command:N/A][socket connect:+]]... cphulkd [[check command:+][socket connect:+]]... cpgreylistd [[check command:N/A][socket connect:N/A]]... cpdavd [[http_service_auth:1][check command:+][socket connect:+]]... cpanellogd [[check command:+][socket connect:N/A]]... cpanel_php_fpm [[check command:+][socket connect:N/A]]... clamd [[check command:+][socket connect:N/A]]... apache_php_fpm [[check command:+][socket connect:N/A]]...Done Service Check Finished Service Check Started Loading services .....apache_php_fpm....clamd....cpanel_php_fpm....cpanellogd....cpdavd....cphulkd....cpsrvd....crond....dnsadmin....exim....ftpd....httpd....imap....ipaliases....lfd....lmtp....mailman....mysql....named....nscd....p0f....pop....queueprocd....rsyslogd....spamd....sshd..Done [2018-05-27 04:20:30 -0400] Disk check .... / (/) [20.99%] ... /var/tmp (/var/tmp) [6.04%] ... /backup (/backup) [18.98%] ... /boot (/boot) [80.09%] ... /tmp (/tmp) [6.04%] ... {status:ok} ... Done [2018-05-27 04:20:30 -0400] OOM check ....Done [2018-05-27 04:20:30 -0400] Service check .... queueprocd [[check command:+][socket connect:N/A]]... sshd [[check command:+][socket connect:N/A]]... spamd [[check command:+][socket connect:N/A]]... rsyslogd [[check command:+][socket connect:N/A]]... pop [[check command:+][socket connect:+]]... p0f [[check command:+][socket connect:N/A]]... nscd [[check command:+][socket connect:N/A]]... named [[check command:+][socket connect:N/A]]... mysql [[check command:+][socket connect:N/A]]... mailman [[check command:+][socket connect:N/A]]... lmtp [[check command:+][socket connect:+]]... lfd [[check command:+][socket connect:N/A]]... ipaliases [[check command:+][socket connect:N/A]]... imap [[socket_service_auth:1][check command:+][socket connect:+]]... httpd [[check command:N/A][socket connect:+]]... ftpd [[socket_service_auth:1][check command:+][socket connect:+]]... exim [[check command:+][socket connect:+]]... dnsadmin [[http_service_auth:1][check command:+][socket connect:+]]... crond [[check command:+][socket connect:N/A]]... cpsrvd [[http_service_auth:1][check command:N/A][socket connect:+]]... cphulkd [[check command:+][socket connect:+]]... cpgreylistd [[check command:N/A][socket connect:N/A]]... cpdavd [[http_service_auth:1][check command:+][socket connect:+]]... cpanellogd [[check command:+][socket connect:N/A]]... cpanel_php_fpm [[check command:+][socket connect:N/A]]... clamd [[check command:+][socket connect:N/A]]... apache_php_fpm [[check command:+][socket connect:N/A]]...Done Service Check Finished Service Check Started Loading services .....apache_php_fpm....clamd....cpanel_php_fpm....cpanellogd....cpdavd....cphulkd....cpsrvd....crond....dnsadmin....exim....ftpd....httpd....imap....ipaliases....lfd....lmtp....mailman....mysql....named....nscd....p0f....pop....queueprocd....rsyslogd....spamd....sshd..Done [2018-05-27 04:25:56 -0400] Disk check .... / (/) [20.99%] ... /backup (/backup) [18.98%] ... /boot (/boot) [74.31%] ... /tmp (/tmp) [11.83%] ... /var/tmp (/var/tmp) [11.83%] ... {status:ok} ... Done [2018-05-27 04:25:56 -0400] OOM check ....Done [2018-05-27 04:25:56 -0400] Service check .... queueprocd [[check command:+][socket connect:N/A]]... sshd [[check command:+][socket connect:N/A]]... spamd [[check command:+][socket connect:N/A]]... rsyslogd [[check command:+][socket connect:N/A]]... pop [[check command:+][socket connect:+]]... p0f [[check command:+][socket connect:N/A]]... nscd [[check command:+][socket connect:N/A]]... named [[check command:+][socket connect:N/A]]... mysql [[check command:+][socket connect:N/A]]... mailman [[check command:+][socket connect:N/A]]... lmtp [[check command:+][socket connect:+]]... lfd [[check command:+][socket connect:N/A]]... ipaliases [[check command:+][socket connect:N/A]]... imap [[socket_service_auth:1][check command:+][socket connect:+]]... httpd [[check command:N/A][socket connect:+]]... ftpd [[socket_service_auth:1][check command:+][socket connect:+]]... exim [[check command:+][socket connect:+]]... dnsadmin [[http_service_auth:1][check command:+][socket connect:+]]... crond [[check command:+][socket connect:N/A]]... cpsrvd [[http_service_auth:1][check command:N/A][socket connect:+]]... cphulkd [[check command:+][socket connect:+]]... cpgreylistd [[check command:N/A][socket connect:N/A]]... cpdavd [[http_service_auth:1][check command:+][socket connect:+]]... cpanellogd [[check command:+][socket connect:N/A]]... cpanel_php_fpm [[check command:+][socket connect:N/A]]... clamd [[check command:+][socket connect:N/A]]... apache_php_fpm [[check command:+][socket connect:N/A]]...Done Service Check Finished
0 -
Hi @H.R. Holmer I'm happy to help as best I can. but that only fixed (removed) the message in WHM "you need to reboot" We shall see how long it stays "fixed"
Definitely let us know if that issue re-occurs. I'm sorry it wasn't fixed for you in 68 as well From the output of the chkservd logs, all service checks are completing normally. I would normally expect to see them failing at some point. Can you tell me what the output of the following is:/scripts/restartsrv_chkservd
As well as:ps faux |grep -i fpm
You'd mentioned that you weren't running php-fpm and I just wanted to make sure Though I am beginning to wonder if the issue is related to aforementioned CageFS issues you were experiencing with the disabling of CageFS.0 -
# /scripts/restartsrv_chkservd # /scripts/restartsrv_chkservd Waiting for "tailwatchd" to restart "waiting for "tailwatchd" to initialize "finished. Service Status tailwatchd (tailwatchd) is running as root with PID 3473578 (systemd+/proc check method). Startup Log Jun 26 17:02:01 dude1.example.com systemd[1]: Starting tailwatchd... Jun 26 17:02:01 dude1.example.com restartsrv_tailwatchd[3473564]: [Tue Jun 26 17:02:01 2018] Starting /usr/local/cpanel/libexec/tailwatch/tailwatchd daemon Jun 26 17:02:01 dude1.example.com restartsrv_tailwatchd[3473564]: Log is at /usr/local/cpanel/logs/tailwatchd_log Jun 26 17:02:01 dude1.example.com systemd[1]: tailwatchd.service: Supervising process 3473578 which is not our child. We'll most likely not notice when it exits. Jun 26 17:02:01 dude1.example.com systemd[1]: Started tailwatchd. tailwatchd restarted successfully.
# ps faux |grep -i fpm root 3474062 0.0 0.0 114752 1012 pts/0 S+ 17:03 0:00 | \_ grep --color=auto -i fpm Re: FPM, I have no PHP-FPM packages running -- nothing PHP-FPM is even compiled into Apache 2.4 using EA24. And I have even removed the CloudLinux "hardened" ALT-PHP options and hidden the CL PHP-selector. This was done because CL has said they only created those ALT-PHP versions until such time as cPanel had its EA-PHP versions "hardened" and CL is now satisfied this is the case. They say there should be no problems running EA-PHP with their LiteSpeed API handler (LSAPI), and that's what I have enabled - EA-PHP56, 71, and 72 (default) only, and in WHM they are set to use only LSAPI as supplied by CL. It works great, it's extremely fast, extremely stable, and I have no problems. Ergo, I have everything PHP-FPM disabled, including in cP's Service Manager where I have disabled (unchecked): PHP-FPM for cPanel Daemons. But then it is not clear to me where cPanel would get its PHP-FPM handlers to run the above service for its own daemons if I don't have anything PHP-FPM provisioned. Does cPanel have its own, secret, internal PHP-FPM handlers? This may be off-topic (although it is related to what you've asked today) but... cPanel's Multi-PHP Manager in WHM is a total and complete NAG about PHP-FPM as if it is some kind of magic formula for speed. It most certainly is not such a thing, nor is it desirable versus LSAPI in any context or fashion I can imagine after doing my homework before choosing -- the FPM handler loses head-to-head in speed, stability, security, flexibility, resource consumption, processes spawned, promptness of process termination, and about any other measure you can find online to compare. I just don't get it. Why the PRESSURE to install PHP-FPM? Look at how this page treats you if you don't have it installed -- it's worse than the ModSecurity shaming... seriously! :) Help me with this one, will you? Anyway, I have been trying and trying and trying to figure out where does cP get the PHP-FPM handlers to run its own FPM service for its own Daemons when a user like me has disabled FPM everywhere he can, and runs LSAPI for reasons very good and valid (and well-documented online)? Can you tell me where that is? 1. Where does cP hide its own PHP version? and 2. What PHP version is it currently? and 3. Where does cP hide its FPM handlers? and 4. If I don't want any problems with running both FPM and LSAPI, shouldn't I have cP's FPM shut off? (especially me, since I have no actual "users" for my accounts, other than me, and a friend or two to whom I give free WordPress hosting that I throttle heavily with CL's LVE) And, (QUESTION) Can I please please please shut off the nagging and shaming in WHM's Multi-PHP Manager about how I durned-well-oughtabe using PHP-FPM since it's the best thing since sliced bread and four-wheel brakes... Sheesh! No, it is NOT that great. And given how close you cooperate with CL, I'm kinda surprised you push it so hard. How about giving me a way to tell that stupid module to "back off with the PHP-FPM, since I found someone else..." (LOL) :-D Anywho, those are the settings, and slight pontification in response to cPanel's PHP-handler-shaming of anyone not running PHP-FPM in that Multi-PHP Manager module. Is it that important to you that your customers run a specific PHP-handler? Really, I just do not get it... OK, thanks again for helping me. :) Mystery continues. Screenshot shows REBOOT NAGGING is back, but this type of nagging is expected. Just shows a new version is out, and if I don't want to wait for KernelCare to handle the upgrade, I can reboot into it. And this type of reboot-nagging isn't a problem. It's the other message that drives us all crazy. The one telling us (essentially) that our running kernel and grub2-default kernel do not match... even if we have KernelCare and theoretically we should never again even NEED a bootloader... 53075 P.S. /var/run/chkservd/ is empty /var/run/chkservd.services_suspend/ has these files: # dir cpbandwd cpdavd-ssl cpgreylistd dovecot exim eximstats mailman mysql spamd tailwatchd Seems not right to me... There is not process running at this time which would suspend tailwatchd or checkservd (that I know of).0 -
Hi @H.R. Holmer This might be the most important information in regards to the issue in the response, I'm glad you included it. P.S. /var/run/chkservd/ is empty
This being empty indicates that the services aren't being monitored or the system doesn't believe they are. If you got to WHM>>Service Configuration>>Service Manager can you show me a screenshot of what you see? One of each of the sections might be helpful In regard to this:/var/run/chkservd.services_suspend/ has these files: # dir cpbandwd cpdavd-ssl cpgreylistd dovecot exim eximstats mailman mysql spamd tailwatchd
That's pretty normal mine does as well:[root@server chkservd]# ls -lah /var/run/chkservd.services_suspend/ total 44K drwx------ 2 root root 260 Jun 27 09:26 . drwxr-xr-x 36 root root 1.9K Jun 27 09:40 .. -rw------- 1 root root 10 Jun 26 02:32 cpbandwd -rw------- 1 root root 10 Jun 26 02:32 cpdavd-ssl -rw------- 1 root root 10 Jun 27 02:34 cpgreylistd -rw------- 1 root root 10 Jun 27 09:25 dovecot -rw------- 1 root root 10 Jun 27 09:25 exim -rw------- 1 root root 10 Jun 26 02:32 eximstats -rw------- 1 root root 10 Jun 26 02:32 httpd -rw------- 1 root root 10 Jun 27 09:25 mailman -rw------- 1 root root 10 Jun 26 02:32 mysql -rw------- 1 root root 10 Jun 27 02:34 spamd -rw------- 1 root root 10 Jun 27 09:26 tailwatchd
I'm working on more information for you for the php-fpm questions and I'll update again with that. Thanks!0 -
Hi @H.R. Holmer As far as cPanel PHP-FPM because of it's superiority of other handlers in terms of speed we do place emphasis on it. This may be off-topic (although it is related to what you've asked today) but... cPanel's Multi-PHP Manager in WHM is a total and complete NAG about PHP-FPM as if it is some kind of magic formula for speed. It most certainly is not such a thing, nor is it desirable versus LSAPI in any context or fashion I can imagine after doing my homework before choosing -- the FPM handler loses head-to-head in speed, stability, security, flexibility, resource consumption, processes spawned, promptness of process termination, and about any other measure you can find online to compare. I just don't get it. Why the PRESSURE to install PHP-FPM? Look at how this page treats you if you don't have it installed -- it's worse than the ModSecurity shaming... seriously! :) Help me with this one, will you?
I discussed this with the Product Owner for the team responsible for this and showed him your screenshot - which resulted in him opening an internal case on this HB-3853 (there's no CPANEL case#) to have it looked into as an issue as he didn't believe that the nag should be there if you don't have PHP-FPM enabled. On top of that he let me know that we are working on a lot of changes one of which would include offering mod_lsapi which would allow you to switch from php-fpm to LSAPI and not see notifications about php-fpm - the issue being on systems without CL or without litespeed php-fpm IS the best handler out there for them which is why it's pushed.Anyway, I have been trying and trying and trying to figure out where does cP get the PHP-FPM handlers to run its own FPM service for its own Daemons when a user like me has disabled FPM everywhere he can, and runs LSAPI for reasons very good and valid (and well-documented online)?
If you're running PHP-FPM for cPanel services and you don't want to, you can disable it at WHM>>Service Configuration>>Service Manager I should point out that cPanel runs it's own php and it's completely separate from the sites php. The only real documentation on this is here: Service Manager - Version 72 Documentation - cPanel Documentation1. Where does cP hide its own PHP version?
. What PHP version is it currently?
The answer for both of these is cPanel's PHP version information is here:/usr/local/cpanel/3rdparty/php/56
. Where does cP hide its FPM handlers?
The configuration for cPanel's PHP-FPM can be found in/var/cpanel/php-fpm /var/cpanel/php-fpm.d
. If I don't want any problems with running both FPM and LSAPI, shouldn't I have cP's FPM shut off?
No, it shouldn't matter since it's separate from anything running on live sites.And, (QUESTION) Can I please please please shut off the nagging and shaming in WHM's Multi-PHP Manager about how I durned-well-oughtabe using PHP-FPM since it's the best thing since sliced bread and four-wheel brakes... Sheesh! No, it is NOT that great. And given how close you cooperate with CL, I'm kinda surprised you push it so hard. How about giving me a way to tell that stupid module to "back off with the PHP-FPM, since I found someone else..." (LOL) :-D
I just kept this in here because it made me laugh - I hope my answers above helped answer this for you. :-D:-D:-DMystery continues. Screenshot shows REBOOT NAGGING is back, but this type of nagging is expected. Just shows a new version is out, and if I don't want to wait for KernelCare to handle the upgrade, I can reboot into it. And this type of reboot-nagging isn't a problem. It's the other message that drives us all crazy. The one telling us (essentially) that our running kernel and grub2-default kernel do not match... even if we have KernelCare and theoretically we should never again even NEED a bootloader...
So that nagging that you're showing me doesn't indicate a reboot nag that's an update notification just letting you know that cPanel's ready to update - not that you need to reboot. Is there another one?0 -
Thanks for that, @cPanelLauren ... You're wonderful! I'm so pleased to have your help here, and I think we're both finding that maybe my question(s), and all that it (they) clearly now encompass, wasn't so dumb after all... First quick question and appeal for help -- I'm sandboxed into moderated status and that keeps me from adjusting posts I've made. Kind of a security issue since this is a public thread and I noticed my servername went out on this public thread by accent, and I couldn't go to Tools >> Edit after posting in this forum to change the servername in the reports to something generic like: myob1.my-server-name-here.tld instead of the actual servername. Now, after they've been accepted by moderator, I can't get in to them at all to edit/adjust them. When can I get un-sandboxed to be able to fix that? It's not a huge issue, but I really don't think anybody should have their servername on public display here since something in the thread could indicate a security hole, and then the bad actor would know right where it is. Can I please please please be un-moderated at this point, at least so I can fix that? (I will resist throwing any tantrums -- at least nothing worse than my kind and reasonable pleading about the PHP-FPM nagging ... LOL.) Also I would like to adjust something in an above post to remove my public identification of servername in this case. Can I do that if you un-sandbox me? :) This being empty indicates that the services aren't being monitored or the system doesn't believe they are. If you got to WHM>>Service Configuration>>Service Manager can you show me a screenshot of what you see? One of each of the sections might be helpful
I'm attaching screenshot of Service Manager at bottom of this post. Reminder that: 1. cP's internal PHP-FPM is off and, ergo, un-monitored (turning it on does -not- fix this problem). Probably after researching the information you gave me on it, and verifying with people much smarter than me that there will be no interference with our own EA-PHP/LSAPI setup for sites, we will turn it on again. Not that we need it, since nobody is ever on cPanel (as in non-WHM) interfaces other than me, and only very occasionally... 2. JailManager is not enabled, nor desired, as we like our free-to-roam linking, and we have nobody else but us on this server so that's fine; also it's in Beta, maybe even Alpha. 3. ModSecLog is not eneabled, since we have ModSecurity neither installed nor even provisioned in our EA package. Reason covered before -- it's just likely to create more problems with some already touchy Rewrites/Redirects if we don't get its rules perfectly written, plus we're just not convinced we need it. We think it likely fruitless, and potentially corrupting to provision it and activating it just to see if this fixes this problem, and we can't imagine how it would be related, unless we need to somehow better inform cPanel to stop looking for it. (Do we?) This is another module cPanel almost nags about. The folks at CSF/LFD just love ModSec and go into severe panic red warnings on their reports if you don't run it... :) That's almost amusing... ;-)I just kept this in here because it made me laugh - I hope my answers above helped answer this for you. :-D:-D:-D
Yes, very helpful and I am pleased to see your "Product Owner" also agrees the PHP-FPM nag is a bit much. Here's something to remind them: LSAPI is Open-Source on its PHP-Handler and those LSAPI versions (not only the CL version) should work equally well with EA-PHP. No reason cPanel can't have those open-source LSAPI handlers in its repositories to choose as a PHP-FPM alternative, especially given its generally agreed superior performance. You could nag (ahem, "recommend") that users choose one or the other... hehe :)So that nagging that you're showing me doesn't indicate a reboot nag that's an update notification just letting you know that cPanel's ready to update - not that you need to reboot. Is there another one?
No more version nagging about kernel reboots -- only the (normal) standard notification that an update is available in case somebody wants it right away, and that notification (rightly) stays there until KernelCare does its update, after which we are on the newest. For example, no longer is it "72.0.5" on the notification ... now it says: Version "72.0.7" is available. Update Now And that's fine, and we just leave it there since there's nothing in that update we're desperate for and can't wait until CL/KernelCare does its thing and automatically moves us. So all is well with that. So far... hehe Screenshot mentioned in first answer: 53123 And Service Status still down: 53127 Thanks, again, @cPanelLauren -- looking forward to those updates on the Multi-PHP Manager in WHM page... Just learning that alone was worth the price of admission... Does our ServiceManager screenshot help any?0 -
I'm sandboxed into moderated status and that keeps me from adjusting posts I've made
New users are restricted until they become more active. You should be able to edit going forward. Keep in mind there is a limit on the time that you can edit your posts after they have been made as well.0 -
Thank you whoever deleted my post and screenshot regarding the PHP-FPM nag messages. I guess some double standard makes it OK for one person to repeatedly mention it in this thread and post screen shots displaying it, but not for someone else to support and concur with how ridiculous and unnecessary it is. I guess you will also be deleting this message as well. Don't worry, I wont be back now I understand how unwelcome any sort of critique is received by cPanel staff. 0 -
I removed it. If you've got an issue to discuss, please feel free to start a new thread of your own to discuss it. There is no need to add on to someone else's thread about an issue they're having. 0 -
New users are restricted until they become more active. You should be able to edit going forward. Keep in mind there is a limit on the time that you can edit your posts after they have been made as well.
Many thanks, @Infopro ... Unfortunately the time has already passed in which I could have edited the post revealing more information about my server than desired. Can you assist me to delete (or edit) that portion of post #7 from this thread -- it's the tailwatchd log printout showing the servername under Startup Log within the code. Very top of the post under: # /scripts/restartsrv_chkservd Five instances of my servier shown. Whole code section could be deleted if it's easier. Shouldn't matter since it wasn't at issue, but probably I'm thinking no need to delete whole post or thread loses some cogency. I'd be thrilled if my servername was simply replaced with: test1.my-server-name-here.tld ...since that's how I would have likely hidden it had I noticed in time before clicking Post Reply. Yep, my fault, totally. I admit: I should have paid closer attention when posting, but discovered the error seconds after doing so, but could not edit. Still, seems like I should have been able to at least edit a not-yet-published post if it's moderated... If it's moderated, where is the harm in my adjusting it before a moderator even sees it? Just a thought. Thanks again for any adjustments you made to my account to let me edit. I think the sensibility of this topic, and questions posed (including tertiary but important issues addressed, such as which internal PHP and PHP-FPM handlers cPanel uses internally that it does not document) may perhaps show that I'm not a complete newbie idiot wasting time here... and my core/primary situation is not only legitimate, but can be found all over a web search as experienced by others to varying degrees... I won't waste anybody's time, duplicate topics, screw around, or cause problems if you'll please un-sandbox me. I am grateful, and a big fan of cPanel... hope I can contribute here, and in the future I'll be less wordy than this post... :)0 -
Your post has been edited. About your other questions about edits, it has to do with you being a new user. There"s a long running thread in the forum lounge asking about this sort of thing. ;) 0 -
I think we're both finding that maybe my question(s), and all that it (they) clearly now encompass, wasn't so dumb after all...
I never said anything about them being dumb, actually I think they were great questions!!! :)Can I please please please be un-moderated at this point, at least so I can fix that?
Hopefully @Infopro was able to answer those questions for you, he manages the forums as a whole and is definitely the best source of information for that kind of stuff. cP's internal PHP-FPM is off and, ergo, un-monitored (turning it on does -not- fix this problem). Probably after researching the information you gave me on it, and verifying with people much smarter than me that there will be no interference with our own EA-PHP/LSAPI setup for sites, we will turn it on again. Not that we need it, since nobody is ever on cPanel (as in non-WHM) interfaces other than me, and only very occasionally...
Yea your screenshot made us aware of this, the PO of that team wasn't aware this was occurring. Just waiting on updates to that case at this point.. JailManager is not enabled, nor desired, as we like our free-to-roam linking, and we have nobody else but us on this server so that's fine; also it's in Beta, maybe even Alpha.
. ModSecLog is not eneabled, since we have ModSecurity neither installed nor even provisioned in our EA package. Reason covered before -- it's just likely to create more problems with some already touchy Rewrites/Redirects if we don't get its rules perfectly written, plus we're just not convinced we need it. We think it likely fruitless, and potentially corrupting to provision it and activating it just to see if this fixes this problem, and we can't imagine how it would be related, unless we need to somehow better inform cPanel to stop looking for it. (Do we?) This is another module cPanel almost nags about. The folks at CSF/LFD just love ModSec and go into severe panic red warnings on their reports if you don't run it... :) That's almost amusing... ;-)
For these two, I don't think they're related to either issue?Here's something to remind them: LSAPI is Open-Source on its PHP-Handler and those LSAPI versions (not only the CL version) should work equally well with EA-PHP. No reason cPanel can't have those open-source LSAPI handlers in its repositories to choose as a PHP-FPM alternative, especially given its generally agreed superior performance. You could nag (ahem, "recommend") that users choose one or the other... hehe :)
Yea the big reason this is happening is that we don't offer mod_lsapi right now, it's available through CloudLinux and if you have litespeed but it's not specifically offered by cPanel. I believe the end result the PO wanted to see was a one or the other situation so it should handle both, nag and offer LSAPIAnd that's fine, and we just leave it there since there's nothing in that update we're desperate for and can't wait until CL/KernelCare does its thing and automatically moves us. So all is well with that. So far... hehe
So that notification is for cPanel update, meaning there's an update avaialable and has nothing to do with KernelCare or CL - it will go away when cPanel itself updates and no reboot is required for that.Yep, my fault, totally. I admit: I should have paid closer attention when posting, but discovered the error seconds after doing so, but could not edit.
I'm going to take some blame on this as well, I should have noticed the hostname in the response and I didn't, I can edit these for you, I'm sorry I didn't notice. For the "all services reporting down" stuff, I'm out of ideas without being able to access the server, could you please open a ticket with us using the link in my signature? Once it's opened let us know the ticket ID here and I'll follow up with updates here. Thanks!0 -
Thanks for the reply @cPanelLauren I will open a ticket at this point, but I appreciate the help. And, yes, in post #7 please go into the code section I submitted under Startup Log section and do an edit to change our hostname to something generic-looking like: test1.our-domain-name-here.tld so it might look like this instead: Startup Log Jun 26 17:02:01 test1.our-domain-name.tld systemd[1]: Starting tailwatchd... Jun 26 17:02:01 test1.our-domain-name.tld restartsrv_tailwatchd[3473564]: [Tue Jun 26 17:02:01 2018] Starting /usr/local/cpanel/libexec/tailwatch/tailwatchd daemon Jun 26 17:02:01 test1.our-domain-name.tld restartsrv_tailwatchd[3473564]: Log is at /usr/local/cpanel/logs/tailwatchd_log Jun 26 17:02:01 test1.our-domain-name.tld systemd[1]: tailwatchd.service: Supervising process 3473578 which is not our child. We'll most likely not notice when it exits. Jun 26 17:02:01 test1.our-domain-name.tld systemd[1]: Started tailwatchd.
It looks like some of them were edited but not all. Many thanks again! A pleasure meeting and conversing with you. :-) We appreciate cPanel and it attempts to accommodate such a wide range of server configurations and user options, and still provide a large number of features. We know this is a massive endeavor that needs constant curating and updating, and we see the whining (we're done some ourselves... LOL), but we're happy to have this broad-scope, robust tool. Cheers! -Holmer0 -
Hi @H.R. Holmer And, yes, in post #7 please go into the code section I submitted under Startup Log section and do an edit to change our hostname to something generic-looking like: test1.our-domain-name-here.tld so it might look like this instead:
Looks like @Infopro beat me to it :)A pleasure meeting and conversing with you. :)
It's been a pleasure conversing with you as well, I just wish I was able to identify what specifically was going on with your system.We appreciate cPanel and it attempts to accommodate such a wide range of server configurations and user options, and still provide a large number of features. We know this is a massive endeavor that needs constant curating and updating, and we see the whining (we're done some ourselves... LOL), but we're happy to have this broad-scope, robust tool.
We do try and listen to what our clients want in the product so please do continue with constructive criticism and suggestions to improve, we're constantly evolving! When you do open a ticket please share the Ticket ID here as well, thank you!0 -
When you do open a ticket please share the Ticket ID here as well, thank you!
Hi @cPanelLauren ... My experiences with cPanel keep getting better and better. Ticket ID# 9821845 was answered within minutes, and first looked over by cP's @SuedeY and then taken over by cP's Kevin Mitsch, who quickly traced the solution -- based on your having noticed this: [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. Here is the solution for those following or finding this thread in the future, and it takes the matter in a different direction:Regarding the issue shown on [WHM Home > Server Status > Service Status]. The tailwatchd log entries which Lauren noted in the forums post are significant: ==================== [09:44:08 server1 root@9821845 ~]cPs# grep -i 'waiting for server' /usr/local/cpanel/logs/tailwatchd_log | tail -n5 [708555] [2018-06-29 00:07:08 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [708718] [2018-06-29 00:07:33 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [709885] [2018-06-29 00:08:30 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [1702460] [2018-06-30 06:46:14 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. [1705113] [2018-06-30 06:50:09 -0400] [Cpanel::TailWatch] [INFO] Waiting for server to finish booting. ==================== They relate to a known issue, CPANEL-20031. As the log entries suggest, the daemon responsible for running service checks thinks the server is still booting. This is due to the systemd multi-user.target appearing in an inactive state: ==================== [09:39:35 server1 root@9821845 ~]cPs# systemctl status multi-user.target " multi-user.target - Multi-User System Loaded: loaded (/lib/systemd/system/multi-user.target; enabled; vendor preset: disabled) Active: inactive (dead) Docs: man:systemd.special(7) ==================== Which is the result of the following plymouth login services, waiting on user input: ==================== [09:44:01 server1 root@9821845 ~]cPs# systemctl --state=waiting UNIT LOAD ACTIVE SUB DESCRIPTION systemd-ask-password-plymouth.path loaded active waiting Forward Password Requests to Plymouth Directory Watch systemd-ask-password-wall.path loaded active waiting Forward Password Requests to Wall Directory Watch ==================== If you were to remotely connect to the server's GUI, I think you would find that it is waiting at a login screen. This is interfering with the startup of some cPanel services. cPanel does not support the use of Desktop Environments such as KDE and MATE. I've seen customers utilize Desktop Environments successfully, but if you wish to continue using one, this problem will need to be addressed. Logging in may provide a temporary resolution and bring chkservd and the Service Status page back to a working state. My recommendation is to remove the Desktop Environment entirely or at least to reconfigure it so the login/waiting state is avoided during boot. Please review the information provided and let me know if I can offer any further assistance. Regards, -- Kevin Mitsch cPanel, Inc. Technical Analyst II / Migrations Specialist
Probably there would need to be a new thread opened to solve this issue, and it's not really cPanel's problem to solve, meaning I would need to find my own solution. I have run a remote MATE desktop (over X2go) for a long time on this server, and the server prior to this one from which I upgrated/migrated. I didn't have this issue before, and the outage doesn't seem to coincide with when I recently upgraded X2go to its newest version on server (and client), and removed MATE, which was crashy and worried me, and instead installing a very stripped-down version of KDE Plasma in its place, which is very, very stable. But we do know the problem, and I would invite solutions if it's not a capital offense to ask for them in this thread... There's no clash between the KDE and cPanel for the very few things I do in KDE, such as running its Dolphin file manager, its file-indexer, and UltraEdit, all so that I can easily and graphically do a Find-and-Replace-within-Files using PCRE RegEx for filemask and/or the search/replace function -- and I only really do it within files located in the directories /home/cpacctname/public_html and I'm not messing with system files. Ergo, to fix the problem in my instance, and this might assist others resolving incompatibility... All that I really need to do is kill these two (listening) processes each time I exit KDE ... systemd-ask-password-plymouth.path systemd-ask-password-wall.path And then the running service will not think the server is still booting, waiting for a login, and it will start fully and I will have back my monitoring. Only occasionally do I use that KDE Desktop, and I may use it for a few hours or days to do a big Find/Replace-in-Files job, and then I get out of it. So those listeners have no purpose -- the other ways I access my server -- SFTP over SSH port using WinSCP, and SSH terminal using PuTTY client -- do not require those listeners, nor does cPanel (can you confirm this?) so it should just be a matter of killing off those processes when exiting my KDE, and -poof-, it's settled, and there should be no issues of conflict going forward. I am unsure of the best way to kill off these processes, although I can think of a few things:- Set up a script that triggers at KDE logoff to kill those 2 processes.
- Manually activate a script from within KDE that kills those 2 processes, then logs off, and that will be my logoff method.
- Set up something in my X2go client (windows version, but shouldn't matter) that sends the server the kill instructions when connection is terminated or logoff is seen by the connected client.
- Set up something in my X2go server that kills those 2 processes when it sees a failed connection.
- Set up a script that I manually run via PuTTY or my SFTP over SSH when I'm done with a KDE session that kills these processes (requires remembering to do this each time).
- Disable these two processes entirely -- I am pretty sure X2go doesn't need them to start up KDE from the client -- since it is used to dealing with desktop crashes and un-restarted processes all the time, and it will find the X2go server which will call and start the desktop even with these processes down -- and restart those processes IF it needs them.
- (Least desirable) Set a cron job to kill these two processes every X number of hours.
0 -
Hi @H.R. Holmer I'm really happy that our analysts were able to help you with this issue. The case Kevin linked has been marked as By Design, meaning that we don't intend for you to be running a Desktop UI on servers with cPanel installed. Ultimately though the resolution seems to be to finish the boot process, I not sure if you do have to kill the processes, it looks like you may need to log into the UI and allow boot to complete, at least per the workaround noted in the case this was the solution. In the example from the case, it was a gnome UI that needed the license accepted: [QUOTE]If the cause is the one outlined in this ticket, the user can complete the license acceptance at the console, and afterwards service checks will work normally. If there is some other reason behind multi-user.target showing as inactive, a server reboot might fix it.
Other than that I don't know of the "best" way to handle this since it's not necessarily a configuration we support. If you've narrowed it down to the two processes here and that is all there is: [QUOTE]systemd-ask-password-plymouth.path systemd-ask-password-wall.path
I think the best suggestion you had would be: [QUOTE]- Set up a script that triggers at KDE logoff to kill those 2 processes.
That way you can ensure that logoff was successful before killing those processes. Though, I wonder if it will affect subsequent logons.0 -
Thanks again, @cPanelLauren ... Unfortunately completing the login doesn't resolve this... And those processes are unneeded by my X2go and unneeded by my KDE desktop. So they could be: 1. Those two processes disabled entirely as processes to run ...or... 2. The systemd/tailwatchd/whateverd configuration .conf file could be set to ignore those specific processes (just ignore that they are running and in wait state -- and just let them sit there doing no harm ...or... Your suggested resolution of: [QUOTE]I think the best suggestion you had would be: - Set up a script that triggers at KDE logoff to kill those 2 processes.
... If this is the only real impediment to running a KDE desktop (very conservatively) on a cPanel RHEL configuration, then it might be nice to figure out the solution and have it posted here in the forums. But if we are on too much of a tangent, I will certainly back off, and take it up elsewhere, perhaps on the (albeit limited) X2go discussions on their site(s). Thanks again! I appreciate you, and all of cPanel for helping me with something that was clearly my fault because I forgot to disclose originally as part of my configuration.0 -
Hi @H.R. Holmer Since this seems to be the least intrusive you might want to at least try this first: [QUOTE]2. The systemd/tailwatchd/whateverd configuration .conf file could be set to ignore those specific processes (just ignore that they are running and in wait state -- and just let them sit there doing no harm
Before you try killing the processes. At this point though, I think the only way to get an answer would be to test but unfortunately, I don't have any real advice to give you on which would be the best solution. Out of curiosity, what do you use the desktop for on this system?0 -
[QUOTE]Out of curiosity, what do you use the desktop for on this system?
KDE Plasma -- with a very minimal installation. I even have the GUI stripped down and un-fancy so as not to sap resources -- although I wouldn't care if it sucked up 16GB of mem. I have 128GB and barely use any of it, and 40 cores of processor, also with extremely low usage. I previously ran MATE but when X2go had an update that let it run KDE Plasma, I removed MATE (crashy, slow!) and I also removed many of the gnome packages, so that when I installed KDE it would be very minimal. I only use it as stated above: Gives me the Dolphin File Manager, and runs UltraEdit beautifully, so from a GUI I can do very simple Find-And-Replace-Inside-Files and I can use RegEx to set the filemask and the content I'm searching/replacing. All much faster than I could ever do it with a combination of sed and grep and Perl one-liners from a command prompt in PuTTY. I don't mess with system files from within the desktop, other than an occasional edit of a .conf file but that's even rare. I don't provision or use Yumex, and do all install/remove/repository-handling from yum command line. I do nothing with the desktop that gets anywhere near cPanel's files. It's a very simple and seemingly innocuous install. Except for its GUI, it's probably less intrusive than Webmin.0 -
I don't know if you've looked at it but the ConfigServer Explorer can do similar tasks: ConfigServer Explorer (cse) Though it's not supported by cPanel either it is made specifically for servers running cPanel and they do make a plugin for cPanel. Other than that, looking through the forums here for advice from others seems to indicate that most just remove it rather than identify a workaround which doesn't help in this instance. 0
Please sign in to leave a comment.
Comments
23 comments