Skip to main content

[CPANEL-23845] cpsrvd fails after 76 upgrade when running a custom Apache service

inteldigital
At around 0700 hours this morning, we were alerted to one of our redundant DNSOnly servers had failed, it has been trying to restart itself for the past 9 hours and we have tried everything to get it restarted but nothing is working. On running the command "/scripts/restartsrv_cpsrvd" we get the following response: Cpanel::Exception::Services::StartError Service Status Service Error (XID 44mh23) The "cpsrvd" service failed to start. Startup Log Nov 08 14:39:08 jade.example.com restartsrv_cpsrvd[18140]: /usr/local/cpanel/libexec/cpsrvd-dormant: Waiting to bind to port 80 (Address already in use).... Nov 08 14:39:09 jade.example.com restartsrv_cpsrvd[18140]: /usr/local/cpanel/libexec/cpsrvd-dormant: Waiting to bind to port 80 (Address already in use).... Nov 08 14:39:10 jade.example.com restartsrv_cpsrvd[18140]: /usr/local/cpanel/libexec/cpsrvd-dormant: Waiting to bind to port 80 (Address already in use).... Nov 08 14:39:11 jade.example.com restartsrv_cpsrvd[18140]: /usr/local/cpanel/libexec/cpsrvd-dormant: Waiting to bind to port 80 (Address already in use).... Nov 08 14:39:12 jade.example.com restartsrv_cpsrvd[18140]: /usr/local/cpanel/libexec/cpsrvd-dormant: Waiting to bind to port 80 (Address already in use).... Nov 08 14:39:13 jade.example.com restartsrv_cpsrvd[18140]: /usr/local/cpanel/libexec/cpsrvd-dormant: Waiting to bind to port 80 (Address already in use).... Nov 08 14:39:13 jade.example.com systemd[1]: cpanel.service start operation timed out. Terminating. Nov 08 14:39:13 jade.example.com systemd[1]: Failed to start cPanel services. Nov 08 14:39:13 jade.example.com systemd[1]: Unit cpanel.service entered failed state. Nov 08 14:39:13 jade.example.com systemd[1]: cpanel.service failed.
Our message log is filled with stuff like this: Nov 8 01:00:06 jade systemd: Stopping User Slice of nmonagent. Nov 8 01:00:23 jade systemd: Stopping tailwatchd... Nov 8 01:00:24 jade restartsrv_tailwatchd: [7708] [2018-11-08 01:00:24 +0000] [main] Current process '6671' stopped Nov 8 01:00:24 jade systemd: Stopped tailwatchd. Nov 8 01:00:24 jade systemd: Starting tailwatchd... Nov 8 01:00:24 jade restartsrv_tailwatchd: [Thu Nov 8 01:00:24 2018] Starting /usr/local/cpanel/libexec/tailwatch/tailwatchd daemon Nov 8 01:00:24 jade restartsrv_tailwatchd: Log is at /usr/local/cpanel/logs/tailwatchd_log Nov 8 01:00:24 jade systemd: PID file /var/run/tailwatchd.pid not readable (yet?) after start. Nov 8 01:00:24 jade systemd: Started tailwatchd. Nov 8 01:00:40 jade systemd: Reloading. Nov 8 01:01:01 jade systemd: Started Session 211372 of user root. Nov 8 01:01:01 jade systemd: Starting Session 211372 of user root. Nov 8 01:01:01 jade systemd: Created slice User Slice of nmonagent. Nov 8 01:01:01 jade systemd: Starting User Slice of nmonagent. Nov 8 01:01:01 jade systemd: Started Session 211371 of user nmonagent. Nov 8 01:01:01 jade systemd: Starting Session 211371 of user nmonagent. Nov 8 01:01:06 jade systemd: Removed slice User Slice of nmonagent. Nov 8 01:01:06 jade systemd: Stopping User Slice of nmonagent. Nov 8 01:01:25 jade systemd: Reloading. Nov 8 01:01:43 jade systemd: Reloading. Nov 8 01:01:43 jade systemd: Stopping PowerDNS Authoritative Server... Nov 8 01:01:43 jade systemd: Starting PowerDNS Authoritative Server... Nov 8 01:01:43 jade pdns_server: Reading random entropy from '/dev/urandom' Nov 8 01:01:43 jade pdns_server: Loading '/usr/lib64/pdns/libbindbackend.so' Nov 8 01:01:43 jade pdns_server: This is a standalone pdns Nov 8 01:01:43 jade pdns_server: Listening on controlsocket in '/var/run/pdns.controlsocket' Nov 8 01:01:43 jade pdns_server: UDP server bound to 0.0.0.0:53 Nov 8 01:01:43 jade pdns_server: UDPv6 server bound to [::]:53 Nov 8 01:01:43 jade pdns_server: TCP server bound to 0.0.0.0:53 Nov 8 01:01:43 jade pdns_server: TCPv6 server bound to [::]:53 Nov 8 01:01:43 jade pdns_server: PowerDNS Authoritative Server 4.1.4 (C) 2001-2018 PowerDNS.COM BV Nov 8 01:01:43 jade pdns_server: Using 64-bits mode. Built using gcc 4.8.2 20140120 (Red Hat 4.8.2-16) on Oct 29 2018 15:56:48 by root@rpmbuild-64-centos-7.dev.cpanel.net. Nov 8 01:01:43 jade pdns_server: PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2. Nov 8 01:01:43 jade pdns_server: Creating backend connection for TCP Nov 8 01:01:43 jade pdns_server: [bindbackend] Parsing 227 domain(s), will report when done Nov 8 01:01:44 jade pdns_server: [bindbackend] Warning! Skipping zone '.' because type 'hint' is invalid Nov 8 01:01:44 jade pdns_server: [bindbackend] Warning! Skipping zone '.' because type 'hint' is invalid Nov 8 01:01:44 jade pdns_server: [bindbackend] Warning! Skipping zone '.' because type 'hint' is invalid Nov 8 01:01:45 jade pdns_server: [bindbackend] Done parsing domains, 3 rejected, 115 new, 0 removed Nov 8 01:01:45 jade systemd: Started PowerDNS Authoritative Server. Nov 8 01:01:45 jade pdns_server: Only asked for 1 backend thread - operating unthreaded Nov 8 01:02:01 jade systemd: Created slice User Slice of nmonagent. Nov 8 01:02:01 jade systemd: Starting User Slice of nmonagent. Nov 8 01:02:01 jade systemd: Started Session 211373 of user nmonagent. Nov 8 01:02:01 jade systemd: Starting Session 211373 of user nmonagent. Nov 8 01:02:03 jade systemd: Stopping cPanel services... Nov 8 01:02:03 jade systemd: Stopped cPanel services. Nov 8 01:02:03 jade systemd: Cannot add dependency job for unit mailman.service, ignoring: Unit not found. Nov 8 01:02:03 jade systemd: Starting cPanel services... Nov 8 01:02:03 jade restartsrv_cpsrvd: License is valid and has already updated recently. Nov 8 01:02:04 jade restartsrv_cpsrvd: /usr/local/cpanel/libexec/cpsrvd-dormant: Waiting to bind to port 80 (Address already in use).... Nov 8 01:02:05 jade systemd: Removed slice User Slice of nmonagent. Nov 8 01:02:05 jade systemd: Stopping User Slice of nmonagent. Nov 8 01:02:05 jade restartsrv_cpsrvd: /usr/local/cpanel/libexec/cpsrvd-dormant: Waiting to bind to port 80 (Address already in use).... Nov 8 01:02:06 jade restartsrv_cpsrvd: /usr/local/cpanel/libexec/cpsrvd-dormant: Waiting to bind to port 80 (Address already in use)....
EDIT: I might add, we have Apache running on these servers too. We've had 0 problems running cPanel DNS Only and Apache until now. However, it seems when I disable httpd I can restart cPanel, but httpd won't run as something is taking up port 80? If I use netstat to try to locate what is running on port 80 it doesn't show anything, do you have any experience with this @cPanelLauren

Comments

7 comments

Date Votes
  • cPanelMichael
    Hello @inteldigital, It looks like the issue you are facing stems from the following change noted in the Tweak Settings interface (WHM >> Home >> Server Configuration >> Tweak Settings). This setting prevents the cpsrvd daemon from taking over the standard HTTP ports when you disable the system"s web server services, such as Apache. This action renders any cPanel & WHM features that depend on the standard HTTP ports partially or entirely unusable. These features include service subdomains, AutoSSL, Mailman, and BoxTrapper.
    To enable this option on a DNSONLY system, you can manually edit the the following value in the /var/cpanel/cpanel.config file: disable_cphttpd=0
    To: disable_cphttpd=1
    Then, save the file and restart cpsrvd with the following command: /usr/local/cpanel/scripts/restartsrv_cpsrvd
    Note that internal case CPANEL-23845 is open to report this behavior and determine if there's a better practice to prevent the unexpected downtime upon the upgrade to version 76. I'll update this thread with more information on the status of this case as it becomes available. Thank you.
    0
  • inteldigital
    @cPanelMichael Thank you so much for this, I didn't even check the changelog. I noticed that the auto upgrade had stopped for some reason but decided to run it manually. Perhaps I should have checked beforehand EDIT: This worked great, thanks again.
    0
  • cPanelMichael
    Hello @inteldigital, I'm glad to see the solution helped. Regarding CPANEL-23845, the change from this case will ensure that cpsrvd only listens on HTTP ports when the Web Server role is disabled. You can read more about roles and profiles at: How to Use Server Profiles - cPanel Knowledge Base - cPanel Documentation This will prevent the issue you encountered from happening on standard cPanel & WHM servers. However, since you are using DNSOnly, manually changing the disable_cphttpd setting using the instructions from the post above remains a requirement. You shouldn't have to make any additional changes now that you've made the change. Thank you.
    0
  • inteldigital
    @cPanelMichael Thanks for updating me on this. Much appreciated. The changes implemented have worked and we've had no further issues with our production DNS Only servers.
    0
  • cPanelMichael
    Hello @inteldigital, I'm glad to see the suggested change helped. For anyone facing this issue on standard cPanel & WHM server, this is fixed in cPanel & WHM version here and see which build is published to each release tier by clicking
    0
  • Juan R. Pozo
    As a suggestion, if there is something listening on port 80/443, maybe you shouldn't stop it and make cpsrvd listen on HTTP ports, even if WebServer role is disabled.
    0
  • cPanelMichael
    As a suggestion, if there is something listening on port 80/443, maybe you shouldn't stop it and make cpsrvd listen on HTTP ports, even if WebServer role is disabled.

    Hello @Juan R. Pozo, The disable_cphttpd setting referenced
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post