Skip to main content

The service “httpd” appears to be down.

LordLiverpool
Hello cPanel In the past 2 weeks my HTTPD has gone down 3 times. Normally it would recover itself. Each time I got this email: 49423 Looks like the reason it can't recover is: "Unable to connect to port 80 on 127.0.0.1: Connection refused: died" I looked on the cPanel forum and out on Google for a solution and found similar threads: [LIST]
  • The service "httpd" appears to be down.
  • cPanel restart service httpd failed
  • The service httpd appears to be down even though apache is running
  • The service "httpd" appears to be down? Following the advice contained within them, I checked the error log at: /usr/local/cpanel/logs/error_log And found this at the time it fell over:
    [2017-11-09 01:17:12 +0000] info [gather-update-logs] File /usr/local/cpanel/logs/update_analysis/2017-11-09T01:17:07Z.tar.gz has been sent successfully to updatelogdrop.cpanel.net. ==> cpsrvd 11.68.0.10 started ==> cpsrvd: loading security policy....Done ==> cpsrvd: Setting up SSL support ... Done ==> cpsrvd: transferred port bindings ==> cpsrvd: bound to ports ==> cpsrvd 11.68.0.10 started ==> cpsrvd: loading security policy....Done ==> cpsrvd: Setting up SSL support ... Done ==> cpsrvd: transferred port bindings ==> cpsrvd: bound to ports ==> cpsrvd 11.68.0.10 started ==> cpsrvd: loading security policy....Done ==> cpsrvd: Setting up SSL support ... Done ==> cpsrvd: transferred port bindings ==> cpsrvd: bound to ports [2017-11-09 01:31:57 +0000] warn [restartsrv_httpd] The 'httpd' service's PID file '/var/run/apache2/httpd.pid' did not appear after 180 seconds. [2017-11-09 01:31:58 +0000] info [queueprocd] chkservd::Notify Notification => ***********@**mail.*** via EMAIL [eventimportance => High (1)] [2017-11-09 01:32:02 +0000] info [cpaddons] Successfully verified signature for cpanel (key types: release). [2017-11-09 01:32:04 +0000] info [cpanelsync] Successfully verified signature for cpanel (key types: release). [2017-11-09 01:32:04 +0000] info [cpanelsync] Successfully verified signature for cpanel (key types: release). [2017-11-09 01:32:05 +0000] info [cpanelsync] Successfully verified signature for cpanel (key types: release). [2017-11-09 01:32:05 +0000] info [cpanelsync] Successfully verified signature for cpanel (key types: release). [2017-11-09 01:32:12 +0000] info [cpaddons] Successfully verified signature for cpanel (key types: release). [2017-11-09 01:36:58 +0000] warn [restartsrv_httpd] The 'httpd' service's PID file '/var/run/apache2/httpd.pid' did not appear after 180 seconds.
    I have CSF firewall installed, but I haven't made any changes to it myself for months. I can post the iptables rules here if you like? But I noticed they were redacted from other posts. I searched the iptable rules for the string "127.0.0.1" but got zero matches. Is that good? 49427 I'm not an expert in server administration so if you want access to my server I can open a support ticket. Any help is greatly appreciated. Best Regards Mike

    • Comments

    8 comments

    Date Votes
    • cPanelMichael
      Hello, Could you open a support ticket so we can take a closer look? Thank you.
      0
    • LordLiverpool
      Michael I really appreciated you helping me. I've opened a ticket: #9019765 and provided a more detailed explanation, as best as I understand. Thanks very much. Mike
      0
    • LordLiverpool
      OK just to give an update on this thread, so hopefully it helps someone else in the future. Apologies in advance to any Level 18 Fire Breathing +5 SysAdmins; this is noob stuff :-) The cPanel team examined my server and determined that Apache had ran out of Semaphores. Each time HTTPD crashed it didn't release the semaphores and eventually my server ran out (I had 128). The cPanel team kindly cleared down the "locked" semaphores as follows:
      [19:48:55 mail root@9019765 ~/cptechsjt]cPs# for i in `ipcs | grep admin | awk {'print $2'}`; do ipcrm -s $i; done [19:49:06 mail root@9019765 ~/cptechsjt]cPs# /scripts/restartsrv_httpd ===
      And the support analyst advised to read this article: Apache: No space left on device: Couldn't create accept lock - major.io After reading it, I googled for corroboratory advice and found this article: Semaphore limits and many Apache instances on Linux | End Point So I decide to increase the number of semaphores from 128 to 256 as follows: How many semaphores do I have?
      # cat /proc/sys/kernel/sem 250 32000 32 128
      Answer = 128 Add this line to /etc/sysctl.conf:
      kernel.sem = 500 64000 64 256
      To make the change take immediate effect, issue this command:
      # sysctl -p
      Alternatively perform a graceful reboot of your server, (If you can.) So how many semaphores do I have now?
      # cat /proc/sys/kernel/sem 500 64000 64 256
      You can also check the number of "running" semaphores with this command:
      # ipcs -s
      I also noticed that one website (wordpress) was creating a disproportionate number of semaphores compared to the other websites (also wordpress) on my server, so it must be a rogue plugin. I will investigate further. Thank you to great team at cPanel for helping me out, its genuinely appreciated. Best Regards Mike
      0
    • oprezy
      Am presently facing similar issues with my web host
      0
    • cPanelMichael
      Am presently facing similar issues with my web host

      Hello @oprezy, Have you reported the issue to your web hosting provider so they can take a closer look? Thank you.
      0
    • Efren Gutierrez Sosa
      Hi!!! i also have this problem y was a really headache. I open a case, however i did"t get a solution, my agent just limit to say that all my services were ok and i needed to hire a system administrator. My principal error was that i didn't believed a could be under a DDoS attack, however that was exactly the problem. To know who was doing this attack i run this command:
      # netstat -an | egrep ':80|:443' | grep ESTABLISHED | awk '{print $5}' | grep -o -E "([0-9]{1,3}[\.]){3}[0-9]{1,3}" | sort -n | uniq -c | sort -nr
      The first data is the number of current connections, the second is the IP doing GET and POST. The mayor number of current connections was 3, however, there was one IP with 279 current connections. First of all, i think the machine has a virus, and i just block connections from this IP using iptables like this:
      DROP all -- dsl-[removed]anywhere
      Once i did it, after a few minutes my server got normal again, but, that was not the end. The problem came back, it became suspicious because after doing an IP tracker, the location of the attack was the same, so, as the time they were working with a dynamic IP it would be a never end history to try to block every new IP, and my server will be falling every 30 minutes. To solve this, i worked again over iptables doing this:
      REJECT tcp -- anywhere anywhere tcp dpt:http flags:FIN,SYN,RST,ACK/SYN #conn src/32 > 10 reject-with tcp-reset REJECT tcp -- anywhere anywhere tcp dpt:https flags:FIN,SYN,RST,ACK/SYN #conn src/32 > 10 reject-with tcp-reset REJECT tcp -- anywhere anywhere tcp dpt:webcache flags:FIN,SYN,RST,ACK/SYN #conn src/32 > 10 reject-with tcp-reset REJECT tcp -- anywhere anywhere tcp dpt:radan-http flags:FIN,SYN,RST,ACK/SYN #conn src/32 > 10 reject-with tcp-reset
      This iptables rules block any new connection from any IP when the current connections are over 10. At the time, this has been enough to stop the attack, there are other solutions blocking after "N" seconds, but, that was not my solution because there are many process in my applications that take a lot of time. As i understand, this error is not caused in all the cases by the same reason, i expect this could be help to some one with the same symptoms.
      0
    • ExpertWebWorld
      Hello, Could you open a support ticket so we can take a closer look? Thank you.

      Same issue we are facing from last days
      Notification The service "httpd" is now operational.
      Notification The service "ftpd" appears to be down.
      0
    • cPRex Jurassic Moderator
      @ExpertWebWorld - this post is 4 years old now - can you make a new thread for your specific issue?
      0

    Please sign in to leave a comment.

    Didn't find what you were looking for?

    New post