Skip to main content

HTTPD Child Connections

DynamiteDJ
Hi, Since the latest forced update early hours this morning my cPanel is closing down the httpd service every 6-7 minutes exceeding the max child connections, I have tried ammended the global apache configuration and increasing the limit however this had little effect. I am receiving the following in my error_log and i have contacted my DC who have advised their is not an issue with incoming traffic and the ports they are monitoring are all showing as normal so i am not being DDoSed according to their logs.
root@server1 [~]# tail -f /var/log/apache2/error_log [Tue Jan 24 09:41:07.689222 2017] [ssl:info] [pid 26406] [client 198.41.226.68:33367] AH01964: Connection to child 254 established (server domain.co.uk:443)
I have opened a support ticket however over 100 clients are showing ahead of me and i wasn't sure if perhaps anyone else had this problem previously and knew of a fix for it. When the Max Client Connections is reached i am receiving the following [Tue Jan 24 08:14:15.094124 2017] [mpm_prefork:error] [pid 27999] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
Can anyone offer any advice with regards to this? or should i keep waiting for the Ticket to be answered and manually keep restarting the httpd service every 6-7 minutes.

Comments

12 comments

Date Votes
  • DynamiteDJ
    Help??? ANYONE?? I've managed to find out that its something related to SNI, however the VHOSTS are all set correct, the SSL Manager is showing the certs correctly. i am still receiving these child connections but JUST the "child connection establishing" I am still getting AH01964 which majority of people are saying is related to vhost being set to *:443 however my vhost is configured to can anyone offer any help with this as it is slowing my server down and most of my websites are taking an age to load up so customers are constantly calling to complain.
    0
  • cPanelMichael
    [Tue Jan 24 08:14:15.094124 2017] [mpm_prefork:error] [pid 27999] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

    Hello, Could you verify if you browsed to "WHM " Service Configuration " Apache Configuration " Global Configuration" and increased the "Max Request Workers" limit? This is the specific limit your server is reaching based on the error output. Thank you.
    0
  • DynamiteDJ
    Michael i did increase the limit which made no difference, after a few attempts of playing with the SSL Certs i've got it now so there are no errors or anything just A LOT of info lines quoting [Tue Jan 24 09:41:07.689222 2017] [ssl:info] [pid 26406] [client 198.41.226.68:33367] AH01964: Connection to child 254 established (server domain.co.uk:443)
    0
  • cPanelMichael
    Hello, It looks like a support ticket (#8154713) is open for this issue. Feel free to update this thread with the outcome once the support ticket is closed. Thank you.
    0
  • did-vmonroig
    Hello, Was this issue solved? I'm experiencing same problem. Thanks.
    0
  • cPanelMichael
    Was this issue solved? I'm experiencing same problem.

    Hello, The outcome of the support ticket suggested the issue was related to the amount of incoming traffic on the affected system. Could you provide some more details about the specific issue you are experiencing? Thank you.
    0
  • did-vmonroig
    The outcome of the support ticket suggested the issue was related to the amount of incoming traffic on the affected system. Could you provide some more details about the specific issue you are experiencing?

    I'm just seeing lots of this lines in /usr/local/apache/logs/error_log:
    [Thu Mar 30 08:49:41.505771 2017] [ssl:info] [pid 19317:tid 139730245093120] [client AAA.BBB.CCC.DDD:35143] AH01964: Connection to child 201 established (server domain.tld:443) [Thu Mar 30 08:49:41.564197 2017] [ssl:info] [pid 19225:tid 139730129704704] [client AAA.BBB.CCC.DDD:35146] AH01964: Connection to child 19 established (server domain.tld:443) [Thu Mar 30 08:49:41.565210 2017] [ssl:info] [pid 19288:tid 139730276562688] [client AAA.BBB.CCC.DDD:35147] AH01964: Connection to child 134 established (server domain.tld:443) [Thu Mar 30 08:49:41.567026 2017] [ssl:info] [pid 20963:tid 139730077255424] [client AAA.BBB.CCC.DDD:35148] AH01964: Connection to child 280 established (server domain.tld:443) [Thu Mar 30 08:49:41.567687 2017] [ssl:info] [pid 19317:tid 139730108724992] [client AAA.BBB.CCC.DDD:35150] AH01964: Connection to child 214 established (server domain.tld:443) [Thu Mar 30 08:49:41.568014 2017] [ssl:info] [pid 25576:tid 139730129704704] [client AAA.BBB.CCC.DDD:35149] AH01964: Connection to child 339 established (server domain.tld:443)
    Sometimes are followed by this other lines:
    [Thu Mar 30 08:49:53.691729 2017] [ssl:info] [pid 19225:tid 139730129704704] (70014)End of file found: [client AAA.BBB.CCC.DDD:35146] AH01991: SSL input filter read failed. [Thu Mar 30 08:49:53.691740 2017] [ssl:info] [pid 19288:tid 139730276562688] (70014)End of file found: [client AAA.BBB.CCC.DDD:35147] AH01991: SSL input filter read failed. [Thu Mar 30 08:49:53.692992 2017] [ssl:info] [pid 20963:tid 139730077255424] (70014)End of file found: [client AAA.BBB.CCC.DDD:35148] AH01991: SSL input filter read failed. [Thu Mar 30 08:49:53.693200 2017] [ssl:info] [pid 19317:tid 139730108724992] (70014)End of file found: [client AAA.BBB.CCC.DDD:35150] AH01991: SSL input filter read failed. [Thu Mar 30 08:49:53.694085 2017] [ssl:info] [pid 25576:tid 139730129704704] (70014)End of file found: [client AAA.BBB.CCC.DDD:35149] AH01991: SSL input filter read failed.
    This happens to only one domain in our shared hosting. Reviewing this issue, i've found this lines, too, but less frequent:
    [Sun Mar 26 10:49:33.153437 2017] [ssl:info] [pid 16204:tid 140315593754368] [remote 127.0.0.1:2096] AH01964: Connection to child 0 established (server server.domain.tld:443)
    Those are from localhost to our server full name. Thanks for your help. :)
    0
  • cPanelMichael
    Hello @did-vmonroig, Do you notice any "Broken pipe" entries in the Apache error log? EX:
    grep "Broken pipe" /usr/local/apache/logs/error_log
    Thank you.
    0
  • did-vmonroig
    Hello,
    Do you notice any "Broken pipe" entries in the Apache error log?

    Yes, lots of them:
    [Thu Mar 30 17:03:42.742381 2017] [ssl:info] [pid 31976:tid 139730129704704] (32)Broken pipe: [client AAA.BBB.CCC.DDD:54394] AH01993: SSL output filter write failed. [Thu Mar 30 17:05:07.727765 2017] [ssl:info] [pid 31436:tid 139728372365056] (32)Broken pipe: [client AAA.BBB.CCC.DDD:54406] AH01993: SSL output filter write failed. [Thu Mar 30 17:05:07.755979 2017] [ssl:info] [pid 31493:tid 139730129704704] (32)Broken pipe: [client AAA.BBB.CCC.DDD:54405] AH01993: SSL output filter write failed. [Thu Mar 30 17:08:10.670363 2017] [ssl:info] [pid 31976:tid 139730213623552] (32)Broken pipe: [client AAA.BBB.CCC.DDD:54434] AH01993: SSL output filter write failed. [Thu Mar 30 17:08:10.675515 2017] [ssl:info] [pid 31463:tid 139730297542400] (32)Broken pipe: [client AAA.BBB.CCC.DDD:54435] AH01993: SSL output filter write failed. [Thu Mar 30 17:11:59.269767 2017] [ssl:info] [pid 31436:tid 139730297542400] (32)Broken pipe: [client EEE.FFF.GGG.HHH:63805] AH01993: SSL output filter write failed. [Thu Mar 30 17:20:02.944319 2017] [ssl:info] [pid 32025:tid 139730276562688] (32)Broken pipe: [client III.JJJ.KKK.LLL:54208] AH01993: SSL output filter write failed.
    0
  • cPanelMichael
    Hello @did-vmonroig, It's possible this relates to an Apache race-condition. Here's a quote from internal case EA-6020 that was opened to make note of this issue, though keep in mind it's an issue with Apache itself: Apache has a race-condition when it kills and restarts its piped-logging processes on graceful restart before all of its children handling ongoing client connections have finished, resulting in a "Broken pipe" error when those children attempt to log to a pipe that no longer exists. Because it is a race-condition, this will be more likely to happen on busier servers where httpd children are servicing a client request in the middle of a graceful restart, and unlikely to be seen on idle servers.
    If you are not noticing any resource usage issues, the only concern would be the potential to lose some domlog/byteslog entries in your Apache logs. You can disable Piped Logging via "WHM >> Apache Configuration >> Piped Log Configuration" to see if the error messages go away. Thank you.
    0
  • did-vmonroig
    If you are not noticing any resource usage issues, the only concern would be the potential to lose some domlog/byteslog entries in your Apache logs. You can disable Piped Logging via "WHM >> Apache Configuration >> Piped Log Configuration" to see if the error messages go away.

    We've no reports from clients notifying any problem, but we saw it reviewing server health. Piped Logging was already disabled, so Broken pipe lines must be for another cause. Please, update this thread if you have any news in your internal case. Thanks!
    0
  • cPanelMichael
    Hello, It's unlikely your issue is related to that case if Piped Logging is disabled on your system. Feel free to open a support ticket if you'd like us to take a closer look and rule out any issues with the cPanel/WHM. Thank you.
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post