Skip to main content

Server hangs, low resource usage, restarting php-fpm fixes it.

mvandemar
One of my clients keeps getting intermittent outages on their server, at least a few times a week. They can last anywhere from 5 to 30 minutes, if I catch them and restart php-fpm services it comes right back up again. According to htop when this is happening there is no unusual stress on the server when this is happening: load average: 0.53, 0.44, 0.50
I have looked through the server logs but am really not adept at reading them, but to my untrained eye I am not seeing anything that would explain it. Any help in figuring out why this is happening or what I should be looking for/at would be appreciated, thanks. -Michael

Comments

20 comments

Date Votes
  • cPRex Jurassic Moderator
    Hey there! It depends on what you mean by "outage" - what is the specific symptom when you try and visit the website? The best thing to do when the issue is actually happening would be to check both the Apache log file at /etc/apache2/logs/error_log and the Apache status with the "apachectl status" command to see what Apache is doing in real-time. One of those two things should show you why things aren't working well - for example, you may find an error in the log, or you may see that the scoreboard in the status output is full and Apache can't handle the traffic.
    0
  • mvandemar
    Hey there! It depends on what you mean by "outage" - what is the specific symptom when you try and visit the website?

    Literally nothing happens, it just hangs. I have looked at the logs as I mentioned, but again I am not sure what I am looking for. There was nothing in the Apache error log for the time period it was down today (a couple Connection refused messages 3 minutes prior to it going down, but those are all throughout the log), just notices in the php-fpm error.log. It's only php pages, static files are served just fine. -Michael
    0
  • cPRex Jurassic Moderator
    Thanks for the additional details. If it the issue only happens with PHP, I'd recommend checking the FPM logs at /opt/cpanel/ea-phpXX/root/usr/var/log/php-fpm. You'll just need to update the "XX" portion of the log data with the specific PHP version you're using on the system. That should have more specific details about the outage than the Apache log.
    0
  • mvandemar
    I'd recommend checking the FPM logs

    Rex, thanks, but as I mentioned I did, and there was nothing there but the same notices I get during the non-downtime period. I'm stumped. -Michael
    0
  • cPRex Jurassic Moderator
    Sounds like it's time to open a ticket, then.
    0
  • mvandemar
    Sounds like it's time to open a ticket, then.

    Thanks. I think I've narrowed it down to being an issue with php-fpm based on the fact that there appears to have been no mysql activity during the latest downtime: and I set up a monitor on a plain php file that had nothing but an echo statement in it, and that too was unresponsive. Is there a way to set the pm.status_path, ping.path, and listen directives in whm, as described in this article?
    0
  • cPRex Jurassic Moderator
    Yes - details on how to adjust those can be found here - Configuration Values of PHP-FPM | cPanel & WHM Documentation
    0
  • mvandemar
    Yes - details on how to adjust those can be found here -
    0
  • cPRex Jurassic Moderator
    It would seem that document I linked is a bit out of date, or at least doens't say you need to create those files manually. We do say that specifically here:
    0
  • mvandemar
    It would seem that document I linked is a bit out of date, or at least doens't say you need to create those files manually. We do say that specifically here:
    0
  • cPRex Jurassic Moderator
    Correct - those files are not required or else they would exist by default, but they are necessary when performing customizations. I have submitted a case to our documentation team to get that cleared up. Personal opinion - we're heading down the wrong direction with this. If there isn't anything in the FPM logs, setting up additional monitoring and watches isn't going to be helpful as we already know it isn't giving us log data. Since we're starting the third week of this thread, it's really time for a ticket so we can do some investigation.
    0
  • mvandemar

    cPRex Wow, the transition to the new system really screwed things up, didn't it... do you have the links to the documentation that disappeared by any chance? Thanks.

    -Michael

    0
  • cPRex Jurassic Moderator

    Honestly, at this time, no, I don't have a good way to find what that link was.  I likely will within a few days, and I'll leave this open as a reminder to myself to reply when I do have that information.

    0
  • cPRex Jurassic Moderator

    Here's that missing link - https://docs.cpanel.net/knowledge-base/php-fpm/php-fpm-code-and-filesystem-layout-for-easyapache-4/

    1
  • Marc LP

    Hi, we are facing the same issue on a certain server we manage.

    Could you guide us on what was the problem told by mvandemar and what did you do to solve it?

    Thanks.

    0
  • cPRex Jurassic Moderator

    Marc LP - I don't have any data available that you aren't seeing in this thread.

    Are your outages also intermittent?  If so, have you reached out to the datacenter to confirm there are no issues there, especially if you're on a VPS server?

    0
  • Marc LP

    Yes, every 3 - 4 days. It’s a dedicated server that we’ve been fully managing for a couple of years now. There have been no recent changes except for regular cPanel updates. It’s running the latest stable version of cPanel. We have other servers using cPanel in the same data center, and we have no issues with them.

    Based on the information in this post, we’ve just modified the PHP-FPM configuration to allow it to use more resources. Hopefully, this will improve the situation. We will keep you posted.

    0
  • cPRex Jurassic Moderator

    I wouldn't have recommended modifying anything until you know more details.

    The most useful area will likely be the sar logs:

    https://support.cpanel.net/hc/en-us/articles/360047748654-How-to-diagnose-high-loads-with-sar-command

    as those are daily logs that capture what the server is doing.  Reviewing those may help point you in the right direction as it hopefully will show at least what process was using the resources during the outage.

    0
  • Marc LP

    Thank you for your recommendations cPRex.

    sar logs did not told me anything useful related to the php-fpm issue.

    But after tweaking the PHP-FPM settings to use more resources no other episode of unresponsive website has occurred (and that's more than 10 days ago). I don't fully understand why that helped, because the server load was really low during the outage...

    Anyway, I'll keep watching the server and check the info with the sar command if it happens again.

    0
  • cPRex Jurassic Moderator

    I'm glad to hear things are working well!  If anything else comes up feel free to reach out at any time!

    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post