PHP FPM not optimal for shared hosting
Hi there
A few versions back FPM was introduced and we blindly activated the feature, as usually whm/cpanel works fine out of the box.
This feature however reacts unexpected in a shared hosting setup.
It seems FPM is meant to run one site (one user). So if we use Cpanel (out of the box) for multiple sites, every client (every site) gets it's own pool and if we activate FPM with max_children=20, every client can expand to 20 processes. If they in a rare case all do so at the same time, the server will run out of memory (and oom kicks in).
The Internet suggests that I should experiment, give some sites like 5 processes, others 2 or put some on "pm=demand" ( not dynamic ) and look at the logs for "you should increase number of childs" to further tune the setup... Is that really the only way?
I found a hint to something like max.processes. This would globally cap the max processes ALL Pools combined. But I can't figure out where to put it. Every configfile is telling me that I should use the "interface".
I could also make all sites run as one user, but I don't think cpanel will support that.
If you know what I can do?
Edit: toned it down a bit
-
If you don't like FPM, just switch to a different PHP handler. PHP Handlers | cPanel & WHM Documentation There are obvious trade-offs to switching, but it can be done. 0 -
Hi Again. I solved it myself. Here is what I found out & what I did. The PHP-FPM will spawn pool processes for every .conf file in the /opt/cpanel/ea-php{php_version}/root/etc/php-fpm.d directory. These are the domains for which you have php-fpm set to "on" in the user interface under "MultiPHP Manger". Each domain will basically get the amount of "max children" ( standard 5 ) set in the "System PHP-FPM" tab. Besides the defaults for all domains, you can individually change the settings using the "Edit PHP-FPM" link for each domain. However, this configuration either sets you for a crash, or leaves many of the server's resources unused. When you set the max children to high, and the sum of all max children of all domains combined is more that your server can handle. It might run smooth, until many domains simultaneously demand more processes. When that happens, you run out of memory and OOM will kill your server, downtime for every client on the server. When you put a number too small, sites will stop working if they reach only a fraction of the load the server could potentially handle. I also found that when a domain reaches it's max children, the domain becomes unresponsive ( ERR Timeout ). This might be another issue altogether, but it also suggest to raise the max children. Either way, you're in a bad place. Here is the list of global php-fpm.conf directives 0 -
Note, the php-fpm.conf file you reference isn't global unless PHP 7.2 is the only version installed. process.max you mentioned is for use with the dynamic process manager. Cpanel defaults to the ondemand on to use less processes / memory so I do not believe that setting will do anything for you. Adjusting the idle timeout down is good for your needs since with the previous setting of 150 seconds that left a lot of time keeping a processing waiting for further use and if you have a really active site that could grow the number of processes a lot. So a shorter timeout allows them a better chance to close. max_requests per process is another way to keep the memory use per process in check for processes that see enough activity to never close. This will make sure they close even if the idle timeout hasn't passed. Max children may not need a high setting if your sites make use of any caching that delivers cached html pages rather than executing php pages. I have a bunch site that get away with 5 and I haven't set any past 15. You'll just need to set a value and watch the process use. 0 -
@kdean Thanks for your reply. Indeed. There is one older version of php running. This will be phased out this week. Could that cause the instabilities? Well, the max_processes setting does trigger when the max processes are reached (saw it once when i stress tested the setup) and it puts a line in the logs, but it also crashes the webserver. Same goes for max children (per domain). If I get the message "server reached max_children setting (15), consider raising it" also the domain is completely down (times out). I have to reboot the server to get it back. Restart fpm alone doesn't fix. Sometimes restarting fpm AND apache helps. Most sites work good with 5 and never use more than 2 children. Just that one site is having 5+ all the time. It's very frustrating to have FPM freeze unexpectedly. And there isn't much to go on, because nobody is talking about using PHP-FPM in a shared environment. The only things I could find were people with similar problems when using it in Cpanel. Lot and lot's of manuals about using it elsewhere, but always servers that serve only one application (or one website), never multiple users. I'm not sure what to think or why Cpanel put it in here when it's not supposed to work this way? :eek: I will try changing OnDemand to Dynamic though (like you pointed), maybe my setting isn't supported and untested. edit: that was a bad idea XD If someone can shed a light on why the Apache or FPM is not serving sites anymore after max_children has been reached, it would be great. I would hate it to have to go back to mod_php for stability. 0 -
PHP-FPM has always been a bug in my side with cPanel, I was asking much of these same questions 3-4 years ago. Yeah, it"s the new hotness, but cPanel has never really provided any decent documentation on how it either works or how to configure it. And you"ve just verified again that they still knowingly do not providing access to all of its settings. I"ve not hit any of Walter"s problems, but once it became default, RAM became cheap enough I just doubled the amount of RAM I normally used for a server and did a swag of a double amount of Max Children any site on the box will ever hit and then used that multiplied by the number of sites on the box to set Max Requests. Which is completely stupid, but works okay enough I don"t ever get sites going dark. Interestingly in CENTOS 7.8, WHM v90.0.15 Process Idle Timeout"s default is now 10. That seems to suggest cPanel is aware of the crashes the webserver issues... Frustrated too, Michael 0 -
As I said in my reply, you can switch handlers. We turn off PHP-FPM and are typically using LSAPI. 0 -
This thread has a very good discussion on handlers, my personal preference is LSAPI but PHP-FPM is also suitable for many instances. @walterdevos I'm glad to see you were able to resolve your issue, please let us know if you have any further issues. 0 -
What do you have set for the pm.process_idle_timeout parameter? ondemand should work. If you're using ondemand and you run up against max_children and assuming you have a reasonable value for max_children (i.e. not 1 or 2) then I might be a little concerned that the websites themselves are just getting flooded with requests. This can happen a lot with various search engine crawling bots that don't really adhere to any form of rate limiting themselves (and are usually some less that recognizable search engine bots). 0 -
@cPanelLaurne, Where did @walterdevos say his issue was solved? Even before his last edit he clearly states, "but it also crashes the webserver" and "If someone can shed a light on why the Apache or FPM is not serving sites anymore after max_children has been reached" # # # Hi All, For a secure, shared environment (e.g. more than one cPanel account per server) we"re left with FCGI and PHP-FPM. Hence this discussion on how to get FPM to work without fubar-ing websites... Here"s background to continue this discussion (from [1]): Note: If I missed a PHP handler that can be installed in WHM/cPanel, please add it. Usable in shared environment FCGI The primary benefit for FastCGI is the ability to be used with SuEXEC like the handler SuPHP. This allows scripts to be processed/executed as the user instead of the default nobody4) user that Apache uses. With EasyApache4, the FCGI handler can be applied to all versions. @[2] The mod_fcgid Apache module is not compatible with the mod_ruid2 or mod_mpm_itk Apache modules. PHP-FPM Requires in-depth configured Can cause performance issues if not configured properly Requires decent amount of server resources available Not meant for low-end servers Not sure if usable without CloudLinux? LSAPI / mod_lsapi CloudLinux is required to take full advantage of LSAPI with mod_lsapi PRO. In a Multi-PHP Environment, the LSAPI handler can be applied to all versions. Improper file permissions can cause a world-readable situation that introduces security risks across websites. @[2] cPanel & WHM's version of LSAPI differs from the one that CloudLinux provides. This version disables Checkpoint/Restore in Userspace (CRIU), connection pooling, and opcache preservation. Note: To utilize this handler's full functionality, your system must run CloudLinux. For more information, read Cloudlinux's Apache mod_lsapi documentation. Not generally considered usable in shared environment CGI This handler is considered insecure due to non-restrictive permissions Apache directives cannot be used. Configuration file (php.ini) required SuPHP Unable to use any PHP Opcode Caching No longer actively maintained by it's developers DSO In a Multi-PHP Environment (EasyApache 4 or using multiple versions of PHP with DirectAdmin's CustomBuild) the DSO+mod_ruid2 handler can only be applied to one PHP version " DSO cannot be used for multiple versions of PHP at the same time. Reference: [1] # # # @walterdevos, You"ve done the most (recent) testing with FPM, would you mind filing a bug report with cPanel detailing how to replicate the issues we all have experienced at one point or another? Buying double RAM isn"t anyone"s long term preferred solution... Thanks, Michael Edit: Additional References: 0 -
What do you have set for the pm.process_idle_timeout parameter?
WHM "Home "Software "MultiPHP Manager Tab: System PHP-FPM Configuration Edit: I read that as 'Where do you set' opps... Here's the right answer:Somehow PHP-FPM keeps processes running iddle, but they don't accept new request. Sites go dark after the max children has been reached. I solved this by setting process iddle timeout to 5 ( from 150 ).
0 -
@Michael-Inet @cPanelLaurne, Where did @walterdevos say his issue was solved? Even before his last edit he clearly states, "but it also crashes the webserver" and "If someone can shed a light on why the Apache or FPM is not serving sites anymore after max_children has been reached"
It says that here:Hi Again. I solved it myself. Here is what I found out & what I did. The PHP-FPM will spawn pool processes for every .conf file in the /opt/cpanel/ea-php{php_version}/root/etc/php-fpm.d directory. These are the domains for which you have php-fpm set to "on" in the user interface under "MultiPHP Manger". Each domain will basically get the amount of "max children" ( standard 5 ) set in the "System PHP-FPM" tab. Besides the defaults for all domains, you can individually change the settings using the "Edit PHP-FPM" link for each domain.
The reason for the crash I assumed in this instance was due to reaching the max workers which would actually be normal when pool limits are reached. But, @walterdevos If you're thinking this issue is related to I would correct suggestions to open a case and say that you should not open a ticket to cPanel for this as this is a known issue with PHP-FPM which is specifically related to PHP-FPM and while we are aware it exists it is not something we can fix. The problem specifically being that kill_all_lockers is attempting to kill processes that belong to a different FPM pool. This from every instance I've seen is only occurring when opcache is enabled as well. The workaround that is suggested for this currently is to add kill_all_lockers to disable functions or alternatively disable/remove opcache. The latter not being an option many are able to do. I've discussed this issue on the forums in multiple other threads, the most memorable for me at the time being0 -
Pasting more notes: You must configure Apache to allow enough server instances to handle the maximum number of PHP-FPM child processes. Set the Start Servers option in WHM"s Global Configuration interface (WHM >> Home >> Service Configuration >> Apache Configuration >> Global Configuration) to a value greater than or equal to the pm_max_children setting in WHM"s MultiPHP Manager interface (WHM >> Home >> Software >> MultiPHP Manager). From: # # # @cPanelLauren, Thanks for the reply! Would you happen to know where to set the 3 settings (above: emergency_restart_threshold, eri, pct) so they are persistent? If I'm wrapping my head around all three, then all three combined might mitigate the I"ve not hit any of Walter"s problems, but once it became default, RAM became cheap enough I just doubled the amount of RAM I normally used for a server and did a swag of a double amount of Max Children any site on the box will ever hit and then used that multiplied by the number of sites on the box to set Max Requests. Which is completely stupid, but works okay enough I don"t ever get sites going dark.
I wanted to edit the original post for this, but the forums are locked down so tight I can"t" To add to my frustration, while the above calculation has been working, it"s not because it was from sound logic. Egg on my face! Max Requests has nothing to do with the maximum request FPM as a whole does per pool before the pool respawns, it"s the "number of requests each child process should execute before respawning."0 -
Would you happen to know where to set the 3 settings (above: emergency_restart_threshold, eri, pct) so they are persistent? If I'm wrapping my head around all three, then all three combined might mitigate the
First I want to note that while it may be worth a try, adding kill_all_lockers to disable_functions is the official workaround from PHP, I would assume that if there were an alternate workaround that didn't involve disabling it they would have suggested that. These changes can be made in WHM>>Software>>MultiPHP Manager -> System PHP-FPM Configuration which is discussed here: MultiPHP INI Editor for WHM | cPanel & WHM Documentation Alternatively you can make them over CLI and this is discussed here: How to Manage Your php.ini Directives with PHP FPM | cPanel & WHM Documentation system-wide modifications can be made here:/var/cpanel/ApachePHPFPM/system_pool_defaults.yaml
Per VirtualHost modifications can be set here:/var/cpanel/userdata/[user]/[domain].php-fpm.yaml
For the items you're looking to modify:process.max int
emergency_restart_threshold int
emergency_restart_interval mixed
process_control_timeout mixed
These are all present in thesystem_pool_defaults
yaml file referenced above. A full list of configuration values can be found in our documentation here: Configuration Values of PHP-FPM | cPanel & WHM DocumentationI wanted to edit the original post for this, but the forums are locked down so tight I can"t"
Forum responses are editable for 24 hours after they're made.0
Please sign in to leave a comment.
Comments
13 comments