Symptoms
Attempting to restart Apache/PHP-FPM shows the below error during startup:
Apr 02 19:16:23 server restartsrv_httpd[24273]: at
/usr/local/cpanel/Cpanel/PHPFPM/Controller.pm line 140.
Apr 02 19:16:23 server restartsrv_httpd[24273]:
Cpanel::Exception::Services::RestartError
Apr 02 19:16:23 server restartsrv_httpd[24273]: Service Status
Apr 02 19:16:23 server restartsrv_httpd[24273]:
Found 1 versions of PHP-FPM: 56
Apr 02 19:16:23 server restartsrv_httpd[24273]: Service Error
Apr 02 19:16:23 server restartsrv_httpd[24273]:
(XID wg98e6) The “apache_php_fpm” service failed to restart because the… error:
Apr 02 19:16:23 server restartsrv_httpd[24273]:
apache_php_fpm has failed. Contact your system administrator if the ser...over.
Apr 02 19:16:23 server restartsrv_httpd[24273]:
restartsrv_apache_php_fpm --graceful exited unclean
Websites will show 500 errors. The Apache error_log will show errors like the below:
Fri Apr 2 20:38:29 2021 (4546): Fatal Error Unable to create lock file:
Bad file descriptor (9)
[Fri Apr 02 20:38:29.030721 2021] [cgid:error] [pid 4363:tid 47947418986240]
[client x.x.x.x:48980] End of script output before headers: ea-php56
Fri Apr 2 20:38:29 2021 (4547): Fatal Error Unable to create lock file:
Bad file descriptor (9)
[Fri Apr 02 20:38:29.126314 2021] [cgid:error] [pid 4363:tid 47947418986240]
[client x.x.x.x:48980] End of script output before headers: ea-php56
It can also be observed that creating files in /tmp fail with an error.
touch /tmp/test
touch: cannot touch ‘/tmp/test’: Read-only file system
Description
In the event of a server crash, it could cause the securetmp mount to get corrupted. When the server is rebooted, the PHP-FPM service will start failing. Other services and functions that need to write to /tmp will also report failures. Incomplete writes generally causes this type of corruption.
Workaround
Note: The below steps should be performed by a certified administrator to ensure nothing goes wrong.
1. Disable the /tmp from /etc/fstab with a text editor like vi, vim, nano.
The tmp path in the fstab file will look similar to the below.
/usr/tmpDSK /tmp ext3 defaults 0 0
/usr/swpDSK swap swap defaults 0 0
/tmp /var/tmp ext3 defaults,bind 0 0
Ensure "noauto" appears next to defaults or bind, so it looks like the below. This may already be configured, so you don't need to perform this step if it's already in place.
/usr/tmpDSK /tmp ext3 defaults,noauto 0 0
/usr/swpDSK swap swap defaults 0 0
/tmp /var/tmp ext3 defaults,bind,noauto 0 0
2. Disable the securetmp service.
systemctl disable securetmp
3. Next, reboot the server.
4. After the reboot has been completed, remove the old tmpDSK file.
/bin/rm /usr/tmpDSK
5. Then run the securetmp command to re-enable and create a new tmpDSK.
/scripts/securetmp
Enter "y" at the prompts to confirm, enable and configure the boot option.
6. Reboot the server one more time, and you are all set.