Rarely, Apache will stop unexpectedly because of a segmentation fault. It stops suddenly, and the error log has an entry like:
[core:notice] [pid 1234] AH00060: seg fault or similar nasty error detected in the parent process
or
[core:notice] [pid 1234] AH00051: child pid 1234 exit signal Segmentation fault (11), possible coredump in /etc/apache2
To debug these types of problems, it's frequently advantageous to examine the "core dump" left by Apache when it ended. To do this, you must know how to use the gdb debugger, and you must turn on capturing core dump files from Apache. It's important to note that the steps below do NOT solve any problems themselves - the process sets up one more tool for determining the root cause.
Procedure
1. Create Backups
Use the following three commands to create backups of the values you're going to change so that you can set them back once you have completed troubleshooting. Core dumps can be large and contain sensitive data, so you probably do not wish to keep them enabled on a production server longer than necessary to capture your troubleshooting information.
mkdir /root/backup_proc_core_values
cp /proc/sys/fs/suid_dumpable /root/backup_proc_core_values/suid_dumpable
cp /proc/sys/kernel/core_pattern /root/backup_proc_core_values/core_pattern
2. Set up /proc
Now you'll set those variables you just backed up the values you need to capture core dumps for Apache. The core pattern can put the core files anywhere you want. Make sure that the directory you choose exists. Since core files may be large, I recommend putting them in /home. The /home partition frequently has the most free space on a cPanel server.
echo 2 > /proc/sys/fs/suid_dumpable
mkdir /home/core_files
echo '/home/core_files/%e.core.%t.%p' > /proc/sys/kernel/core_pattern
3. Set cPanel and Apache to generate core dumps in Tweak Settings & restart Apache
Go to Tweak Settings, and on the Security tab, there's an option to "Generate Core Dumps". It is off by default. Turn it on and use the save button at the bottom of the page.
Next, run systemctl edit to create a drop-in systemd configuration file to override the LimitCORE values:
systemctl edit httpd
Paste the following in the empty text editor that opens:
[Service]
LimitCORE=infinity
Save the changes and then run systemd daemon-reload:
systemctl daemon-reload
Daemon-reload reloads all unit files, so the latest changes go into effect. After this, confirm the LimitCORE value is infinity:
systemctl show httpd | grep LimitCORE
LimitCORE=infinity
LimitCORESoft=infinity
Restart Apache using systemctl:
systemctl restart httpd
Now the server is ready to generate core dumps from Apache. Please don't forget to turn these features off when you're done, as core dumps can contain sensitive data and be very large.
To disable core dumps when troubleshooting is completed:
- Change the "Generate Core Dumps" setting to "Off" in Tweak Settings
- Set the suid_dumpable back to its previous value (probably 0):
cp /root/backup_proc_core_values/suid_dumpable /proc/sys/fs/suid_dumpable
Remove the line with LimitCORE in /etc/systemd/system/httpd.service.d/override.conf file and run the systemctl daemon-reload command again:
systemctl daemon-reload
Optionally, set the core pattern back to its previous value as well:
cp /root/backup_proc_core_values/core_pattern /proc/sys/kernel/core_pattern