HTTPd and PHP OOM - Out of Memory
Appealing to some cpanel + WHM experts.
We experienced Out of Memory and MYsql restarts PHP and httpd on our Centos 6 platform of WHM version 60 and Centos 6.8 with server memory 8GB
/var/log/messages
Jan 12 Restart 1
php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jan 30 Restart 2
httpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
After Restart 1
We ran the mysqltuner.pl everyone talks about for tweaking mysql.cnf
Also we increase server memory from 8 - 12GB
Then added these settings as recommended.
After Restart 1
We ran the mysqltuner.pl it now recommends this ?
-------- InnoDB Metrics ----------------------------------------------------------------------------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 8
[OK] InnoDB File per table is activated
[OK] InnoDB buffer pool / data size: 9.0G/2.1G
[!!] Ratio InnoDB log file size / InnoDB Buffer pool size (2.77777777777778 %): 128.0M * 2/9.0G should be equal 25%
[!!] InnoDB buffer pool instances: 8
[--] InnoDB Buffer Pool Chunk Size not used or defined in your version
[OK] InnoDB Read buffer efficiency: 99.98% (345174626 hits/ 345244918 total)
[!!] InnoDB Write Log efficiency: 65.86% (109652 hits/ 166496 total)
[OK] InnoDB log waits: 0.00% (0 waits / 56844 writes)
Variables to adjust:
table_open_cache (> 2000)
key_buffer_size (> 88.1M)
innodb_log_file_size should be equals to 1/4 of buffer pool size (=2G) if possible.
innodb_buffer_pool_instances(=9)
Has anyone experience enough to verify the above configuration or experience with sar analysis for server and memory optimisation ?
Architecturally should we stick with this mix of Apache 2.4 / Mysql 5.6 or considering cloud Linux trialling it on a dev server found the migration to be painful. We would also consider a dedicated mysql server and webserver load balance if thats the better answer than struggling inside one server ?
BTW Apache logs the Restart 2 was triggered by a script coming from a GCP hosted server.
Kind Regards
Matt
-
Hello, You may find the following threads helpful when troubleshooting this issue: SOLVED - The process "php-cgi" was terminated because the system is low on memory Hang, Failed, Recovered and Out of Memory Let us know if this helps. Thank you. 0 -
Thanks Michael for the links I appreciated this command. ps -e -o pid,user,cpu,size,rss,cmd --sort -size,-rss | head Notable Results : Size in (kb) 4113 mysql - 8500944 /usr/sbin/mysqld 10511 named - 56472 /usr/sbin/named -u named 5300 root - 24000 /usr/bin/newrelic-daemon -c 4624 root - 442460 /usr/local/cpanel/3rdparty/bin/clamd 7798 nobody - 138296 /usr/local/apache/bin/httpd -k start 8002 nobody - 136508 /usr/local/apache/bin/httpd -k start What I am tracking is the mysqld service isn't good at releasing memory and is on a constant upward trend until oom killer is invoked. Do you have any information on MQL memory handling to reclaim this space back ? We have considered writing a memory monitor which restarts a service when it gets to a threshold level similar to oom killer yet with more variables and settings. Unless you'd consider this a waste if there is configuration for oom Killer ? Also want to carefully consider the impacts of implementing cloud Linux vs lightspeed as we are using some of the Apache modules. 0 -
Used the 'atop' -m for memory stats its very good. According to this tool we have 512MB swap and 0 free. For a server with 16 GB the recommended memory is higher does Cpanel have any guidelines ? Linux: Should You Use Twice the Amount of Ram as Swap Space? How do we safely increase the swap size ? our /etc/fstab : dev/xvda / ext4 discard,usrjquota=quota.user,jqfmt=vfsv0 1 1 /dev/xvdb none swap sw 0 0 none /dev/pts devpts gid=5,mode=620 0 0 0 -
Hello, Have you considered switching to MariaDB via "WHM >> MySQL/MariaDB Upgrade"? I've seen reports of improved memory performance when switching to MariaDB. Thank you. 0
Please sign in to leave a comment.
Comments
4 comments