Mysqld terminates, system low on memory
I've been getting this error repeatedly: "The process "mysqld" was terminated because the system is low on memory".
I have a blog running on WordPress and a forum running on XenForo
In the log, I see that too many php processes are there. This usually happens once or twice in the day, and the load spikes so much that the server goes out of memory and mysqld terminates.
As suggested in the famous thread here, I tried checking the access logs in case this increased load is caused by increased traffic, but that does not seem to be the case as traffic is not high when this occurs. The load is very high, there are too many php processes open, there's the WordPress's index.php with a high CPU usage in the WHM's daily process log.
Even the mysql tables report some tables as crashed, but I do have a weekly cron job that repairs them, apart from the InnoDB ones I think.
Can anyone suggest what could be the problem?
Have attached some logs.
-
Hello, Could you let us know the contents of your /etc/my.cnf file? Thank you. 0 -
Hello, Could you let us know the contents of your /etc/my.cnf file? Thank you.
Hello, Yes, my.cnf contents:[mysqld] innodb=on default-storage-engine=MyISAM interactive_timeout=600 key_buffer_size=624M key_cache_block_size=4096 max_heap_table_size=96M max_join_size=1000000000 max_allowed_packet=268435456 open_files_limit=10000 query_cache_size=96M thread_cache_size=100 tmp_table_size=96M wait_timeout=7800 max_user_connections=200 myisam_recover_options=FORCE innodb_file_per_table=1 innodb_flush_log_at_trx_commit=0 innodb_purge_threads=1 innodb_support_xa=0 innodb_force_recovery=0 innodb_thread_concurrency=8 pid-file="/var/lib/mysql/mysqld.pid" innodb_buffer_pool_size=256M innodb_log_file_size=32M table_open_cache=450
I did change some parameters here after running the mysqltuner.pl script. Recent PHP changes were also made from mod_fcgid to suphp as the php handler because a script wasn't executing with fcgid. Though the error was occurring on mod_fcgid too. The ouf of memory error can be viewed here: - Removed - (I wasn't able to attach the file here).0 -
Out of memory log: [350298.620131] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0 [350298.624705] php cpuset=/ mems_allowed=0 [350298.625787] Pid: 17274, comm: php Not tainted 2.6.32-642.1.1.el6.x86_64 #1 [350298.649328] Call Trace: [350298.650646] [] ? dump_header+0x90/0x1b0 [350298.651862] [] ? security_real_capable_noaudit+0x3c/0x70 [350298.653133] [] ? oom_kill_process+0x82/0x2a0 [350298.654555] [] ? select_bad_process+0xe1/0x120 [350298.655656] [] ? out_of_memory+0x220/0x3c0 [350298.656867] [] ? __alloc_pages_nodemask+0x93c/0x950 [350298.658083] [] ? alloc_pages_vma+0x9a/0x150 [350298.659325] [] ? handle_pte_fault+0x73d/0xb20 [350298.660488] [] ? generic_file_aio_read+0x380/0x700 [350298.661615] [] ? handle_mm_fault+0x299/0x3d0 [350298.662762] [] ? perf_event_task_sched_out+0x33/0x70 [350298.664097] [] ? __do_page_fault+0x146/0x500 [350298.665289] [] ? schedule+0x3ee/0xb70 [350298.666395] [] ? mutex_lock+0x1e/0x50 [350298.667892] [] ? generic_file_llseek_size+0x8c/0xd0 [350298.669136] [] ? do_page_fault+0x3e/0xa0 [350298.670292] [] ? page_fault+0x25/0x30 [350298.671394] Mem-Info: [350298.672543] Node 0 DMA per-cpu: [350298.674122] CPU 0: hi: 0, btch: 1 usd: 0 [350298.675239] CPU 1: hi: 0, btch: 1 usd: 0 [350298.676308] Node 0 DMA32 per-cpu: [350298.677517] CPU 0: hi: 186, btch: 31 usd: 81 [350298.678545] CPU 1: hi: 186, btch: 31 usd: 166 [350298.679583] Node 0 Normal per-cpu: [350298.680669] CPU 0: hi: 186, btch: 31 usd: 143 [350298.681790] CPU 1: hi: 186, btch: 31 usd: 150 [350298.682882] active_anon:713777 inactive_anon:181639 isolated_anon:1504 [350298.682883] active_file:203 inactive_file:880 isolated_file:261 [350298.682884] unevictable:0 dirty:0 writeback:53 unstable:0 [350298.682885] free:22157 slab_reclaimable:5543 slab_unreclaimable:12270 [350298.682886] mapped:456 shmem:65 pagetables:25031 bounce:0 [350298.688560] Node 0 DMA free:15724kB min:248kB low:308kB high:372kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15312kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [350298.694818] lowmem_reserve[]: 0 3512 4017 4017 [350298.696525] Node 0 DMA32 free:63932kB min:58868kB low:73584kB high:88300kB active_anon:2658944kB inactive_anon:531784kB active_file:672kB inactive_file:3356kB unevictable:0kB isolated(anon):2560kB isolated(file):768kB present:3596500kB mlocked:0kB dirty:0kB writeback:168kB mapped:1388kB shmem:236kB slab_reclaimable:13132kB slab_unreclaimable:21136kB kernel_stack:10896kB pagetables:66728kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:66400 all_unreclaimable? no [350298.703573] lowmem_reserve[]: 0 0 505 505 [350298.705278] Node 0 Normal free:8352kB min:8464kB low:10580kB high:12696kB active_anon:196104kB inactive_anon:195128kB active_file:144kB inactive_file:1092kB unevictable:0kB isolated(anon):3328kB isolated(file):276kB present:517120kB mlocked:0kB dirty:0kB writeback:72kB mapped:400kB shmem:24kB slab_reclaimable:9040kB slab_unreclaimable:27944kB kernel_stack:5152kB pagetables:33396kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:60069 all_unreclaimable? no [350298.711828] lowmem_reserve[]: 0 0 0 0 [350298.713873] Node 0 DMA: 3*4kB 0*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15724kB [350298.724363] Node 0 DMA32: 952*4kB 580*8kB 327*16kB 146*32kB 553*64kB 47*128kB 2*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 62320kB [350298.730353] Node 0 Normal: 1374*4kB 6*8kB 1*16kB 0*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 8952kB [350298.736946] 20887 total pagecache pages [350298.738348] 18243 pages in swap cache [350298.739916] Swap cache stats: add 17697337, delete 17679094, find 10809983/12511189 [350298.742325] Free swap = 0kB [350298.743395] Total swap = 4194252kB [350298.777644] 1048575 pages RAM [350298.780166] 67960 pages reserved [350298.781357] 36048 pages shared [350298.782784] 947214 pages non-shared [350298.784523] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [350298.786654] [ 490] 0 490 2680 1 1 -17 -1000 udevd [350298.789306] [ 790] 0 790 2662 1 1 -17 -1000 udevd [350298.791688] [ 1342] 0 1342 62320 64 0 0 0 rsyslogd [350298.794406] [ 1357] 0 1357 4563 72 1 0 0 irqbalance [350298.796445] [ 1376] 25 1376 59678 161 0 0 0 named [350298.798547] [ 1394] 81 1394 5359 13 0 0 0 dbus-daemon [350298.800875] [ 1432] 0 1432 1020 1 0 0 0 acpid [350298.803071] [ 1451] 28 1451 336791 222 1 0 0 nscd [350298.806086] [ 1569] 0 1569 16086 36 0 -17 -1000 sshd [350298.808433] [ 1580] 38 1580 6262 39 1 0 0 ntpd [350298.811308] [ 2053] 0 2053 4662 11 1 0 0 dovecot [350298.814059] [ 2065] 401 2065 10639 2 0 0 0 pop3-login [350298.816606] [ 2066] 401 2066 10637 16 0 0 0 imap-login [350298.819023] [ 2067] 97 2067 3368 13 1 0 0 anvil [350298.821178] [ 2068] 0 2068 3401 14 1 0 0 log [350298.823629] [ 2070] 401 2070 10655 2 1 0 0 pop3-login [350298.826213] [ 2071] 401 2071 10637 16 1 0 0 imap-login [350298.828541] [ 2072] 0 2072 3659 16 1 0 0 config [350298.830667] [ 2082] 47 2082 18238 29 1 0 0 exim [350298.832887] [ 2100] 0 2100 49742 402 0 0 0 /usr/local/cpan [350298.835547] [ 2102] 0 2102 52925 184 0 0 0 spamd child [350298.837546] [ 2129] 0 2129 45352 2 1 0 0 abrtd [350298.840235] [ 2139] 0 2139 45162 12 1 0 0 abrt-dump-oops [350298.842453] [ 2150] 0 2150 19189 164 1 0 0 httpd [350298.844936] [ 2165] 0 2165 34191 5 0 0 0 pure-ftpd [350298.847513] [ 2167] 0 2167 34036 3 1 0 0 pure-authd [350298.849849] [ 2260] 0 2260 29271 34 1 0 0 crond [350298.852149] [ 2275] 0 2275 4857 1 1 0 0 atd [350298.854527] [ 2375] 0 2375 26952 158 1 0 0 cpsrvd (SSL) - [350298.857220] [ 2381] 0 2381 8172 72 1 0 0 queueprocd - wa [350298.859662] [ 2391] 0 2391 27506 32 1 0 0 dnsadmin - serv [350298.861835] [ 2419] 0 2419 20808 236 1 0 0 tailwatchd [350298.864235] [ 2433] 0 2433 19586 37 0 0 0 cPhulkd - proce [350298.866265] [ 2476] 0 2476 33028 74 1 0 0 cpdavd - accept [350298.869020] [ 2512] 0 2512 5915 80 0 0 0 cpanellogd - sl [350298.871438] [ 2540] 0 2540 1016 2 1 0 0 mingetty [350298.873466] [ 2542] 0 2542 1016 2 0 0 0 mingetty [350298.876566] [ 2545] 0 2545 1016 2 1 0 0 mingetty [350298.878716] [ 2547] 0 2547 2679 1 0 -17 -1000 udevd [350298.881694] [ 2548] 0 2548 1016 2 1 0 0 mingetty [350298.884167] [ 2550] 0 2550 1016 2 1 0 0 mingetty [350298.886919] [ 2552] 0 2552 1016 2 1 0 0 mingetty [350298.889212] [13473] 0 13473 3367 2 0 0 0 ssl-params [350298.891793] [ 9484] 0 9484 2394 15 0 0 0 mysqld_safe [350298.894148] [10131] 0 10131 146212 16 0 0 0 clamd [350298.897015] [10833] 402 10833 565594 43704 1 0 0 mysqld [350298.899710] [10101] 99 10101 19189 91 0 0 0 httpd [350298.903219] [10102] 0 10102 19061 16 0 0 0 leechprotect [350298.905757] [10105] 99 10105 334311 1433 0 0 0 httpd [350298.908375] [10106] 99 10106 334148 1290 1 0 0 httpd [350298.910609] [10107] 99 10107 334132 1629 0 0 0 httpd [350298.913125] [10193] 99 10193 334447 1283 0 0 0 httpd [350298.915317] [10572] 99 10572 334296 1454 1 0 0 httpd [350298.917957] [11640] 99 11640 334172 1098 0 0 0 httpd [350298.940859] [17266] 500 17266 67391 4405 0 0 0 php [350298.943043] [17267] 500 17267 68176 5563 0 0 0 php [350298.945111] [17268] 500 17268 67258 3739 0 0 0 php [350298.947643] [17269] 500 17269 67207 3737 0 0 0 php [350298.950179] [17270] 500 17270 69232 7128 0 0 0 php [350298.952471] [17271] 500 17271 67207 4163 1 0 0 php [350298.954654] [17272] 500 17272 66872 4234 0 0 0 php [350298.957165] [17273] 500 17273 67201 4464 0 0 0 php [350298.959682] [17274] 500 17274 67006 4889 1 0 0 php [350298.962353] [17275] 500 17275 67564 5368 0 0 0 php [350298.965809] [17276] 500 17276 67207 3914 0 0 0 php [350298.968943] [17277] 500 17277 67385 5173 1 0 0 php [350298.971422] [17278] 500 17278 67258 3936 1 0 0 php [350298.973697] [17279] 500 17279 67194 4690 0 0 0 php [350298.976801] [17280] 500 17280 67194 4433 0 0 0 php [350298.979837] [17281] 500 17281 67391 4686 0 0 0 php [350298.982474] [17282] 500 17282 67271 4478 0 0 0 php [350298.984817] [17283] 500 17283 66195 3007 0 0 0 php [350298.987409] [17284] 500 17284 67455 5214 1 0 0 php [350298.990438] [17285] 500 17285 67271 5281 0 0 0 php [350298.992784] [17286] 500 17286 67650 5195 0 0 0 php [350298.995364] [17287] 500 17287 66324 3838 0 0 0 php [350298.997480] [17288] 500 17288 69394 7827 0 0 0 php [350298.999829] [17289] 500 17289 67207 4614 0 0 0 php [350299.001774] [17290] 500 17290 66997 4676 1 0 0 php [350299.003670] [17291] 500 17291 67194 4353 0 0 0 php [350299.006868] [17292] 500 17292 67450 4823 0 0 0 php [350299.009223] [17293] 500 17293 67391 5577 0 0 0 php [350299.011355] [17294] 500 17294 67452 4386 0 0 0 php [350299.014289] [17295] 500 17295 69600 7275 0 0 0 php [350299.016437] [17296] 500 17296 69496 7707 0 0 0 php [350299.018945] [17297] 500 17297 67391 4847 0 0 0 php [350299.021336] [17298] 500 17298 67383 5149 0 0 0 php [350299.023280] [17299] 500 17299 66809 4182 0 0 0 php [350299.025469] [17300] 500 17300 67207 4029 0 0 0 php [350299.027403] [17301] 500 17301 67207 3717 1 0 0 php [350299.029556] [17302] 500 17302 67391 4422 0 0 0 php [350299.031995] [17303] 500 17303 67459 4402 0 0 0 php [350299.034431] [17304] 500 17304 69411 7612 0 0 0 php [350299.036560] [17305] 500 17305 67455 4531 0 0 0 php [350299.039460] [17306] 500 17306 66547 3994 1 0 0 php [350299.041403] [17307] 500 17307 69406 7885 0 0 0 php [350299.043566] [17308] 500 17308 68052 5806 0 0 0 php [350299.045967] [17309] 500 17309 67207 5305 1 0 0 php [350299.049183] [17310] 500 17310 67455 5329 0 0 0 php [350299.051991] [17311] 500 17311 67455 4668 0 0 0 php [350299.054280] [17312] 500 17312 67391 4033 0 0 0 php [350299.056262] [17313] 500 17313 67384 4708 0 0 0 php [350299.059027] [17314] 500 17314 67391 5346 0 0 0 php [350299.061119] [17315] 500 17315 67387 4926 0 0 0 php [350299.063469] [17316] 500 17316 67391 4523 0 0 0 php [350299.066010] [17317] 500 17317 67391 5182 0 0 0 php [350299.068024] [17318] 500 17318 67455 5264 0 0 0 php [350299.070347] [17319] 500 17319 67391 4299 0 0 0 php [350299.072337] [17320] 500 17320 69012 7721 0 0 0 php [350299.074420] [17321] 500 17321 67520 5352 0 0 0 php [350299.076530] [17357] 500 17357 67455 5333 0 0 0 php [350299.078552] [17360] 500 17360 68754 6829 1 0 0 php [350299.080401] [17363] 500 17363 67391 5124 0 0 0 php [350299.082341] [17364] 500 17364 69600 8627 0 0 0 php [350299.084885] [17365] 500 17365 68199 7222 0 0 0 php [350299.087587] [17368] 500 17368 68754 7460 0 0 0 php [350299.089945] [17369] 500 17369 67391 5297 1 0 0 php [350299.091945] [17375] 500 17375 69672 8672 0 0 0 php [350299.094660] [17377] 500 17377 67395 5649 0 0 0 php [350299.096602] [17380] 500 17380 69492 8154 0 0 0 php [350299.098716] [17382] 500 17382 69536 8141 0 0 0 php [350299.100626] [17383] 500 17383 69664 8606 0 0 0 php [350299.102990] [17422] 500 17422 70186 11312 1 0 0 php [350299.105022] [17423] 99 17423 334211 2205 0 0 0 httpd [350299.107096] [17453] 500 17453 70056 12985 0 0 0 php [350299.109248] [17454] 500 17454 69995 12786 0 0 0 php [350299.112064] [17457] 500 17457 71827 15152 0 0 0 php [350299.114144] [17468] 500 17468 60674 8959 1 0 0 php [350299.116146] [17469] 500 17469 60738 9081 1 0 0 php [350299.118310] [17470] 500 17470 54427 3423 0 0 0 php [350299.120502] [17476] 500 17476 60738 9171 1 0 0 php [350299.122413] [17477] 500 17477 59667 8199 0 0 0 php [350299.124641] [17482] 500 17482 54259 3346 1 0 0 php [350299.126649] [17484] 0 17484 35567 84 0 0 0 crond [350299.129146] [17485] 500 17485 54007 3355 0 0 0 php [350299.131367] [17487] 99 17487 334166 1209 0 0 0 httpd [350299.133445] [17491] 99 17491 334201 1362 1 0 0 httpd [350299.135701] [17494] 500 17494 53333 2409 0 0 0 php [350299.137930] [17496] 500 17496 58559 7913 0 0 0 php [350299.140207] [17497] 99 17497 334138 1328 0 0 0 httpd [350299.142168] [17505] 500 17505 57589 6978 0 0 0 php [350299.144212] [17506] 500 17506 54215 3280 0 0 0 php [350299.146347] [17507] 500 17507 54215 3433 0 0 0 php [350299.148410] [17508] 500 17508 55350 4696 0 0 0 php [350299.170789] [17509] 500 17509 56907 6248 0 0 0 php [350299.173387] [17511] 500 17511 54280 3159 1 0 0 php [350299.175506] [17590] 500 17590 59788 9075 0 0 0 php [350299.177513] [17591] 500 17591 60735 10096 0 0 0 php [350299.179530] [17593] 500 17593 58943 8408 0 0 0 php [350299.181537] [17595] 500 17595 54378 3372 1 0 0 php [350299.183874] [17596] 500 17596 58943 8353 1 0 0 php [350299.185941] [17598] 500 17598 60671 9975 1 0 0 php [350299.187915] [17599] 500 17599 55435 4792 1 0 0 php [350299.189908] [17600] 500 17600 49128 1241 0 0 0 php [350299.192339] [17605] 500 17605 54115 2900 0 0 0 php [350299.194391] [17607] 500 17607 55834 5115 0 0 0 php [350299.196391] [17608] 500 17608 54215 3350 1 0 0 php [350299.198412] [17609] 500 17609 54280 3444 0 0 0 php [350299.200637] [17610] 500 17610 58943 8282 0 0 0 php [350299.203100] [17612] 500 17612 55480 4790 0 0 0 php [350299.205249] [17614] 500 17614 59724 9067 0 0 0 php [350299.207379] [17615] 500 17615 54797 4082 0 0 0 php [350299.209676] [17616] 500 17616 58369 7816 0 0 0 php [350299.211744] [17619] 500 17619 56842 6144 0 0 0 php [350299.213619] [17620] 500 17620 56782 6085 0 0 0 php [350299.215524] [17622] 500 17622 56486 5757 1 0 0 php [350299.217405] [17625] 500 17625 55687 4968 0 0 0 php [350299.219521] [17626] 500 17626 54045 3036 0 0 0 php [350299.221424] [17629] 500 17629 59072 8347 0 0 0 php [350299.223704] [17631] 500 17631 55020 4084 0 0 0 php [350299.226940] [17633] 500 17633 54422 3939 0 0 0 php [350299.229236] [17636] 500 17636 54376 3714 1 0 0 php [350299.232666] [17637] 500 17637 60735 10040 1 0 0 php [350299.234645] [17638] 500 17638 55552 4865 0 0 0 php [350299.236683] [17641] 0 17641 26526 50 0 0 0 sh [350299.238738] [17642] 500 17642 54743 4099 0 0 0 php [350299.240741] [17643] 500 17643 54010 3313 1 0 0 php [350299.242787] [17644] 500 17644 55676 4917 0 0 0 php [350299.244824] [17646] 500 17646 54749 3935 1 0 0 php [350299.247230] [17647] 500 17647 60031 9405 0 0 0 php [350299.249504] [17649] 500 17649 54742 4040 0 0 0 php [350299.251636] [17650] 500 17650 50192 1162 0 0 0 php [350299.253612] [17651] 500 17651 57490 6852 1 0 0 php [350299.256050] [17652] 500 17652 54428 3704 1 0 0 php [350299.258169] [17653] 500 17653 60096 9409 0 0 0 php [350299.260601] [17654] 0 17654 35506 953 1 0 0 /usr/local/cpan [350299.262765] [17655] 500 17655 54742 4065 0 0 0 php [350299.265208] [17658] 500 17658 54180 3133 1 0 0 php [350299.267654] [17659] 500 17659 54202 3720 0 0 0 php [350299.269827] [17661] 500 17661 57069 6365 0 0 0 php [350299.272417] [17662] 500 17662 54200 3733 0 0 0 php [350299.275127] [17663] 500 17663 54002 3318 0 0 0 php [350299.277601] [17664] 500 17664 54490 3957 0 0 0 php [350299.280010] [17667] 0 17667 20908 730 0 0 0 tailwatchd - ch [350299.282374] [17668] 500 17668 54294 3806 0 0 0 php [350299.284645] [17671] 500 17671 55103 4433 0 0 0 php [350299.286942] [17672] 500 17672 58943 8258 0 0 0 php [350299.288975] [17673] 500 17673 55088 4527 1 0 0 php [350299.291023] [17700] 500 17700 54552 4162 0 0 0 php [350299.292977] [17706] 500 17706 54078 3218 1 0 0 php [350299.294971] [17719] 500 17719 53672 3173 1 0 0 php [350299.296938] [17772] 99 17772 252176 846 1 0 0 httpd [350299.299512] [17788] 99 17788 219408 664 0 0 0 httpd [350299.301524] [17789] 99 17789 235792 1147 0 0 0 httpd [350299.303549] [17791] 99 17791 235792 666 1 0 0 httpd [350299.305550] [17809] 500 17809 50976 2203 0 0 0 php [350299.307718] [17810] 500 17810 54202 3819 1 0 0 php [350299.310657] [17813] 99 17813 219408 749 1 0 0 httpd [350299.313010] [17814] 99 17814 235792 692 1 0 0 httpd [350299.315186] [17966] 500 17966 54422 3964 1 0 0 php [350299.317185] [17967] 500 17967 32182 645 0 0 0 php [350299.319346] [17968] 500 17968 54486 4054 1 0 0 php [350299.321413] [17969] 500 17969 55334 4740 1 0 0 php [350299.323495] [17999] 500 17999 54550 4063 0 0 0 php [350299.325636] [18000] 500 18000 54422 3871 0 0 0 php [350299.327666] [18002] 500 18002 54567 4004 1 0 0 php [350299.330300] [18023] 500 18023 54422 3911 0 0 0 php [350299.332789] [18025] 500 18025 54488 4079 1 0 0 php [350299.335951] [18026] 500 18026 54142 3743 0 0 0 php [350299.338709] [18029] 500 18029 53767 3292 1 0 0 php [350299.341568] [18030] 500 18030 54294 3896 1 0 0 php [350299.344237] [18063] 500 18063 32182 636 0 0 0 php [350299.346566] [18065] 500 18065 32347 827 0 0 0 php [350299.349040] [18066] 500 18066 32149 563 0 0 0 php [350299.351194] [18067] 500 18067 32116 521 0 0 0 php [350299.353195] [18069] 500 18069 32347 827 0 0 0 php [350299.355329] [18070] 500 18070 32281 756 0 0 0 php [350299.357368] [18072] 500 18072 52608 1915 1 0 0 php [350299.359529] [18074] 500 18074 32347 820 0 0 0 php [350299.361566] [18075] 500 18075 30393 519 0 0 0 php [350299.363873] [18076] 0 18076 26952 264 0 0 0 cpsrvd (SSL) - [350299.365889] [18078] 500 18078 2754 21 0 0 0 php [350299.368501] [18079] 500 18079 32248 693 1 0 0 php [350299.370901] [18080] 500 18080 32182 647 1 0 0 php [350299.398974] [18082] 500 18082 32182 606 0 0 0 php [350299.401369] [18084] 500 18084 32347 808 0 0 0 php [350299.403551] [18085] 500 18085 2754 20 0 0 0 php [350299.405727] [18105] 500 18105 52351 1829 0 0 0 php [350299.408946] [18106] 500 18106 32347 801 0 0 0 php [350299.411218] [18107] 0 18107 35567 105 0 0 0 crond [350299.413639] [18109] 500 18109 32347 821 0 0 0 php [350299.417419] [18113] 0 18113 31058 138 0 0 0 clean_techproxy [350299.419412] [18114] 99 18114 334148 1297 0 0 0 httpd [350299.421328] [18142] 500 18142 13982 43 0 0 0 php [350299.423495] [18143] 500 18143 5500 26 1 0 0 php [350299.425361] [18144] 500 18144 2754 18 0 0 0 php [350299.428537] [18145] 500 18145 2754 19 0 0 0 php [350299.431746] [18146] 500 18146 4463 23 0 0 0 php [350299.433815] [18147] 500 18147 4980 24 0 0 0 php [350299.435957] [18148] 0 18148 16010 109 1 0 0 sshd [350299.438044] [18149] 500 18149 5500 23 1 0 0 php [350299.440161] [18150] 0 18150 35506 123 0 0 0 crond [350299.443130] [18151] 99 18151 1532 18 1 0 0 suphp [350299.445118] [18152] 500 18152 3819 21 0 0 0 php [350299.447159] [18153] 500 18153 5500 26 0 0 0 php [350299.449159] [18154] 500 18154 5500 25 0 0 0 php [350299.451606] [18155] 500 18155 5500 26 0 0 0 php [350299.454054] [18156] 99 18156 334148 1413 0 0 0 httpd [350299.456683] [18157] 500 18157 2754 18 0 0 0 php [350299.458958] [18158] 47 18158 5187 29 0 0 0 exim [350299.460931] [18159] 500 18159 2754 19 0 0 0 php [350299.464651] [18160] 500 18160 2754 18 0 0 0 php [350299.466686] [18161] 500 18161 2754 19 0 0 0 php [350299.468658] [18162] 500 18162 3116 92 1 0 0 suphp [350299.470853] [18163] 500 18163 2754 18 0 0 0 php [350299.473249] [18164] 99 18164 2067 19 1 0 0 suphp [350299.476234] [18165] 500 18165 2754 18 0 0 0 php [350299.478515] [18166] 0 18166 35506 122 0 0 0 crond [350299.480537] [18167] 0 18167 35506 121 1 0 0 crond [350299.483045] [18168] 0 18168 35506 120 1 0 0 crond [350299.485540] [18169] 0 18169 35506 119 1 0 0 crond [350299.488173] [18170] 0 18170 35506 119 1 0 0 crond [350299.491015] [18171] 0 18171 35506 119 0 0 0 crond [350299.493674] [18172] 0 18172 35506 119 0 0 0 crond [350299.495627] [18173] 0 18173 35506 119 0 0 0 crond [350299.497897] [18174] 0 18174 26952 296 0 0 0 cpsrvd (SSL) - [350299.500545] [18175] 99 18175 334296 1416 0 0 0 httpd [350299.502775] Out of memory: Kill process 10833 (mysqld) score 38 or sacrifice child [350299.505430] Killed process 10833, UID 402, (mysqld) total-vm:2262376kB, anon-rss:174524kB, file-rss:172kB
There are a lot of php processes running.0 -
Another out of memory log: [562074.326710] mysqld invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [562074.329646] mysqld cpuset=/ mems_allowed=0 [562074.330882] Pid: 21351, comm: mysqld Not tainted 2.6.32-642.1.1.el6.x86_64 #1 [562074.333171] Call Trace: [562074.334870] [] ? dump_header+0x90/0x1b0 [562074.336812] [] ? security_real_capable_noaudit+0x3c/0x70 [562074.338692] [] ? oom_kill_process+0x82/0x2a0 [562074.340270] [] ? select_bad_process+0xe1/0x120 [562074.341582] [] ? out_of_memory+0x220/0x3c0 [562074.343229] [] ? __alloc_pages_nodemask+0x93c/0x950 [562074.344930] [] ? ext4_get_block+0x0/0x120 [ext4] [562074.346436] [] ? alloc_pages_current+0xaa/0x110 [562074.347604] [] ? __page_cache_alloc+0x87/0x90 [562074.348889] [] ? find_get_page+0x1e/0xa0 [562074.350660] [] ? filemap_fault+0x1a7/0x500 [562074.352207] [] ? __do_fault+0x54/0x530 [562074.353872] [] ? handle_pte_fault+0xf7/0xb20 [562074.355966] [] ? rwsem_down_failed_common+0x95/0x1d0 [562074.357531] [] ? rwsem_down_read_failed+0x26/0x30 [562074.359503] [] ? handle_mm_fault+0x299/0x3d0 [562074.361105] [] ? call_rwsem_down_read_failed+0x14/0x30 [562074.362522] [] ? __do_page_fault+0x146/0x500 [562074.364590] [] ? lock_sock_nested+0xac/0xc0 [562074.387996] [] ? _spin_unlock_bh+0x1b/0x20 [562074.389187] [] ? pvclock_clocksource_read+0x58/0xd0 [562074.390457] [] ? kvm_clock_read+0x1c/0x20 [562074.391586] [] ? kvm_clock_get_cycles+0x9/0x10 [562074.392673] [] ? getnstimeofday+0x58/0xf0 [562074.394697] [] ? do_page_fault+0x3e/0xa0 [562074.395974] [] ? page_fault+0x25/0x30 [562074.397218] Mem-Info: [562074.399103] Node 0 DMA per-cpu: [562074.400605] CPU 0: hi: 0, btch: 1 usd: 0 [562074.402129] CPU 1: hi: 0, btch: 1 usd: 0 [562074.403506] Node 0 DMA32 per-cpu: [562074.405629] CPU 0: hi: 186, btch: 31 usd: 0 [562074.406749] CPU 1: hi: 186, btch: 31 usd: 30 [562074.407914] Node 0 Normal per-cpu: [562074.409325] CPU 0: hi: 186, btch: 31 usd: 0 [562074.410787] CPU 1: hi: 186, btch: 31 usd: 30 [562074.411941] active_anon:717855 inactive_anon:182000 isolated_anon:5083 [562074.411943] active_file:32 inactive_file:192 isolated_file:64 [562074.411944] unevictable:0 dirty:0 writeback:259 unstable:0 [562074.411944] free:21353 slab_reclaimable:5762 slab_unreclaimable:10906 [562074.411945] mapped:63 shmem:43 pagetables:21560 bounce:0 [562074.418218] Node 0 DMA free:15724kB min:248kB low:308kB high:372kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15312kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes [562074.424382] lowmem_reserve[]: 0 3512 4017 4017 [562074.426602] Node 0 DMA32 free:61060kB min:58868kB low:73584kB high:88300kB active_anon:2667432kB inactive_anon:524160kB active_file:4kB inactive_file:396kB unevictable:0kB isolated(anon):15980kB isolated(file):384kB present:3596500kB mlocked:0kB dirty:0kB writeback:560kB mapped:232kB shmem:172kB slab_reclaimable:15184kB slab_unreclaimable:17264kB kernel_stack:6736kB pagetables:66680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:64 all_unreclaimable? no [562074.432920] lowmem_reserve[]: 0 0 505 505 [562074.434773] Node 0 Normal free:8392kB min:8464kB low:10580kB high:12696kB active_anon:203604kB inactive_anon:203616kB active_file:124kB inactive_file:320kB unevictable:0kB isolated(anon):4992kB isolated(file):128kB present:517120kB mlocked:0kB dirty:0kB writeback:476kB mapped:20kB shmem:0kB slab_reclaimable:7864kB slab_unreclaimable:26360kB kernel_stack:6832kB pagetables:19560kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no [562074.441562] lowmem_reserve[]: 0 0 0 0 [562074.443251] Node 0 DMA: 3*4kB 0*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15724kB [562074.449972] Node 0 DMA32: 492*4kB 4530*8kB 511*16kB 140*32kB 89*64kB 3*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 60784kB [562074.455851] Node 0 Normal: 1455*4kB 68*8kB 3*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 8396kB [562074.461844] 22462 total pagecache pages [562074.463320] 21989 pages in swap cache [562074.464571] Swap cache stats: add 34726956, delete 34704967, find 25405257/28809084 [562074.466666] Free swap = 0kB [562074.467793] Total swap = 4194252kB [562074.508867] 1048575 pages RAM [562074.510066] 67960 pages reserved [562074.511918] 7658 pages shared [562074.513082] 947230 pages non-shared [562074.514178] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [562074.516236] [ 490] 0 490 2680 0 1 -17 -1000 udevd [562074.518561] [ 790] 0 790 2662 0 1 -17 -1000 udevd [562074.520752] [ 1342] 0 1342 62368 50 1 0 0 rsyslogd [562074.522842] [ 1357] 0 1357 4563 23 0 0 0 irqbalance [562074.525130] [ 1376] 25 1376 59678 129 0 0 0 named [562074.527174] [ 1394] 81 1394 5359 1 0 0 0 dbus-daemon [562074.529698] [ 1432] 0 1432 1020 0 0 0 0 acpid [562074.532021] [ 1451] 28 1451 336791 177 0 0 0 nscd [562074.534053] [ 1569] 0 1569 16086 24 1 -17 -1000 sshd [562074.536171] [ 1580] 38 1580 6262 27 1 0 0 ntpd [562074.539282] [ 2053] 0 2053 4662 1 0 0 0 dovecot [562074.542072] [ 2065] 401 2065 10639 1 0 0 0 pop3-login [562074.545036] [ 2066] 401 2066 10637 1 0 0 0 imap-login [562074.547647] [ 2067] 97 2067 3368 1 0 0 0 anvil [562074.550029] [ 2068] 0 2068 3401 1 1 0 0 log [562074.552190] [ 2070] 401 2070 10655 1 0 0 0 pop3-login [562074.554224] [ 2071] 401 2071 10670 1 0 0 0 imap-login [562074.556886] [ 2072] 0 2072 3659 1 0 0 0 config [562074.560112] [ 2082] 47 2082 18238 1 1 0 0 exim [562074.562160] [ 2100] 0 2100 49742 335 1 0 0 /usr/local/cpan [562074.564197] [ 2102] 0 2102 53119 223 0 0 0 spamd child [562074.566209] [ 2129] 0 2129 45352 1 0 0 0 abrtd [562074.568273] [ 2139] 0 2139 45197 1 0 0 0 abrt-dump-oops [562074.570489] [ 2150] 0 2150 19189 124 0 0 0 httpd [562074.572607] [ 2165] 0 2165 34191 0 0 0 0 pure-ftpd [562074.574719] [ 2167] 0 2167 34036 1 0 0 0 pure-authd [562074.577609] [ 2260] 0 2260 29271 27 1 0 0 crond [562074.580212] [ 2275] 0 2275 4857 5 1 0 0 atd [562074.582301] [ 2375] 0 2375 26952 125 0 0 0 cpsrvd (SSL) - [562074.584483] [ 2381] 0 2381 8172 69 0 0 0 queueprocd - wa [562074.586500] [ 2391] 0 2391 27506 53 0 0 0 dnsadmin - serv [562074.589509] [ 2419] 0 2419 20822 258 0 0 0 tailwatchd [562074.592045] [ 2433] 0 2433 19586 1 0 0 0 cPhulkd - proce [562074.594295] [ 2476] 0 2476 33028 60 0 0 0 cpdavd - accept [562074.597210] [ 2512] 0 2512 5915 102 1 0 0 cpanellogd - sl [562074.600503] [ 2540] 0 2540 1016 1 1 0 0 mingetty [562074.603689] [ 2542] 0 2542 1016 1 0 0 0 mingetty [562074.610302] [ 2545] 0 2545 1016 1 1 0 0 mingetty [562074.615154] [ 2547] 0 2547 2679 0 0 -17 -1000 udevd [562074.617989] [ 2548] 0 2548 1016 1 1 0 0 mingetty [562074.622920] [ 2550] 0 2550 1016 1 1 0 0 mingetty [562074.625853] [ 2552] 0 2552 1016 1 1 0 0 mingetty [562074.628530] [30499] 0 30499 147473 23 0 0 0 clamd [562074.631934] [ 8712] 0 8712 2394 1 1 0 0 mysqld_safe [562074.634865] [ 9408] 402 9408 564397 42943 0 0 0 mysqld [562074.637620] [24553] 0 24553 27283 1 1 0 0 sshd [562074.641062] [24562] 0 24562 13893 1 0 0 0 sftp-server [562074.643785] [25296] 0 25296 27283 1 1 0 0 sshd [562074.646513] [25312] 0 25312 26649 1 0 0 0 bash [562074.649571] [25954] 99 25954 19222 60 0 0 0 httpd [562074.652579] [25955] 0 25955 19061 1 0 0 0 leechprotect [562074.655407] [25958] 99 25958 334134 795 0 0 0 httpd [562074.657912] [25959] 99 25959 334193 806 1 0 0 httpd [562074.660551] [25960] 99 25960 334438 802 0 0 0 httpd [562074.663110] [26044] 99 26044 334439 803 0 0 0 httpd [562074.665618] [26206] 99 26206 334191 844 1 0 0 httpd [562074.668698] [26361] 99 26361 334134 886 0 0 0 httpd [562074.671241] [28324] 0 28324 27283 1 1 0 0 sshd [562074.673759] [28333] 0 28333 26649 1 0 0 0 bash [562074.676257] [28546] 99 28546 334167 821 0 0 0 httpd [562074.702428] [ 1091] 0 1091 27283 1 1 0 0 sshd [562074.704538] [ 1130] 0 1130 26649 1 0 0 0 bash [562074.706652] [ 5276] 500 5276 77474 7801 1 0 0 php [562074.708866] [ 5278] 500 5278 77582 7649 0 0 0 php [562074.711662] [ 5281] 500 5281 77580 7259 1 0 0 php [562074.714235] [ 5290] 500 5290 77601 7474 1 0 0 php [562074.716490] [ 5295] 500 5295 77538 7540 1 0 0 php [562074.718824] [ 5299] 500 5299 77414 7551 1 0 0 php [562074.721384] [ 5302] 500 5302 77518 8104 1 0 0 php [562074.724114] [ 5304] 500 5304 77414 7522 1 0 0 php [562074.726653] [ 5308] 500 5308 77582 7241 0 0 0 php [562074.729482] [ 5309] 500 5309 77542 7294 0 0 0 php [562074.731758] [ 5367] 500 5367 79215 8223 0 0 0 php [562074.734948] [ 5369] 500 5369 79230 7073 1 0 0 php [562074.738000] [ 5389] 500 5389 74670 6863 1 0 0 php [562074.740609] [ 5407] 500 5407 74996 5832 0 0 0 php [562074.742809] [ 5415] 500 5415 75850 6233 1 0 0 php [562074.745813] [ 5416] 500 5416 75061 5948 1 0 0 php [562074.748265] [ 5418] 500 5418 74992 5796 1 0 0 php [562074.750694] [ 5420] 500 5420 76190 7332 0 0 0 php [562074.753004] [ 5430] 500 5430 74932 6113 1 0 0 php [562074.755384] [ 5434] 500 5434 75460 6480 1 0 0 php [562074.757639] [ 5439] 500 5439 74928 5879 1 0 0 php [562074.761562] [ 5440] 500 5440 75125 5701 1 0 0 php [562074.764306] [ 5442] 500 5442 75914 6693 1 0 0 php [562074.766600] [ 5446] 500 5446 75214 5475 1 0 0 php [562074.768955] [ 5448] 500 5448 74705 6309 1 0 0 php [562074.771372] [ 5450] 500 5450 73976 6564 0 0 0 php [562074.773778] [ 5451] 500 5451 73831 6459 1 0 0 php [562074.776178] [ 5458] 500 5458 74611 7603 0 0 0 php [562074.778666] [ 5459] 500 5459 75781 8240 1 0 0 php [562074.781649] [ 5465] 500 5465 73772 7310 1 0 0 php [562074.784412] [ 5470] 500 5470 74414 7797 1 0 0 php [562074.788349] [ 5476] 500 5476 74864 8214 1 0 0 php [562074.790625] [ 5480] 500 5480 73510 8366 0 0 0 php [562074.793613] [ 5482] 500 5482 73708 8698 0 0 0 php [562074.796689] [ 5492] 500 5492 72444 11027 1 0 0 php [562074.799184] [ 5496] 500 5496 72784 12415 1 0 0 php [562074.802301] [ 5499] 500 5499 73768 11388 0 0 0 php [562074.804993] [ 5500] 500 5500 72582 11302 0 0 0 php [562074.808597] [ 5503] 500 5503 70752 9687 0 0 0 php [562074.810941] [ 5513] 500 5513 73447 10543 0 0 0 php [562074.813474] [ 5519] 500 5519 72728 12005 1 0 0 php [562074.815596] [ 5520] 500 5520 72710 11134 1 0 0 php [562074.818435] [ 5523] 500 5523 72246 11313 0 0 0 php [562074.821278] [ 5533] 500 5533 73576 12705 0 0 0 php [562074.824227] [ 5536] 500 5536 70305 11674 1 0 0 php [562074.826292] [ 5541] 500 5541 73378 13119 1 0 0 php [562074.828638] [ 5545] 500 5545 69769 10932 1 0 0 php [562074.831461] [ 5548] 500 5548 69382 10296 0 0 0 php [562074.833803] [ 5549] 500 5549 73458 13978 1 0 0 php [562074.836589] [ 5564] 500 5564 69708 11383 1 0 0 php [562074.839962] [ 5568] 500 5568 69380 12267 0 0 0 php [562074.842304] [ 5570] 500 5570 69580 13324 1 0 0 php [562074.844533] [ 5574] 500 5574 69057 13993 0 0 0 php [562074.847082] [ 5577] 500 5577 69450 13875 1 0 0 php [562074.849309] [ 5578] 500 5578 70048 14829 1 0 0 php [562074.851653] [ 5587] 500 5587 69514 15274 1 0 0 php [562074.854140] [ 5589] 500 5589 69382 16196 1 0 0 php [562074.856537] [ 5593] 500 5593 68012 14763 0 0 0 php [562074.858820] [ 5597] 500 5597 71343 18344 0 0 0 php [562074.861408] [ 5598] 500 5598 66115 12838 1 0 0 php [562074.864285] [ 5603] 500 5603 69319 16732 1 0 0 php [562074.866510] [ 5612] 500 5612 63573 11969 0 0 0 php [562074.868753] [ 5613] 500 5613 61574 9964 1 0 0 php [562074.871009] [ 5616] 500 5616 63637 12150 0 0 0 php [562074.874032] [ 5624] 500 5624 63726 12707 1 0 0 php [562074.876563] [ 5625] 500 5625 55276 4013 1 0 0 php [562074.879156] [ 5630] 500 5630 52951 2211 0 0 0 php [562074.881476] [ 5631] 500 5631 53736 3070 1 0 0 php [562074.883968] [ 5633] 500 5633 53245 2474 0 0 0 php [562074.886305] [ 5634] 500 5634 54533 3868 1 0 0 php [562074.888585] [ 5635] 500 5635 54650 3839 0 0 0 php [562074.890872] [ 5636] 500 5636 56802 6104 0 0 0 php [562074.893228] [ 5637] 500 5637 54189 3375 1 0 0 php [562074.895530] [ 5638] 500 5638 54563 3901 0 0 0 php [562074.897784] [ 5639] 500 5639 52667 1958 1 0 0 php [562074.900122] [ 5640] 500 5640 54533 3711 0 0 0 php [562074.902475] [ 5641] 500 5641 52922 2197 0 0 0 php [562074.904697] [ 5642] 500 5642 52667 1954 1 0 0 php [562074.907013] [ 5643] 500 5643 52958 2226 0 0 0 php [562074.909381] [ 5644] 500 5644 52667 1927 0 0 0 php [562074.911628] [ 5645] 500 5645 54015 3402 1 0 0 php [562074.913914] [ 5646] 500 5646 52951 2165 0 0 0 php [562074.916614] [ 5647] 500 5647 54635 3863 1 0 0 php [562074.919109] [ 5648] 99 5648 219408 400 1 0 0 httpd [562074.921510] [ 5649] 500 5649 53928 3331 1 0 0 php [562074.924011] [ 5650] 500 5650 56653 5958 0 0 0 php [562074.926651] [ 5651] 500 5651 52951 2223 0 0 0 php [562074.929157] [ 5652] 500 5652 52735 1997 1 0 0 php [562074.954567] [ 5653] 500 5653 52457 1707 1 0 0 php [562074.957251] [ 5654] 99 5654 235792 481 1 0 0 httpd [562074.959480] [ 5660] 500 5660 53928 3321 0 0 0 php [562074.961682] [ 5686] 500 5686 54417 3563 0 0 0 php [562074.964573] [ 5712] 500 5712 52958 2216 0 0 0 php [562074.966804] [ 5713] 500 5713 53887 3247 0 0 0 php [562074.969044] [ 5714] 500 5714 52951 2206 0 0 0 php [562074.971840] [ 5715] 500 5715 53736 3103 1 0 0 php [562074.974827] [ 5716] 500 5716 52663 1938 1 0 0 php [562074.977527] [ 5717] 500 5717 56118 5385 0 0 0 php [562074.980867] [ 5726] 500 5726 57172 6396 1 0 0 php [562074.984658] [ 5727] 500 5727 52523 1760 0 0 0 php [562074.987267] [ 5728] 500 5728 52587 1869 1 0 0 php [562074.989641] [ 5730] 500 5730 52430 1711 1 0 0 php [562074.992423] [ 5731] 500 5731 53672 3061 0 0 0 php [562074.995140] [ 5732] 500 5732 52587 1860 1 0 0 php [562074.998136] [ 5733] 500 5733 52770 2016 0 0 0 php [562075.002104] [ 5734] 500 5734 53216 2537 1 0 0 php [562075.004987] [ 5735] 500 5735 54351 3615 1 0 0 php [562075.007912] [ 5737] 500 5737 54329 3665 0 0 0 php [562075.010646] [ 5738] 500 5738 53864 3231 1 0 0 php [562075.013388] [ 5750] 500 5750 53992 3332 1 0 0 php [562075.016259] [ 5751] 500 5751 53992 3356 1 0 0 php [562075.018239] [ 5754] 500 5754 52457 1707 0 0 0 php [562075.020361] [ 5755] 500 5755 53672 3008 0 0 0 php [562075.022769] [ 5758] 500 5758 53928 3334 1 0 0 php [562075.024717] [ 5759] 500 5759 53800 3153 0 0 0 php [562075.026820] [ 5760] 500 5760 52457 1715 1 0 0 php [562075.029743] [ 5761] 500 5761 53672 2999 1 0 0 php [562075.031868] [ 5767] 500 5767 52430 1709 0 0 0 php [562075.033922] [ 5768] 500 5768 53672 3015 1 0 0 php [562075.035960] [ 5772] 500 5772 52457 1708 1 0 0 php [562075.038017] [ 5774] 500 5774 53928 3307 0 0 0 php [562075.040342] [ 5777] 500 5777 52430 1709 0 0 0 php [562075.042585] [ 5780] 500 5780 52922 2200 1 0 0 php [562075.044900] [ 5781] 500 5781 29874 419 0 0 0 php [562075.046949] [ 5782] 500 5782 52430 1710 0 0 0 php [562075.049163] [ 5787] 500 5787 53736 3111 0 0 0 php [562075.051348] [ 5788] 500 5788 52428 1681 0 0 0 php [562075.053411] [ 5789] 500 5789 16226 45 1 0 0 php [562075.055455] [ 5790] 500 5790 53992 3361 0 0 0 php [562075.057385] [ 5796] 500 5796 53800 3145 0 0 0 php [562075.059354] [ 5802] 500 5802 51933 1164 0 0 0 php [562075.061403] [ 5803] 500 5803 53736 3088 1 0 0 php [562075.063319] [ 5828] 500 5828 52457 1712 1 0 0 php [562075.065328] [ 5830] 500 5830 53992 3362 1 0 0 php [562075.067304] [ 5833] 500 5833 52457 1711 0 0 0 php [562075.069272] [ 5834] 500 5834 52457 1710 0 0 0 php [562075.071342] [ 5838] 500 5838 25576 65 0 0 0 php [562075.073811] [ 5839] 99 5839 121104 248 1 0 0 httpd [562075.075876] [ 5848] 99 5848 121104 279 1 0 0 httpd [562075.078224] [ 5849] 500 5849 29230 282 1 0 0 php [562075.080532] [ 5850] 500 5850 29118 217 1 0 0 php [562075.082861] [ 5851] 500 5851 53352 2643 0 0 0 php [562075.084933] [ 5852] 99 5852 121104 281 1 0 0 httpd [562075.087091] [ 5854] 500 5854 52430 1708 0 0 0 php [562075.089180] [ 5856] 500 5856 51703 882 0 0 0 php [562075.091154] [ 5863] 99 5863 121104 286 1 0 0 httpd [562075.093112] [ 5970] 500 5970 28797 186 1 0 0 php [562075.095140] [ 5986] 0 5986 5651 60 0 0 0 upcp-running [562075.097396] [ 5988] 500 5988 29118 223 0 0 0 php [562075.099621] [ 5991] 99 5991 104720 241 0 0 0 httpd [562075.101929] [ 5993] 500 5993 3116 43 0 0 0 suphp [562075.104055] [ 5994] 500 5994 29329 408 0 0 0 php [562075.106361] [ 6026] 500 6026 29118 223 0 0 0 php [562075.108660] [ 6027] 500 6027 29118 231 1 0 0 php [562075.111327] [ 6028] 500 6028 7365 23 1 0 0 php [562075.113563] [ 6032] 99 6032 121104 279 1 0 0 httpd [562075.115682] Out of memory: Kill process 9408 (mysqld) score 44 or sacrifice child [562075.117786] Killed process 9408, UID 402, (mysqld) total-vm:2257588kB, anon-rss:171860kB, file-rss:544kB
PHP seems to be eating all the swap, and kicking mysql out of memory. Any suggestions on what optimizations I should do?0 -
Hello, How much memory is available on the system? You can check this with a command such as: free -m
Thank you.0 -
Hello, How much memory is available on the system? You can check this with a command such as:
free -m
Thank you.
free -m output:total used free shared buffers cached Mem: 3830 1445 2385 0 23 438 -/+ buffers/cache: 983 2847 Swap: 4095 227 38680 -
Hello, Have you noticed any changes since adjusting your MySQL configuration after running the tuner? Have you tried running the tuner again after making the adjustments to your /etc/my.cnf file after the last tuner run? Thank you. 0 -
Hello, Have you noticed any changes since adjusting your MySQL configuration after running the tuner? Have you tried running the tuner again after making the adjustments to your /etc/my.cnf file after the last tuner run? Thank you.
I took the mysqltuner suggestions about 2-3 times and faced repeated failure, i.e. the out of memory error would occur twice everyday when making modifications based on the tuner. I read some articles and increased table_open_cache to 4000 from 450 and reduced the key_buffer_size to 384M from 624M. I'm no server admin so I don't know if this was the right move or not. According to the out of memory log, too many php processes are spawned. On examination during a crash, it appears that they were WordPress' index.php processes, reaching CPU usage of 100% twice everyday. I reduced the amount of memory allotted to php, and adjusted the mysql config as mentioned above. This has reduced the frequency of the out of memory error, but not quite solved the problem. I would imagine the problem requires the correct php configuration, WordPress optimisation, and the correct mysql configuration for my website. I'm working on these three with whatever knowledge I can find. Would appreciate any inputs on the configuration front. Thanks!0 -
Hello, You may also want to consider using a third-party application such as Cloud Linux to manage and control the amount of memory/CPU available to each account if you plan on hosting additional accounts in the future. Another consideration is increasing the amount of memory or CPU power on the system. Thank you. 0 -
Though cloudlinux is great option, I would personally go for moving to Litespeed in this specific scenario. The reason for this is the php processes. These will be limited by Litespeed, far better than CL can, honestly. cPanel (by default) doesn't even bother to limit them. 0
Please sign in to leave a comment.
Comments
10 comments