cpsrvd.so segfault and recurring page allocation failures

Hey guys, so I just came back from Christmas holidays and got to reading through the server logs. The past few log reports (emailed) contain warnings about page allocation failures: - kernel: swapper: page allocation failure. order:1, mode:0x20 - kernel: kswapd0: page allocation failure. order:1, mode:0x20 - kernel: imap: page allocation failure. order:1, mode:0x20 Checking previous logs this seems to stem from a cpsrvd issue (botched update causing out of memory??). The VPS has been running fine and the website is still up, which is kind of why I'm only noticing this now as I haven't had any warnings about server downtime, etc. The VPS doesn't seem to be under any heavy load or running out of resources: CPU Load: average 0.10 MEM: 26.3% (3.74G total) SWAP: 3.0% (1024M total) Following are logs for each incident: cpsrvd problem (2013-12-22 00:44:01)

[12/22/2013:00:44:01 -0000] info [cpsrvd] reloading config based on -HUP signal
[2013-12-22 00:44:22 +0000] warn [cpsessetup] Failed to create: Cpanel::Cpses::Setup::Postgres: could not determine postgres user at /usr/local/cpanel/Cpanel/Cpses/Setup/Postgres.pm line 21.
 at /usr/local/cpanel/bin/cpsessetup line 99
	bin::cpsessetup::process_cpses_setup_modules() called at /usr/local/cpanel/bin/cpsessetup line 34
	bin::cpsessetup::script('bin::cpsessetup') called at /usr/local/cpanel/bin/cpsessetup line 24
[2013-12-22 00:44:34 +0000] die [realadduser] User cpanel already exists
[2013-12-22 00:44:34 +0000] die [realadduser] User ftp already exists
This is exceeding 8192 pixels and color palette issue may be expected when its associated sprite file is used:
	 "/usr/local/cpanel/base/frontend/x3/branding/the_beach/heading_sprites_bg_snap_to_smallest_width.png".
This is exceeding 8192 pixels and color palette issue may be expected when its associated sprite file is used:
	 "/usr/local/cpanel/base/frontend/x3/branding/motor_city/ui_sprites_bg_snap_to_smallest_width.png".
This is exceeding 8192 pixels and color palette issue may be expected when its associated sprite file is used:
	 "/usr/local/cpanel/base/frontend/x3/branding/crimson_smoke/heading_sprites_bg_snap_to_smallest_width.png".
This is exceeding 8192 pixels and color palette issue may be expected when its associated sprite file is used:
	 "/usr/local/cpanel/base/frontend/x3mail/branding/the_beach/heading_sprites_bg_snap_to_smallest_width.png".
This is exceeding 8192 pixels and color palette issue may be expected when its associated sprite file is used:
	 "/usr/local/cpanel/base/frontend/x3mail/branding/motor_city/ui_sprites_bg_snap_to_smallest_width.png".
This is exceeding 8192 pixels and color palette issue may be expected when its associated sprite file is used:
	 "/usr/local/cpanel/base/frontend/x3mail/branding/crimson_smoke/heading_sprites_bg_snap_to_smallest_width.png".
[2013-12-22 01:00:56 +0000] warn [whostmgr2] Failed to locate postgresql CLI application
Restarting cpsrvd daemon process 2011 via /usr/local/cpanel/cpsrvd-ssl
==> cpsrvd 11.40.1.7 started
==> cpsrvd: preloaded ssl_write_all!
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up native SSL support ... Done
==> cpsrvd: transfered port bindings
==> cpsrvd: bound to ports
Out of memory!

From then on (2013-12-22 00:44:01) I initially had the segfault with cpsrvd.so, then numerous page allocation failrues: cpsrvd.so segfault (2013-12-23 03:52:21)

Dec 23 03:52:21 442248 kernel: cpsrvd.so[21715]: segfault at 0 ip 000000361248eff2 sp 00007fff8e325e50 error 4 in libperl.so[3612400000+cf000]
Dec 23 03:52:27 442248 abrtd: Directory 'ccpp-2013-12-23-03:52:21-21715' creation detected
Dec 23 03:52:27 442248 abrt[21724]: Saved core dump of pid 21715 (/usr/local/cpanel/perl/Cpanel/cpsrvd.so) to /var/spool/abrt/ccpp-2013-12-23-03:52:21-21715 (483356672 bytes)
Dec 23 03:52:28 442248 abrtd: Executable '/usr/local/cpanel/perl/Cpanel/cpsrvd.so' doesn't belong to any package and ProcessUnpackaged is set to 'no'
Dec 23 03:52:28 442248 abrtd: 'post-create' on '/var/spool/abrt/ccpp-2013-12-23-03:52:21-21715' exited with 1
Dec 23 03:52:28 442248 abrtd: Deleting problem directory '/var/spool/abrt/ccpp-2013-12-23-03:52:21-21715'

page allocation failure: swapper (2013-12-23 12:01:03)

Dec 23 12:01:03 442248 kernel: swapper: page allocation failure. order:1, mode:0x20
Dec 23 12:01:03 442248 kernel: Pid: 0, comm: swapper Not tainted 2.6.32-279.14.1.el6.x86_64 #1
Dec 23 12:01:03 442248 kernel: Call Trace:
Dec 23 12:01:03 442248 kernel:   [] ? __alloc_pages_nodemask+0x77f/0x940
Dec 23 12:01:03 442248 kernel: [] ? virtnet_remove+0x11/0x3f [virtio_net]
Dec 23 12:01:03 442248 kernel: [] ? kmem_getpages+0x62/0x170
Dec 23 12:01:03 442248 kernel: [] ? fallback_alloc+0x1ba/0x270
Dec 23 12:01:03 442248 kernel: [] ? cache_grow+0x2cf/0x320
Dec 23 12:01:03 442248 kernel: [] ? ____cache_alloc_node+0x99/0x160
Dec 23 12:01:03 442248 kernel: [] ? kmem_cache_alloc+0x11b/0x190
Dec 23 12:01:03 442248 kernel: [] ? sk_prot_alloc+0x48/0x1c0
Dec 23 12:01:03 442248 kernel: [] ? sk_clone+0x22/0x2e0
Dec 23 12:01:03 442248 kernel: [] ? inet_csk_clone+0x16/0xd0
Dec 23 12:01:03 442248 kernel: [] ? tcp_create_openreq_child+0x23/0x450
Dec 23 12:01:03 442248 kernel: [] ? tcp_v4_syn_recv_sock+0x4d/0x310
Dec 23 12:01:03 442248 kernel: [] ? tcp_check_req+0x226/0x460
Dec 23 12:01:03 442248 kernel: [] ? tcp_v4_do_rcv+0x35b/0x430
Dec 23 12:01:03 442248 kernel: [] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
Dec 23 12:01:03 442248 kernel: [] ? tcp_v4_rcv+0x4fe/0x8d0
Dec 23 12:01:03 442248 kernel: [] ? ip_local_deliver_finish+0x0/0x2d0
Dec 23 12:01:03 442248 kernel: [] ? ip_local_deliver_finish+0xdd/0x2d0
Dec 23 12:01:03 442248 kernel: [] ? ip_local_deliver+0x98/0xa0
Dec 23 12:01:03 442248 kernel: [] ? ip_rcv_finish+0x12d/0x440
Dec 23 12:01:03 442248 kernel: [] ? ip_rcv+0x275/0x350
Dec 23 12:01:03 442248 kernel: [] ? __netif_receive_skb+0x49b/0x6f0
Dec 23 12:01:03 442248 kernel: [] ? netif_receive_skb+0x58/0x60
Dec 23 12:01:03 442248 kernel: [] ? virtnet_poll+0x5ed/0x8e0 [virtio_net]
Dec 23 12:01:03 442248 kernel: [] ? net_rx_action+0x103/0x2f0
Dec 23 12:01:03 442248 kernel: [] ? skb_recv_done+0x39/0x40 [virtio_net]
Dec 23 12:01:03 442248 kernel: [] ? __do_softirq+0xc1/0x1e0
Dec 23 12:01:03 442248 kernel: [] ? handle_IRQ_event+0x60/0x170
Dec 23 12:01:03 442248 kernel: [] ? call_softirq+0x1c/0x30
Dec 23 12:01:03 442248 kernel: [] ? do_softirq+0x65/0xa0
Dec 23 12:01:03 442248 kernel: [] ? irq_exit+0x85/0x90
Dec 23 12:01:03 442248 kernel: [] ? do_IRQ+0x75/0xf0
Dec 23 12:01:03 442248 kernel: [] ? ret_from_intr+0x0/0x11
Dec 23 12:01:03 442248 kernel:   [] ? native_safe_halt+0xb/0x10
Dec 23 12:01:03 442248 kernel: [] ? default_idle+0x4d/0xb0
Dec 23 12:01:03 442248 kernel: [] ? cpu_idle+0xb6/0x110
Dec 23 12:01:03 442248 kernel: [] ? rest_init+0x7a/0x80
Dec 23 12:01:03 442248 kernel: [] ? start_kernel+0x424/0x430
Dec 23 12:01:03 442248 kernel: [] ? x86_64_start_reservations+0x125/0x129
Dec 23 12:01:03 442248 kernel: [] ? x86_64_start_kernel+0xfa/0x109

page allocation failure: kswapd0 (2013-12-23 20:01:00)

Dec 23 20:01:00 442248 kernel: kswapd0: page allocation failure. order:1, mode:0x20
Dec 23 20:01:00 442248 kernel: Pid: 38, comm: kswapd0 Not tainted 2.6.32-279.14.1.el6.x86_64 #1
Dec 23 20:01:00 442248 kernel: Call Trace:
Dec 23 20:01:00 442248 kernel:   [] ? __alloc_pages_nodemask+0x77f/0x940
Dec 23 20:01:00 442248 kernel: [] ? dev_hard_start_xmit+0x1e0/0x3f0
Dec 23 20:01:00 442248 kernel: [] ? kmem_getpages+0x62/0x170
Dec 23 20:01:00 442248 kernel: [] ? fallback_alloc+0x1ba/0x270
Dec 23 20:01:00 442248 kernel: [] ? cache_grow+0x2cf/0x320
Dec 23 20:01:00 442248 kernel: [] ? ____cache_alloc_node+0x99/0x160
Dec 23 20:01:00 442248 kernel: [] ? kmem_cache_alloc+0x11b/0x190
Dec 23 20:01:00 442248 kernel: [] ? sk_prot_alloc+0x48/0x1c0
Dec 23 20:01:00 442248 kernel: [] ? sk_clone+0x22/0x2e0
Dec 23 20:01:00 442248 kernel: [] ? inet_csk_clone+0x16/0xd0
Dec 23 20:01:00 442248 kernel: [] ? tcp_create_openreq_child+0x23/0x450
Dec 23 20:01:00 442248 kernel: [] ? tcp_v4_syn_recv_sock+0x4d/0x310
Dec 23 20:01:00 442248 kernel: [] ? tcp_check_req+0x226/0x460
Dec 23 20:01:00 442248 kernel: [] ? tcp_v4_do_rcv+0x35b/0x430
Dec 23 20:01:00 442248 kernel: [] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
Dec 23 20:01:00 442248 kernel: [] ? tcp_v4_rcv+0x4fe/0x8d0
Dec 23 20:01:00 442248 kernel: [] ? ip_local_deliver_finish+0x0/0x2d0
Dec 23 20:01:00 442248 kernel: [] ? ip_local_deliver_finish+0xdd/0x2d0
Dec 23 20:01:00 442248 kernel: [] ? ip_local_deliver+0x98/0xa0
Dec 23 20:01:00 442248 kernel: [] ? ip_rcv_finish+0x12d/0x440
Dec 23 20:01:00 442248 kernel: [] ? ip_rcv+0x275/0x350
Dec 23 20:01:00 442248 kernel: [] ? __netif_receive_skb+0x49b/0x6f0
Dec 23 20:01:00 442248 kernel: [] ? netif_receive_skb+0x58/0x60
Dec 23 20:01:00 442248 kernel: [] ? virtnet_poll+0x5ed/0x8e0 [virtio_net]
Dec 23 20:01:00 442248 kernel: [] ? pvclock_clocksource_read+0x58/0xd0
Dec 23 20:01:00 442248 kernel: [] ? net_rx_action+0x103/0x2f0
Dec 23 20:01:00 442248 kernel: [] ? skb_recv_done+0x39/0x40 [virtio_net]
Dec 23 20:01:00 442248 kernel: [] ? __do_softirq+0xc1/0x1e0
Dec 23 20:01:00 442248 kernel: [] ? handle_IRQ_event+0x60/0x170
Dec 23 20:01:00 442248 kernel: [] ? __do_softirq+0x11f/0x1e0
Dec 23 20:01:00 442248 kernel: [] ? call_softirq+0x1c/0x30
Dec 23 20:01:00 442248 kernel: [] ? do_softirq+0x65/0xa0
Dec 23 20:01:00 442248 kernel: [] ? irq_exit+0x85/0x90
Dec 23 20:01:00 442248 kernel: [] ? do_IRQ+0x75/0xf0
Dec 23 20:01:00 442248 kernel: [] ? ret_from_intr+0x0/0x11
Dec 23 20:01:00 442248 kernel:   [] ? _spin_lock+0x12/0x30
Dec 23 20:01:00 442248 kernel: [] ? jbd2_journal_release_jbd_inode+0xc8/0x100 [jbd2]
Dec 23 20:01:00 442248 kernel: [] ? ext4_clear_inode+0x3c/0x50 [ext4]
Dec 23 20:01:00 442248 kernel: [] ? clear_inode+0xac/0x140
Dec 23 20:01:00 442248 kernel: [] ? dispose_list+0x40/0x120
Dec 23 20:01:00 442248 kernel: [] ? shrink_icache_memory+0x274/0x2e0
Dec 23 20:01:00 442248 kernel: [] ? shrink_slab+0x12a/0x1a0
Dec 23 20:01:00 442248 kernel: [] ? balance_pgdat+0x57d/0x7e0
Dec 23 20:01:00 442248 kernel: [] ? isolate_pages_global+0x0/0x350
Dec 23 20:01:00 442248 kernel: [] ? zone_watermark_ok_safe+0xad/0xc0
Dec 23 20:01:00 442248 kernel: [] ? kswapd+0x136/0x3b0
Dec 23 20:01:00 442248 kernel: [] ? autoremove_wake_function+0x0/0x40
Dec 23 20:01:00 442248 kernel: [] ? kswapd+0x0/0x3b0
Dec 23 20:01:00 442248 kernel: [] ? kthread+0x96/0xa0
Dec 23 20:01:00 442248 kernel: [] ? child_rip+0xa/0x20
Dec 23 20:01:00 442248 kernel: [] ? kthread+0x0/0xa0
Dec 23 20:01:00 442248 kernel: [] ? child_rip+0x0/0x20
Dec 23 20:01:00 442248 kernel: kswapd0: page allocation failure. order:1, mode:0x20
Dec 23 20:01:00 442248 kernel: Pid: 38, comm: kswapd0 Not tainted 2.6.32-279.14.1.el6.x86_64 #1
Dec 23 20:01:00 442248 kernel: Call Trace:
Dec 23 20:01:00 442248 kernel:   [] ? __alloc_pages_nodemask+0x77f/0x940
Dec 23 20:01:00 442248 kernel: [] ? lapic_next_event+0x10/0x30
Dec 23 20:01:00 442248 kernel: [] ? kmem_getpages+0x62/0x170
Dec 23 20:01:00 442248 kernel: [] ? fallback_alloc+0x1ba/0x270
Dec 23 20:01:00 442248 kernel: [] ? cache_grow+0x2cf/0x320
Dec 23 20:01:00 442248 kernel: [] ? ____cache_alloc_node+0x99/0x160
Dec 23 20:01:00 442248 kernel: [] ? kmem_cache_alloc+0x11b/0x190
Dec 23 20:01:00 442248 kernel: [] ? sk_prot_alloc+0x48/0x1c0
Dec 23 20:01:00 442248 kernel: [] ? sk_clone+0x22/0x2e0
Dec 23 20:01:01 442248 kernel: [] ? inet_csk_clone+0x16/0xd0
Dec 23 20:01:01 442248 kernel: [] ? tcp_create_openreq_child+0x23/0x450
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_syn_recv_sock+0x4d/0x310
Dec 23 20:01:01 442248 kernel: [] ? tcp_check_req+0x226/0x460
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_do_rcv+0x35b/0x430
Dec 23 20:01:01 442248 kernel: [] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_rcv+0x4fe/0x8d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver_finish+0x0/0x2d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver_finish+0xdd/0x2d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver+0x98/0xa0
Dec 23 20:01:01 442248 kernel: [] ? ip_rcv_finish+0x12d/0x440
Dec 23 20:01:01 442248 kernel: [] ? ip_rcv+0x275/0x350
Dec 23 20:01:01 442248 kernel: [] ? __netif_receive_skb+0x49b/0x6f0
Dec 23 20:01:01 442248 kernel: [] ? netif_receive_skb+0x58/0x60
Dec 23 20:01:01 442248 kernel: [] ? virtnet_poll+0x5ed/0x8e0 [virtio_net]
Dec 23 20:01:01 442248 kernel: [] ? pvclock_clocksource_read+0x58/0xd0
Dec 23 20:01:01 442248 kernel: [] ? net_rx_action+0x103/0x2f0
Dec 23 20:01:01 442248 kernel: [] ? skb_recv_done+0x39/0x40 [virtio_net]
Dec 23 20:01:01 442248 kernel: [] ? __do_softirq+0xc1/0x1e0
Dec 23 20:01:01 442248 kernel: [] ? handle_IRQ_event+0x60/0x170
Dec 23 20:01:01 442248 kernel: [] ? __do_softirq+0x11f/0x1e0
Dec 23 20:01:01 442248 kernel: [] ? call_softirq+0x1c/0x30
Dec 23 20:01:01 442248 kernel: [] ? do_softirq+0x65/0xa0
Dec 23 20:01:01 442248 kernel: [] ? irq_exit+0x85/0x90
Dec 23 20:01:01 442248 kernel: [] ? do_IRQ+0x75/0xf0
Dec 23 20:01:01 442248 kernel: [] ? ret_from_intr+0x0/0x11
Dec 23 20:01:01 442248 kernel:   [] ? _spin_lock+0x12/0x30
Dec 23 20:01:01 442248 kernel: [] ? jbd2_journal_release_jbd_inode+0xc8/0x100 [jbd2]
Dec 23 20:01:01 442248 kernel: [] ? ext4_clear_inode+0x3c/0x50 [ext4]
Dec 23 20:01:01 442248 kernel: [] ? clear_inode+0xac/0x140
Dec 23 20:01:01 442248 kernel: [] ? dispose_list+0x40/0x120
Dec 23 20:01:01 442248 kernel: [] ? shrink_icache_memory+0x274/0x2e0
Dec 23 20:01:01 442248 kernel: [] ? shrink_slab+0x12a/0x1a0
Dec 23 20:01:01 442248 kernel: [] ? balance_pgdat+0x57d/0x7e0
Dec 23 20:01:01 442248 kernel: [] ? isolate_pages_global+0x0/0x350
Dec 23 20:01:01 442248 kernel: [] ? zone_watermark_ok_safe+0xad/0xc0
Dec 23 20:01:01 442248 kernel: [] ? kswapd+0x136/0x3b0
Dec 23 20:01:01 442248 kernel: [] ? autoremove_wake_function+0x0/0x40
Dec 23 20:01:01 442248 kernel: [] ? kswapd+0x0/0x3b0
Dec 23 20:01:01 442248 kernel: [] ? kthread+0x96/0xa0
Dec 23 20:01:01 442248 kernel: [] ? child_rip+0xa/0x20
Dec 23 20:01:01 442248 kernel: [] ? kthread+0x0/0xa0
Dec 23 20:01:01 442248 kernel: [] ? child_rip+0x0/0x20
Dec 23 20:01:01 442248 kernel: kswapd0: page allocation failure. order:1, mode:0x20
Dec 23 20:01:01 442248 kernel: Pid: 38, comm: kswapd0 Not tainted 2.6.32-279.14.1.el6.x86_64 #1
Dec 23 20:01:01 442248 kernel: Call Trace:
Dec 23 20:01:01 442248 kernel:   [] ? __alloc_pages_nodemask+0x77f/0x940
Dec 23 20:01:01 442248 kernel: [] ? kmem_getpages+0x62/0x170
Dec 23 20:01:01 442248 kernel: [] ? fallback_alloc+0x1ba/0x270
Dec 23 20:01:01 442248 kernel: [] ? cache_grow+0x2cf/0x320
Dec 23 20:01:01 442248 kernel: [] ? ____cache_alloc_node+0x99/0x160
Dec 23 20:01:01 442248 kernel: [] ? kmem_cache_alloc+0x11b/0x190
Dec 23 20:01:01 442248 kernel: [] ? sk_prot_alloc+0x48/0x1c0
Dec 23 20:01:01 442248 kernel: [] ? sk_clone+0x22/0x2e0
Dec 23 20:01:01 442248 kernel: [] ? inet_csk_clone+0x16/0xd0
Dec 23 20:01:01 442248 kernel: [] ? tcp_create_openreq_child+0x23/0x450
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_syn_recv_sock+0x4d/0x310
Dec 23 20:01:01 442248 kernel: [] ? tcp_check_req+0x226/0x460
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_do_rcv+0x35b/0x430
Dec 23 20:01:01 442248 kernel: [] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_rcv+0x4fe/0x8d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver_finish+0x0/0x2d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver_finish+0xdd/0x2d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver+0x98/0xa0
Dec 23 20:01:01 442248 kernel: [] ? ip_rcv_finish+0x12d/0x440
Dec 23 20:01:01 442248 kernel: [] ? ip_rcv+0x275/0x350
Dec 23 20:01:01 442248 kernel: [] ? __netif_receive_skb+0x49b/0x6f0
Dec 23 20:01:01 442248 kernel: [] ? netif_receive_skb+0x58/0x60
Dec 23 20:01:01 442248 kernel: [] ? virtnet_poll+0x5ed/0x8e0 [virtio_net]
Dec 23 20:01:01 442248 kernel: [] ? pvclock_clocksource_read+0x58/0xd0
Dec 23 20:01:01 442248 kernel: [] ? net_rx_action+0x103/0x2f0
Dec 23 20:01:01 442248 kernel: [] ? skb_recv_done+0x39/0x40 [virtio_net]
Dec 23 20:01:01 442248 kernel: [] ? __do_softirq+0xc1/0x1e0
Dec 23 20:01:01 442248 kernel: [] ? handle_IRQ_event+0x60/0x170
Dec 23 20:01:01 442248 kernel: [] ? __do_softirq+0x11f/0x1e0
Dec 23 20:01:01 442248 kernel: [] ? call_softirq+0x1c/0x30
Dec 23 20:01:01 442248 kernel: [] ? do_softirq+0x65/0xa0
Dec 23 20:01:01 442248 kernel: [] ? irq_exit+0x85/0x90
Dec 23 20:01:01 442248 kernel: [] ? do_IRQ+0x75/0xf0
Dec 23 20:01:01 442248 kernel: [] ? ret_from_intr+0x0/0x11
Dec 23 20:01:01 442248 kernel:   [] ? _spin_lock+0x12/0x30
Dec 23 20:01:01 442248 kernel: [] ? jbd2_journal_release_jbd_inode+0xc8/0x100 [jbd2]
Dec 23 20:01:01 442248 kernel: [] ? ext4_clear_inode+0x3c/0x50 [ext4]
Dec 23 20:01:01 442248 kernel: [] ? clear_inode+0xac/0x140
Dec 23 20:01:01 442248 kernel: [] ? dispose_list+0x40/0x120
Dec 23 20:01:01 442248 kernel: [] ? shrink_icache_memory+0x274/0x2e0
Dec 23 20:01:01 442248 kernel: [] ? shrink_slab+0x12a/0x1a0
Dec 23 20:01:01 442248 kernel: [] ? balance_pgdat+0x57d/0x7e0
Dec 23 20:01:01 442248 kernel: [] ? isolate_pages_global+0x0/0x350
Dec 23 20:01:01 442248 kernel: [] ? zone_watermark_ok_safe+0xad/0xc0
Dec 23 20:01:01 442248 kernel: [] ? kswapd+0x136/0x3b0
Dec 23 20:01:01 442248 kernel: [] ? autoremove_wake_function+0x0/0x40
Dec 23 20:01:01 442248 kernel: [] ? kswapd+0x0/0x3b0
Dec 23 20:01:01 442248 kernel: [] ? kthread+0x96/0xa0
Dec 23 20:01:01 442248 kernel: [] ? child_rip+0xa/0x20
Dec 23 20:01:01 442248 kernel: [] ? kthread+0x0/0xa0
Dec 23 20:01:01 442248 kernel: [] ? child_rip+0x0/0x20
Dec 23 20:01:01 442248 kernel: kswapd0: page allocation failure. order:1, mode:0x20
Dec 23 20:01:01 442248 kernel: Pid: 38, comm: kswapd0 Not tainted 2.6.32-279.14.1.el6.x86_64 #1
Dec 23 20:01:01 442248 kernel: Call Trace:
Dec 23 20:01:01 442248 kernel:   [] ? __alloc_pages_nodemask+0x77f/0x940
Dec 23 20:01:01 442248 kernel: [] ? kmem_getpages+0x62/0x170
Dec 23 20:01:01 442248 kernel: [] ? fallback_alloc+0x1ba/0x270
Dec 23 20:01:01 442248 kernel: [] ? cache_grow+0x2cf/0x320
Dec 23 20:01:01 442248 kernel: [] ? ____cache_alloc_node+0x99/0x160
Dec 23 20:01:01 442248 kernel: [] ? kmem_cache_alloc+0x11b/0x190
Dec 23 20:01:01 442248 kernel: [] ? sk_prot_alloc+0x48/0x1c0
Dec 23 20:01:01 442248 kernel: [] ? sk_clone+0x22/0x2e0
Dec 23 20:01:01 442248 kernel: [] ? inet_csk_clone+0x16/0xd0
Dec 23 20:01:01 442248 kernel: [] ? tcp_create_openreq_child+0x23/0x450
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_syn_recv_sock+0x4d/0x310
Dec 23 20:01:01 442248 kernel: [] ? tcp_check_req+0x226/0x460
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_do_rcv+0x35b/0x430
Dec 23 20:01:01 442248 kernel: [] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_rcv+0x4fe/0x8d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver_finish+0x0/0x2d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver_finish+0xdd/0x2d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver+0x98/0xa0
Dec 23 20:01:01 442248 kernel: [] ? ip_rcv_finish+0x12d/0x440
Dec 23 20:01:01 442248 kernel: [] ? ip_rcv+0x275/0x350
Dec 23 20:01:01 442248 kernel: [] ? __netif_receive_skb+0x49b/0x6f0
Dec 23 20:01:01 442248 kernel: [] ? netif_receive_skb+0x58/0x60
Dec 23 20:01:01 442248 kernel: [] ? virtnet_poll+0x5ed/0x8e0 [virtio_net]
Dec 23 20:01:01 442248 kernel: [] ? pvclock_clocksource_read+0x58/0xd0
Dec 23 20:01:01 442248 kernel: [] ? net_rx_action+0x103/0x2f0
Dec 23 20:01:01 442248 kernel: [] ? skb_recv_done+0x39/0x40 [virtio_net]
Dec 23 20:01:01 442248 kernel: [] ? __do_softirq+0xc1/0x1e0
Dec 23 20:01:01 442248 kernel: [] ? handle_IRQ_event+0x60/0x170
Dec 23 20:01:01 442248 kernel: [] ? __do_softirq+0x11f/0x1e0
Dec 23 20:01:01 442248 kernel: [] ? call_softirq+0x1c/0x30
Dec 23 20:01:01 442248 kernel: [] ? do_softirq+0x65/0xa0
Dec 23 20:01:01 442248 kernel: [] ? irq_exit+0x85/0x90
Dec 23 20:01:01 442248 kernel: [] ? do_IRQ+0x75/0xf0
Dec 23 20:01:01 442248 kernel: [] ? ret_from_intr+0x0/0x11
Dec 23 20:01:01 442248 kernel:   [] ? _spin_lock+0x12/0x30
Dec 23 20:01:01 442248 kernel: [] ? jbd2_journal_release_jbd_inode+0xc8/0x100 [jbd2]
Dec 23 20:01:01 442248 kernel: [] ? ext4_clear_inode+0x3c/0x50 [ext4]
Dec 23 20:01:01 442248 kernel: [] ? clear_inode+0xac/0x140
Dec 23 20:01:01 442248 kernel: [] ? dispose_list+0x40/0x120
Dec 23 20:01:01 442248 kernel: [] ? shrink_icache_memory+0x274/0x2e0
Dec 23 20:01:01 442248 kernel: [] ? shrink_slab+0x12a/0x1a0
Dec 23 20:01:01 442248 kernel: [] ? balance_pgdat+0x57d/0x7e0
Dec 23 20:01:01 442248 kernel: [] ? isolate_pages_global+0x0/0x350
Dec 23 20:01:01 442248 kernel: [] ? zone_watermark_ok_safe+0xad/0xc0
Dec 23 20:01:01 442248 kernel: [] ? kswapd+0x136/0x3b0
Dec 23 20:01:01 442248 kernel: [] ? autoremove_wake_function+0x0/0x40
Dec 23 20:01:01 442248 kernel: [] ? kswapd+0x0/0x3b0
Dec 23 20:01:01 442248 kernel: [] ? kthread+0x96/0xa0
Dec 23 20:01:01 442248 kernel: [] ? child_rip+0xa/0x20
Dec 23 20:01:01 442248 kernel: [] ? kthread+0x0/0xa0
Dec 23 20:01:01 442248 kernel: [] ? child_rip+0x0/0x20
Dec 23 20:01:01 442248 kernel: kswapd0: page allocation failure. order:1, mode:0x20
Dec 23 20:01:01 442248 kernel: Pid: 38, comm: kswapd0 Not tainted 2.6.32-279.14.1.el6.x86_64 #1
Dec 23 20:01:01 442248 kernel: Call Trace:
Dec 23 20:01:01 442248 kernel:   [] ? __alloc_pages_nodemask+0x77f/0x940
Dec 23 20:01:01 442248 kernel: [] ? kmem_getpages+0x62/0x170
Dec 23 20:01:01 442248 kernel: [] ? fallback_alloc+0x1ba/0x270
Dec 23 20:01:01 442248 kernel: [] ? cache_grow+0x2cf/0x320
Dec 23 20:01:01 442248 kernel: [] ? ____cache_alloc_node+0x99/0x160
Dec 23 20:01:01 442248 kernel: [] ? kmem_cache_alloc+0x11b/0x190
Dec 23 20:01:01 442248 kernel: [] ? sk_prot_alloc+0x48/0x1c0
Dec 23 20:01:01 442248 kernel: [] ? sk_clone+0x22/0x2e0
Dec 23 20:01:01 442248 kernel: [] ? inet_csk_clone+0x16/0xd0
Dec 23 20:01:01 442248 kernel: [] ? tcp_create_openreq_child+0x23/0x450
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_syn_recv_sock+0x4d/0x310
Dec 23 20:01:01 442248 kernel: [] ? tcp_check_req+0x226/0x460
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_do_rcv+0x35b/0x430
Dec 23 20:01:01 442248 kernel: [] ? ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
Dec 23 20:01:01 442248 kernel: [] ? tcp_v4_rcv+0x4fe/0x8d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver_finish+0x0/0x2d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver_finish+0xdd/0x2d0
Dec 23 20:01:01 442248 kernel: [] ? ip_local_deliver+0x98/0xa0
Dec 23 20:01:01 442248 kernel: [] ? ip_rcv_finish+0x12d/0x440
Dec 23 20:01:01 442248 kernel: [] ? ip_rcv+0x275/0x350
Dec 23 20:01:01 442248 kernel: [] ? __netif_receive_skb+0x49b/0x6f0
Dec 23 20:01:01 442248 kernel: [] ? netif_receive_skb+0x58/0x60
Dec 23 20:01:01 442248 kernel: [] ? virtnet_poll+0x5ed/0x8e0 [virtio_net]
Dec 23 20:01:01 442248 kernel: [] ? pvclock_clocksource_read+0x58/0xd0
Dec 23 20:01:01 442248 kernel: [] ? net_rx_action+0x103/0x2f0
Dec 23 20:01:01 442248 kernel: [] ? skb_recv_done+0x39/0x40 [virtio_net]
Dec 23 20:01:01 442248 kernel: [] ? __do_softirq+0xc1/0x1e0
Dec 23 20:01:01 442248 kernel: [] ? handle_IRQ_event+0x60/0x170
Dec 23 20:01:01 442248 kernel: [] ? __do_softirq+0x11f/0x1e0
Dec 23 20:01:01 442248 kernel: [] ? call_softirq+0x1c/0x30
Dec 23 20:01:01 442248 kernel: [] ? do_softirq+0x65/0xa0
Dec 23 20:01:01 442248 kernel: [] ? irq_exit+0x85/0x90
Dec 23 20:01:01 442248 kernel: [] ? do_IRQ+0x75/0xf0
Dec 23 20:01:01 442248 kernel: [] ? ret_from_intr+0x0/0x11
Dec 23 20:01:01 442248 kernel:   [] ? _spin_lock+0x12/0x30
Dec 23 20:01:01 442248 kernel: [] ? jbd2_journal_release_jbd_inode+0xc8/0x100 [jbd2]
Dec 23 20:01:01 442248 kernel: [] ? ext4_clear_inode+0x3c/0x50 [ext4]
Dec 23 20:01:01 442248 kernel: [] ? clear_inode+0xac/0x140
Dec 23 20:01:01 442248 kernel: [

cPanelMichael

December 30, 2013 16:23

Hello :) Could you let us know the version of cPanel installed on this system? You can find out the version via: cat /usr/local/cpanel/version
Try running the following command manually: /usr/local/cpanel/perl/Cpanel/cpsrvd.so
Does it complete successfully, or does it hang and continue to run? Thank you.

JerryJ.

December 30, 2013 19:07

If your version reports as 11.40.1.7, please run upcp. There was a bug in build 7 which caused cpsrvd.so to segfault. This is resolved in 11.40.1.8 You can find the change log at

s2s

January 02, 2014 08:25

Thanks for getting back guys. This issue looks to have resolved itself, the last 2 logs don't contain any issues and my current version is : 11.40.1.8

cpsrvd.so segfault and recurring page allocation failures

Comments

Didn't find what you were looking for?