Skip to main content

Bug report: Out Of Memory (OOM) messages: 3 times over 2 days for 1 incident

rbairwell

I have just tried to log this via https://support.cpanel.net/hc/en-us/requests/new only to get the "cPanel Technical Support is available for cPanel Partners and Direct customers" message with no way of reaching cPanel tech support directly - is this intentional? (it's a recent change).

Basically, I've just got an OOM notification from the server (via email and Slack notifications) within the last hour - with the email having a date string of "Date: Thu, 21 Dec 2023 08:48:59 GMT" and the Slack message being within a few seconds of that.

However, the OOM event time as detailed in both notifications is "Wednesday, December 20, 2023 at 1:33:38 PM UTC" (although /var/log/messages says Dec 20 14:04:44)

Looking at /var/log/messages/ , there are no (non-firewall) related entries on Dec 20 at 13:3X  - (server is running on UTC).

So - I've got 3 different times across 2 days...why? And why was the latest notification delayed by over 19 hours before the email was even generated.

Comments

3 comments

Date Votes
  • cPRex Jurassic Moderator

    Hey there!  We have changed the support portal to detect where the license was purchased.  If the license was purchased directly through cPanel, it gives you the option to open a ticket.  If not, it displays that message indicating the Partner where you purchased the license through should be providing that frontline support.

    As far as the OOM issues, is it possible a service failed and that caused the delay, such as Exim?  And then it went into the retry cycle until things were more stable on the machine?  Obviously I'm guessing a bit, but I don't have another good explanation off the top of my head.

    0
  • rbairwell

    Support portal: I do see and understand the reasoning, but most (if not all) of my issues tend to be stuff "partner frontline support" can't assist with - and hence it'll just introduce a delay (I tend to be the person providing the support anyway). I did like the old system where it could be "overriden".

    OOM: The indication is that PHP running under the Event MPM caused the problem (I've switched back to Worker MPM now). And, like you, I did suspect a mail backlog or similar (even though load/memory usage returned to nominal values within 10 minutes) - but not only does the "Date:" header of the email give today's date/time, but at the bottom of the email "The system generated this notice on Thursday, December 21, 2023 at 8:48:59 AM UTC." with the event time "    Wednesday, December 20, 2023 at 1:33:38 PM UTC". So something caused chkservd not to record it even generated the message correctly:

    According to /var/log/chkservd.log

    [2023-12-20 14:06:50 +0000] OOM check ......OOM Event:[anon_rss=125608kB,file_rss=0kB,is_cgroup=0,pid=2028861,seconds_since_boot=21161243.250826,time=1703079219,total_vm=2525220kB,uid=27,user=mysql]....OOM Event:[anon_rss=34464kB,file_rss=4kB,is_cgroup=0,pid=2882281,seconds_since_boot=21161335.915182,time=1703079311,total_vm=740724kB,uid=1008,user=XXXXX].....Sent OOM Notification......Sent OOM Notification...... Done
    [2023-12-20 14:12:23 +0000] OOM check ......OOM Event:[anon_rss=125608kB,file_rss=0kB,is_cgroup=0,pid=2028861,seconds_since_boot=21161243.250826,time=1703079219,total_vm=2525220kB,uid=27,user=mysql]....OOM Event:[anon_rss=34464kB,file_rss=4kB,is_cgroup=0,pid=2882281,seconds_since_boot=21161335.915182,time=1703079311,total_vm=740724kB,uid=1008,user=XXXXX]....OOM Event:[anon_rss=131136kB,file_rss=0kB,is_cgroup=0,pid=2882380,seconds_since_boot=21161512.749764,time=1703079488,total_vm=1724412kB,uid=27,user=mysql].....Skipped OOM Notification (too soon)......Skipped OOM Notification (too soon)......Skipped OOM Notification (too soon)...... Done
    [2023-12-20 14:17:24 +0000] OOM check ......OOM Event:[anon_rss=125608kB,file_rss=0kB,is_cgroup=0,pid=2028861,seconds_since_boot=21161243.250826,time=1703079219,total_vm=2525220kB,uid=27,user=mysql]....OOM Event:[anon_rss=34464kB,file_rss=4kB,is_cgroup=0,pid=2882281,seconds_since_boot=21161335.915182,time=1703079311,total_vm=740724kB,uid=1008,user=XXXXX]....OOM Event:[anon_rss=131136kB,file_rss=0kB,is_cgroup=0,pid=2882380,seconds_since_boot=21161512.749764,time=1703079488,total_vm=1724412kB,uid=27,user=mysql].....Skipped OOM Notification (too soon)......Skipped OOM Notification (too soon)......Skipped OOM Notification (too soon)...... Done

    The same message repeats for every 5 minutes for 18 hours until:

    [2023-12-21 08:43:36 +0000] OOM check ......OOM Event:[anon_rss=125608kB,file_rss=0kB,is_cgroup=0,pid=2028861,seconds_since_boot=21161243.250826,time=1703079218,total_vm=2525220kB,uid=27,user=mysql]....OOM Event:[anon_rss=34464kB,file_rss=4kB,is_cgroup=0,pid=2882281,seconds_since_boot=21161335.915182,time=1703079310,total_vm=740724kB,uid=1008,user=XXXXX]....OOM Event:[anon_rss=131136kB,file_rss=0kB,is_cgroup=0,pid=2882380,seconds_since_boot=21161512.749764,time=1703079487,total_vm=1724412kB,uid=27,user=mysql].....Skipped OOM Notification (too soon)......Skipped OOM Notification (too soon)......Skipped OOM Notification (too soon)...... Done
    [2023-12-21 08:48:59 +0000] OOM check ......OOM Event:[anon_rss=125608kB,file_rss=0kB,is_cgroup=0,pid=2028861,seconds_since_boot=21161243.250826,time=1703079218,total_vm=2525220kB]....OOM Event:[anon_rss=34464kB,file_rss=4kB,is_cgroup=0,pid=2882281,seconds_since_boot=21161335.915182,time=1703079310,total_vm=740724kB,uid=1008,user=XXXXX]....OOM Event:[anon_rss=131136kB,file_rss=0kB,is_cgroup=0,pid=2882380,seconds_since_boot=21161512.749764,time=1703079487,total_vm=1724412kB,uid=27,user=mysql].....Sent OOM Notification......Skipped OOM Notification (too soon)......Skipped OOM Notification (too soon)...... Done

     

    0
  • cPRex Jurassic Moderator

    That's interesting, and likely something we'd want to see a ticket on to get some more details for sure.

    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post