Skip to main content

VPS Restarting during backups

dazeck
During the last couple of days my VPS has restarted during backups. This has happened a few times before so I logged a ticket but then closed it as a fsck seemed to have fixed the problem, however, it's happening a lot more now. My backups are scheduled for 2am and I get notifications that my server is down about 10 minutes later and then server is back up shortly after that. In the past, I had to start the server manually when I woke up so I have an app installed on my phone, VPSMON, that will restart if necessary so I assume this app is forcing the server back online. After restart, dmesg log file contains the following :
[ 1.187951] EXT4-fs (vda1): INFO: recovery required on readonly filesystem [ 1.187954] EXT4-fs (vda1): write access will be enabled during recovery [ 1.368994] EXT4-fs (vda1): orphan cleanup on readonly fs [ 1.369001] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 1712446 [ 1.369047] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 286509 [ 1.428414] Refined TSC clocksource calibration: 3292.466 MHz. [ 1.701677] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 1044515 [ 1.701709] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 1044497 [ 1.756150] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 1711037 [ 1.768686] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 1709486 [ 1.787361] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 1708687 [ 1.787394] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 1708685 [ 1.828075] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 1706559 [ 1.828108] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 271866 [ 1.843338] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 271855 [ 1.843363] EXT4-fs (vda1): ext4_orphan_cleanup: deleting unreferenced inode 271740 [ 1.843369] EXT4-fs (vda1): 12 orphan inodes deleted [ 1.843371] EXT4-fs (vda1): recovery complete [ 1.893035] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts:
So it does imply it's an issue with the disk, but how do I go about digging further and why does this only happen during backups ? Any help appreciated.

Comments

10 comments

Date Votes
  • Eric
    Howdy, What version of cPanel and Centos are you on? There is a kernel bug that can cause this in older OS's but you're using ext4 which is why I ask. Thanks!
    0
  • dazeck
    Hi It's CLOUDLINUX 6.7 x86_64 kvm " WHM 11.50.2 (build 2). Update to latest version of cpanel is blocked until tomorrow morning.
    0
  • Eric
    Howdy, Go ahead and force the update and if it crashes again open up a ticket. Thanks!
    0
  • dazeck
    It's not the update that killed the server, it was nightly backups.
    0
  • Eric
    Correct there are some fixes to the backups that should help.
    0
  • dazeck
    Oh great, I'll force an update later tonight and see what happens at 2am. Thanks for the help.
    0
  • cPanelMichael
    Oh great, I'll force an update later tonight and see what happens at 2am. Thanks for the help.

    Hello :) You can also force a generation of backups via:
    /usr/local/cpanel/bin/backup --force
    Your earlier post suggests you are already using cPanel version 11.50.2.2, so please open a support ticket and post the ticket number here if the issue persists with the next backup generation so we can take a closer look. Thank you.
    0
  • dazeck
    I forced the update of cpanel last night, and the backup ran fine when scheduled. It's done this in the past though where it only used to restart when doing the backup on a Sunday morning, I'll monitor the situation further. Thanks for the help .
    0
  • dazeck
    OK my VPS restarted this morning around 10am, I can rule backups out as they ran at 2am. This is so infuriating, I cannot rely on my VPS at all now as it may decide to reboot when it feels like it. Where can I start to trace what the issue is ?
    0
  • cPanelMichael
    I cannot rely on my VPS at all now as it may decide to reboot when it feels like it. Where can I start to trace what the issue is ?

    I suggest reaching out to your VPS hosting provider to see if there's any activity on the VPS node that would have lead to a restart of your VPS at that time. It's possible a system administrator is restarting the VPS to reduce load on the node, or there might be a hardware issue with the VPS node itself. Thank you.
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post