v66 backups take significantly longer to complete

Hello, I notice that after switching to v66, my backups take up to 2 hours longer to complete on most servers. I'm using the "new" backup system with compressed backups (non incremental). I have compared some logs from before and after the upgrade. What is strange is that creating the backup takes the same amount of time, but then the process sits there for 7,5 minutes doing... nothing? I'm not doing any remote copies, no mounting, ... just a basic compressed full backup. Have a look at the timestamps after "pkgacct completed". Old log, truncated:

[2017-08-26 03:55:59 +0200] info [backup] Calling pkgacct under cpuwatch to backup user "lighth"
[2017-08-26 03:55:59 +0200] pkgacct started.
[2017-08-26 03:55:59 +0200] pkgacct version 10 - user : lighth - tarball: 1 - target mysql : default - split: 0 - incremental: 0 - homedir: 1 - mailman: 1 - backup: 1 - archive version: 3 - running with uid 0
[2017-08-26 03:55:59 +0200] pkgacct using '/usr/local/cpanel/3rdparty/bin/pigz -2 --processes 4 --blocksize 16384 --rsyncable' to compress archives
[2017-08-26 03:56:52 +0200] Creating Archive .
[2017-08-26 04:08:46 +0200] Done
[2017-08-26 04:08:46 +0200] pkgacctfile is: /backup/2017-08-26/accounts/lighth.tar.gz
[2017-08-26 04:08:46 +0200] size is: 17803247613
[2017-08-26 04:08:46 +0200] homesize is: 20483817472
[2017-08-26 04:08:46 +0200] homefiles is: 31373
[2017-08-26 04:08:46 +0200] pkgacct completed
[2017-08-26 04:08:47 +0200] info [backup] Queuing daily backup copy for transport

New log, truncated:


[2017-08-31 02:05:20 +0200] info [backup] checking backup for lighth
[2017-08-31 02:05:20 +0200] info [backup] Backups ARE enabled for lighth
[2017-08-31 02:05:20 +0200] info [backup] Calling pkgacct under cpuwatch to backup user "lighth"
[2017-08-31 02:05:20 +0200] pkgacct started.
[2017-08-31 02:05:20 +0200] pkgacct version 10 - user : lighth - tarball: 1 - target mysql : default - split: 0 - incremental: 0 - homedir: 1 - mailman: 1 - backup: 1 - archive version: 3 - running with uid 0
[2017-08-31 02:05:20 +0200] pkgacct using '/usr/local/cpanel/3rdparty/bin/pigz -2 --processes 4 --blocksize 16384 --rsyncable' to compress archives
[2017-08-31 02:05:20 +0200] pkgacct working dir : /backup/2017-08-31/accounts/lighth
[2017-08-31 02:05:20 +0200] pkgacct started.
[2017-08-31 02:05:44 +0200] Creating Archive
[2017-08-31 02:16:16 +0200] Done
[2017-08-31 02:16:16 +0200] pkgacctfile is: /backup/2017-08-31/accounts/lighth.tar.gz
[2017-08-31 02:16:16 +0200] size is: 17806622636
[2017-08-31 02:16:16 +0200] homesize is: 20484128768
[2017-08-31 02:16:16 +0200] homefiles is: 31665
[2017-08-31 02:16:16 +0200] pkgacct completed
>>> [2017-08-31 02:23:43 +0200] info [backup] Queuing daily backup copy of "lighth" for transport of "/backup/2017-08-31/accounts/lighth.tar.gz" to "2017-08-31/accounts/lighth.tar.gz" <<<
[2017-08-31 02:25:48 +0200] info [backup] Queuing transport of file: /backup/2017-08-31/accounts/lighth.tar.gz
[2017-08-31 02:25:48 +0200] info [backup] no_transport = 0 .. and queueid = TQ:TaskQueue:80961
[2017-08-31 02:25:49 +0200] info [backup] leaving queue_backup_transport_item
[2017-08-31 02:25:49 +0200] info [backup] Queuing transport of file: /backup/2017-08-31/accounts/lighth-=-meta
[2017-08-31 02:25:49 +0200] info [backup] no_transport = 0 .. and queueid = TQ:TaskQueue:80962

Any explanation for the large gap? Edit: - I saw some "gzip -d" (decompress) running, maybe this is some check to see if the backup is correct? - The time difference is sometimes a few seconds, sometimes 10 minutes. Seems to be related to the size of the account?

Rawhide

September 06, 2017 14:06

I am also experiencing a major increase in the amount of time to backup my site since upgrading to WHM v66. I am using the Compressed backup type (not Incremental). My site is quite large: 111 GB, 489,642 files. 91 GB of the 111 GB is being used by mail. It used to take 8.5 hours to complete a backup. After upgrading to WHM v66, it now takes 22.5 hours to complete a backup. I rebooted my server thinking that a rogue process was slowing things down, but the problem remains.

cPanelMichael

September 06, 2017 16:15

Hello, Could you open a support ticket using the link in my signature so we can take a closer look at the affected system to see why backups might be slower since the update? Thank you.

Evolvermeister

September 12, 2017 11:51

Hello, I'm also noticed much longer backups after upgrade to 66. I have two servers with CL6 and CL7 doing incremental backups. Both of servers became do backups 2-4 hours longer than before upgrade. If earlier it took 7-8 hours to backup, now it takes up to 12 hours in some days. By the way, I faced similar issue after upgrading to version 64, but changing some ext4 mount options (noatime,nodiratime,commit=60) on my /backup partition made backups run a bit faster. But now I don't know what to do else. Any help would be appreciated.

September 12, 2017 14:17

Hi @Evolvermeister, Could you open a support ticket using the link in my signature so we can take a closer look at one of the affected systems? You can post the ticket number here and we will update this thread with the outcome. Thank you.

September 14, 2017 14:14

Hello, If you are using CloudLinux, please note the following thread: Backups taking longer on CloudLinux and cPanel 66 Thank you.

roliboli

October 10, 2017 06:41

Hi Do you have any information on this? On all servers the backup duration is much more (around 1h more). The space of the accounts-dir is 82GB. The problem occured after update to v66. There is no cloudlinux. Maybe I found something different in the first line of the backup log: in v64 : info [backup] Setting I/O priority to reduce system load: best-effort: prio 3 in v66 : info [backup] Setting I/O priority to reduce system load: unknown: prio 0 Thanks Roland

October 10, 2017 15:46

Hello @roliboli, The existing case relates to the use of CloudLinux. If you are not using CloudLinux, feel free to open a support ticket using the link in my signature so we can take a closer look at your system. Thank you.

v66 backups take significantly longer to complete

Comments

Didn't find what you were looking for?