Long lasting backup - Settings to make it faster ?

SecondSight

• December 28, 2015 23:46

Hello :) I looked at the backup log on my server, because backups are lasting too long. I found the archive creation for my biggest website is lasting more than 6 hours : [2015-12-28 05:33:07 +0100] Creating Archive ............................................. [2015-12-28 11:57:55 +0100] Done [2015-12-28 11:57:55 +0100] pkgacctfile is: /backup/2015-12-28/accounts/mysite.tar.gz => When I do it myself, using a simple tar command, it doesn't take more than 10 minutes (I only backup public_html directory). Backing up the database is also taking more than 3 hours : [2015-12-28 02:17:05 +0100] Storing mysql dbs............ mysite_forums......... ......... ......... [2015-12-28 05:33:06 +0100] ...Done => When I do it myself, using a mysqldump command, it lasts 5 minutes. Do you know the reasons why the WHM backup tool takes so long (more than 9 hours) to backup one account ? I do it myself in not more than 15 minutes and in the past, the backup lasted 45 minutes. I noticed these problems after I had CloudLinux installed but I was told it has nothing to do with it... What can I do to find the reasons why backup is so slow ? Can you tell me what are the settings I can change in WHM to make the backups faster ? Thank you :)

• 

  cPanelMichael

  • December 29, 2015 16:24

  Hello :) Do you notice any additional entries in the backup log within the /usr/local/cpanel/logs/cpbackup directory during the backup process? What's the server load when the automatic backups are generating? Thank you.

  0
• 

  SecondSight

  • December 30, 2015 21:47

  Hello ! :) When I watched the server load, I saw it is between 1 and 4 when the backup is ran (that is to say from 2 am to 16 pm...). The only error message is about the database : ERROR: Failed to dump one or more databases In the error_log file I found : [2015-12-30 03:32:20 +0100] warn [pkgacct] mysite_forums: mysqldump: Error 2013: Lost connection to MySQL server during query when dumping table `ibf_posts` at row: 1658647 at bin/pkgacct.pl line 1953, <$fh> line 1. This table, ibf_posts is the biggest table. Here is the content of the log file sent to me by email : (I removed a lot of lines of dots and my site's/account's real name)
  [2015-12-30 02:10:02 +0100] info [backup] Setting I/O priority to reduce system load: best-effort: prio 3 [2015-12-30 02:10:02 +0100] info [backup] Hard Linking available on "/backup" [2015-12-30 02:10:02 +0100] info [backup] Running account backup with target : /backup/2015-12-30/accounts [2015-12-30 02:10:02 +0100] info [backup] Calling pkgacct under cpuwatch to backup user "mysite" [2015-12-30 02:10:02 +0100] pkgacct started. [2015-12-30 02:10:02 +0100] pkgacct version 10 - user : mysite - tarball: 1 - target mysql : default - split: 0 - incremental: 0 - homedir: 1 - mailman: 1 - backup: 1 - archive version: 3 - running with uid 0 [2015-12-30 02:10:02 +0100] pkgacct using '/usr/local/cpanel/3rdparty/bin/pigz -6 --processes 12 --blocksize 128 --rsyncable' to compress archives [2015-12-30 02:10:02 +0100] pkgacct working dir : /backup/2015-12-30/accounts/mysite [2015-12-30 02:10:03 +0100] Copying Reseller Config...[2015-12-30 02:10:03 +0100] Done [2015-12-30 02:10:03 +0100] Copying Suspension Info (if needed)...[2015-12-30 02:10:03 +0100] Done [2015-12-30 02:10:03 +0100] Copying SSL certificates, CSRs, and keys...[2015-12-30 02:10:03 +0100] Done [2015-12-30 02:10:03 +0100] Copying Domain Keys....[2015-12-30 02:10:03 +0100] Done [2015-12-30 02:10:03 +0100] Copying Counter Data....[2015-12-30 02:10:03 +0100] Copying Bandwidth Data....Summary databases " done! [2015-12-30 02:10:10 +0100] Copying Dns Zones.......mysite.com...[2015-12-30 02:10:11 +0100] Done [2015-12-30 02:10:11 +0100] Copying Mail files....[2015-12-30 02:10:11 +0100] Done [2015-12-30 02:10:11 +0100] Copying proftpd file....[2015-12-30 02:10:11 +0100] Done [2015-12-30 02:10:11 +0100] Copying www logs.......log file sizes [369148419 byte(s)]............ ...mysite.com-bytes_log......ftp.mysite.com-ftp_log.offsetftpbytes......mysite.com............ ......... [2015-12-30 02:18:20 +0100] Done [2015-12-30 02:18:20 +0100] Copy userdata...[2015-12-30 02:18:20 +0100] Done [2015-12-30 02:18:20 +0100] Copy custom virtualhost templates...[2015-12-30 02:18:20 +0100] Done [2015-12-30 02:18:20 +0100] Copying mailman lists and archives.... [2015-12-30 02:18:20 +0100] Done copying mailman lists and archives. [2015-12-30 02:18:20 +0100] Copying homedir............. ......... [2015-12-30 02:18:27 +0100] Done [2015-12-30 02:18:27 +0100] Calculating nobody files............ ......... [2015-12-30 02:18:34 +0100] Done [2015-12-30 02:18:34 +0100] Entering timeout safety mode for MySQL (suspending cpuwatch) Load watching suspended due to SIGUSR1 [2015-12-30 02:18:34 +0100] Determining mysql dbs...[2015-12-30 02:18:34 +0100] ...mysqldump version: 5.5.47...[2015-12-30 02:18:34 +0100] ...mysql version: 5.5...[2015-12-30 02:18:34 +0100] Saving mysql privs...[2015-12-30 02:18:34 +0100] Done [2015-12-30 02:18:34 +0100] ...Done [2015-12-30 02:18:34 +0100] Storing mysql dbs............ mysite_forums......... ......... Here, you have the Database tables list. I removed it. ......... [2015-12-30 05:05:48 +0100] (2638923665 bytes) ERROR: Failed to dump one or more databases [2015-12-30 05:05:48 +0100] ...Done [2015-12-30 05:05:48 +0100] Leaving timeout safety mode for MySQL (unsuspending cpuwatch) Load watching resumed due to SIGUSR2 [2015-12-30 05:05:48 +0100] Copying cpuser file.......[2015-12-30 05:05:48 +0100] Done [2015-12-30 05:05:48 +0100] Copying crontab file.......[2015-12-30 05:05:48 +0100] Done [2015-12-30 05:05:48 +0100] Performing "Quota" component....[2015-12-30 05:05:48 +0100] Completed "Quota" component. [2015-12-30 05:05:48 +0100] Storing Subdomains.... [2015-12-30 05:05:50 +0100] Done [2015-12-30 05:05:50 +0100] Storing Parked Domains.... [2015-12-30 05:05:50 +0100] Done [2015-12-30 05:05:50 +0100] Storing Addon Domains.... [2015-12-30 05:05:50 +0100] Done [2015-12-30 05:05:50 +0100] Storing SSL domain......[2015-12-30 05:05:50 +0100] Done [2015-12-30 05:05:50 +0100] Copying password.......[2015-12-30 05:05:50 +0100] Done [2015-12-30 05:05:50 +0100] Copying shell.......[2015-12-30 05:05:50 +0100] Done [2015-12-30 05:05:50 +0100] Creating Archive ................................... .................... [2015-12-30 10:50:09 +0100] Done [2015-12-30 10:50:09 +0100] pkgacctfile is: /backup/2015-12-30/accounts/mysite.tar.gz [2015-12-30 10:55:11 +0100] md5sum is: 53d94b548ee3106666fd36f56108fb3b [2015-12-30 10:55:11 +0100] size is: 14666831266 [2015-12-30 10:55:11 +0100] homesize is: 35683127296 [2015-12-30 10:55:11 +0100] pkgacct completed [2015-12-30 10:55:12 +0100] info [backup] Queuing daily backup copy for transport [2015-12-30 10:55:12 +0100] info [backup] Queuing transport of file: /backup/2015-12-30/accounts/mysite.tar.gz [2015-12-30 10:55:46 +0100] info [backup] Calling pkgacct under cpuwatch to backup user "anothersite" [2015-12-30 10:55:47 +0100] pkgacct started. ... (I removed information about other sites backup here) [2015-12-30 12:33:26 +0100] info [backup] Completed at Wed Dec 30 12:33:26 2015 [2015-12-30 12:33:26 +0100] info [backup] Final state is Backup::Success (0) [2015-12-30 12:33:26 +0100] info [backup] Sent Backup::Success notification.
  Thank you for your help. :)

  0
• 

  SecondSight

  • January 05, 2016 03:35

  Hello ! :) I've found that when I run /scripts/pkgacct mysite, the backup is done in 18 minutes... (while it's lasting more than 9 hours with the automatic backup system) and I don't get the "ERROR: Failed to dump one or more databases" error message. When I compare the two logs I see that I don't have this kind of lines when I use /scripts/pkgacct : Setting I/O priority to reduce system load: best-effort: prio 3 Calling pkgacct under cpuwatch to backup user "mysite" Do you think it's cpuwatch which makes the backup last long ? Thank you ! :)

  0
• 

  SecondSight

  • January 06, 2016 05:49

  Hello ! :) My server management company techs told me : - the account is 36 Gb in size. - the backup lasts long because it is halted due tot the server load, - mysql times out when performing the backup of my biggest database's table. Anyway, /scripts/pkgacct works... (but I have not tested the backup file to see if it's ok). What can I do now to backup this 36 Gb account ? Can you give me pieces of advice ? Thank you ! :)

  0
• 

  SecondSight

  • January 07, 2016 16:59

  Hello ! :) Still trying to find a solution. 1) Running backups as root works. 2) The automatic WHM backup system doesn't work (slow and error messages). What are the differences between the two ? Can you tell me which user the automatic WHM backup task runs ? Thank you for your help. :)

  0
• 

  cPanelMichael

  • February 02, 2016 20:54

  You may want to consider excluding databases from account backups and only backing up the entire MySQL directory: Backup Configuration - Documentation - cPanel Documentation Thank you.

  0

Please sign in to leave a comment.