Backup large accounts

veronicabend

• April 09, 2015 07:19

Hi, I have a dedicated server with many domains, some of those are small but a few are about 4-10GB in size. I run cpanel backup daily to keep backup copies on a 2nd hard drive. I'm having issues because the backup process is running for many hours to get all accounts backed up, and it is causing high server load, particularly when packing up the large accounts. The large accounts have many image and video files. How should I configure the backups to run? Is it necessary to use a tool other than builtin WHM Backups? I need to keep sites backups without affecting server performance this much. Thanks for any help.

• 

  LostNerd

  • April 09, 2015 13:22

  What transport method are you using? (e.g. FTP/Amazon/WebDAV) - I personally now use a custom backup script thanks to @jgillmanjr and his script allows me to manage the amount of threads to allow in an instance whilst uploading to an S3 platform.

  0
• 

  madmanmachines

  • April 09, 2015 14:57

  Hi, This is likely due to I/O. You can use the `sar' command line utility to confirm. You should check the accounts for directories with many(1,000+) files directly within. You should also check mail accounts for the same issue. If mail accounts are large, you should impose mailbox limits or encourage POP for large accounts. You may also check Tweak Settings I/O priority and try pigz compression. As well, if you have the space, you may consider disabling compression altogether. Thanks,

  0
• 

  veronicabend

  • April 09, 2015 15:20

  What transport method are you using? (e.g. FTP/Amazon/WebDAV) .

  
  Currently these backups are being generated to a 2nd hard drive on the same server. The problem is that cpanel takes about 10 hours to complete the backups, during this period several time with high load, especially when it is packing the big accounts.

  0
• 

  veronicabend

  • April 09, 2015 15:32

  Hi, This is likely due to I/O. You can use the `sar' command line utility to confirm. You should check the accounts for directories with many(1,000+) files directly within. You should also check mail accounts for the same issue. If mail accounts are large, you should impose mailbox limits or encourage POP for large accounts. You may also check Tweak Settings I/O priority and try pigz compression. As well, if you have the space, you may consider disabling compression altogether. Thanks,

  
  Yes I think it is due to cpanel backup copying the files over and then gzipping them. That's why I was thinking, if there is a better way to backup cpanel accounts?

  0
• 

  madmanmachines

  • April 09, 2015 15:52

  Yes I think it is due to cpanel backup copying the files over and then gzipping them. That's why I was thinking, if there is a better way to backup cpanel accounts?

  
  If you plan to use compression with your backups, you will likely see similar issues/delay with other methods. You should check for large directories. Thanks,

  0
• 

  cPanelMichael

  • April 13, 2015 15:13

  Hello, Have you considered excluding certain large photo and video files and backing them up through another method? If that's an option, then the following document explains how you can exclude certain files from backups:

  0

Please sign in to leave a comment.