Skip to main content

Multi-Threaded Remote FTP Backup?

seenBEST
I backup a VPS remotely to FTP via the "additional destinations" area of the built-in WHM Backup Configuration tool. It's slow! I find that it takes nearly 14 hours to transfer 40GB of backup data from the VPS to the FTP destination which works out to about 1MB/s sustained rate. The VPS is from a major provider with nearly unlimited speed and the remote FTP server has a 25MB connection, so I know that I'm not being limited by the pipe in either direction. However, the built-in FTP script used by the backup utility uploads a single file one at a time via FTP. This single thread file upload, for whatever reason, never exceeds 1MB/s. I've ran a test by downloading the entire 40GB backup directory via the same FTP server using a FileZilla client set to 6 simultaneous threads and it completes much faster (a couple of hours). So that leads to my question, is there a way to configure the WHM remote FTP backup script to upload files using multiple threads so that I'm not waiting for files 1 at a time? In addition does anyone know if there is some configuration somewhere from within WHM that could be limiting a single FTP upload to 1MB/s? If I could increase the speed at which the FTP uploads then it wouldn't be so bad.

Comments

3 comments

Date Votes
  • cPanelMichael
    Hello :) cPanel does not limit the backup transfer speed with FTP. It simply uses the FTP protocol on your server as it's configured. In your test, did you use FTP to upload the files to the remote server from the cPanel server? That would be the most reliable way to determine the transfer speeds. Thank you.
    0
  • seenBEST
    [quote="cPanelMichael, post: 1669601">In your test, did you use FTP to upload the files to the remote server from the cPanel server? That would be the most reliable way to determine the transfer speeds.
    Thank you for the reply, cPanelMichael. In my first speed test, I monitored the VPS server during the backup operation to get FTP speeds sent from the VPS to the remote server FTP. Transfers hovered around 1MB/s. This was my (PUSH) test. In my second test, I used a Filezilla client from the remote server FTP to download the backup files from the VPS server. Transfers again hovered around 1MB/s. This was my (PULL) test. Finally, I did a multi-threaded FTP download where I downloaded 6 of the backup files at a time using FileZilla client. Each thread obtained 1MB/s, which made it possible for me to transfer all of the backup files much faster than waiting for 1 file at a time. I am hoping there is some hidden variable somewhere that can be adjusted to make the Backup to Remote FTP option send backup files via FTP using multiple threads. It could reduce my time to transfer backups to FTP from 12 hours to 3.
    0
  • cPanelMichael
    There are no such configuration options in the cPanel backup interface. You are welcome to submit a feature request for that via: Submit A Feature Request Thank you.
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post