Incremental Backups and S3

tbutler

• December 03, 2022 22:48

If I use incremental backups, the documentation says that those backups are only supported with rsync destinations. Does that mean I cannot use S3 as an additional destination, or does it simply mean the backups that arrive at S3 will not be incremental? I don't mind if the S3 backups are just standard ones, but I don't have enough room on my server for a bunch of full backups, which makes incremental much more appealing locally. Assuming one cannot use S3 as an additional destination at all if incremental is turned on, does anyone happen to have a good guide for spinning up a nice little AWS instance that could be an rsync destination while using as little CPU time on the Amazon side as possible? Thanks!

• 

  tbutler

  • December 04, 2022 05:58

  Scratch the AWS instance idea, I think -- that looks like it'd cost at least $10/month before storage costs, which is significantly more than storing to S3. Rsync.net looks like an interesting option for that price range, but I'd really like to stick to just S3 or something like it (e.g. Wasabi). Assuming I'd have to use rsync, I'm wondering if there's any unforeseen difficulties with writing a script that does the following each day after cpbackup completes:

  • Compresses the latest copy of each incremental backup into a tar.gz archive.
  • Uses the aws-cli tool to send it to an S3 compatible storage server
  • Deletes the tar.gz archive

  That seems to me something I could code fairly easily and would allow incremental backups, since the tarballs of a directory with hard links should be a full fledge, genuine backup. The only question would be: how would these then be restored since they aren't cPanel's standard archives? If the system were to go down, could I decompress all of these individual tarballs in the /backups directory and cPanel would accept them?

  1
• 

  lhr

  • December 11, 2022 11:15

  Its a nice idea, but I would like to see technical possibility of the same.

  0
• 

  cPRex Jurassic Moderator

  • December 12, 2022 16:26

  Hey there! Yes - with the plan you outlined you would restore all the backups you have available after uncompressing them in the backup directory, and cPanel can handle that.

  0
• 

  tbutler

  • December 12, 2022 17:07

  That's wonderful to hear. I've been working on a script, only hindered by a tad of issue with the Perl S3 API I'm trying to figure out. I believe there's a hook with cPanel to run a script after cpbackup finishes, isn't there? I think I ran across that the other day. Thanks for the help!

  0
• 

  cPRex Jurassic Moderator

  • December 12, 2022 17:10

  I believe this is what you're referring to: Guide to Standardized Hooks - Hookable Events

  0
• 

  tbutler

  • December 20, 2022 02:44

  Thanks! I'm pleased to share that my S3 backup rotation script is working if anyone else would like incremental backups to be automatically compressed, sent to S3 and rotated there. I posted it to GitHub: GitHub - trbutler/ftBackup: A tool to take daily incremental backups from cPanel, upload them to S3 and rotate the copies on S3.

  0

Please sign in to leave a comment.