S3 Backups not being deleted

platinum

• June 04, 2015 08:47

There's a few threads I've seen, but as far as I could work out - the current version of cPanel with S3 backups is meant to behave the same was as regular backups, i.e. only retain the latest set of weekly/monthly/daily. My daily backups are just building up each day, causing a ton of space to be used. I don't really want to use the auto-expire feature of files, because it would be better to explicitly have them deleted by the backup system. grep BACKUP_DAILY_RETENTION /var/cpanel/backups/config = 1 Also checked that my Amazon bucket policy allows for deletion of files, it's happening the same on 2 servers I'm testing with. Edit: should also add, on WHM 11.50.0 (build 9)

• 

  cPanelMichael

  • June 04, 2015 14:53

  Hello, There's a thread on this at:

  0
• 

  platinum

  • June 04, 2015 22:39

  Ah I see - I thought that might have been implemented as the last reply on that feature request says it does work, and the most recent thread about it here was asking about how to stop it deleting files. I also get these messages in the transporter log, which say it's attempting to prune?
  [2015-06-05 08:04:16 +1000] info [cpbackup_transporter] Starting a "prune" operation on the "Server-Name" destination ID XXXXX
  

  0
• 

  cPanelMichael

  • June 05, 2015 12:32

  [2015-06-05 08:04:16 +1000] info [cpbackup_transporter] Starting a "prune" operation on the "Server-Name" destination ID XXXXX

  
  Hello, Yes, this message indicates it's pruning the expired backups on the remote destination. Thank you.

  0
• 

  platinum

  • June 06, 2015 03:43

  Well this is confusing haha. So it does prune expired backups on a remote destination (in this case S3) ? What qualifies an "expired" backup - as at the moment it seems like it's just storing everything.

  0
• 

  cPanelMichael

  • June 08, 2015 19:25

  Yes, for instance, daily backups are replaced with new daily backups. There is an internal case open (#111053) to address an issue where Amazon backups are not being pruned from Amazon S3 destination servers, however you should see errors in /usr/local/cpanel/logs/cpbackup_transporter.log when this happens. Thank you.

  0
• 

  LostNerd

  • June 09, 2015 10:11

  There is also another internal case open (#187065) which has recently been discovered to be relevant to the S3 platform, not just FTP:
  187065 - Pruning remote backups can fail when FTP directory listing format is different than expected
  The definition I've received from P. Elsner is: When pruning old backups, we list the directory contents and we look for a specific directory structure.. $string !~ /^[rwxsStT-]{9}$/ That means the files have to begin with a permissions string that starts with rwxsSrT-
  For cPanel Staff: Support Ticket ID: 6578307 for more info on what I have quoted.

  0
• 

  platinum

  • June 11, 2015 07:03

  Yes, for instance, daily backups are replaced with new daily backups. There is an internal case open (#111053) to address an issue where Amazon backups are not being pruned from Amazon S3 destination servers, however you should see errors in /usr/local/cpanel/logs/cpbackup_transporter.log when this happens. Thank you.

  
  Here's my log:
  [2015-06-11 12:08:11 +1000] info [cpbackup_transporter] Starting a "prune" operation on the "XXXXXX" destination ID "XXXXXX". [2015-06-11 12:08:12 +1000] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks [2015-06-11 12:08:12 +1000] info [cpbackup_transporter] cpbackup_transporter - Processing next task [2015-06-11 12:08:12 +1000] info [cpbackup_transporter] cpbackup_transporter - Checking queue for tasks [2015-06-11 12:08:12 +1000] info [cpbackup_transporter] cpbackup_transporter - Exiting - the queue has been emptied; no more work to do [2015-06-11 12:08:12 +1000] info [cpbackup_transporter] cPanel Backup Transporter Queue Daemon is being stopped.
  Nothing has been removed, I've got a backlog of about 10 daily backups built up again. Is there anything else I can check or is this a bug?

  0
• 

  cPanelMichael

  • June 11, 2015 13:20

  Feel free to open a support ticket using the link in my signature so we can take a closer look. You can post the ticket number here so we can update this thread with the outcome. Thank you.

  0

Please sign in to leave a comment.