Skip to main content

Backup destination config for S3

bloatedstoat
  • Edited

Hello,

I'm battling with cpanel backup to create a workable S3 solution.

Due to limited local disk space our daily, weekly and monthly backups have a retention policy of 1 in each case on the local server.

Daily retention   = 1
Weekly retention  = 1
Monthly retention = 1
Strict retention  = OFF

The difficulty is that cPanel's native retention/pruning policy is applying the pruning to the remote S3 destination too.

The current structure we have in place is:

bucket/
└── server/
    ├── 2026-08-16/
    │   ├── accounts/
    │   └── system/
    │
    ├── 2026-08-15/
    │   ├── accounts/
    │   └── system/
    │
    ├── weekly/
    │   ├── 2026-08-16/
    │   │   ├── accounts/
    │   |    └── system/
    └── monthly/
        ├── 2026-08-01/
        │   ├── accounts/
        |   └── system/

The ideal structure would be where daily has its own explicit prefix and then we could configure a retention life cycle policy better on S3.

bucket/server/daily/
bucket/server/weekly/
bucket/server/monthly/

Can the cPanel generated object names be configured to give us enough information to distinguish between the three sets, daily, weekly, monthly using a daily explicit prefix?

OR

Can cpanel backup objects have tags? 

backup-type=daily
backup-type=weekly
backup-type=monthly

The simple fix would be having the option to change cPanel's remote pruning behaviour and apply the pruning policy to local only. From what I've read this is not currently achievable though right?

Thanks.

 

 

 

Comments

9 comments

Date Votes
  • cPRex Jurassic Moderator

    Hey there!  As far as I'm aware, this is currently not an option - the backup schedule can't be adjusted to have local with one value and remote with another.  

    Would you like to submit (or have me submit for you) a feature request to update this behavior?

    1
  • bloatedstoat

    Yes please mate, may as well join the queue.

    I can see from older posts that this has been requested.

    Thank you! 

    0
  • bloatedstoat

    There are two feature requests for the S3 backup.

    1. The ability to disable remote pruning in S3 bucket backups within the S3 configuration within WHM.

    Currently I have a delete object denied rule in place in S3 which achieves what I need with the pruning. Pruning is denied but we get a "Backup Transport Error" email.

    2. Daily backups have their own prefix as per weekly and monthly.

    Having the daily prefix would make the transition of S3 backup objects to Glacier trivial. Presently it's created a bit of a headache.

    Current example structure is:

    bucket/
    └── server01/
        ├── 2026-08-16/
        │   ├── accounts/
        │   └── system/
        │
        ├── 2026-08-15/
        │   ├── accounts/
        │   └── system/
        │
        ├── weekly/
        │   ├── 2026-08-16/
        │   │   ├── accounts/
        │   │   └── system/
        │   └── 2026-08-09/
        │       ├── accounts/
        │       └── system/
        │
        └── monthly/
            ├── 2026-08-01/
            │   ├── accounts/
            │   └── system/
            └── 2026-07-01/
                ├── accounts/
                └── system/

     

    We can apply lifecycle rules by prefix to weekly and monthly on S3 but not to server01 as it would group weekly and monthly under the same policy, so having trouble here applying lifecycle rules to the daily backups.

    Desired structure

    server01/
    ├── daily/
    ├── weekly/
    └── monthly/

    I did look at S3 versioning but each backup is not a version of the same backup thus versioning falls on its face too.

    If anyone else here has tackled this successfully I'd appreciate a heads up.

    Thank you.

     

    0
  • cPRex Jurassic Moderator

    1 - So you have a rule that does this for you, and it works, but cPanel doesn't like it and sends you an error?  Is that where we're at?

    2 - Are we sure this isn't an S3 limitation out of cPanel's control?  If that's the case, I don't believe we're going to be expanding the backup tools to offer additional functions with options like Comet backup available now.

    0
  • bloatedstoat

    Hello,

    Thanks for the reply.

    1. Yes, that's the case. We explicitly deny the cpanel backup operation from pruning any backups on S3 through an IAM user. The cpanel backup process complains that it cannot prune and the following email is received.

    The system encountered errors during transport of the backup files. Below is a preview of the attached log file.

    Preview of transport errors log:
    Unable to prune transport “Amazon_S3”

    Error pruning “/server01/weekly/2026-08-15” from “Amazon_S3”: AccessDenied: User: arn:aws:iam::redacted:user/server01-cpanel-backups is not authorized to perform: s3:DeleteObject on resource: "arn:aws:s3:::server01-cpanel-backups-redacted-ap-southeast-2-an/server01/weekly/2026-08-15/system/system_files.tar" with an explicit deny in an identity-based policy: arn:aws:iam::redacted:policy/CpanelServer01BackupToS3

    The system could not prune the “/server01/weekly/2026-08-15” directory due to an error.

    Read the go.cpanel.net/directorypruning documentation for solutions to successfully prune the directory.

    We achieve the outcome we aimed for in that our backups moved to S3 are retained and our local cpanel policy of retain 1 for daily, weekly and monthly work too. We save on local disk space and importantly, our own lifecycle retention policy on S3 works too. If we didn't have this policy in place cpanel would prune the remote backups.

    2. No. There is no daily sub-directory provided, whereas weekly and monthly are provided

    If you consider the following:

    Cpanel's default backup structure on the local machine

    /backup <-- no daily sub-directory 
    /backup/weekly
    /backup/monthly

    Desired:

    backup/
    ├── daily/
    ├── weekly/
    └── monthly/

    Then on S3 we get

    server01/
    ├── daily/
    ├── weekly/
    └── monthly/

    We've managed to hock a process together using Lambda to achieve what we want but it was really convoluted. If the backup behaviour on cpanel was to create a daily directory for the daily backups then life would be rosy.

    I'm hoping this helps clarify. 

    Thank you.

     

     

     

    0
  • cPRex Jurassic Moderator

    I'm looking into this today and I hope to have more details soon!

    0
  • cPRex Jurassic Moderator

    I talked with our team and they let me know two things.

    The first, is that we *could* make this change, but then existing daily backups would be stuck unless we also changed the pruning system as well to compensate.  It would likely be a large change and could potentially be disruptive to other users.

    As a potential workaround, even though it's not ideal, could you create multiple backup jobs to handle the daily/weekly/monthly tasks, or multiple S3 buckets?

    0
  • cPRex Jurassic Moderator

    There's also the option of using a more advanced tool like Comet Backups to handle all of your backup needs.

    0
  • bloatedstoat

    Thanks for your time looking in to this, appreciate the time. We so far have what we need working which is ultimately the desired outcome, it's just not as elegant as we would have liked.

    Cheers.

    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post