How to backup 600GB of email data?
Hi there,
I've just taken over a client's cPanel system, and one of their main concerns are backup and redundancy.
They are currently hosting on a RAID 5 dedicated server. The email has been coming for about 20 years and is now 600GB big.
I just need some ideas on what you guys propose I do to back this up. On my Plesk and Virtualmin systems we do incrementals daily and weekly full backups. But I'm afraid that 600GB will just take down the system. Please give me some ideas.
-
I would use cPanel's incremental backup system. Initial backup will take a while but subsequent ones won't be bad. You might also consider switching them over to mdbox. This will store the email in large collections in single files as opposed to what must be hundreds of thousads of files in mbox format. 0 -
Large email accounts are a big part of the reason we began utilizing mdbox, it might help substantially, along with archiving. @GOT is right, the initial incremental backup will take a considerable amount of time but subsequent ones shouldn't be too bad at all, just changes. In case you hadn't checked it out already, the following is our documentation on automated backups: Backup Configuration - Version 84 Documentation - cPanel Documentation 0 -
Wow guys thanks so much these are great replies! I will start my research straight away. I need info on mdbox conversion of existing system, and archiving: Is archiving built into cPanel, or is that a menu? Or where can I get more info on the ability to archive mailboxes? And then if there is any advice for *converting* an existing system from whatever to MDBOX, that would be incredibly useful :-) Thank you so much for the help. 0 -
The conversion is right in WHM in email section called mailbox conversion. 0 -
Hi @eugenevdm.host The docs on email archiving might help, you can find them here: Archive - Version 84 Documentation - cPanel Documentation Also for reference, here is the mailbox conversion documentation for maildir/mdbox: Mailbox Conversion - Version 84 Documentation - cPanel Documentation 0 -
Hi @cPanelLauren, Thank you so much, I am making a lot of progress with the backups. For interest, a 180GB backup takes 12 hours. It looks like our 600GB backup will take 50 hours. Next I'm trying uncompressed because really I'm after incremental. I'm a bit confused why incremental is uncompressed, is it due to the rsync nature? My main question though is: In the mailbox conversion documentation quoted on your previous message: > Due to this change in UIDs, mailbox conversions can cause POP email users to receive previously-downloaded messages as though they were new. Does this apply to IMAP as well? We have 1000s of accounts and resetting this flag could have major customer service implications for our staff and clients. warm regards, Eugene 0 -
Incremental backups aren't compressed because they don't actually contain ALL of the data - only updates and they're hardlinked to the previous backup. The backup configuration documentation I sent you previously explains how they work as well. Does this apply to IMAP as well?
Just POP3 as they download mail where IMAP synchronizes it0 -
For those reaching this post, I have feedback. 1. We converted all users from MAILDIR to MDBOX without any disruption to the user base. On both servers it took very long, as in about a day or two. Not sure how much disk space we're saving because of this. 2. The archiving feature is useful, but it's mostly user and age of message controlled so the real use is only really for individual users that are using too much. Not really an ISP-wide feature. 3. The incremental backups are working like a charm: a. We installed a USB drive on the Mail server. b. The mail server which uses over 600 GB is doing incremental backups and these are only taking 30 minutes. c. We're using same USB drive to backup the Web server. d. The web server's backup, although has about same amount of data, takes 4 hours. I believe it's because it's 100s of domains and also because USB is a bit further way. So we're happy, backups are sorted. We had to move away from Backblaze just because of the size of the backups. Backblaze RSYNCs were running 24/7 and also the large files because unmanageable at the Backblaze UI forcing us to use the command line. Anyway, this was a poor solution, not because Backblaze is bad, but because you don't back up 600 GB to far away from your local disk in terms of latency. 0 -
@eugenevdm.host Thank you so much for the update on this. It's really nice to know when changes to things help! I'm glad the backup issues are sorted. As far as the MDBox items, it's typically a considerable amount on servers with large email accounts, pending once you ensure that there were no issues with the conversion you removed the maildir files that were present. 0
Please sign in to leave a comment.
Comments
9 comments