Skip to main content

Mailbox Compression and Maildir-->MDBox conversion

mtindor
Hello, I just enabled Mailbox compression today. 99% of the mailboxes on said server are using Maildir. I want to gradually migrate, per account, users' email from the maildir format to the mdbox format. New emails coming in are automatically being compressed. Old emails are not automatically compressed (and yes, I realized this is documented). I wanted to convert an account from maildir to mdbox and have all messages compressed. So I ran the maildir-to-mdbox conversion, then ran an mdbox-to-maildir conversion, and then ran a maildir-to-mdbox conversion. No matter what I do, the conversion process does not seem to be compressing all of the old email for an account. But, from what I've read of others on the forum, simply converting from maildir to mdbox (or vice versa) and back should cause all of the email for a specific account to be compressed. What am I doing wrong? Mike

Comments

11 comments

Date Votes
  • cPanelMichael
    Hi Mike, It's important to note that we don't offer support for the compression of existing messages at this time. The feature request for this is open at: Ability to compress existing email I encourage you to vote and add your feedback to this request. As far as a temporary workaround to compress existing messages, some users have reported success with this by switching from the Maildir format to the Mdbox format, back to the Maildir format, and then to the Mdbox format again. However, it's important to note the following warning from our
    0
  • mtindor
    I recognize that cPanel doesn't support the ability to compress existing email at this time. I have followed exactly that process which others have outlined in the forums. Yes I'm aware of message indexes getting reset for POP3 users. I verified that it is not actually compressing all mail when converting back to maildir and then converting to mdbox again by attempting to view the various m.* files in the storage folder. I can read all of the messages in all of the m.* files using cat / less without issue. In the last m.* file (in this case m.91), there is a mixture of uncompressed and compressed messages because that particular file contains messages from prior to this morning + a couple from this morning [after I enabled compression]. If the conversion back to maildir and then forward to mdbox were actually compressing all of the email, I would not be able to view all of the messages in any of the m.* files using cat / less. Mike
    0
  • cPanelMichael
    I verified that it is not actually compressing all mail when converting back to maildir and then converting to mdbox again by attempting to view the various m.* files in the storage folder. I can read all of the messages in all of the m.* files using cat / less without issue. In the last m.* file (in this case m.91), there is a mixture of uncompressed and compressed messages because that particular file contains messages from prior to this morning + a couple from this morning [after I enabled compression]. If the conversion back to maildir and then forward to mdbox were actually compressing all of the email, I would not be able to view all of the messages in any of the m.* files using cat / less.

    Could you open a support ticket using the link in my signature so we can take a closer look at the system? Thank you.
    0
  • mtindor
    I did upvote the feature request. Thanks for showing me that. Of course, unless a lot of people upvote it, it'll be useless. At any rate, after further searching, I see that even now after dovecot has supported compression for years there is still no simple, tried-and-true reliable way of compressing existing mail. I have to wonder if those who report successful compression of all emails have really checked and confirmed that every indidivual email file (maildir) or m.* file (mdbox) is indeed compressed. Perhaps the space savings they are seeing was simply a savings due to conversion from maildir to mdbox. Of course, I saw one person mention going from 5.5 GB to 3.5 GB of space used. I can't imagine that one could realize that kind of space savings in any other way but having full compressed mail files. I can't imagine saving that amount of space purely by changing from maildir to mdbox (without compression of existing email). Mike
    0
  • mtindor
    Could you open a support ticket using the link in my signature so we can take a closer look at the system? Thank you.

    Hi Michael, I've opened a support ticket. Thanks Mike
    0
  • mtindor
    As I suspected, cPanel support has verified that switching back and forth between the maildir / mdbox formats is not going to magically make all of your mail become compressed, even if you are sure compression is enabled in dovecot, since the conversion scripts do not pass anything through Dovecot and that is the only way Dovecot would compress them. Mike
    0
  • Hedloff
    Hi Mike, It's important to note that we don't offer support for the compression of existing messages at this time. The feature request for this is open at:
    0
  • cPanelMichael
    Hello @Hedloff, Thanks for the report. It looks to have been removed as part of the feature request website's hardware failure in March: [QUOTE]On March 28th, 2018 at approximately 1:30 pm CST the server housing the cPanel Feature Request site"s hypervisor unexpectedly failed. This failure resulted in the loss of data back to September of 2017. Read more in our We apologize for the inconvenience. Thank you.
    0
  • petersphilo
    Sorry for reviving an old thread.. Does anyone have an idea of how much space can be saved, and if users connected using IMAP will also be forced to re-download all emails? (note that docs explicitly state that POP users will re-download all emails present on the server because of new UUIDs on each message) i have clients with mailboxes from 20+GB to 70+GB -- before you ask, i have no idea what in the world they keep in there, but i'm pretty sure it's not giant videos; they're more like pack-rats, keeping every email they ever got, all those email with small files attached add up over time... How much space can i reasonably expect to save? is it 1%; 8%; 25%? i already have compression level set to 6 in 'Mailserver Configuration'... Also, i can't ask my clients to re-download their entire 70GB worth of emails; especially if all of it is going to show up as unread... (On that note, what about custom folders; will their contents also be re-downloaded?) Any help would be greatly appreciated!! Thank you!
    0
  • cPanelMichael
    Hello @petersphilo, I've merged your post into the existing thread on this topic. You can review the previous posts to this thread for discussion related to your questions. Thank you.
    0
  • VIRTBIZ CHRIS
    I just stumbled upon this old thread and have to say I'm surprised there hasn't been more traction. Email consumes a pretty large amount of space on our systems, and the ability to compress existing stored mail would be very welcome.
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post