Skip to main content

[CPANEL-28981] Rare error when converting from MDBOX back to MAILDIR

Denis Gomes Franco
Hey everyone. I think I found a rather unusual, and possibly quite rare, bug. For quite some time I've been using the MDBOX format due to its efficiency. But I had to convert some accounts back to MAILDIR format. And then I found out - the hard way, after a customer's complaint - that while the conversion from MAILDIR to MDBOX was flawless, the opposite has a weird bug: all mails stored in folders that had accented characters (eg. FINALIZA""O) were lost. Thing is, during conversion Cpanel didn't show any warnings. Those folders simply weren't converted and now they are gone. Below I'll summarize the steps to reproduce this issue: 1. Create a CPANEL account using the MDBOX format 2. Create an e-mail account 3. Open that e-mail account in Roundcube 4. Create two folders, one named FINALIZACAO (without accents) and another named FINALIZA""O (with accents) 5. Save some messages to both these folders 6. Run the conversion from MDBOX to MAILDIR 7. Fire up Roundcube again 8. Notice there is only the folder named FINALIZACAO. Here is the log of the conversion process I ran in order to test this bug: Starting "RESTORE" for "Account" "domaintest". The original files will be removed after the conversion. Temporarily lifting quota restrictions for "domaintest". Changing directory to "/home/domaintest/mail". Target "/home/domaintest/mail" on host "cp001.sitesexclusivos.com.br" has 99.16 GB free and requires at least 0 bytes free, which includes space for temporary files. Creating conversion lock file for "_mainaccount@domaintest.com" Cleaning up "/home/domaintest/mail" for "_mainaccount@domaintest.com". Syncing while existing sessions for "_mainaccount@domaintest.com" are using mdbox (pass 1). Syncing "_mainaccount@domaintest.com" mdbox -> maildir (Attempt: 1/10) For those of you playing at home, here's what we're about to run in order to convert the mailbox: /usr/bin/dsync -o mail_fsync=never -o maildir_broken_filename_sizes=yes -o mailbox_list_index=no -o mail_location=maildir:/home/domaintest/mail -u _mainaccount@domaintest.com -v mirror -x INBOX..* -x *@*_* mdbox:/home/domaintest/mail Syncing while existing sessions for "_mainaccount@domaintest.com" are using mdbox (pass 2). Syncing "_mainaccount@domaintest.com" mdbox -> maildir (Attempt: 1/10) For those of you playing at home, here's what we're about to run in order to convert the mailbox: /usr/bin/dsync -o mail_fsync=never -o maildir_broken_filename_sizes=yes -o mailbox_list_index=no -o mail_location=maildir:/home/domaintest/mail -u _mainaccount@domaintest.com -v mirror -x INBOX..* -x *@*_* mdbox:/home/domaintest/mail Removing conversion lock files for "_mainaccount@domaintest.com" Flushing the authentication cache for dovecot. Bouncing existing session for "_mainaccount@domaintest.com" in order to have them reload in the new format. Syncing again to pickup any new mail that was delivered while the system converted "_mainaccount@domaintest.com" to maildir. Syncing "_mainaccount@domaintest.com" mdbox -> maildir (Attempt: 1/10) For those of you playing at home, here's what we're about to run in order to convert the mailbox: /usr/bin/dsync -o maildir_broken_filename_sizes=yes -o mailbox_list_index=no -o mail_location=maildir:/home/domaintest/mail -u _mainaccount@domaintest.com -v mirror -x INBOX..* -x *@*_* mdbox:/home/domaintest/mail Removing mdbox files for "_mainaccount@domaintest.com" from /home/domaintest/mail. rmdir /home/domaintest/mail/mailboxes rmdir /home/domaintest/mail/storage _mainaccount@domaintest.com: success Changing directory to "/home/domaintest/mail/domaintest.com/acct". Target "/home/domaintest/mail/domaintest.com/acct" on host "cp001.sitesexclusivos.com.br" has 99.16 GB free and requires at least 67.94 KB free, which includes space for temporary files. Creating conversion lock file for "acct@domaintest.com" Cleaning up "/home/domaintest/mail/domaintest.com/acct" for "acct@domaintest.com". Syncing while existing sessions for "acct@domaintest.com" are using mdbox (pass 1). Syncing "acct@domaintest.com" mdbox -> maildir (Attempt: 1/10) For those of you playing at home, here's what we're about to run in order to convert the mailbox: /usr/bin/dsync -o mail_fsync=never -o maildir_broken_filename_sizes=yes -o mailbox_list_index=no -o mail_location=maildir:/home/domaintest/mail/domaintest.com/acct -u acct@domaintest.com -v mirror -x INBOX..* -x *@*_* mdbox:/home/domaintest/mail/domaintest.com/acct Syncing while existing sessions for "acct@domaintest.com" are using mdbox (pass 2). Syncing "acct@domaintest.com" mdbox -> maildir (Attempt: 1/10) For those of you playing at home, here's what we're about to run in order to convert the mailbox: /usr/bin/dsync -o mail_fsync=never -o maildir_broken_filename_sizes=yes -o mailbox_list_index=no -o mail_location=maildir:/home/domaintest/mail/domaintest.com/acct -u acct@domaintest.com -v mirror -x INBOX..* -x *@*_* mdbox:/home/domaintest/mail/domaintest.com/acct Removing conversion lock files for "acct@domaintest.com" Flushing the authentication cache for dovecot. Bouncing existing session for "acct@domaintest.com" in order to have them reload in the new format. Syncing again to pickup any new mail that was delivered while the system converted "acct@domaintest.com" to maildir. Syncing "acct@domaintest.com" mdbox -> maildir (Attempt: 1/10) For those of you playing at home, here's what we're about to run in order to convert the mailbox: /usr/bin/dsync -o maildir_broken_filename_sizes=yes -o mailbox_list_index=no -o mail_location=maildir:/home/domaintest/mail/domaintest.com/acct -u acct@domaintest.com -v mirror -x INBOX..* -x *@*_* mdbox:/home/domaintest/mail/domaintest.com/acct Removing mdbox files for "acct@domaintest.com" from /home/domaintest/mail/domaintest.com/acct. unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Trash/dbox-Mails/dovecot.index.log unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Trash/dbox-Mails/dovecot.index.cache rmdir dbox-Mails rmdir Trash unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/spam/dbox-Mails/dovecot.index.log rmdir dbox-Mails rmdir spam unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Archive/dbox-Mails/dovecot.index.log unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Archive/dbox-Mails/dovecot.index.cache rmdir dbox-Mails rmdir Archive unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Aten""o/dbox-Mails/dovecot.index.log unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Aten""o/dbox-Mails/dovecot.index.cache rmdir dbox-Mails rmdir Aten""o unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Atencao/dbox-Mails/dovecot.index.log unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Atencao/dbox-Mails/dovecot.index.cache rmdir dbox-Mails rmdir Atencao unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Sent/dbox-Mails/dovecot.index.log unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Sent/dbox-Mails/dovecot.index.cache rmdir dbox-Mails rmdir Sent unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Drafts/dbox-Mails/dovecot.index.log unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/Drafts/dbox-Mails/dovecot.index.cache rmdir dbox-Mails rmdir Drafts unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/INBOX/dbox-Mails/dovecot.index.log unlink /home/domaintest/mail/domaintest.com/acct/mailboxes/INBOX/dbox-Mails/dovecot.index.cache rmdir dbox-Mails rmdir INBOX rmdir /home/domaintest/mail/domaintest.com/acct/mailboxes unlink /home/domaintest/mail/domaintest.com/acct/storage/dovecot.map.index.log unlink /home/domaintest/mail/domaintest.com/acct/storage/m.1 rmdir /home/domaintest/mail/domaintest.com/acct/storage acct@domaintest.com: success Restoring quota restrictions for "domaintest". Success.
Notice there are no mentions whatsoever that some folders were skipped or could not be read. And unfortunately I trusted Cpanel too much on this one that I didn't click the KEEP OLD MAILBOX FILES checkbox.

Comments

4 comments

Date Votes
  • cPanelLauren
    Hello, You're right, this is definitely an issue and while it is partially fixed in a later version of cPanel/WHM with the internal case ID - CPANEL-28981 there are a number of open cases being worked to address this issue. The full resolution for the issue will be in the internal cases CPANEL-26708 and CPANEL-26762 which are currently unresolved. We'll do our best to update here when the issues are addressed but you can also follow along in our changelogs. You can check them here: Change Logs - Change Logs - cPanel Documentation
    0
  • Denis Gomes Franco
    That's great news. However in the meantime, while that bug isn't fixed, maybe someone could put up a simple warning before conversion (this warning might get skimped over, but then you guys did your part so...), or add a check before conversion, or make the KEEP OLD MAILBOX FILES option mandatory.
    0
  • cPanelLauren
    That's great news. However in the meantime, while that bug isn't fixed, maybe someone could put up a simple warning before conversion (this warning might get skimped over, but then you guys did your part so...), or add a check before conversion, or make the KEEP OLD MAILBOX FILES option mandatory.

    A UI warning such as that is suggested in one of the cases but not implemented yet.
    0
  • cPanelLauren
    Hello, This is just an update to let you know that this issue is marked as resolved in v86 of cPanel & WHM and can be referenced in our changelogs here: Change Logs - Change Logs - cPanel Documentation
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post