Skip to main content

email disk space issue

coursevector
I ran into this issue today on a server of ours. We have a cPanel account with a disk quota of 5.5GB. That was adjusted up from 4GB in Apr 2017. I log in today and WHM reports that the account is at 5520/5500 and thus it's at max capacity. I go into the cPanel to investigate what the user did and I find that cPanel is reporting they have 11GB worth of email. Now... in Apr 2017 they were below the 4GB limit. We increased it to give them time to clean up their email which apparently they never did. But how did they exceed the limit by almost double? Shouldn't WHM be stopping mail delivery once its full? Additionally, in the breakdown screenshot, all the accounts summed is about 7.1GB. I can't find where the other 3.8GB is coming from that cPanel is reporting to make the full 11GB. I had to manually suspend the account today since the website was still up and functional and they were still receiving email without an issue at all. Any input on what happened is appreciated. 51271 51275

Comments

26 comments

Date Votes
  • cPanelMichael
    Hello, Can you verify if "Compress Messages" is enabled under "WHM Home " Service Configuration " Mailserver Configuration" on this system? Thank you.
    0
  • coursevector
    It is not enabled
    0
  • coursevector
    I also opened a support ticket with our host LiquidWeb, and they reported this: "I am running a fix quotas script. From what I can tell cpanel just sees the email folder are being much larger than it actually is for no reason. It looks like the default account is the issue. Cpanel thinks it is 6.6G, and I am not seeing it as using any real space, so what I will need to do is use this ticket so that we can investigate this to see what can be done to fix it."
    0
  • cPanelMichael
    Hello, Feel free to update this thread with the outcome once they respond with further information. Thank you.
    0
  • coursevector
    The outcome regarding that was inconclusive. I believe cPanel ran the /scripts/fixquotas and rebooted the server. But at the same time the client deleted a ton of email. LW concluded that the script resolved the issue but I wasn't convinced but had no means to prove otherwise so I dropped it. Today I have another account with a similar issue. I have already run the /scripts/fixquotas script and rebooted the server and the issue is still there. While they have not yet exceeded their quota, the mail is still reporting erroneous numbers. See the images attached. The sum of all the email in the various accounts is 2.5gb, but cPanel reports 3.3GB being using. Which means ~800mb is separate from the email accounts listed, but it shows 800mb in the global account. Logging into the global account shows only 2 emails at all. Where is cPanel coming up with 800mb from? 51463 Here is a screenshot of the two emails, just so you know they are not 800mb worth of emails.
    0
  • cPanelMichael
    Hello @coursevector, Do you have email compression enabled? If so, note that internal case CPANEL-17326 was implemented in cPanel & WHM version 70 to help reduce the confusion when customers check their disk usage in cPanel: Fixed case CPANEL-17326: Clarify meaning of "Email Accounts" in cPanel Disk Usage. The "Disk Usage" in cPanel will display the following note explaining the difference in the amount of disk space reported: [QUOTE] "Email account storage may occupy less space on the disk if you use compression or hard-link optimizations designed to save space. Email account storage does not include the metadata that the system uses to store email. "The files outside of your home directory, the metadata that the system uses to store email in the mail directory, or the files that you do not have permission to access.
    Thank you.
    0
  • coursevector
    No, compression is not enabled.
    0
  • cPanelMichael
    Hello, We'll need you to open a support ticket or reply to the previous ticket so we can take a closer look to see why the disk space reported isn't accurate. Thank you.
    0
  • coursevector
    Your Support Request ID is: 9390147
    0
  • cPanelMichael
    Hello, To update, our Technical Analysts were unable to find a discrepancy between what was reported in the cPanel UI versus the actual usage on the server. Thank you.
    0
  • coursevector
    This was resolved by running this script: /scripts/generate_maildirsize --confirm --verbose USERNAME "appeared to be an issue with what is called the "maildirsize" file which carries the data. In the case of the main user for the cPanel account, it hadn't been regenerated with the current data. I forced the account to update, and after that the size showed up properly as shown in the screenshot." "The maildirsize files I brought up will sometimes become corrupted which can prevent them from being regenerated with up-to-date information, like was happening here. On the cPanel side of things, to the UI, it just looks "stuck" like this where the currently incorrect size gets displayed because of it. The fix is to repopulate the file with current data, and from there it should continue automatically maintaining new updated information. Some public documentation/information about the maildirsize data can be found here:
    0
  • Adam Reece | WebBox
    Neither the "fixquotas" or "generate_maildirsize" scripts resolved this issue for a mailbox on one of our servers. Compression is not enabled. cPanel is adamant that the size of the mailbox in question is 9153 MB, but when inspecting the disk usage there is only 5187 MB of data really on the file system. This is preventing the mailbox from receiving new emails because cPanel wrongly thinks it is exceeding its 6 GB quota.
    0
  • cPanelMichael
    Hi @adamreece.webbox, Can you let us know your ticket number so we can take a closer look? Thank you.
    0
  • Adam Reece | WebBox
    Hi Michael, Ticket number is 9413305.
    0
  • cPanelLauren
    Hi @adamreece.webbox I took a look at your ticket for you, It looks like your issue was specifically related to an internal case CPANEL-17326 which was opened due to mail disk usage discrepancies this has been resolved in v70 of cPanel. While you may not want to update to v70 until it's in RELEASE, if you do choose to do so, please let us know if the issue persists once running v70 Thank you!
    0
  • Adam Reece | WebBox
    I'm very tempted, though the server in question hosts approximately 150 websites, so if V70 is planned for mid April I think we can wait until then. Thanks for looking into our ticket so promptly.
    0
  • cPanelLauren
    I understand and it is planned for mid-April. You're very welcome and I'm happy we were able to get you information on the issue. Thank you!
    0
  • Adam Reece | WebBox
    I've just updated all our servers to version 70 today, I like the feature to clamp mailbox quotas. :) I noticed that this issue doesn't fix itself, so I ran both "fixquotas" and "generate_maildirsize" again for the 2 accounts impacted. Unfortunately they still have mailboxes cPanel thinks are way bigger than they are though. (Issue not resolved.)
    0
  • cPanelLauren
    Hi @adamreece.webbox Would it be possible for you to re-open Ticket ID 9413305 so that we can take another look at this?
    0
  • Adam Reece | WebBox
    Hi Lauren, Sure -- I've done that now. :)
    0
  • cPanelLauren
    Thank you, I'll continue watching the ticket. I'm very sorry that the issue is still present.
    0
  • Adam Reece | WebBox
    NP, thanks for following this up. Fortunately only 2 mailboxes (that we've been informed about) are impacted by this, and those 2 clients are not abusing the increased quotas.
    0
  • cPanelLauren
    Of course! I've also discussed with the analyst looking at the issue and updated him as well.
    0
  • cPanelLauren
    HI @adamreece.webbox I just took a look at the ticket and it appears that your disk usage discrepancy was due to compression being enabled at some point for email. The solution was to decompress the message as disabling this setting does not automatically do so. It looks like you created a one-liner to do this for all your messages, if you don't mind sharing it here, it may help others in the future. Thanks!
    0
  • Adam Reece | WebBox
    Hi Lauren, Yes, it appears that the issue in this server's case was either
    • the email compression setting was enabled when the mailbox was created (and during much of its life), but the setting no longer is, or
    • the hosting account was migrated from another cPanel server that has email compression enabled.
    Rex gave me the following command to help determine if a mailbox has compressed contents. This should be executed with the working directory set to a mailbox maildir location, e.g. "/home/{account}/mail/{domainname}/{mailbox}". find . -type f -exec file {} \+ | awk -F ':[ ]+' '{ print $2; }' | sort | uniq -c | sort -n If the mailbox contains any compressed messages you'll see this line in the response: gzip compressed data, from Unix To decompress these messages I built this command. -- Obviously this will increase the mailbox disk usage.
    • Make a backup copy of the {mailbox} directory first.
    • Really. Make a backup copy of the whole {mailbox} directory first.
    • Are you sure you have a backup copy of the {mailbox} directory?
    Again, run this with the working directory set to a mailbox maildir location. for MSGFILE in $(find . -type f -exec file {} \+ | grep gzip | awk -F ':[ ]+' '{ print $1; }' | sort); do echo "${MSGFILE}"; gzip -c -d "${MSGFILE}" > "${MSGFILE}.plain"; if [ -s "${MSGFILE}.plain" ]; then echo "Success"; rm -f "${MSGFILE}"; mv "${MSGFILE}.plain" "${MSGFILE}"; else echo "Failed"; rm -f "${MSGFILE}.plain"; fi; done; I'm sure a Bash guru could shorten that further, particularly the use of a temporary ".plain" file. You will start seeing a list of email files, with either "Success" or "Failed" indicating if the message has been decompressed. Bare in mind there will be an expected (short) delay for the cPanel disk usage component to notice these changes. ---- You could wrap these commands up to work with all mailboxes for an account (all domains within) as follows, with your working directory to "/home/{account}" for DOMAINDIR in $(find . -maxdepth 1 -type d | sort); do if [ ! -d "${DOMAINDIR}/mail" ]; then continue; fi; for MSGFILE in $(find "${DOMAINDIR}/mail" -type f -exec file {} \+ | grep gzip | awk -F ':[ ]+' '{ print $1; }' | sort); do echo "${MSGFILE}"; gzip -c -d "${MSGFILE}" > "${MSGFILE}.plain"; if [ -s "${MSGFILE}.plain" ]; then echo "Success"; rm -f "${MSGFILE}"; mv "${MSGFILE}.plain" "${MSGFILE}"; else echo "Failed"; rm -f "${MSGFILE}.plain"; fi; done; done; I've not tested this for multiple mailboxes at once though! Verify what I've written, and backup the mailbox directories first. (Only 2 mailboxes seemed to be impacted on the server in question so I just sorted those individually.) ---- Thanks again for your support on this issue. :)
    0
  • cPanelLauren
    Hi @adamreece.webbox This is incredibly helpful, thank you so much for sharing and including so much detailed information. I'm so happy we were able to help and that you were able to identify a solution :D ! Thank you!!!
    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post