Skip to main content

CPANEL-42825 - Random address mass mailing

coolcom
Greetings... Can anyone who is experiencing or has experienced this, comment on it and share your experience? Over the last few weeks our mail servers have been getting mail bombed with large amounts of email to random (non-existant) addresses at some of the domains we host. I am not referring to alphabet login attacks... these are just attempts to send email to random addresses. There is no problem rejecting the emails (Sender verification, some TLD rejections etc.)... but I'm curious whether others are experiencing this recently? Has this practice suddenly increased, or have we just been included? I'm also open to suggestions on how to best relieve our mail servers of some of the pressure of having to deal with all these rejections. The sending mail server IP's are all over the place.... Russia, China, Bolivia, India, US.. so compromised mail servers from around the globe....no sense even trying to add IP's to a block list. Possibly a use of AI... don't know. I've experimentally set some of our mail servers to only receive email from Canada, US (and a few countries that process email over in Europe for Microsoft etc.) by using the "Filter Incoming Emails By Country" in WHM>Email to see how that works. The filter incoming emails by domain works (*.ru for example), but the server mail log still lists all of these... whereas using the country rejection prevents them from even arriving apparently as there is no record of them in the logs at all. I'm not sure why "Exim Configuration Manager>ACL Options>Dictionary attack protection" does not stop the incoming after 4 failed recipients... I thought that it would. There are usually about 100 attempts before they move to another domain. So any advice, suggestions or sharing of current or past experiences welcomed. Thanks

Comments

17 comments

Date Votes
  • cPRex Jurassic Moderator
    Hey there! This sounds normal to me, and it does seem to have picked up recently. I also recommend ensuring that cPanel >> Default Address is configured in a way to either reject those or send them to an account you filter.
    0
  • coolcom
    Hey there! This sounds normal to me, and it does seem to have picked up recently. I also recommend ensuring that cPanel >> Default Address is configured in a way to either reject those or send them to an account you filter.

    Thanks for the reply. Yes, default is set to reject. Would you be able to help me understand why the senders are able to mail up to 100 addresses before they move on to the next domain? I thought the Exim Configuration Manager>ACL Options>Dictionary attack protection... would stop that, but perhaps I am not understanding it correctly. I have tried using Maximum message recipients (soft limit) and Maximum message recipients before disconnect (hard limit), but I believe that only limits the amount of addresses an email can have in the To:, Cc: and Bcc: fields. If the spammers are sending individual emails, this will have no effect... correct?
    0
  • cPRex Jurassic Moderator
    I don't understand either as that setting defaults to 4 failed attempts and then it should be blocking that sender: Block dictionary attacks by dropping and ratelimiting hosts with more than 4 failed recipients
    Yes, if the spammers are sending individual emails the Max recipient settings would not apply. If you don't see the dictionary attack setting performing as you expect, it might be best to create a ticket with our team so we can take a look, as we'd need to examine the Exim logs on the machine to see why that isn't triggering.
    0
  • coolcom
    I don't understand either as that setting defaults to 4 failed attempts and then it should be blocking that sender: Block dictionary attacks by dropping and ratelimiting hosts with more than 4 failed recipients
    Yes, if the spammers are sending individual emails the Max recipient settings would not apply. If you don't see the dictionary attack setting performing as you expect, it might be best to create a ticket with our team so we can take a look, as we'd need to examine the Exim logs on the machine to see why that isn't triggering.

    Thanks.... Ticket has been submitted.
    0
  • cPRex Jurassic Moderator
    Can you post the number here so I can follow along?
    0
  • coolcom
    Can you post the number here so I can follow along?

    Certainly... #95118856 It appears that "...an open internal case that might be adding to this issue. Specifically, it seems that when the default email address is set to "fail" incoming mails for non-existent addresses, the senders are not properly getting scanned or added to blacklists. This in turn prevents future emails from being rejected."
    0
  • cPRex Jurassic Moderator
    Thanks for that - the case is CPANEL-42825 and I'll be sure to post updates once I hear more!
    0
  • bayden10

    Any updates on this issue as we are experiencing the same and have tried many variations to remedy this. Exim servers are being hammered. Sometimes the rate limit works then 1,000s are hitting all at the same time, DOS on Exim should be shielded much better than what is currently available to block dictionary and rate limit connections.

    Also the case CPANEL-42825 cannot be found. Is there a new case ? Please provide a link as search produces nothing.

    0
  • coolcom

    Hi bayden10... I checked back and went through the Support link and community chat... 

    Support says case is "Solved"... dont know why... there was no response to my last post... and its been about 10 months (#95118856)... and I see no solutions posted outside of one user suggesting setting "default" unassignable mail to "blackhole."

    We've moved on to a provider with "managed" services for a very small additional fee... and now I spend my time on marketing and sales and let them monitor mail issues. Much more fruiful, and easier on the heart... lol.

    Sorry I could not be of additional help. I did see a drop off in those types of attacks, but more likely becasue they moved on to other servers and domains.

    0
  • coolcom
    • Edited

    Oh... this is the only link I find to the Support case... but it's confusing as to what is Support and what is Community chat... and then there is a reference to my request not being the same as the case... etc. etc... so... sorry.... not much help.

    CPANEL-42825 - Ratelimit incoming mail to non-existent accounts – cPanel

    0
  • bayden10

    coolcom thanks for the reply, yes I get the same search results. Might have to open a ticket as I cannot seem to find the best combination of rate limits etc. to mitigate this DoS. Been a couple of weeks now and we seem to be the next target with our ASN..

    0
  • bayden10

    The only way rate limit seems to work, in a crippled manner, is you must not select discard for the accounts default, select Forward to your system account. This will now produce a 452 too many recipients but still 100 connections hit with reject. This is still not optimal. Haven't opened a ticket yet as this appears to be an issue with the design.

    So far my adjustments to try and limit this DoS

    CSF - Relay Tracking
    --------------------
    RT_RELAY_LIMIT = 100  to --> RT_RELAY_LIMIT = 50
    RT_AUTHRELAY_LIMIT = 100  to --> RT_AUTHRELAY_LIMIT = 50
    RT_LOCALRELAY_LIMIT = 100 --> RT_LOCALRELAY_LIMIT = 50
    RT_LOCALHOSTRELAY_LIMIT = 100 to --> RT_LOCALHOSTRELAY_LIMIT = 45

    Messenger_Rate = 100/s --> Messenger_Rate = 35/s
    Messenger_Burst = 150 --> Messenger_Burst = 50

    CONNLIMIT = 25;35

    Exim Basic Config
    -----------------
    Maximum message recipients before disconnect (hard limit) = 35
    Maximum message recipients (soft limit) = 50 

    Exim Advanced Config
    --------------------
    smtp_accept_max = 10

    Any comments welcome.Perhaps a workflow diagram would help with the design, I cannot make sense of it.

    0
  • cPRex Jurassic Moderator

    Hey there!  Does this seem like the same behavior here?

    https://support.cpanel.net/hc/en-us/community/posts/29632844615191

    If so, I've noted a case in that thread with more details that our team is looking into options to help with this behavior.

    0
  • bayden10
    • Edited

    Hey cPRex

    Yes same behavior and I've managed to tweak enough where I now only see single Exim log entries
    except if the sender domain has been banned which I had been previously updating so I then would remove those as needed now  Removing the ban then triggers the limiting in Exim.

    ```

    452 too many recipients
    Too many recipients specified. Come back in a few hours.
    Rejected relay attempt: 


    ```

    The only way this would work is if the domain is not banned and the default domain email does not use drop but instead set to Forward to your system account which is not ideal so there seems to be some logic issues to scan before that drop and rate limiting if ban is found on the domain backscatter.

    My current tweaked setup which is working except for what I outlined:
    .

    CSF - Relay Tracking
    -----------------------------

    RT_RELAY_LIMIT = 50
    RT_AUTHRELAY_LIMIT = 50
    RT_LOCALRELAY_LIMIT = 50
    RT_LOCALHOSTRELAY_LIMIT = 45

    Messenger_Rate = 35/s
    Messenger_Burst = 50

    CONNLIMIT = 25;20


    Exim Basic Config
    ------------------------

    Maximum message recipients before disconnect (hard limit) = 35
    Maximum message recipients (soft limit) = 50 


    Exim Advanced Config
    ------------------------------

    smtp_accept_queue_per_connection = 10
    smtp_accept_max = 15
    smtp_accept_max_per_connection = 15
    smtp_accept_max_per_host = 15


    Tweaks
    -----------

    Initial default/catch-all forwarder destination = System account --> Fail
    If sender domain is banned still hits 100 connections, remove that domain then limiting works.

    Hope a solution is found with this particular behavior.

    0
  • bayden10

    Hello again, an update. Since we had so many Sender domains banned it was decided to disable this in Exim until this case is resolved. Not ideal by any means.

    Exim Advanced Config
    ------------------------------
    blockeddomains = disabled

    With the above configuration and this addition has greatly reduced processing and log entries against this backscatter/DoS.

    cPRex where can we signup to monitor that internal case or can you shed some light to its resolution?

     

    0
  • cPRex Jurassic Moderator

    I'm following the case on my end and I'll be sure to update the threads as I hear things, but since it's so new there hasn't been any action just yet.  Our team is still exploring the best way to deal with these type of attacks that have recently appeared.

    Since that is the case I don't have a way for you to follow along for updates, but feel free to reach out to me at any time and ask!

    0
  • bayden10

    Ok thanks, just in case any packet hunters out there I've noticed some interesting behavior with the backscatter packets. Here is a sample with notable characteristics. Any comments welcome. Of course the Source MAC is the previous hop from our upstream peering.

     

    0

Please sign in to leave a comment.

Didn't find what you were looking for?

New post