A better filter for spam domain names
I have a word filter to avoid spam from some .domains inside /usr/local/cpanel/etc/exim/sysfilter/options/custom_words
it works well, although I experience problems with a few legitimate senders who are using ".top" in their email adrresses like: "richard.top.sales@legitimatedomain.com" Their messages also are discarded. My only way to avoid this problem is including exceptions in this way:
although I'm looking for a better automatic way to solve this issue. I cannot find enough information inside Exim docs about the $h_from variable. I wonder if there is some valid way to specify a better expression, something like: $h_from: contains "@*.top" to delimitate the filter action to the last part of the sender address thanks for any help
if
$h_from: contains ".top" or
$h_from: contains ".pro" or
$h_from: contains ".xyz" or
$h_from: contains ".bid"
and $h_from: does not contain "legitimatedomain.com"
then
seen finish
endifit works well, although I experience problems with a few legitimate senders who are using ".top" in their email adrresses like: "richard.top.sales@legitimatedomain.com" Their messages also are discarded. My only way to avoid this problem is including exceptions in this way:
and $h_from: does not contain "legitimatedomain.com"although I'm looking for a better automatic way to solve this issue. I cannot find enough information inside Exim docs about the $h_from variable. I wonder if there is some valid way to specify a better expression, something like: $h_from: contains "@*.top" to delimitate the filter action to the last part of the sender address thanks for any help
-
Something new that may interest you might be Filter Incoming Emails by Domain | cPanel & WHM Documentation Have you taken a look at this to see if it would work for you? 0 -
CSF mailscanner everytime for me. It's not free, but not expensive and does exactly this with minimal of fuss. 0 -
@Mise It's quite new, which is why I pointed it out! Let me know if it works for you :) 0 -
@Mise It's quite new, which is why I pointed it out! Let me know if it works for you :)
bad news I'm here again. It seems the whm filter window has the same problem, and these messages are discarded. It is not able to distinguish the second part of the address. I have included the chains in this way: *.pro *.bid *.buzz *.desi *.gdn and Exim shows "discarded" when looking "something.pro@legitimatedomain.com": 2020-01-11 11:31:31.241 [941] 1iqENa-0000ED-2N => discarded (system filter) also, it seems in the filter windows there is some delay in applying the changes. After emptying the filter window and restart exim, the changes weren't applied immediately or perhaps in a definitive way. I have need to apply two times and restart again. Hope you can solve this issue. There are some legitimate address which can use the format "='whatever.professional@somedomain.com'>whatever.pro@somedomain.com", "richard.biden@somedomain.com" which would be discarded (not sure in the case of "richard.biden@somedomain.com" but the first "='whatever.professional@somedomain.com'>whatever.pro@somedomain.com" is sure) I suppose there is some way to limit the filter after the "@" although I cannot find how to do it. thanks! :)0 -
Would two wildcards work. eg X@X.pro Just a thought (forum wouldnt allow me to use stars) 0 -
Would two wildcards work. eg X@X.pro Just a thought (forum would allow me to use stars)
that's not allowed, it shows the error: ='x@x.pro'>x@x.pro: The label must contain only non-ASCII characters and the following: a-z, A-Z, 0-9 y -.0 -
@Mise Can you show me the output from the logs where it discarded a message with the first part? If that is actually the case this would be considered something that needs to be resolved. 0 -
Can you show me the output from the logs where it discarded a message with the first part? If that is actually the case this would be considered something that needs to be resolved.
logs only shows this line: 2020-01-11 11:31:31.241 [941] 1iqENa-0000ED-2N => discarded (system filter) that's all. Please, do a test blocking domain *.pro and then with any sender address "mike.pro.sales@anydomain.com" Yes, I believe you should do something or maybe contact with Exim developers in case the regular expressions are not allowed inside "$h_from" variable. While these lists of spammer domains grows in internet probably more people can be affected. And the "discard" option don't send notifications, at least in my case I would add more logs although I don't want to activate these filters because I ignore if this can affect more chain addresses like "mike.prolan.sales@domain.com" or similar. I experienced a bad problem with a customer just because only one message, and I fear the filter activation can affect to some entering message from a new legitimate sender. waiting news, Thanks so much, :)0 -
Hello, I was hoping you'd be able to provide the log excerpt for this from the exim logs at /var/log/exim_mainlog0 -
here is one real excerpt. I have made minimal replacements, the real email addresses involved are exactly with the same format - my server: 2.2.2.2
- sender: sender.prolan@gmail.com
- receiver: receiver@mydomain.com
2020-01-07 08:11:26.436 [9518] SMTP connection from mail-ua1-f49.google.com [209.85.222.51]:39494 I=[2.2.2.2]:25 closed by QUIT 2020-01-07 08:11:26.490 [9522] 1iaira-0004Pa-1P DKIM: d=gmail.com s=20161025 c=relaxed/relaxed a=rsa-sha256 b=2048 [verification succeeded] 2020-01-07 08:11:26.545 [9522] 1iaira-0004Pa-1P <= sender.prolan@gmail.com H=mail-vs1-f53.google.com [209.85.217.51]:32976 I=[2.2.2.2]:25 P=esmtps L- X=TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128 CV=no SNI="mydomain.com" S=64060 M8S=0 DKIM=gmail.com RT=0.397s id=CEFfk+A7iF9fx+U_v_fy36Ak0c2rBf5BOThf5s@mail.gmail.com T="Mr. Sender" from for receiver@mydomain.com 2020-01-07 08:11:26.550 [9574] cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1iaira-0004Pa-1P 2020-01-07 08:11:26.553 [9574] 1iaira-0004Pa-1P => discarded (system filter) 2020-01-07 08:11:26.553 [9574] 1iaira-0004Pa-1P Completed QT=0.509s
thanks to your message about logs I have found more senders discarded in past times because they contained the chain ".pro" and others in some place of the first part address. Then now this is sure: any sender like "sender.prolan@gmail.com" is discarded because the chain *.pro is detected It can affects legitimate email addresses depending of the blocked domains ( .pro, .xyz, .bid and etc.. ) Please, update the WHM filter. thanks! :)0 -
@Mise In the opening of the thread here you noted that you had modified the exim system filter but this behavior was occurring. Did you remove your system filter prior to enabling this setting? I ask because of two reasons: - the output you've noted indicates this is a system filter:
2020-01-07 08:11:26.553 [9574] 1iaira-0004Pa-1P => discarded (system filter)
- I cannot replicate this behavior in any fashion:
- First I created and sent from an account with no period in the user: laurenprolific
[root@server ~]# exigrep 1irpgq-0005BL-7W /var/log/exim_mainlog
- Then I created an account with a period in the user lauren.prolific and used protonmail just to be thorough:
[root@server ~]# exigrep 1irprJ-0006RC-0Z /var/log/exim_mainlog 2020-01-15 15:05:54.147 [24842] cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1irprJ-0006RC-0Z 2020-01-15 15:05:54.107 [24750] 1irprJ-0006RC-0Z H=mail-40130.protonmail.ch [185.70.40.130]:33267 I=[MyIPAddress]:25 Warning: "SpamAssassin as lauren detected message as NOT spam (1.8)" 2020-01-15 15:05:54.136 [24750] 1irprJ-0006RC-0Z H=mail-40130.protonmail.ch [185.70.40.130]:33267 I=[MyIPAddress]:25 Warning: Message has been scanned: no virus or other harmful content was found 2020-01-15 15:05:54.139 [24750] 1irprJ-0006RC-0Z <= lauren.prolific@protonmail.com H=mail-40130.protonmail.ch [185.70.40.130]:33267 I=[MyIPAddress]:25 P=esmtps L- X=TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256 CV=no S=3890 M8S=0 RT=0.139s id=MNP0OUXeXN9qby-ddb28WXCf9-kB3LkwpfY8AhuTkilG571wbx_rFl_QP_ohpIM8wupL62toOHk4-YRvQ6LaPNoh9GbQ2sum-hA_tcvhOTI=@protonmail.com T="prolific testing" from for lauren@mydomain.us 2020-01-15 15:05:54.215 [24842] 1irprJ-0006RC-0Z => lauren F= P= R=virtual_user T=dovecot_virtual_delivery S=4107 C="250 2.0.0 GHgZDLJ+H16RXwAA9Z/phw Saved" QT=5s DT=0.013s 2020-01-15 15:05:54.216 [24842] 1irprJ-0006RC-0Z Completed QT=5s
0 - the output you've noted indicates this is a system filter:
-
that's strange. You are right, it is not emptied, inside /usr/local/cpanel/etc/exim/sysfilter/options/custom_words I have: if $h_from: contains "hacker" or $h_from: contains "anonym" or $h_subject: contains "Viagra" or $h_subject: contains "Cialis" then #fail text "Unable to delivery" seen finish endif
Do you mean is there some incompatibility between the existence of both filters? Should I delete the custom_words file completely? thanks! *PD: I have no any trace of ".pro" inside /usr/local/cpanel/etc/exim folders# grep -i "\.pro" /usr/local/cpanel/etc/exim -R0 -
You noted initially that you had the following: I have a word filter to avoid spam from some .domains inside /usr/local/cpanel/etc/exim/sysfilter/options/custom_words
That had the following:if $h_from: contains ".top" or $h_from: contains ".pro" or $h_from: contains ".xyz" or $h_from: contains ".bid" and $h_from: does not contain "legitimatedomain.com" then seen finish endif
I am unaware if you removed this or not but it would explain the filtering of these. As you can see in my test it is clearly not filtering .pro from the first part of the message. I can't replicate this at all and it looks like a configuration issue. In the filter you had previously you did not have a / before .pro so a grep for "/.pro" would be invalid. Can you send me a screenshot of the settings in WHM>>Service Configuration>>Exim Configuration Manager->Filters?0 -
In the filter you had previously you did not have a / before .pro so a grep for "/.pro" would be invalid.
please note in the grep command I wrote it was "\.pro" instead "/.pro". In my terminal it works;# grep -i "\.pro" /usr/local/cpanel/etc/exim -R
the backslash "\" can include the character "." and the search is able to find ".pro" instead only "pro" [QUOTE] I am unaware if you removed this or not but it would explain the filtering of these. As you can see in my test it is clearly not filtering .pro from the first part of the message. I can't replicate this at all and it looks like a configuration issue. ... Can you send me a screenshot of the settings in WHM>>Service Configuration>>Exim Configuration Manager->Filters?
I can see. Although I'm not sure if you are replicating exactly what I say Please, to replicate this bug follow these 3 steps: (1)- Write the following inside /usr/local/cpanel/etc/exim/sysfilter/options/custom_wordsif $h_from: contains "hacker" or $h_from: contains "anonym" or $h_subject: contains "Viagra" or $h_subject: contains "Cialis" then #fail text "Unable to delivery" seen finish endif
Save and restart Exim (2) - write the following inside WHM -> "Filter Incoming Emails by Domain" : (see the attached image) *.top *.pro *.xyz *.bid Save and restart Exim (3) - now send to yourself a message from some email address including the characters .pro in the first part: In example:From: "richard.professional@domain.com" To: "myaddress@mydomain.com"
now check the logs. At least in my case, the message is discarded. please, confirm this Thanks!0 -
I don't mind being told to shut up :) And I did ask, Would a double wildcard work. ? star@star.pro (the forum must see the actual string as an sql injection or something) 0 -
- None of our testing is done with custom configurations in place. My testing (and our internal testing) absolutely will not take into account custom configurations you may have in place. Customizations such as a custom filter being present are things you would need to troubleshoot.
- You can clearly see in the output I provided that with the default settings in place the .pro in the first part of my email address was not filtered.
- I requested the following screenshot but I don't see it in your response:
Can you send me a screenshot of the settings in WHM>>Service Configuration>>Exim Configuration Manager->Filters?
- What I'm trying to surmise is if you still have the following custom filter in place which would account for the behavior you're seeing:
if $h_from: contains ".top" or $h_from: contains ".pro" or $h_from: contains ".xyz" or $h_from: contains ".bid" and $h_from: does not contain "legitimatedomain.com" then seen finish endif
0 -
I don't mind being told to shut up :) And I did ask, Would a double wildcard work. ? star@star.pro (the forum must see the actual string as an sql injection or something)
Actually I believe it italicizes it :) and two asterisks makes it bold - I'll shut that feature off (it was useful for me but i can see how it wouldn't be in this instance)0 -
- None of our testing is done with custom configurations in place. My testing (and our internal testing) absolutely will not take into account custom configurations you may have in place. Customizations such as a custom filter being present are things you would need to troubleshoot.
- You can clearly see in the output I provided that with the default settings in place the .pro in the first part of my email address was not filtered.
- I requested the following screenshot but I don't see it in your response: WHM>>Service Configuration>>Exim Configuration Manager->Filters?
- What I'm trying to surmise is if you still have the following custom filter in place which would account for the behavior you're seeing:
if $h_from: contains ".top" or $h_from: contains ".pro" or $h_from: contains ".xyz" or $h_from: contains ".bid" and $h_from: does not contain "legitimatedomain.com" then seen finish endif
I attach you an screenshot of WHM>>Service Configuration>>Exim Configuration Manager->Filters: and this is the code of the active /etc/cpanel_exim_system_filter (without comments):root># grep ^[^#] /etc/cpanel_exim_system_filter if not first_delivery then finish endif if error_message and $header_from: contains "Mailer-Daemon@" then finish endif if $header_content-type: matches "(?:file)?name=(\"[^\">+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|exe|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc])\")" then fail text "This message has been rejected because it has\n\ potentially executable content $1\n\ This form of attachment has been used by\n\ recent viruses or other malware.\n\ If you meant to send this file then please\n\ package it up as a zip file and resend it." seen finish endif if $header_content-type: matches "(?:file)?name=(\\\\S+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|exe|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc]))([\\\\s;]|\\$)" then fail text "This message has been rejected because it has\n\ potentially executable content $1\n\ This form of attachment has been used by\n\ recent viruses or other malware.\n\ If you meant to send this file then please\n\ package it up as a zip file and resend it." seen finish endif if $message_body matches "(?:Content-(?:Type:(?>\\\\s*)[\\\\w-]+/[\\\\w-]+|Disposition:(?>\\\\s*)attachment);(?>\\\\s*)(?:file)?name=|begin(?>\\\\s+)[0-7]{3,4}(?>\\\\s+))(\"[^\">+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|exe|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc])\")[\\\\s;]" then fail text "This message has been rejected because it has\n\ a potentially executable attachment $1\n\ This form of attachment has been used by\n\ recent viruses or other malware.\n\ If you meant to send this file then please\n\ package it up as a zip file and resend it." seen finish endif if $message_body matches "(?:Content-(?:Type:(?>\\\\s*)[\\\\w-]+/[\\\\w-]+|Disposition:(?>\\\\s*)attachment);(?>\\\\s*)(?:file)?name=|begin(?>\\\\s+)[0-7]{3,4}(?>\\\\s+))(\\\\S+\\\\.(?:ad[ep]|ba[st]|chm|cmd|com|cpl|crt|eml|exe|hlp|hta|in[fs]|isp|jse?|lnk|md[be]|ms[cipt]|pcd|pif|reg|scr|sct|shs|url|vb[se]|ws[fhc]))[\\\\s;]" then fail text "This message has been rejected because it has\n\ a potentially executable attachment $1\n\ This form of attachment has been used by\n\ recent viruses or other malware.\n\ If you meant to send this file then please\n\ package it up as a zip file and resend it." seen finish endif if $h_from: contains "Hacker" or $h_from: contains "nonym" or $h_subject: contains "masturb" or $h_subject: contains "Viagra" or $h_subject: contains "Cialis" then #fail text "Unable to delivery" seen finish endif if "${if def:header_X-Spam-Subject: {there}}" is there then headers remove Subject headers add "Subject: $rh_X-Spam-Subject:" headers remove X-Spam-Subject endif
well, this is no an enough urgence to open a ticket. For that reason I write here in this thread. Just to discard the problem in my server. And I believe if there is a bug it can affect more people using custom word filters. Meanwhile I have disabled the WHM filter. I don't want to delete the custom_word file because it is effective in my server for the spam coming with these words, which is worse. Although it worked better with both joined because most times these domains and words appears in the same message, although not always. Sometimes spammers writes special characters to bypass word filters, then the .domain filter is useful. Other times it is the inverse case. thanks0 -
I don't mind being told to shut up :) And I did ask, Would a double wildcard work. ? star@star.pro (the forum must see the actual string as an sql injection or something)
Exim documentation seems to allow to delimitate better the parts (See "22. String testing conditions") :0 -
@Mise Just to confirm: - I added the following custom filter "custom_words" with the contents as follows:
if $h_from: contains "hacker" or $h_from: contains "anonym" or $h_subject: contains "Viagra" or $h_subject: contains "Cialis" then #fail text "Unable to delivery" seen finish endif
- Enabled it in the Exim Configuration Manager:
Your changes have been saved. Restarting cPanel daemons...done. Updating your system to reflect any changes... Creating new setting for "filter_custom_words" of "On". "filter_custom_words" was updated. Done. Your configuration changes have been saved! Waiting for "exim" to restart "waiting for "exim" to initialize "finished. Service Status exim (/usr/sbin/exim -ps -bd -q1h -oP /var/spool/exim/exim-daemon.pid) is running as mailnull with PID 16326 (systemd+/proc check method). exim (/usr/sbin/exim -qG) is running as root with PID 16328 (systemd+/proc check method). Startup Log Jan 17 15:07:39 server.cptechsupport.us systemd[1]: Starting Exim is a Mail Transport Agent, which is the program that moves mail from one machine to another.... Jan 17 15:07:39 server.cptechsupport.us systemd[1]: Started Exim is a Mail Transport Agent, which is the program that moves mail from one machine to another.. exim restarted successfully.
- Sent a test email from my test account I created the other day:
[root@server options]# exigrep 1isYsK-0004Rx-Gr /var/log/exim_mainlog 2020-01-17 15:09:56.573 [17121] cwd=/var/spool/exim 3 args: /usr/sbin/exim -Mc 1isYsK-0004Rx-Gr 2020-01-17 15:09:56.540 [17109] 1isYsK-0004Rx-Gr H=mail4.protonmail.ch [185.70.40.27]:20226 I=[MyIPAddress]:25 Warning: "SpamAssassin as lauren detected message as NOT spam (1.8)" 2020-01-17 15:09:56.556 [17109] 1isYsK-0004Rx-Gr H=mail4.protonmail.ch [185.70.40.27]:20226 I=[MyIPAddress]:25 Warning: Message has been scanned: no virus or other harmful content was found 2020-01-17 15:09:56.558 [17109] 1isYsK-0004Rx-Gr <= lauren.prolific@protonmail.com H=mail4.protonmail.ch [185.70.40.27]:20226 I=[MyIPAddress]:25 P=esmtps L- X=TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256 CV=no S=3924 M8S=0 RT=0.136s id=oM-u53fPDhV323SShIqElFpuYHTRaEcHq_7cytbNh7Uqxlpv5ikaCV0pO5jCFdcq9qoN59L5T6TMIpcUZa0RWptGOlPf-WblAoAOXnlbsOo=@protonmail.com T="test" from for lauren@mydomain.us 2020-01-17 15:09:56.666 [17121] 1isYsK-0004Rx-Gr => lauren F= P= R=virtual_user T=dovecot_virtual_delivery S=4142 C="250 2.0.0 Ob5YJqQiIl7mQgAA9Z/phw Saved" QT=4.144s DT=0.038s 2020-01-17 15:09:56.667 [17121] 1isYsK-0004Rx-Gr Completed QT=4.144s
- Note that the delivery was successful.
0 - I added the following custom filter "custom_words" with the contents as follows:
-
Just a test for CpanelLauren. Please ignore *@*.me 0 -
- Note that the delivery was successful.
Do you have also the domain *.pro inside the WHM Filter domain? I'm sorry but you didn't mention this. Just to be sure0 -
Do you have also the domain *.pro inside the WHM Filter domain? I'm sorry but you didn't mention this. Just to be sure
I have the *.pro included in the Blocked domains: Which was the entire point of testing this...0 -
then there is something strange in my Exim config that I cannot see. If I can find what happens I will post here. Thank you for your time in testing in another server! 0 -
Feel free to open a ticket as well @Mise and we can take a more in-depth look at your configuration. If you do open a ticket please update this thread with the Ticket ID and I'll update here with the outcome. 0 -
Does the filter support pattern anchors: $h_from: contains ".top$" or
In bash, this would only match .top when at the end of a string.0 -
Does the filter support pattern anchors:
$h_from: contains ".top$" or
In bash, this would only match .top when at the end of a string.
If he was using his filter it, this would work. Using the cPanel/WHM feature this would not work. *.pro works without issue as has been evidenced in this thread and domains aren't entered in the standard filter format like you would if you were creating a filter file.0 -
Hi Lauren finally I have found the problem When I add some change inside the custom filter: /usr/local/cpanel/etc/exim/sysfilter/options/custom_words this is not added automatically inside: /etc/cpanel_exim_system_filter so what happens is, I was not following the instructions in comments at the end of file /etc/cpanel_exim_system_filter: # BEGIN - Included from /usr/local/cpanel/etc/exim/sysfilter/options/custom_words # (Use the Basic Editor in the Exim Configuration Manager in WHM to change) # or manually edit /etc/exim.conf.localopts and run /scripts/buildeximconf
I believed the restart Exim process was doing that work automatically. Although this is not the case. And the portion of the custom filter with an old error was present from many days ago inside file /etc/cpanel_exim_system_filter Just by editing the error in /etc/cpanel_exim_system_filter, saving and restart Exim, and all is working, the messages are no discarded. The error was this: $h_from: contains ".pro" or when the chain ".pro" is present in the sender this is discarded. Therefore name.professional@domain.com is discarded. However, when including ".pro" in the WHM filter and I use the custom word filter only for $h_subject then both works:if $h_subject : contains "Hacker" or $h_subject : contains "nonym" #...(etc) then #fail text "Unable to delivery" seen finish endif
. I would suggest including also another comment inside /usr/local/cpanel/etc/exim/sysfilter/options/custom_words to warn about the process to follow. error solved :) thanks for all the help!0 -
Hi @Mise I'm really glad you found the source of the issue! This is not necessary due to the way we explain how to edit this in the custom filter documentation How to Customize the Exim System Filter File | cPanel & WHM Documentation - the purpose for suggesting that you enable/disable the filter through the exim configuration editor is because it rebuilds the exim configuration. If you're going to edit it manually, once you make your edits (and after you're sure the filter is enabled) you need to run the following to rebuild the exim conf: /scripts/buildeximconf
then restart exim. It, for some reason, didn't occur to me that you might be editing this file manually then not making the modification in the exim configuration manager. Either way, thanks for reporting back and letting us know what happened. I'd love to hear if you run into any issues with the domain blocking feature if you're moving forward with using it.0
Please sign in to leave a comment.
Comments
30 comments