[CPANEL-31754] SPF false positive, fails on valid senders after upgrade
After upgrading to CentOS7 with cPanel 86.0.04 some incoming email bounce due to SPF check.
Most of those emails have a valid sender IP address that matches domain SPF record but for some reason Exim's SPF fail to match but if I run a test on numerous validators (online or command line) that same sender / spf record are correct.
From what I've noticed here's two cases where Exim's SPF fails:
- SPF check does not follow the 'a' tag to the respective IP address but if the record contains ip4:xxx (with the same IP as the A record) it passes;
- The 'exists:' tag is not processed correctly PTR records with wildcard records
Here's some proof / test so someone can take a look at it.
Example rejection:
H=smtp1.example.pt (anubis04.example.pt) []:11970 X=TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256 CV=no F=<******@example.pt> rejected RCPT <******@******>: SPF: is not allowed to send mail from example.pt
SPF record for example.pt:
v=spf1 a:smtp1.example.pt a:smtp2.example.pt a:smtp3.example.pt a:smtp4.example.pt a:smtp5.example.pt a:newsletter.example.pt include:spf.protection.outlook.com -all
Sender IP address matches the SPF record:
smtp1.example.pt. 21600 IN A
How can I tweak / fix this? Is this check some external call?
Thanks,
PM
-
Can you show the entire exim transaction? You should be able to see the MessageID in the logs and then run something like the following with it: exigrep /var/log/exim_mainlog
Another thing I noticed is that while the domain has a valid A record the domain doesn't actually have a valid txt record in place:dig txt smtp1.example.pt +short
Has no return0 -
Can you show the entire exim transaction? You should be able to see the MessageID in the logs and then run something like the following with it:
Hi, thanks for reply, domain names were edited by forum review.0 -
We are seeing the same here, email is being rejected due to false SPF positives. It's only happening on the two servers that have been upgraded to 86.0.4. The other servers that are still on 84.0.21 are working correctly. For example: 2020-02-25 10:39:46 SMTP connection from []:30200 (TCP/IP connection count = 39) 2020-02-25 10:40:27 H=smtp1.domain.cat [ rejected RCPT <******@******.com>: SPF: is not allowed to send mail from domain.cat 2020-02-25 10:40:32 SMTP connection from smtp1.domain.cat []:30200 closed by QUIT The IP address corresponds to on of the MX records which should be allowed by the SPF record: domain.cat. 1562 IN TXT "v=spf1 mx ?a:postari.txtdomain.cat ?ip4: -all" 0 -
Hi, thanks for reply, domain names were edited by forum review.
I removed the domain names as we remove all personally identifying information from posts, this is discussed here: . Regardless though I can see your SPF record now:"v=spf1 a:smtp1.example.pt a:smtp2.example.pt a:smtp3.cp.example a:smtp4.example.pt a:smtp5.cp.pt a:newsletter.example.pt include:spf.protection.outlook.com -all"
You're matching all domains and IP's but the originating domain, example.pt, does this persist if you add a:example.pt to the record? I noticed both of you are hard failing addresses that don't match with "-all" while testing issues with the SPF you may want to soft fail instead "~all"- - Fail, an IP that matches a mechanism with this qualifier will fail SPF.
- ~ SoftFail, an IP that matches a mechanism with this qualifier will soft fail SPF, which means that the host should accept the mail, but mark it as an SPF failure.
0 -
Hi Lauren, I don't understand your reply, sorry. The IP address of the sender host matches the mx mechanism of the domain's SPF record: "This mechanism matches if is one of the MX hosts for a domain name." as is the case here. Therefore it is a match and the check should not fail. There are many other false positives, this was just the first report I got yesterday. 0 -
What I'm trying to reconcile here is that there have been no significant changes to SPF validations which would cause this as far as I am aware. The update to v86 had no changes to exim or SPF per the release notes and changelogs here:
As far as the A record goes, I can't rule out an issue with the configuration with one of those IP's timing out returning records for the domain if you could humor me for one second and ensure that DNS resolution for the domain is working properly on all IP's as well as add a or +a I'd like to see if that moves you in the right direction. @Juan R. Pozo mx in that record is the only thing being looked at because of the "?" qualifier you're using for a and ip4 - if you change mx to +mx does this resolve anything? (I realize that it should be fine with mx as mx should be equivalent to +mx but I'd still like to know.0 -
I'm opening a case for the exim bug report you found and thank you for that, it seems to be related to having multiple (> 1) MX references and DNS RR's not being returned for one of them [QUOTE]When a TXT SPF record contains two MX references, and the second reference returns no DNS RRs, Exim erases the results retrieved from the first lookup, treating it as also null. This causes the SPF ACL to return "fail" even though the first MX clause should short-circuit the result as "pass".
it seems that based on that the ip4: reference should suffice, which leads me to wonder, I'm curious, why you are using the "?" qualifier in that record and not the "+"? I'm unaware of a reason for that in this context and genuinely curious. You could not include the ip4: and -all since it's a neutral result could you not? Keep in mind I fully understand that the initital mx match should produce a success not a fail. I'm primarily just curious and wondering if it could be a potential workaround until this is resolved.0 -
I've opened CPANEL-31754 for this issue thanks to your information. I chose to go with an internal case rather than an upstream case due to the fact that exim does have a patch for this but it's not available in CentOS upstream I fully understand how this is replicable now. I did clarify I understand that the initial match on mx should be a success - I was just thinking as far as my own curiosity and potentially a workaround until it's resolved. 0 -
Thanks @cPanelLauren, in the meanwhile we'll be invoking spfquery from exim.conf instead of using the built-in spf facility. 0 -
@Pedro Marques Looking at the domain's DNS records I found that it resolves to two separate IP addresses - one of which does not return any DNS records for the domain. Which is why I was unable to see the SPF yesterday.
[root@server templates]# dig a +short example.pt 80.251.XXX.XXX 81.90.XX.XXX [root@server templates]# dig txt +short @80.251.XXX.XXX example.pt "v=spf1 a:smtp1.example.pt a:smtp2.example.pt a:smtp3.example.pt a:smtp4.example.pt a:smtp5.example.pt a:newsletter.example.pt include:spf.protection.outlook.com -all" "OJsOzrrhyuyFNpPNraj9vrlhH9EcI2rVOJjsFERpednPBReUj4JcoTLjSMD0wzbEDb5IJwekTBV6zXVHReWubA==" [root@server templates]# dig txt +short @81.90.XX.XXX example.pt ;; connection timed out; no servers could be reached
As far as the A record goes, I can't rule out an issue with the configuration with one of those IP's timing out returning records for the domain if you could humor me for one second and ensure that DNS resolution for the domain is working properly on all IP's as well as add a or +a I'd like to see if that moves you in the right direction.
Sender from domain @example.pt. SPF record obtained from example.pt which contains the tag 'a:smtp1.example.pt' as allowed host to send from this domain. When the sender reach Exim, the IP address of smtp1.example.com is rejected because SPF did not match the IP address of ANY 'a' tags in SPF record for that domain. If the SPF record states the same host IP as 'ip4:XXX.XXX.XXX.XXX' it passes. Proof is in the original post that was shadowed but I can easily replicate with other examples if you want and send you with real data. Had to disable SPF check because it was denying a lot of good, valid messages.0 -
Sender from domain @example.pt. SPF record obtained from example.pt which contains the tag 'a:smtp1.example.pt' as allowed host to send from this domain. When the sender reach Exim, the IP address of smtp1.example.com is rejected because SPF did not match the IP address of ANY 'a' tags in SPF record for that domain. If the SPF record states the same host IP as 'ip4:XXX.XXX.XXX.XXX' it passes. Proof is in the original post that was shadowed but I can easily replicate with other examples if you want and send you with real data. Had to disable SPF check because it was denying a lot of good, valid messages.
Thanks! I agree, the second notice of this was a bit easier to prove in this instance, any time I'm looking at a potential defect I need the most reliable and straightforward way to replicate it, in the instance of yours there were a few variables that made it less easy to be replicated. None the less, as you can see I was able to get a case opened on the information provided, furthermore it's a known issue with exim as is noted in previous responses. The case I opened was to apply the patch that exim has put out for this. I'll update here when there is more information.Thanks @cPanelLauren, in the meanwhile we'll be invoking spfquery from exim.conf instead of using the built-in spf facility.
If you are able to provide the workaround you employ here, I'm sure it would be much appreciated.0 -
Hello @Juan R. Pozo and @Pedro Marques Can you both let me know if you have any custom ACL's in the exim configuration? You'd find these at /etc/exim.conf.local. I ask this specifically because though the functionality to do this exists in exim, and we will still be adding the patch based on the case I opened, cPanel's version of Exim does not by default do any SPF denials, instead that's handled by SpamAssassin. 0 -
We have the same issue. We have some custom ACL's because of spamexperts. But for SPF check we have this in "Exim Config Manager" -> Advanced Editor -> custom_begin_mailauth: deny message = SPF: $sender_host_address is not allowed to send mail from $sender_address_domain !condition = ${if match_ip{$sender_host_address}{iplsearch;/etc/trustedmailhosts}{1}{0}} spf = fail
Any idea when you will push a fix for this? What is the easiest workaround for this?0 -
Hello @Juan R. Pozo and @Pedro Marques Can you both let me know if you have any custom ACL's in the exim configuration? You'd find these at /etc/exim.conf.local.
Hi, the only custom config I had was the SPF in custom_begin_mailauth, but I had to remove it as it was blocking too many messages from hosts with valid SPF records.# BEGIN INSERT spf_bl deny message = SPF: $sender_host_address is not allowed to send mail from $sender_address_domain spf = fail # END INSERT spf_bl
Thanks.0 -
That's what I'd figured but had wanted to confirm. For all instances of this, we've seen so far, it's due to a custom ACL being present that validates SPF records + exim with the version that fails to properly perform SPF lookups as a result of the Exim case. I have the patch included in the internal case I opened, exim should be able to facilitate these DNS lookups successfully whether or not we provide a mechanism for it. I believe a similar issue is why we originally removed the original setting to reject SPF failures from the exim configuration (a long time ago). We rely on SpamAssassin for that functionality now and it does provide quite a bit of configurability though won't perform a rejection at SMTP time for SPF. 0 -
Hi, this is our workaround, in case it may help someone: domainlist skip_spf_check_domains = lsearch;/etc/skipspfcheckdomains [...] # BEGIN INSERT custom_begin_mailauth warn set acl_m_spfquery = ${run{/usr/lib/exim/bin/spfquery -sender $sender_address -ip $sender_host_address -helo $sender_helo_name}{$value}} set acl_m_spfheader = ${if match{$value}{Received-SPF: (.+)}{$1}} # If SPF result is fail then $runrc is 3 deny condition = ${if == {$runrc}{3}} !sender_domains = +skip_spf_check_domains message = SPF: $acl_m_spfheader warn condition = ${if == {$runrc}{3}} sender_domains = +skip_spf_check_domains log_message = Ignoring SPF result: $acl_m_spfheader warn !condition = ${if == {$runrc}{3}} condition = ${if def:acl_m_spfheader} add_header = :at_start:Received-SPF: $acl_m_spfheader # END INSERT custom_begin_mailauth0 -
Do you have any eta when this will be released? We cannot upgrade any more servers to version 86 before this has been released. 0 -
Do you have any eta when this will be released? Needed to disable SPF rules due to filter not respecting common SPF policies like "exists" and rejecting lots of valid and legit emails... Please, update your exim software so this situation is fixed. 0 -
I don't have an ETA I do know this is fixed just not implemented in a v86 build yet, I'd wager that the next release of v86 will include the patch but will update here to confirm when it is resolved. 0 -
Any update soon? We really need this fixed so we can upgrade to v.86! 0 -
It's still waiting on publish to v86 as I mentioned before I'll update here when it is available. 0 -
Hi... more than 1 month with a bugged EXIM version not running ok with simple SPF policies, while Exim developers published a fix months ago.... and no urgency at all. 0 -
Hi... more than 1 month with a bugged EXIM version not running ok with simple SPF policies, while Exim developers published a fix months ago.... and no urgency at all.
I'm going to be pretty straight forward on this. 1. We removed this functionality in our version of Exim some time ago because of issues with the way it performed DNS Lookups. 2. We offset the responsibility for this behavior to SpamAssassin, where you have far more control, it's reliable and can weigh the scores or auto-delete. 3. You're using a customization that bypasses the methods which we have decided work best. With that being clear I do hope you understand that the configuration you've chosen is not something that we provide or support which can sometimes take a back seat to issues with supported items and while this IS fixed in the current EDGE build of cPanel & WHM and you're welcome to update to it as of right now it has not been ported into a RELEASE version of the product and I don't have an ETA of when it will be. As soon as it is I will update this thread.0 -
2. We offset the responsibility for this behavior to SpamAssassin, where you have far more control, it's reliable and can weigh the scores or auto-delete.
Not all customers use SpamAssasin. Maybee you could add a switch in WHM where we can add this so it's handled by SpamAssain? A on/off button for all accounts that are using spamassasin, all emails that do not send from servers that are included in the domains SPF will get rejected/deleted. We really would like to upgrade to v.86 before v.88 is released.0 -
@Hedloff My suggestions were there as a workaround to retain the same functionality until this is resolved. Furthermore, as an administrator you can force SpamAssassin on for all accounts, this is a function that is performed more reliably through SpamAssassin, there's no way to use this without it enabled so I don't think your suggestion would be plausible. I'm asking again today when we think this will be sent to v86 as it's certainly been some time. I'll update here when I have more information. 0 -
Hello, I just wanted to update, for those of you watching this thread that right now the plan for v88 of cPanel to go to CURRENT is estimated around May 4th (this is absolutely not a date that is set in stone) - but I wanted to be as transparent as possible about the timeframe. At that time, the resolution for this will also be patched to v86 of cPanel & WHM As soon as I have any more information on updates to this issue I will update here as well. Thanks! 0 -
v88 of cPanel to go to CURRENT is estimated around May 4th
Does today happen this Tier?0
Please sign in to leave a comment.
Comments
37 comments