[COBRA-13435] AutoSSL not renewing domain certificate when some hosts fail DCV
I have created a private cPanel ticket for this issue but wanted to post it here so other customers with similar issues could track its progress in case they experience something similar:
Verifying "cPanel (powered by Sectigo)""s authorization on 12 domains via DNS CAA records "
11:18:59 PM "customer.example.com" is managed.
"www.customer.example.com" is managed.
"mail.customer.example.com" is managed.
"cpanel.customer.example.com" is managed.
"webdisk.customer.example.com" is managed.
"webmail.customer.example.com" is managed.
"cpcontacts.customer.example.com" is managed.
"mail.customer.com" is managed.
"www.customer.com" is managed.
"customer.com" is managed.
"cpcalendars.customer.example.com" is managed.
"autodiscover.customer.example.com" is managed.
All of this user"s 12 domains are managed.
CA authorized: "customer.example.com"
CA authorized: "www.customer.example.com"
CA authorized: "mail.customer.example.com"
CA authorized: "cpanel.customer.example.com"
CA authorized: "webdisk.customer.example.com"
CA authorized: "webmail.customer.example.com"
CA authorized: "cpcontacts.customer.example.com"
CA authorized: "cpcalendars.customer.example.com"
CA authorized: "autodiscover.customer.example.com"
CA authorized: "customer.com"
CA authorized: "www.customer.com"
CA authorized: "mail.customer.com"
"cPanel (powered by Sectigo)" is authorized to issue certificates for 12 of this user"s 12 domains.
11:18:59 PM Performing HTTP DCV (Domain Control Validation) on 12 domains "
11:18:59 PM Local HTTP DCV OK: customer.com
Local HTTP DCV OK: www.customer.com
WARN Local HTTP DCV error (mail.customer.com): The system queried for a temporary file at "http://mail.customer.com/.well-known/pki-validation/60AB477AA88A4AD2AF59E3F3474255E6.txt", but the web server responded with the following error: 404 (Not Found). A DNS (Domain Name System) or web server misconfiguration may exist. The domain "mail.customer.com" resolved to an IP address "207.204.50.27" that does not exist on this server.
Local HTTP DCV OK: customer.example.com
Local HTTP DCV OK: www.customer.example.com
Local HTTP DCV OK: mail.customer.example.com
Local HTTP DCV OK: cpanel.customer.example.com
Local HTTP DCV OK: webdisk.customer.example.com
Local HTTP DCV OK: webmail.customer.example.com
Local HTTP DCV OK: cpcontacts.customer.example.com
Local HTTP DCV OK: cpcalendars.customer.example.com
Local HTTP DCV OK: autodiscover.customer.example.com
11:18:59 PM Verifying local authority for 1 domain "
11:18:59 PM No local authority: "mail.customer.com"
It seems that AutoSSL/Sectigo couldn't find the "mail.customer.com" host, probably because the user didn't set it up at the registrar or wherever their DNS points to.
The email notification for this failure sent on 12/30/2021 had this message: "AutoSSL would normally renew this certificate now, but 1 of the website"s secured domains just failed DCV. To provide you with more time to resolve this problem, AutoSSL will defer the renewal until Jan 1, 2022 at 12:00:00 AM UTC. After that time, AutoSSL will request a replacement certificate that excludes any domains that fail DCV. At the time of this notice, the certificate will expire in 3 days, 19 hours, 40 minutes, and 19 seconds." However, it did not renew the other domains on the certificate, since going to "customer.com" gave an expired SSL message. I had this happen to another customer about a month ago but it had never happened before over several years of using AutoSSL and customer domains that only have an A record that points to our server. I had to manually run another check for that domain and it corrected itself, but it should do it before the current cert expires so there is no downtime for the customer's site. It seems something has changed with either AutoSSL or with Sectigo recently. Based on the email message, it would seem that AutoSSL expects to renew the valid parts of the cert regardless of the failure, but ultimately does not do so before the cert expires. As I did last time, I ran an AutoSSL check manually on just the "customer" user account and it did issue the cert without the missing "mail.customer.com" domain, as it should. The problem is that it didn't do this automatically prior to the old certificate's expiration, leaving the customer without access to their site until my manual intervention. I'm assuming this manual intervention should not be necessary, particularly provided the email message implying that AutoSSL would provide "a replacement certificate that excludes any domains that fail DCV." cPanel ticket:
The email notification for this failure sent on 12/30/2021 had this message: "AutoSSL would normally renew this certificate now, but 1 of the website"s secured domains just failed DCV. To provide you with more time to resolve this problem, AutoSSL will defer the renewal until Jan 1, 2022 at 12:00:00 AM UTC. After that time, AutoSSL will request a replacement certificate that excludes any domains that fail DCV. At the time of this notice, the certificate will expire in 3 days, 19 hours, 40 minutes, and 19 seconds." However, it did not renew the other domains on the certificate, since going to "customer.com" gave an expired SSL message. I had this happen to another customer about a month ago but it had never happened before over several years of using AutoSSL and customer domains that only have an A record that points to our server. I had to manually run another check for that domain and it corrected itself, but it should do it before the current cert expires so there is no downtime for the customer's site. It seems something has changed with either AutoSSL or with Sectigo recently. Based on the email message, it would seem that AutoSSL expects to renew the valid parts of the cert regardless of the failure, but ultimately does not do so before the cert expires. As I did last time, I ran an AutoSSL check manually on just the "customer" user account and it did issue the cert without the missing "mail.customer.com" domain, as it should. The problem is that it didn't do this automatically prior to the old certificate's expiration, leaving the customer without access to their site until my manual intervention. I'm assuming this manual intervention should not be necessary, particularly provided the email message implying that AutoSSL would provide "a replacement certificate that excludes any domains that fail DCV." cPanel ticket:
-
After cPanel investigation, it appears that Sectigo was responding in the logs that "The "cPanel (powered by Sectigo)" provider cannot currently accept incoming requests. The system will try again later." But unfortunately, it wasn't "trying again later" soon enough to get a new cert in place prior to the old one expiring. According to this page: and will actually try again prior to expiration. This failed to occur twice in the past 5 weeks for my customers and apparently for other users here as well. The strange part is that this only started recently. I have used AutoSSL for a few years with Sectigo and never had issues of certs expiring due to these types of errors. In both cases I experienced, after the customer notified me that their site wasn't working, I was able to run the "AutoSSL Check" on just their account and it did finally issue the cert. But this occurred only with manual intervention after the cert had expired and the customer noticed their site was not accessible via https. 0 -
We are also getting this issue.. When we run a check on the domains waiting we get a message like the one below on all of them. We currently have 4 accounts that have expired ssl certificates that are stuck in the queue. The provider "cPanel (powered by Sectigo)""s AutoSSL queue already contains a certificate request for "accountname""s website "domain.com". The request"s start time is Jan 4, 2022, 12:35:06 AM UTC. Two of them use cloudflare but the other two do not. 0 -
Update to this but cloudflare appears to be unrelated. We currently have 45 domains and sub domains with expired certificates all waiting in Sectigo's queue. 0 -
Update to this but cloudflare appears to be unrelated. We currently have 45 domains and sub domains with expired certificates all waiting in Sectigo's queue.
You see those in WHM -> SSL/TLS -> Manage AutoSSL -> Pending Queue ? We had that kind of problem in one server, and after waiting about 1-2 hours I switched to Let's Encrypt, removed the expired certificates and Let's Encrypt installed them in a few seconds. :rolleyes:0 -
You see those in WHM -> SSL/TLS -> Manage AutoSSL -> Pending Queue ? We had that kind of problem in one server, and after waiting about 1-2 hours I switched to Let's Encrypt, removed the expired certificates and Let's Encrypt installed them in a few seconds. :rolleyes:
Yes.. This is now up to 86 :(0 -
Hello! I will inquire to see if I can get more information about these ongoing AutoSSL issues. 0 -
As a temporary fix I changed from Sectigo to Lets Encrypt, which works for the moment but I suspect it might not for long. It at least gives us time to migrate the sites to a newer server. 0 -
Thank you so much for the updates and your patience. 0 -
cPanel definitely changed something in a recent update. A lot of clients has been contacting us the last few weeks because they received emails about some domains failing DCV. These domains have always failed DCV but the certificate was still issued. This is no longer the case. Even though AutoSSL says the certificate will be renewed (while excluding the domains that failed DCV), renewal doesn't work. This is a huge issue since we have a lot of clients who's using an external DNS provider. They haven't created the cpcalendar, webmail, cpcontacts and autodiscover subdomains. cPanel can no longer renew these certificates - we have to exclude the subdomains from AutoSSL before the certificate can be renewed. The past month, I've basically just been excluding domains from AutoSSL and telling our clients about the issue all day long. What a headache! 0 -
cPanel definitely changed something in a recent update. A lot of clients has been contacting us the last few weeks because they received emails about some domains failing DCV. These domains have always failed DCV but the certificate was still issued. This is no longer the case. Even though AutoSSL says the certificate will be renewed (while excluding the domains that failed DCV), renewal doesn't work. This is a huge issue since we have a lot of clients who's using an external DNS provider. They haven't created the cpcalendar, webmail, cpcontacts and autodiscover subdomains. cPanel can no longer renew these certificates - we have to exclude the subdomains from AutoSSL before the certificate can be renewed. The past month, I've basically just been excluding domains from AutoSSL and telling our clients about the issue all day long. What a headache!
Hello! Which specific DCV errors were you seeing in the logs?0 -
Hello! Which specific DCV errors were you seeing in the logs?
Either that the DNS record doesn't exist or that it's pointing to an external IP. We have clients that have had made no changes to DNS for years, and renewal of SSL certificates has always worked. Just not any more.0 -
Either that the DNS record doesn't exist or that it's pointing to an external IP. We have clients that have had made no changes to DNS for years, and renewal of SSL certificates has always worked. Just not any more.
Thank you for the clarification. As there have been multiple AutoSSL issues, would it be possible to open a ticket with the link in my signature so we can investigate?0 -
This has become a serious issue for us now. Happens every day. It's not just domains that have their DNS hosted elsewhere - it's all of them. Every day a client will contact us to say their certificate didn't get renewed automatically. We have found that running the following does seem to provide some improvement but not always... /usr/local/cpanel/bin/autossl_check --all Often we will see the error mentioned below... "The "cPanel (powered by Sectigo)" provider cannot currently accept incoming requests. The system will try again later." I think there are various other threads on the go about these issues. 0 -
Not a solution, but it may help in the meantime to change the server's AutoSSL cron file (/etc/cron.d/cpanel_autossl) to run the AutoSSL check more frequently. cPanel support suggested this, and although I still see a number of "cannot accept incoming requests" messages in the daily logs, I'm not aware of any cases of SSL certs expiring before renewal on our server since modifying AutoSSL to run more frequently. I set it to run every 6 hours. I guess this quadruples the chances that the AutoSSL checking process will issue the request during a time when Sectigo's server actually can accept requests. The other workaround is to ensure AutoSSL is configured to NOT replace existing valid certs and switch to Let's Encrypt as your SSL provider, which seems to be more reliable but has the downside of rate limits. If those are an issue for you, you can appeal to LE to increase your specific rate limit. I did request an increase last week but have yet to hear back from them. The best solution, of course, would be if Sectigo's service could simply be corrected to work as reliably as it did in the past. They have a number of recent reports of certificate issuance delays/issues on their site (Sectigo) and each one is marked as resolved after clearing the backlog. But the same issue appears several times in the past few months, so it seems that it is not truly resolved. 0 -
@swbrains - I wish I had more positive things to say, but it's definitely not resolved. 0 -
I was able to resolve the problem for now by moving to LetsEncrypt as our AutoSSL provider. Instructions: Step 1 : Install Lets Encrypt cpanel plugin Run the following command in ssh : /usr/local/cpanel/scripts/install_lets_encrypt_autossl_provider Step 2: In WHM -> Manage AutoSSL -> Providers tab -> Select Lets Encrypt and click save. Step 3: Run AutoSSL For All Users (Alternatively you may just do it for the domain where ssl did not renew) 0 -
I was able to resolve the problem for now by moving to LetsEncrypt as our AutoSSL provider. Instructions:
A tip for those that do this: Ensure that the AutoSSL option to replace existing valid certs is OFF before running a check-all operation under Lets Encrypt, particularly if you have a lot of accounts. I did this and quickly ran into the Lets Encrypt rate limit. Had to wait a week to try again after turning off the replacement setting. :(0 -
That's a good tip, @swbrains, as the Let's Encrypt limits are much lower than Sectigo and can cause issues for some users. 0 -
I was able to resolve the problem for now by moving to LetsEncrypt as our AutoSSL provider. Instructions: Step 1 : Install Lets Encrypt cpanel plugin Run the following command in ssh : /usr/local/cpanel/scripts/install_lets_encrypt_autossl_provider Step 2: In WHM -> Manage AutoSSL -> Providers tab -> Select Lets Encrypt and click save. Step 3: Run AutoSSL For All Users (Alternatively you may just do it for the domain where ssl did not renew)
Thank you for this! I have no idea why suddenly my AutoSSL stopped working, but this fix worked. I have only one question related to the authority of LetsEncrypt vs cPanel. Do you think that using a LetsEncrypt certificate for my subdomains vs. a cPanel certificate would impart anything negative to google or other search engines? I know this may seem like a silly question, but I've heard that the quality of your SSL certificate issuer is important, but then again, this could all be just marketing BS.0 -
The SSLs issued by both providers are the same type of domain-validated certificate, so I don't see why there would be a difference. cPanel certificates are provided by Sectigo. 0 -
Have the same problem here. I have received a complaint and It forces to enter inside WHM to renew it manually and restart Apache & php-fm. Yesterday 22 the certificate of the domain (here domain1.com) was expired. And this the log from today 23 before my manual renew: 02:08:03 Analyzing "domain1""s domains " 02:08:03 Analyzing "domain1.com" (website) " 02:08:03 ERROR TLS Status: Defective ERROR Certificate expiry: 22/6/22 0:00 UTC (1,01 days ago) ERROR Defect: OPENSSL_VERIFY: The certificate chain failed OpenSSL"s verification (0:10:CERT_HAS_EXPIRED). 02:08:03 Attempting to ensure the existence of necessary CAA records " 02:08:03 No CAA records were created. 02:08:03 Verifying 8 domains" management status " Verifying "cPanel (powered by Sectigo)""s authorization on 8 domains via DNS CAA records " 02:08:03 "cpanel.domain1.com" is managed. "mail.domain1.com" is managed. "cpcontacts.domain1.com" is managed. "webmail.domain1.com" is managed. "cpcalendars.domain1.com" is managed. CA authorized: "domain1.com" CA authorized: "www.domain1.com" CA authorized: "mail.domain1.com" CA authorized: "cpanel.domain1.com" CA authorized: "webdisk.domain1.com" CA authorized: "webmail.domain1.com" CA authorized: "cpcontacts.domain1.com" CA authorized: "cpcalendars.domain1.com" "cPanel (powered by Sectigo)" is authorized to issue certificates for 8 of this user"s 8 domains. "www.domain1.com" is managed. "webdisk.domain1.com" is managed. "domain1.com" is managed. All of this user"s 8 domains are managed. 02:08:03 Performing HTTP DCV (Domain Control Validation) on 8 domains " 02:08:03 Local HTTP DCV OK: domain1.com Local HTTP DCV OK: www.domain1.com Local HTTP DCV OK: mail.domain1.com Local HTTP DCV OK: cpanel.domain1.com Local HTTP DCV OK: webdisk.domain1.com Local HTTP DCV OK: webmail.domain1.com Local HTTP DCV OK: cpcontacts.domain1.com Local HTTP DCV OK: cpcalendars.domain1.com 02:08:03 No local DNS DCV is necessary.
The tabs config: Providers: AutoSSL providers: (x) cPanel (powered by Sectigo) Options: (x) Allow AutoSSL to replace invalid or expiring non-AutoSSL certificates. Manage users: (x) Reset function list configuration ------- it seems the certificates are not renewed at the scheduled day of expiration0 -
The original COBRA-13435 case was resolved in all versions of 102, so I wouldn't expect this to be happening on any supported version of cPanel at this point. I also wouldn't expect PHP-FPM to be involved at all in the AutoSSL process. The only error I'm seeing from your log is that the SSL has expired, but I don't see anywhere showing the actual problem with the renewal not working. Are there any additional logs that show why AutoSSL failed? 0 -
The solution is that you need to add a CAA text record to your DNS zone for that domain. The record should look like this, and it allows the SSL issuer to modify your DNS zone: domain1.com. 14400 CAA Flags: 0 Tag: issue Value: sectigo.com 0 -
The only error I'm seeing from your log is that the SSL has expired, but I don't see anywhere showing the actual problem with the renewal not working. Are there any additional logs that show why AutoSSL failed?
no errors inside /usr/local/cpanel/logs . Just it seems day 22 was finished and the certificate was not renewed. Are there more logs to look? I will wait for the next domain to be renewed to see what happens.The solution is that you need to add a CAA text record to your DNS zone for that domain. The record should look like this, and it allows the SSL issuer to modify your DNS zone:
thank you. I save your message in case this is repeated in the future.0
Please sign in to leave a comment.
Comments
25 comments