SSL, Vhost creation and cPanel userdata

alexis_

• October 20, 2017 07:40

Hello, I'm wondering, have you change something related to the VHOST generation for SSL domains and the way that data are stored in /var/cpanel/userdata ? Let me explain, we used to parse /var/cpanel/userdata for our custom script for vhost generation for our Nginx web server, in front of the apache one. We knew it was not the best practice and I was aware that one day it will cause us some trouble. In the "old format" (I assumed something changed), we used to have data like this :
# from /var/cpanel/userdata/XXXXXX/domain_SSL --- documentroot: /homeX/USERNAME/DOCROOT group: USERNAME hascgi: 1 homedir: /homeX/USERNAME ip: xxx.xxx.xxx.xxx ipv6: ~ owner: root phpopenbasedirprotect: 1 port: 4430 secruleengineoff: ~ serveradmin: webmaster@xxxxx.com serveralias: www.xxxxxxx.com servername: xxxxxxxxx ssl: 1 sslcacertificatefile: /var/cpanel/ssl/installed/cabundles/Let_s_Encrypt_d5a69d0f2effae8513e08eaced2ccf28_1615999246.cabundle sslcertificatefile: /var/cpanel/ssl/installed/certs/xxxxxxxx.crt sslcertificatekeyfile: /var/cpanel/ssl/installed/keys/xxxxxxx.key usecanonicalname: 'Off' user: USERNAME userdirprotect: ''
Now, a few lines are missing for the new domains with SSL (post last-update I think), it's the "ssl" stuff (ca, key, crt). That force use to re-implement our Vhost generation mechanism with the WHM API (which is not bad, on the contrary). The main problem now is two apaches servers refused to start because of a SSL related error (this never happened before) with errors saying that the key or cabundle was missing. The other problem is that the /script/rebuildhttpconf does not work either :
[root@server certs]# /scripts/rebuildhttpdconf info [rebuildhttpdconf] Skipping SSL VirtualHost for domain DOMAIN.fr, missing certificate file /var/cpanel/ssl/installed/certs/DOMAIN_fr_a864e_35355_1513509900_4ccb5ec314309fa5422c19eec4907b58.crt info [rebuildhttpdconf] Skipping SSL VirtualHost for domain DOMAIN.fr, missing certificate file /var/cpanel/ssl/installed/certs/DOMAIN_fr_bbdd3_002a5_1512676500_33ff5df8b95e57e035fd5a97aaeec6db.crt Initial configuration generation failed with the following message: The "/usr/sbin/httpd" command (process 673380) reported error number 1 when it ended. Configuration problem detected on line 37408 of file /etc/apache2/conf/httpd.conf.work.E8wfXM9c: SSLCACertificateFile: file '/var/cpanel/ssl/installed/cabundles/Let_s_Encrypt_d5a69d0f2effae8513e08eaced2ccf28_1615999246.cabundle' does not exist or is empty --- /etc/apache2/conf/httpd.conf.work.E8wfXM9c --- 37402 37403 SSLEngine on 37404 37405 SSLCertificateFile /var/cpanel/ssl/installed/certs/DOMAIN_fr_bb4d3_31e7d_1515830510_6c68aa940d2219d0132ee3f8ca8fd81c.crt 37406 37407 SSLCertificateKeyFile /var/cpanel/ssl/installed/keys/bb4d3_31e7d_091989ead1f6d2aa8e759d262d98177e.key 37408 ===> SSLCACertificateFile /var/cpanel/ssl/installed/cabundles/Let_s_Encrypt_d5a69d0f2effae8513e08eaced2ccf28_1615999246.cabundle <=== 37409 SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown 37410 37411 SSLOptions +StdEnvVars 37412 37413 37414 --- /etc/apache2/conf/httpd.conf.work.E8wfXM9c --- Rebuilding configuration without any local modifications. info [rebuildhttpdconf] Skipping SSL VirtualHost for domain DOMAIN.fr, missing certificate file /var/cpanel/ssl/installed/certs/DOMAIN_fr_a864e_35355_1513509900_4ccb5ec314309fa5422c19eec4907b58.crt info [rebuildhttpdconf] Skipping SSL VirtualHost for domain DOMAIN.fr, missing certificate file /var/cpanel/ssl/installed/certs/DOMAIN_fr_bbdd3_002a5_1512676500_33ff5df8b95e57e035fd5a97aaeec6db.crt Failed to generate a syntactically correct Apache configuration. Bad configuration file located at /etc/apache2/conf/httpd.conf.work.E8wfXM9c Error: The "/usr/sbin/httpd" command (process 673388) reported error number 1 when it ended. Configuration problem detected on line 37408 of file /etc/apache2/conf/httpd.conf.work.E8wfXM9c: SSLCACertificateFile: file '/var/cpanel/ssl/installed/cabundles/Let_s_Encrypt_d5a69d0f2effae8513e08eaced2ccf28_1615999246.cabundle' does not exist or is empty --- /etc/apache2/conf/httpd.conf.work.E8wfXM9c --- 37402 37403 SSLEngine on 37404 37405 SSLCertificateFile /var/cpanel/ssl/installed/certs/DOMAIN_fr_bb4d3_31e7d_1515830510_6c68aa940d2219d0132ee3f8ca8fd81c.crt 37406 37407 SSLCertificateKeyFile /var/cpanel/ssl/installed/keys/bb4d3_31e7d_091989ead1f6d2aa8e759d262d98177e.key 37408 ===> SSLCACertificateFile /var/cpanel/ssl/installed/cabundles/Let_s_Encrypt_d5a69d0f2effae8513e08eaced2ccf28_1615999246.cabundle <=== 37409 SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown 37410 37411 SSLOptions +StdEnvVars 37412 37413 37414 --- /etc/apache2/conf/httpd.conf.work.E8wfXM9c ---
I'm worried because this kind of stuff never happened before, when the httpd.conf file was in error, the rebuildscript always managed to repair it. In this case, we managed this "by hand", by adding/removing some file related to the SSL certificate (in this case, we took the Let's Encrypt CA from another server). I checked the forum, one other user seems to have similar problem : In Progress - SSLCertificateKeyFile empty causes apache to not start In my opinion, the two things are probably related, have you changed the way that SSL configuration are stored ? It does not seem to be in /var/cpanel/userdata anymore. It looks like the /var/cpanel/ssl/installed/certs is parsed/list directly but when a crt exist without key/cabundle, it causes trouble. Thanks in advance for your response and advice, Alexis

• 

  alexis_

  • October 24, 2017 08:39

  Hello, To answer myself, if anyone has the same problem, it seems the datastore changed in V68 : 68 Release Notes - Version 68 Documentation - cPanel Documentation The VHOST changed too, now it's a "combined" file instead of two files :
  # V68 SSLCertificateFile /var/cpanel/ssl/apache_tls/DOMAIN.COM/combined # PRE-v68 SSLCertificateFile /var/cpanel/ssl/installed/certs/XXXXXXXXXXXX.crt SSLCertificateKeyFile /var/cpanel/ssl/installed/keys/XXXXXXXXXX.key
  Right now, the datastore is a mix of V1 (pre-68) files and V2 (post-68), to check this, you can use this command :
  # check "v2" file which are the one without the SSL* inside for f in /var/cpanel/userdata/*/*_SSL ; do if ! fgrep -q 'sslc' $f ; then echo $f ; fi ; done # for the "v1" for f in /var/cpanel/userdata/*/*_SSL ; do if fgrep -q 'sslc' $f ; then echo $f ; fi ; done
  It does not explain why apache crash randomly, some certificates seems messed up (empty key, empty cabundle etc...). I created a small script to check the SSL files, right now I don't know if this script is enough to check everything as I have not tested yet on a crashed server. It found some weird certificate files.
  #!/bin/bash for f in /var/cpanel/userdata/*/*_SSL ; do fgrep 'sslc' $f done | tr -s ' ' | cut -d ' ' -f 2 | while read i ; do ! test -f $i && echo "$i is missing" if echo "$i" | fgrep -q '.crt' ; then openssl x509 -in $i -text -noout > /dev/null || echo "$i is not a valid CRT" continue fi if echo "$i" | fgrep -q '.key' ; then openssl rsa -in $i -check -noout > /dev/null || echo "$i is not a valid KEY" continue fi if echo "$i" | fgrep -q '.cab' ; then openssl x509 -in $i -text -noout > /dev/null || echo "$i is not a valid CABUNDLE" continue fi done
  

  0
• 

  cPanelMichael

  • October 25, 2017 13:32

  It does not explain why apache crash randomly, some certificates seems messed up (empty key, empty cabundle etc...).

  
  Hello, Feel free to open a support ticket using the link in my signature so we can take a closer look at this. Thank you.

  0

Please sign in to leave a comment.