Extremely slow SSL after migration to new server
We're in the process of migrating a bunch of sites to a new linode cpanel server.
Server is setup running CentOS7, 16Gb of RAM.
We wanted MPM ITK with the new server and no longer had the option to use 2.2, so we're using 2.4 and Easyapache4. The old server was using 2.2 and Easyapache 3. This is basically the only difference between the setup of the 2 servers.
Basically the issue right now is when serving over SSL, it is extremely slow on the new server. Basically it takes 15 - 30 seconds just for the SSL handshake alone.
Have checked /proc/sys/kernel/random/entropy_avail and it's not a problem.
Output from connecting via curl from another server:
Apache configuration: TraceEnable Off ServerSignature Off ServerTokens ProductOnly FileETag None AllowOverride All Options ExecCGI FollowSymLinks IncludesNOEXEC SymLinksIfOwnerMatch StartServers 5 MinSpareServers 5 MaxSpareServers 10 ServerLimit 256 MaxRequestWorkers 150 MaxConnectionsPerChild 10000 KeepAlive On KeepAliveTimeout 5 MaxKeepAliveRequests 100 Timeout 300 # cipher and protocol directives can be set in WHM under 'Apache Configuration' -> 'Global Configuration' SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA SSLProtocol All -SSLv2 -SSLv3 SSLPassPhraseDialog builtin SSLUseStapling on SSLStaplingCache shmcb:/etc/apache2/run/stapling_cache_shmcb(256000) # Prevent browsers from failing if an OCSP server is temporarily broken. SSLStaplingReturnResponderErrors off SSLStaplingErrorCacheTimeout 60 SSLSessionCache shmcb:/etc/apache2/run/ssl_gcache_data_shmcb(1024000) SSLSessionCache dbm:/etc/apache2/run/ssl_gcache_data_dbm SSLSessionCacheTimeout 300 Mutex file:/etc/apache2/run ssl-cache SSLRandomSeed startup builtin SSLRandomSeed connect builtin # The Listen port can be updated using 'Tweak Settings' -> 'System', # However, if you have any Apache Reserved IPs, then this Tweak setting will # be ignored. Instead, each IP on your system (excluding Apache Reserved IPs) # will be listed here. Listen 0.0.0.0:443 Listen [::]:443 AddType application/x-x509-ca-cert .crt AddType application/x-pkcs7-crl .crl I'm not even sure where to go from here. There's nothing I can find that would be causing a 30 second SSL handshake. We even re-created the certificates and private keys completely just to be sure it wasn't something there. Nothing relevant in any of the server's logs. We've tried disabling the firewall, and no difference there either. If I test using pingdom.com or another took, it typically takes a full 60 seconds to load a page. I haven't had a chance to revert apache from MPM-ITK to see if that might be the problem, will probably be my next attempt. But wanted to see if there's something I might be missing here. Something is obviously misconfigured on this server.
root@web2 [~]# time curl -Iv https://www.MYSITE.COM
* About to connect() to www.MYSITE.COM port 443 (#0)
* Trying MY_IP_ADDRESS... connected
* Connected to www.MYSITE.COM (MY_IP_ADDRESS) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject: CN=www.MYSITE.COM,OU=PositiveSSL,OU=Domain Control Validated
* start date: May 02 00:00:00 2016 GMT
* expire date: May 02 23:59:59 2017 GMT
* common name: www.MYSITE.COM
* issuer: CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB
> HEAD / HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: www.MYSITE.COM
> Accept: */*
>
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Date: Wed, 01 Jun 2016 15:38:43 GMT
Date: Wed, 01 Jun 2016 15:38:43 GMT
< Server: Apache
Server: Apache
< X-Frame-Options: SAMEORIGIN
X-Frame-Options: SAMEORIGIN
< Set-Cookie: PHPSESSID=v705s1cud67u90um3d75vl8dn5; expires=Fri, 03-Jun-2016 15:38:43 GMT; Max-Age=172800; path=/
Set-Cookie: PHPSESSID=v705s1cud67u90um3d75vl8dn5; expires=Fri, 03-Jun-2016 15:38:43 GMT; Max-Age=172800; path=/
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
Pragma: no-cache
< Set-Cookie: MYSITE=YTo0OntzOjY6InNvdXJjZSI7aTo4O3M6ODoiY2FtcGFpZ24iO047czo3OiJjb250ZW50IjtOO3M6NDoidGVybSI7Tjt9; expires=Fri, 01-Jul-2016 15:38:43 GMT; Max-Age=2592000; path=/; secure
Set-Cookie: MYSITE=YTo0OntzOjY6InNvdXJjZSI7aTo4O3M6ODoiY2FtcGFpZ24iO047czo3OiJjb250ZW50IjtOO3M6NDoidGVybSI7Tjt9; expires=Fri, 01-Jul-2016 15:38:43 GMT; Max-Age=2592000; path=/; secure
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
<
* Connection #0 to host www.MYSITE.COM left intact
* Closing connection #0
real 0m26.640s
user 0m0.034s
sys 0m0.022s
Apache configuration: TraceEnable Off ServerSignature Off ServerTokens ProductOnly FileETag None AllowOverride All Options ExecCGI FollowSymLinks IncludesNOEXEC SymLinksIfOwnerMatch StartServers 5 MinSpareServers 5 MaxSpareServers 10 ServerLimit 256 MaxRequestWorkers 150 MaxConnectionsPerChild 10000 KeepAlive On KeepAliveTimeout 5 MaxKeepAliveRequests 100 Timeout 300 # cipher and protocol directives can be set in WHM under 'Apache Configuration' -> 'Global Configuration' SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA SSLProtocol All -SSLv2 -SSLv3 SSLPassPhraseDialog builtin SSLUseStapling on SSLStaplingCache shmcb:/etc/apache2/run/stapling_cache_shmcb(256000) # Prevent browsers from failing if an OCSP server is temporarily broken. SSLStaplingReturnResponderErrors off SSLStaplingErrorCacheTimeout 60 SSLSessionCache shmcb:/etc/apache2/run/ssl_gcache_data_shmcb(1024000) SSLSessionCache dbm:/etc/apache2/run/ssl_gcache_data_dbm SSLSessionCacheTimeout 300 Mutex file:/etc/apache2/run ssl-cache SSLRandomSeed startup builtin SSLRandomSeed connect builtin # The Listen port can be updated using 'Tweak Settings' -> 'System', # However, if you have any Apache Reserved IPs, then this Tweak setting will # be ignored. Instead, each IP on your system (excluding Apache Reserved IPs) # will be listed here. Listen 0.0.0.0:443 Listen [::]:443 AddType application/x-x509-ca-cert .crt AddType application/x-pkcs7-crl .crl I'm not even sure where to go from here. There's nothing I can find that would be causing a 30 second SSL handshake. We even re-created the certificates and private keys completely just to be sure it wasn't something there. Nothing relevant in any of the server's logs. We've tried disabling the firewall, and no difference there either. If I test using pingdom.com or another took, it typically takes a full 60 seconds to load a page. I haven't had a chance to revert apache from MPM-ITK to see if that might be the problem, will probably be my next attempt. But wanted to see if there's something I might be missing here. Something is obviously misconfigured on this server.
-
So, I restarted apache, and the time went back down to below 1 second. After a few minutes, it's bouncing between about 5 and 10, and now it's back up to 30 seconds. 0 -
Hello, The following third-party URL offers some additional commands you can use to measure the amount of time it's taking for the SSL connections: SSL handshake latency and HTTPS optimizations. :: semicomplete.com - Jordan Sissel Have you tested from multiple locations to rule out any network issues from the test server you are using to make the connection? Were you able to test with MPM-ITK disabled to see if that makes a difference? Thank you. 0
Please sign in to leave a comment.
Comments
2 comments