CPANEL-40433 - WHM Backup Configuration page loading slowly; previously it was timing out
Hi,
CentOS v7.9.2009
WHM/cPanel v100.0.12
The situation has improved from "Backup Configuration" not loading at all (timing out), to the present situation which is, sometimes it loads right away, other times it takes about 40 seconds to load. I haven't experienced a delay loading any other WHM pages.
The timeout seems to not be happening anymore, since I made some modifications to our custom iptables firewall.
But this intermittent delay loading the page is now happening.
It seems there is some resource that "Backup Configuration" needs, which is involved in causing this delay.
Does anyone know what this is?
thanks,
JC
-
Hi, CentOS v7.9.2009 WHM/cPanel v100.0.12 The situation has improved from "Backup Configuration" not loading at all (timing out), to the present situation which is, sometimes it loads right away, other times it takes about 40 seconds to load. I haven't experienced a delay loading any other WHM pages. The timeout seems to not be happening anymore, since I made some modifications to our custom iptables firewall. But this intermittent delay loading the page is now happening. It seems there is some resource that "Backup Configuration" needs, which is involved in causing this delay. Does anyone know what this is? thanks, JC
In playing with this, I do see opportunity for us to load things in a more optimized way (I used browser network throttling to do this). Can you help us understand what kind of network bandwidth you are working with so that we can test our fixes using those settings?0 -
I tested again last night on a very fast cable broadband connection, and got the timeout again after about 1 min. 40 sec. Timeout while trying to load the "Backup Configuration" page. Tested at ~1:40am Pacific. I don't think this has anything to do with the network. Tested again at 1:44am Pacific... this time page loads immediately. This is looking a lot like a cPanel resource is sometimes overloaded, or perhaps there are one or more cPanel IP addresses that I don't know about, which are currently not open in the firewall for http or https. I can imagine possibly that WHM is trying to reach some remote cPanel service, at an IP address, and there are a set of mirror addresses, and none of them are available, hence the timeout. At random times, the top ones on the list are available, and the page loads immediately. Do I need to open an account to view this? 0 -
There isn't a good reason that should be happening, and I don't see that on my servers at this time. Could you make a ticket with our team so we can investigate this directly on the system? 0 -
@cPRex If we turn off our custom firewall, there are no problems. With our firewall enabled, "Backup Configuration" sometimes (as right now) times out loading after 1 min. 40 sec. This happens consistently. When my testing produces this result, I also see a delay of 20 sec. loading the home page of WHM. Loading other pages, such as "List Accounts" takes 1 second. I have opened the required ports (INPUT, OUTPUT) for cPanel in our firewall, and whitelisted 17 cPanel IP addresses for inbound http:// (80) and https:// (443). Can you tell me what ports, specifically, need to be open for "Backup Configuration" to load normally? (i.e. other than 80, 443) If that isn't the issue, then which service is Backup Configuration trying to access which would require a port 80 (http) or port 443 (https) connection, either inbound or outbound? Note also that WHM doesn't report in the error log why it timed out. thanks, JC 0 -
There aren't any special ports or connections happening with the Backup Configuration page. Do you have everything open from this page? How to Configure Your Firewall for cPanel & WHM Services | cPanel & WHM Documentation 0 -
@cPRex I have just discovered the problem... our firewall is restricting access for FTP to specific IPs and IP ranges. When I remove the FTP rules, WHM is working normally. Specifically, it's the "passive mode" FTP port range that needs to be open, 49152:65534. Can you tell me what cPanel IPs need to be whitelisted for this port range, 49152:65534? Is cPanel using one or more ports in the range for something other than FTP? Note that the "How to Configure..." page doesn't say anything about which cPanel IP addresses need to be whitelisted for inbound requests on TCP ports. We currently have 17 cPanel IPs whitelisted for inbound http and https requests (80, 443). Can you provide me with your official list of which IPs need to be whitelisted? thanks, JC 0 -
I wouldn't expect any passive ports need to be open for the page itself to load. Those ports are used by FTP clients to make connections, and not something that is needed for the WHM interface to function. 0 -
I wouldn't expect any passive ports need to be open for the page itself to load. Those ports are used by FTP clients to make connections, and not something that is needed for the WHM interface to function.
The problems I'm seeing with WHM are intermittent. Sometimes I see delays, e.g. it takes ~20 sec. to reach home page, and the issue of timeout loading "Backup Configuration" page. These conditions happen together. When this occurs, and I open the FTP passive port range, 49152:65534, WHM immediately works normally again. At the moment, I'm not seeing the WHM problems. I will keep trying to test this.0 -
@cPRex I just finished another round of testing on this issue. This time I monitored the TCP ports and found something very interesting. I should say first that I also verified again that turning off our firewall immediately eliminated the delays and timeout in WHM. What I found is that WHM is, in fact, using ports that are within the FTP passive port range, and it's not using those for FTP " it's using them for https (443). In tonight's testing, after I clicked on "Backup Configuration", I then monitored TCP ports from the shell... This is the relevant output from the command, /usr/sbin/ss -tpn (show here vertically for clarity): [QUOTE] State: SYN-SENT Recv-Q: 0 Send-Q: 1 Local Address:Port: 67.231.17.190:51196 Peer Address:Port: 104.18.17.164:443 users ("whostmgr - back",pid=21791,fd=4)
67.231.17.190 is the local host, vps.wwk.com. 104.18.17.164 is cpanel.net's IP address on Cloudflare (also 104.18.16.164). (See www.cpanel.net) In the /usr/sbin/ss output above, we see that the local host has TCP port 51196 open for communication with the peer, cpanel.net. The remote peer is using port 443 (https). The local process using this connection is whostmgr, which is WHM. Another process I've seen using ports in this range for https is queueprocd. For now I will solve this problem by opening 49152:65534 for output from localhost, and for input from 104.18.17.164, 104.18.16.164, and localhost. Tested and is working.0 -
Do you happen to have a remote FTP server configured on this particular WHM instance? 0 -
Do you happen to have a remote FTP server configured on this particular WHM instance?
We are using WHM to manage the FTP service on our VPS. We are using Pure-FTPd. My client is connecting via FTP to the VPS. We are running a custom firewall that whitelists my client's IP addresses for FTP and SSH.0 -
That's definitely odd as I just don't have a good explanation. Could you submit a ticket to our team so we can check this out directly? 0 -
@cPRex I recommend your team look at the code that loads the Backup Configuration page and confirm what I have observed, which is that WHM is communicating with cpanel.net using https, with the remote port being 443, and the local port being in the FTP passive mode range. You should be able to use CSF to block the passive mode range, which should allow you to see the delays and timeout in WHM. 0 -
By default, when CSF is installed it does not include the passive port ranges. I did the following for my test: -created a fresh cPanel 102 install on AlamLinux 8 -made sure an FTP server was installed in WHM -installed CSF At this point I checked /etc/csf/csf.conf and found this configuration, showing that the passive ports aren't allowed by default: # Allow incoming TCP ports TCP_IN = "20,21,22,25,53,80,110,143,443,465,587,993,995,2077,2078,2079,2080,2082,2083,2086,2087,2095,2096,8443" # Allow outgoing TCP ports TCP_OUT = "20,21,22,25,37,43,53,80,110,113,443,587,873,993,995,2086,2087,2089,2703"
This really does seem to be a unique situation to your server environment. If the page worked as you described, every server that doesn't have the passive FTP ports enabled would be experiencing this issue, and so far this is the only report we have of this behavior.0 -
@cPRex Thanks for testing. I see some inconsistency with your CSF allowed ports and what I have programmed in our firewall. First, why is port 8443 open for input? Isn't this used by Plesk? Port 8443 is not listed in the
Tested again and still working OK. You wrote: [QUOTE]This really does seem to be a unique situation to your server environment. If the page worked as you described, every server that doesn't have the passive FTP ports enabled would be experiencing this issue, and so far this is the only report we have of this behavior.
I think the situation is unique because we are running a custom firewall, not CSF or whatever else WHM/cPanel offers. Your test shows that the CSF defaults are different than the official knowledge base article for setting up ports for WHM/cPanel in a firewall, so this may help explain why nobody else is seeing what we've seen.0 -
Right - I just did the default CSF install with no additional configuration necessary. The point of that exercise was to show that the passive ports aren't included by default, and don't need to be opened for the WHM interface to work properly. I do agree the issue is with the custom firewall. 0 -
@cPRex I asked several basic questions in my last post, and you appear unable to answer any of them. We are considering switching to Plesk. 0 -
I'm sorry, I didn't think I needed to answer them as it wasn't relevant to our testing. The ports listed at How to Configure Your Firewall for cPanel & WHM Services | cPanel & WHM Documentation are still the definitive list required for WHM and cPanel services. 0 -
cPanel staff has now determined that cPanel's own firewall defaults are inconsistent with cPanel's own definitive list of required ports. Nothing to see here. 0 -
cPanel doesn't install or administer a firewall. cPanel has no affiliation with CSF or ConfigServer. You mentioned you had used CSF before so I installed it on a test machine to see the default configuration options and post them here to see if that would help clarify the issue. Your issue with the interface not loading well is due to your custom firewall configuration, since once you disable that things work well. I've already confirmed the passive ports are not required for the cPanel interface to function. 0 -
Sure, CSF is supported by WHM, and your documentation recommends CSF. No, we have never used CSF. Once I opened our firewall to the FTP passive port range, for localhost and specific cpanel.net IPs, then WHM worked properly. 0
Please sign in to leave a comment.
Comments
23 comments