Hostaccess control suddenly empty
We discovered a strange configuration change with Host Access Control, and we have no clue when it happened or why.
We had set up an input chain in Host Access Control when we first deployed the server cluster. As we normally never have to edit this after initial setup, we simply loaded our default config and did not bother with it again.
By accident last week, I tried to access port 2087 from a remote location instead of 2083, and I was able to get to the WHM page. After using the nft list command, I noticed there were no handles present anymore at all.
I have since set up the same config as last time, but there are some strange things to notice. For one, the handle starts at number 4 for the chain itself, and then handle 7 for the first IP. But we have had more IPs than that in the previous listing, and we know the IP handle numbers are always increased and not re-used.
So now we have 7 to 20 as handle numbers (IP's, for example, replaced for privacy, security, as well as a custom port number)
table inet filter {
chain cPanel-HostAccessControl { # handle 4
ip saddr 1.2.3.4 ct state new tcp dport 2087 counter packets 0 bytes 0 accept # handle 7
ip saddr 1.2.3.4 ct state new udp dport 2087 counter packets 0 bytes 0 accept # handle 8
ip saddr 5.6.7.8 ct state new tcp dport 2087 counter packets 0 bytes 0 accept # handle 9
ip saddr 5.6.7.8 ct state new udp dport 2087 counter packets 0 bytes 0 accept # handle 10
ip saddr 9.10.11.12 ct state new tcp dport 2087 counter packets 0 bytes 0 accept # handle 11
ip saddr 9.10.11.12 ct state new udp dport 2087 counter packets 0 bytes 0 accept # handle 12
ct state new tcp dport 2087 counter packets 0 bytes 0 reject # handle 14
ct state new udp dport 2087 counter packets 0 bytes 0 reject # handle 16
ip saddr 1.2.3.4 ct state new tcp dport 99 counter packets 0 bytes 0 accept # handle 17
ip saddr 5.6.7.8 ct state new tcp dport 99 counter packets 0 bytes 0 accept # handle 18
ip saddr 9.10.11.12 ct state new tcp dport 99 counter packets 0 bytes 0 accept # handle 19
ct state new tcp dport 99 counter packets 0 bytes 0 reject # handle 20
}
}
To my knowledge, that is a correctly configured input chain.
We had our datacenter look at the situation from their end, but there has been no noticed successful attempt to edit the files according to their information from unknown IP's. And we ourselves have no documented cases of maintenance with this either in the logs. We did a complete audit and have found no further abnormal behavior or any data exfiltration, so we are ruling out that we are dealing with bad actors.
I am still wondering if this behavior has been reported before by a customer to your knowledge in the past, or if this is an entirely new anomaly.
Running on Cloud Linux v8.10.0 STANDARD kvm
cPanel Version 1.20.14
-
Hey there! Unfortunately (or, maybe fortunately) I haven't heard of this area just getting wiped in the past, and I don't see any similar reports when I checked our ticket system just now.
The only place I could think of to check would be /var/log/secure as that would show any SSH logins to the machine.
1 -
Hello cPRex,
I share the fortunately in this case, and hope it stays an anomaly.
We checked the /var/log/secure and additional logs, and we are not seeing anything strange besides lots of wp-toolkit activity.Which we do not even use at all, but fortunately it is nothing related to this. I have since then removed wp-toolkit completely because having logs filled with pointless log rotate messages is not helping to audit stuff.
Thank you for the quick response, and I hope this topic can start to gather dust then.0 -
Sounds good - let me know if you do end up finding anything else!
0 -
Unfortunately, the issue came back after a restart of the system.
This time, however, we were able to find some more details.
It seems for unknown reason the /etc/sysconfig/nftables.conf had malformed data as in 4 lines of
Warning: Extension owner is not supported, missing kernel module?
Warning: Extension owner is not supported, missing kernel module?
Warning: Extension owner is not supported, missing kernel module?
Warning: Extension owner is not supported, missing kernel module?
Removed those than did a service nftables restart
nftables seems to be running again, though I still have to verify if it saves the records correctly.
I have a suspicion that by having this malformed, it automatically drops the hostacces controls added by the GUI. I have no idea how an error message is able to be added to a conf file. Furthermore, I have never seen this behavior in more than twenty years.I am seeing that I am not the first one suffering from this, although this is Almalinux 9. Our situation is CloudLinux v8.10.0: https://support.cpanel.net/hc/en-us/community/posts/21480665266327-Installa-new-Cpanel-on-Almalinux-9-nftables-error
However, kcare seems to be functioning as grep -i kcare /var/log/messages shows entries and the security advisor says kcare 4.18.0-553.5.1.lve.1.el8, so I have no idea either what missing kernel module it is even rambling about.
I will reboot the server tonight after adding the rules back again and see if it stays this time.
Either case, I am scratching my head at this configurations malformation.0 -
I'm not sure what would be happening here, but at least it doesn't sound like it's related to cPanel, so there's some good news.
0 -
I just noticed something similar. I was setting up a new server and had made some entries in host access control. I did a restart and the entries in host access control are gone. I took those steps again and it happens again reliably. I do notice that the services continue to work from the hosts in question even if they aren't listed (such as SSH). I'm not sure if that's because something else is allowing them.
0 -
rinkleton - if you have a way to reproduce this, can you open a ticket so we can take a look?
0 -
I usually have issues with that since we get our servers through LiquidWeb. What's the best way to coordinate that?
0 -
You'd just contact their team and they would escalate to us if necessary. So far we haven't been able to reproduce this on our end.
0 -
LiquidWeb seems to indicate that is happening because CloudLinux 8 does not support Host Access Control. This seems to contradict you're documentation (link below). Can you confirm this?
https://docs.cpanel.net/whm/security-center/host-access-control/
0 -
Host Access Control is still supported, it's just handled differently in version 8 by the operating system than it was in previous versions, so the interface also looks a bit different than you may remember.
0 -
Yeah I noticed it looked different and you specify the port rather than the service... that's all fine. But you're saying entries added there should take effect and would not be erased on a reboot when using the current version of WHM and CloudLinux 8?
0 -
Correct - nothing there should be getting automatically removed.
0 -
LiquidWeb did point me to this article https://support.cpanel.net/hc/en-us/articles/4482286014999-Host-Access-Control-doesn-t-work-when-using-another-firewall
That explains why the rules aren't working, and maybe that's why they are being erased on reboot? But either way, there are a few places in the documentation this should probably be noted. It would also be nice to have a note on the Host Access Control page in WHM that says that these rules won't have an effect if there is another firewall present on certain OSes, and list those affected.
0 -
Were you using another firewall tool on the system? But yes, I think we should have something about this on the main docs page at https://docs.cpanel.net/whm/security-center/host-access-control/. I've reached out to the documentation team about this so they can get that updated!
0 -
Yeah the server was using CSF. Thanks for your help.
0 -
Good to know - we'll update that docs page with more details soon!
0 -
Just wanted to acknowledge we also use CSF before closing.
So that atleast explains it on paper and I guess it is time to retire host access control for us.Thank you both for the additional supplied information we are going to update our internal documentation.
0
Please sign in to leave a comment.
Comments
18 comments