CloudLinux came back??
Not to long ago, somehow, CloudLinux was installed on 2 of our servers... No idea how or who did it.
We had it uninstalled on both servers.. logging in today, I have noticed that its installed back in again.
According to logs, no one other than myself has been on the server, how is it possible that cloudlinux reinstalled by itself? I don't even think that is possible ?
-
Hello :) When is the last time your system was rebooted? Is this a dedicated server or a VPS? How are you determining that it's installed? Thank you. 0 -
In looking at the top when logging into WHM: CLOUDLINUX 6.5 x86_64 standard " It did not state this a day or so ago... It had stated: CENTOS 6.5 x86_64 standard " rebooted as in "graceful reboot" ? It was rebooted fully the last time we had CloudLinux uninstalled, everything was working fine and great after that, till just recently, when I noticed that CloudLinux was reinstalled again. 0 -
For instance, please post the output from: uptime uname -a cat /var/cpanel/envtype
Thank you.0 -
[~]# uptime 13:02:35 up 2:03, 1 user, load average: 0.61, 0.56, 0.48 [~]# uname -a Linux servername.com 2.6.32-531.11.2.lve1.2.55.el6.x86_64 #1 SMP Fri Apr 18 09:06:31 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux [~]# cat /var/cpanel/envtype standardroot@chicago [~]#
Nothing came up for that last one.. But the looks of that stuff there... it doesnt have cloudLinux installed... but if its not installed, then why does it show it at the top of WHM ? It didnt before.0 -
Given uname -a --> it is installed. Sorry about the misterry, but I also really want to know how it can get installed by itself. 0 -
I want to know how it can get installed by itself too. Because, I know that I did not install it. it was installed on our servers before by itself, and we had it removed, because all it did was cause major complications with everything, to us, it is just plain out awful. And all the problems we had before, I would not even think about reinstalling it again. 0 -
I have not heard of that happening before, and I can't think of anything cPanel-related that would automatically install Cloud Linux and reboot the system. Are you sure no other companies or administrators have access to your server? If so, I recommend opening a support ticket so we can take a closer look. You can post the ticket number here so we can update this thread with the outcome. Thank you. 0 -
There's only 1 other company that has access to our servers (MyCPAdmin), in which they did not install it. We had them remove it the last time, and will probably have them remove it again. We are going to see what they are going to do, and if it doesnt work out, we will submit a ticket here. Is there a log that maybe im not looking at, that would show who/how it got installed? 0 -
You could try reviewing the bash history: /root/.bash_history
Thanks.0 -
root@servername [~]# /root/.bash_history -bash: /root/.bash_history: Permission denied
What am I missing there? I am logged in as root0 -
That's the path, not the command. EX: cat /root/.bash_history
Or;grep cl /root/.bash_history
Thank you.0 -
ok so in searching bash_history, I found this: #1401389872 /usr/local/cpanel/bin/cloudlinux_system_install -c which seems was ran to install CL... But, I don't know when or who ran it.. No IP, no date, no info... Unless im just not reading the file correctly, but thats of no help there ? 0 -
Depending on how often the command line is utilized, and how it's configured, you may find the date/time of the entry with the "history" command. EX: history|grep cloudlinux_system_install
However, that does explain how it was installed. You may need to consult with everyone who has root access to your system to determine how it might have happened. Thank you.0 -
root@servername [~]# history|grep cloudlinux_system_install 272 [2014-05-29 20:57:52] /usr/local/cpanel/bin/cloudlinux_system_install -c 522 [2014-06-12 01:56:11] /usr/local/cpanel/bin/cloudlinux_system_install -c 529 [2014-06-12 18:22:08] history|grep cloudlinux_system_install
That is what it shows.... and it looks as if it was ran 1am this morning, however, it is currently not installed. There are only 3 of us that have access to Root... In which I know the other 2 did not install it, and I know I did not install it either.. I found a license update in Cron.d... If that runs, would that somehow automatically reinstall CL ? I did go through and found an actual tutorial on removing "everything" of CloudLinux from the server, and while going through that, I found several items that were still sitting on the server. And removed them as well. However, The part where I get confused at, is this whole "Kernel" thing. How do I know that CentOs is using the correct kernel? And to be sure that CL kernel is no longer on the server.0 -
You can verify the installed kernel with the "uname -a" command. Remember you have to reboot the machine back into the stock kernel after uninstalling Cloud Linux. Thank you. 0 -
root@servername [~]# uname -a Linux servername 2.6.32-531.17.1.lve1.2.56.el6.x86_64 #1 SMP Mon Jun 2 05:54:37 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux0 -
Right, it's still the Cloud Linux kernel. You would have to reboot into the stock kernel. Thank you. 0 -
I have rebooted though.. =\ several times... because when I did... we had several dozen tickets come in complaining their site was down lol... but I must not be setting it correctly or something.. I don't know. I could only go by what I read on how to uninstall that gawd awful thing. 0 -
You can review the following file: /boot/grub/grub.conf
Grub orders the entries starting at zero. If you wanted to utilize the first kernel listed in this file upon boot, make sure it's configured with "default=0". Using "default=1" would boot the second kernel in the file, and so forth. Thank you.0 -
Ugh.. it got installed "AGAIN"... but no one accessed it???? 531 [2014-06-13 04:44:29] history|grep cloudlinux_system_install And at the top on WHM has CLOUDLINUX 6.5 x86_64 standard " servername How in the world does this keep getting installed??? 0 -
Okay, I think we figured out how this was getting installed... It appears that someone had hacked the server (about 2 months ago), and we didnt know it, but they installed "Webmin" on the server... And was using that to access all the servers information... So when we would block access, they would get right back in again... it appears, this is how they were doing it. Since we didnt know that was even on there, we just found it tonight, and have removed it. But that is how it kept getting reinstalled.... And how they kept knowing our password changes and our ssh-key 0 -
Please keep in mind that if your server was rooted, it's a good idea to determine the source of the attack, address it, and then reinstall the OS/cPanel. You can then restore the accounts from backup, or transfer them to/from another server. Otherwise, you may never fully know what system changes the hacker could have made that will lead to future exploits. Thank you. 0 -
[quote="Mark_CFH, post: 1664071">Okay, I think we figured out how this was getting installed... It appears that someone had hacked the server (about 2 months ago), and we didnt know it, but they installed "Webmin" on the server... And was using that to access all the servers information... So when we would block access, they would get right back in again... it appears, this is how they were doing it.
Webmin has nothing to do with CloudLinux and it makes no sense why you would attribute the installation of it to "hackers" since it would in no way benefit them. It's likely this is not related at all and just coincidental. The only thing that makes sense here is that CloudLinux is installed but rather than uninstall it, it was "disabled" via the boot menu. Perhaps every time there's an update for it, it's defaulting itself in your grub.conf and a reboot is making it take effect? [quote]Since we didnt know that was even on there, we just found it tonight, and have removed it. But that is how it kept getting reinstalled.... And how they kept knowing our password changes and our ssh-key
Again, sorry but this makes zero sense and CloudLinux has nothing to do with being compromised.0 -
[quote="Echelon17, post: 1664731">Webmin has nothing to do with CloudLinux and it makes no sense why you would attribute the installation of it to "hackers" since it would in no way benefit them. It's likely this is not related at all and just coincidental. The only thing that makes sense here is that CloudLinux is installed but rather than uninstall it, it was "disabled" via the boot menu. Perhaps every time there's an update for it, it's defaulting itself in your grub.conf and a reboot is making it take effect? Again, sorry but this makes zero sense and CloudLinux has nothing to do with being compromised.
Lets see if I can put it in a way you can understand then... they hacked our servers, installed Webmin... and through webmin installed cloudlinux.. because webmin has an internal SSH which you can run installs when using it... That is how CloudLinux kept getting installed... At the time BEFORE we found Webmin, we had no idea how or who kept reinstalling it... Make sense now? It wasnt "disabled" it was uninstalled, following the full instructions on uninstalling it... not "disabling" it.0 -
I understood fully. No need to become aggressive and defensive - because I was't challenging you, just disagreeing with your appraisal of the situation. I'll take the leap of faith here and assume that you are correct in that "hackers" installed webmin. This seems highly illogical because it's too obvious and would/should be noticed by any half competent administrator, but OK, I'll take that leap of faith... "Hackers" then installed a new kernel (CloudLinux) and rebooted your machine to take advantage of that Kernel? My first thought/question is how would you not notice your machine being rebooted for this? My second question would be, "why"? CloudLinux offers absolutely no benefit or incentive for "hackers" to install. Additionally it's a far too obvious and blatant system change for it to be the mark of a hacker. This sounds far more like an employee or someone inexperienced is messing around here. "Hackers" would not be so obvious and in your face about modifications to a system. 0 -
How would I not notice? because of the time frame of when they made the installs, which was usually when I was in bed sleeping (I don't stay away for 24 hours) When I would wake up... and view WHM, it would say cloudlinux at the top, where as before I went to bed, it stated CentOS... So I guess im not even half competent, according to your words.. And as you are quoting "hackers" There was hackers... it all first started in April, when they hacked our WHMCS, and then both of our servers... installed Webmin on both... and CloudLinux on Both... There was over 75 "fake" accounts created on one our servers... Which we removed... There are only 3 of us with access to the servers.. Which neither one of them installed it... We use to get emails when accounts were created, removed, suspended.. But we no longer got those... Also, Several items were changed with in WHM itself, that caused several problems in us receiving any emails, alerts, etc. If you think I am imagining "hackers" I can assure you there was one, as I have screen shots, IP's, and other things of the person that hacked us. [COLOR="silver">- - - Updated - - - So, if all you are going to do is criticize people, that are "imagining" hackers, Why even post anything at all if its not helpful. 0 -
You really need to calm down and stop being so defensive! If you don't want discussion or critique of your posts, then perhaps you shouldn't post in a public forum? I was using the term "hackers" loosely because I don't believe these are what one would typically class as such. Making such blatant and obvious (also obviously detected) changes to your system does not fit the MO or the behavior of a typical "hacker", who would instead be much much more subtle to the point you would probably not even know you had been compromised. This sounds more like something else entirely. Before you jump on your defensive bandwagon for the third time and claim I don't know what I'm talking about and that you have logs and screenshots indicating it was "hackers" I would have to say that I believe our definitions are very different. In my mind, "hackers" here seems to fit the target of blame but the behaviour does not sound like anything I have experienced before. So to critique your other points; 1. Do you not have monitoring in place that would alert you to your server being down? The reboot would have taken at least a few minutes and would definitely have been long enough for any monitoring system to have detected. 2. How did you obtain logs and screenshots of an attacker, if you didn't even know they had installed webmin? Was this in post-investigation from a third party? 3. How was the compromise achieved? I presume your logs show this? 4. I would resolve the problem with receiving new account e-mails as soon as possible. 5. Try installing csf+lfd, which can also alert you to logins to your server (including root, which is very useful). 6. Hire an administration team who can harden your servers. Monitoring and administration seems to be your key issue here. 0
Please sign in to leave a comment.
Comments
27 comments