Need to find the cause of website aceess lock up
My websites and WHM and account control panels on my server lock up and are inaccessible for few minutes every hour each day. This is the case each time I can't access the sites or the control panels.
It starts at like xx:00 and ends at xx:04 every hour.
I've been trying to find the cause of this without success so I need some assistence.
What I know is that there are several crons that run at this time and all of them except one are system related like cPanel (11.46) or OS (CentOS 6.6 i686 virtuozzo) crons.
There is one cron by a user (my own). It's feed update script that runs each 30 minutes. I have not looked very closely at the server resources this process consumes, but I haven't seen any visible sign that it is consuming to much, like when I monitor the server processes with ps at the same time the lock up is going on and the feed update script process is running. When the script runs at say xx:30 there is no problem accessing the sites, but only when it is among the other crons at xx:00. I see that the domain where the script runs under seems to be using very little of the server resources (0.00) according to Daily Process Log and this is my most active domain on the server for my personal use.
Nov 23 02:55:01 server CROND[11242]: (root) CMD (/usr/local/cpanel/bin/dcpumon >/dev/null 2>&1)
Nov 23 02:58:01 server CROND[11288]: (root) CMD (/usr/local/bandmin/bandmin)
Nov 23 03:00:03 server CROND[11320]: (one-user-account) CMD (/usr/local/bin/php /home/xxxxx/public_html/xxxxxxx/update.php --feeds --quiet)
Nov 23 03:00:03 server CROND[11321]: (root) CMD (/usr/local/cpanel/bin/dcpumon >/dev/null 2>&1)
Nov 23 03:00:03 server CROND[11322]: (root) CMD (/usr/lib/sa/sa1 1 1)
Nov 23 03:01:01 server CROND[11367]: (root) CMD (run-parts /etc/cron.hourly)
Nov 23 03:01:02 server run-parts(/etc/cron.hourly)[11367]: starting 0anacron
Nov 23 03:01:06 server anacron[11376]: Anacron started on 2014-11-23
Nov 23 03:01:06 server run-parts(/etc/cron.hourly)[11378]: finished 0anacron
Nov 23 03:01:24 server anacron[11376]: Will run job `cron.daily' in 35 min.
Nov 23 03:01:24 server anacron[11376]: Jobs will be executed sequentially
Nov 23 03:02:01 server CROND[11404]: (root) CMD (/usr/local/bandmin/bandmin)
Nov 23 03:05:02 server CROND[11439]: (root) CMD (/usr/local/cpanel/bin/dcpumon >/dev/null 2>&1)There is one cron by a user (my own). It's feed update script that runs each 30 minutes. I have not looked very closely at the server resources this process consumes, but I haven't seen any visible sign that it is consuming to much, like when I monitor the server processes with ps at the same time the lock up is going on and the feed update script process is running. When the script runs at say xx:30 there is no problem accessing the sites, but only when it is among the other crons at xx:00. I see that the domain where the script runs under seems to be using very little of the server resources (0.00) according to Daily Process Log and this is my most active domain on the server for my personal use.
-
Among those crons are the bandmin cron which is logged every hour at xx:58:01 and xx:02:01. It might perhaps be worth mentioning that I haven't enabled quotas in WHM (Initial quotas setup) on the virtuozzo VPS, but it's in progress now. 0 -
So, load, I/O wait, and RAM is all good during this time frame? Have you not simply used `top' at this time to monitor load and I/O wait? You may also try using the `free' command or `sar -r' command to monitor RAM usage. Additionally, if you feel that the scheduling of the crons is the issue, why not test this by changing cron times one by one? You may also be overlooking crons from other users. You can run the following command from root to look at all cron's: -- for user in $(cut -f1 -d: /etc/passwd); do echo $user && crontab -u $user -l; done -- Regards, 0 -
]So, load, I/O wait, and RAM is all good during this time frame? Have you not simply used `top' at this time to monitor load and I/O wait? You may also try using the `free' command or `sar -r' command to monitor RAM usage. Additionally, if you feel that the scheduling of the crons is the issue, why not test this by changing cron times one by one? You may also be overlooking crons from other users. You can run the following command from root to look at all cron's: -- for user in $(cut -f1 -d: /etc/passwd); do echo $user && crontab -u $user -l; done -- Regards,
There is no user cron at this time except for the feed update script I/O wait seems be normal RAM is normal, no rise nor insufficiency at the time, 1.3GB of available 6GB. CPU time spent idle is about 9x.xx% at this time But the load gets higher, it rises from 0.1x and up to 2.00 for few minutes. Today I saw it rise up to 2.00 at xx:02 clock when the bandmin process appeared. This is a bit high for a 2 core VPS production machine (Xeon E5-2660 @ 2.20GHz), but I have managed to browse my sites when it's even higher, but not comfortably. I'm thinking about changing cron times, but I haven't found the right time for the job. I have overview of all the crons and when they run. I think it's best to find out what can be done with the bandmin cron and if there is something that needs to be improved with this process on the server. If I change the time for the bandmin cron, is there anything else I need to change that is related to this process? I have these crons usually every hour: bandmin xx:58:01 and xx:02:01 user feed script xx:00:03 dcpumon xx:00:03 sa1 xx:00:03 cron.hourly xx:01:010 -
Hello :) I suggest temporarily disabling the user's PHP script that's run on a cron job to rule that out as the cause of the issue. Also, yes, you can change the time that your bandmin cron job runs without effecting any other services. Thank you. 0 -
]Hello :) I suggest temporarily disabling the user's PHP script that's run on a cron job to rule that out as the cause of the issue. Also, yes, you can change the time that your bandmin cron job runs without effecting any other services. Thank you.
Hi, this appears to be in order now starting just few hours ago. It's unlikely that the feed script has been the cause of this as it is not using that much resources. I got email today from the hosting company CEO informing me of abusive clients on the server node and the host was in the process of resolving the issue and rebooting the server (the physical one). For few hours now, my issue has not been going on, but if this starts again I will contact the host and inform them of this.0 -
I am happy to see the issue is now resolved. Thank you for updating us with the outcome. 0
Please sign in to leave a comment.
Comments
6 comments