"cPanel & WHM update terminated" (over 6 hours)
Last week one of the servers I maintain ran out of disk space during the backup procedure. I added a new disk and configured the backups to use it instead. They are not retained locally, they are offloaded to Amazon S3 and then removed. There is plenty of space on the new disk.
Every night since I did this though I have been receiving 2 emails back to back. The first one has the subject line, "[vps.domain.com] cPanel & WHM update terminated", and in the body it says this:
And attached is a 0 byte update_log.txt. The second email has the subject line, "Cron (/usr/local/cpanel/scripts/fix-cpanel-perl; /usr/local/cpanel/scripts/upcp --cron > /dev/null)", and this in the body (plaintext, unformatted):
There are backups being uploaded to Amazon, so that seems to be working fine now, and I am not sure if it's related at all to the update process failing, but it did start right after that incident. If I ssh in and run "ps aux | grep upcp" this is what I see currently:
Any thoughts where the hangup might be? Thanks. -Michael
The system ran the "/usr/local/cpanel/scripts/upcp" script as PID "6818" for longer than 6 hours. The system will terminate this process and run the script again.
Update log preview:
number_of_preview_linesAnd attached is a 0 byte update_log.txt. The second email has the subject line, "Cron (/usr/local/cpanel/scripts/fix-cpanel-perl; /usr/local/cpanel/scripts/upcp --cron > /dev/null)", and this in the body (plaintext, unformatted):
/bin/sh: line 1: 6818 Killed /usr/local/cpanel/scripts/upcp --cron > /dev/nullThere are backups being uploaded to Amazon, so that seems to be working fine now, and I am not sure if it's related at all to the update process failing, but it did start right after that incident. If I ssh in and run "ps aux | grep upcp" this is what I see currently:
[root@vps ~]# ps aux | grep upcp
root 7638 0.0 0.0 113280 1196 ? Ss 03:40 0:00 /bin/sh -c (/usr/local/cpanel/scripts/fix-cpanel-perl; /usr/local/cpanel/scripts/upcp --cron > /dev/null)
root 7643 0.0 0.0 113280 568 ? S 03:40 0:00 /bin/sh -c (/usr/local/cpanel/scripts/fix-cpanel-perl; /usr/local/cpanel/scripts/upcp --cron > /dev/null)
root 7658 0.0 0.2 152344 10772 ? S 03:40 0:01 cPanel Update (upcp) - Master
root 7662 0.0 0.0 0 0 ? Z 03:40 0:00 [upcp]
root 13718 0.0 0.0 112808 972 pts/0 S+ 05:41 0:00 grep --color=auto upcp
Any thoughts where the hangup might be? Thanks. -Michael
-
Hey there! It might be best to kill all of those active upcp processes, and then force an update with "/scripts/upcp --force" As to the original issue, do you see anything in the logs in /var/cpanel/updatelogs? I know you mentioned you are getting a zero-byte message in the email notification, but there could be older versions that show more detail. 0 -
As to the original issue, do you see anything in the logs in /var/cpanel/updatelogs? I know you mentioned you are getting a zero-byte message in the email notification, but there could be older versions that show more detail.
The last update log was from 9/29 at 3:40am UTC, and seems to have completed successfully:[root@vps ~]# tail /var/cpanel/updatelogs/update.4679.715695505.1625729847.log [2021-09-29 03:40:37 +0000] 95% complete [2021-09-29 03:40:37 +0000] Running Standardized hooks [2021-09-29 03:40:37 +0000] 100% complete [2021-09-29 03:40:37 +0000] [2021-09-29 03:40:37 +0000] cPanel update completed [2021-09-29 03:40:37 +0000] A log of this update is available at /var/cpanel/updatelogs/update.4679.715695505.1625729847.log [2021-09-29 03:40:37 +0000] Removing upcp pidfile [2021-09-29 03:40:37 +0000] [2021-09-29 03:40:37 +0000] Completed all updates => Log closed Wed Sep 29 03:40:37 2021
Hey there! It might be best to kill all of those active upcp processes, and then force an update with "/scripts/upcp --force"
Trying that now, got this far and appears to just be hanging:[root@vps ~]# /scripts/upcp --force [2021-10-05 22:19:09 +0000] W previous PID (7658) has been running more than 6 hours. Killing processes. Use of uninitialized value $file in open at /usr/local/cpanel/Cpanel/LoadFile.pm line 47. Sending kill signal to process group for 7658 Waiting for processes to die Done! Stale PID file '/var/run/upcp.pid' (pid=7658) [2021-10-05 22:19:10 +0000] Detected cron=0 (--force passed on command line) Can't use an undefined value as a symbol reference at /scripts/upcp line 268.
Googling that error in quotes, minus the line number, brings up nothing. Also, I had already killed 7658, not sure why it said it was still running, I confirmed there were no more processes after I killed them all. -Michael0 -
Note: 6 hours later and no change, killing the process. Any thoughts on what to try next? -Michael 0 -
I don't have any other ideas on my end for that, as I also don't have anything on my side when I search for that particular error. Could you create a ticket with our team so we can check this directly on the system? If you can do that, just post the ticket number here so I can follow along. 0 -
@cPRex OPened, the ticket is #94370867. Thanks. -Michael 0 -
@cPRex They were able to fix it by running /scripts/rpmup
first, and then/scripts/upcp --force
ran just fine. -Michael0 -
Thanks for the update - I'm glad that's all it took! 0
Please sign in to leave a comment.
Comments
7 comments