Yum hangs following upcp Python update

Hello, Yesterday yum installed some Python packages during the nightly upcp run. It could be a red herring and the issue lies with the CLN but clearly conjecture. I am unable to successfully complete any yum transactions on the cli, any attempt hangs. Here's what I know. UPCP log reports updates to Python. 49 [2021-04-24 04:21:06 +1000] - Processing command `/usr/local/cpanel/scripts/rpmup` 50 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] checkyum version 22.3 (excludes: bind-chroot kernel) 51 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Loaded plugins: fastestmirror, rhnplugin, universal-hooks 52 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Setting up Update Process 53 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Loading mirror speeds from cached hostfile 54 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] * EA4: 27.50.85.8 55 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] * cpanel-addons-production-feed: 27.50.85.8 56 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] * cpanel-plugins: 27.50.85.8 57 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] * cloudlinux-x86_64-server-6: cl-mirror.syd.ventraip.net.au 58 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Resolving Dependencies 59 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] --> Running transaction check 60 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package cloudlinux-release.x86_64 0:6-6.10.2.el6 will be updated 61 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package cloudlinux-release.x86_64 0:6-6.10.3.el6 will be an update 62 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python.x86_64 0:2.6.6-68.el6_10 will be updated 63 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python.x86_64 0:2.6.6-69.el6.cloudlinux.els6 will be an update 64 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-devel.x86_64 0:2.6.6-68.el6_10 will be updated 65 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-devel.x86_64 0:2.6.6-69.el6.cloudlinux.els6 will be an update 66 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-libs.x86_64 0:2.6.6-68.el6_10 will be updated 67 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-libs.x86_64 0:2.6.6-69.el6.cloudlinux.els6 will be an update 68 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-tools.x86_64 0:2.6.6-68.el6_10 will be updated 69 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-tools.x86_64 0:2.6.6-69.el6.cloudlinux.els6 will be an update 70 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package screen.x86_64 0:4.0.3-19.el6 will be updated 71 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package screen.x86_64 0:4.0.3-20.el6.cloudlinux.els6 will be an update 72 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package tkinter.x86_64 0:2.6.6-68.el6_10 will be updated 73 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package tkinter.x86_64 0:2.6.6-69.el6.cloudlinux.els6 will be an update 74 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] --> Finished Dependency Resolution 75 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 76 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 77 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Total download size: 7.2 M 78 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Downloading Packages: 79 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] -------------------------------------------------------------------------------- 80 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Total 6.9 MB/s | 7.2 MB 00:01 81 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Running rpm_check_debug 82 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Running Transaction Test 83 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Transaction Test Succeeded 84 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Running Transaction 85 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 86 [2021-04-24 04:23:20 +1000] Updating : python-2.6.6-69.el6.cloudlinux.els6.x86_64 1/14 87 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 88 [2021-04-24 04:23:20 +1000] Updating : python-libs-2.6.6-69.el6.cloudlinux.els6.x86_64 2/14 89 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 90 [2021-04-24 04:23:20 +1000] Updating : tkinter-2.6.6-69.el6.cloudlinux.els6.x86_64 3/14 91 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 92 [2021-04-24 04:23:20 +1000] Updating : python-tools-2.6.6-69.el6.cloudlinux.els6.x86_64 4/14 93 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 94 [2021-04-24 04:23:20 +1000] Updating : python-devel-2.6.6-69.el6.cloudlinux.els6.x86_64 5/14 95 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 96 [2021-04-24 04:23:20 +1000] Updating : cloudlinux-release-6-6.10.3.el6.x86_64 6/14 97 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 98 [2021-04-24 04:23:20 +1000] Updating : screen-4.0.3-20.el6.cloudlinux.els6.x86_64 7/14 99 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 100 [2021-04-24 04:23:20 +1000] Cleanup : python-devel-2.6.6-68.el6_10.x86_64 8/14 101 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 102 [2021-04-24 04:23:20 +1000] Cleanup : python-tools-2.6.6-68.el6_10.x86_64 9/14 103 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 104 [2021-04-24 04:23:20 +1000] Cleanup : cloudlinux-release-6-6.10.2.el6.x86_64 10/14 105 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 106 [2021-04-24 04:23:20 +1000] Cleanup : tkinter-2.6.6-68.el6_10.x86_64 11/14 107 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 108 [2021-04-24 04:23:20 +1000] Cleanup : python-2.6.6-68.el6_10.x86_64 12/14 109 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 110 [2021-04-24 04:23:20 +1000] Cleanup : python-libs-2.6.6-68.el6_10.x86_64 13/14 111 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 112 [2021-04-24 04:23:20 +1000] Cleanup : screen-4.0.3-19.el6.x86_64 14/14 113 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 114 [2021-04-24 04:23:20 +1000] Verifying : screen-4.0.3-20.el6.cloudlinux.els6.x86_64 1/14 115 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 116 [2021-04-24 04:23:20 +1000] Verifying : python-libs-2.6.6-69.el6.cloudlinux.els6.x86_64 2/14 117 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 118 [2021-04-24 04:23:20 +1000] Verifying : python-2.6.6-69.el6.cloudlinux.els6.x86_64 3/14 119 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 120 [2021-04-24 04:23:20 +1000] Verifying : tkinter-2.6.6-69.el6.cloudlinux.els6.x86_64 4/14 121 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 122 [2021-04-24 04:23:20 +1000] Verifying : python-devel-2.6.6-69.el6.cloudlinux.els6.x86_64 5/14 123 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 124 [2021-04-24 04:23:20 +1000] Verifying : cloudlinux-release-6-6.10.3.el6.x86_64 6/14 125 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 126 [2021-04-24 04:23:20 +1000] Verifying : python-tools-2.6.6-69.el6.cloudlinux.els6.x86_64 7/14 127 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 128 [2021-04-24 04:23:20 +1000] Verifying : python-libs-2.6.6-68.el6_10.x86_64 8/14 129 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 130 [2021-04-24 04:23:20 +1000] Verifying : python-devel-2.6.6-68.el6_10.x86_64 9/14 131 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 132 [2021-04-24 04:23:20 +1000] Verifying : cloudlinux-release-6-6.10.2.el6.x86_64 10/14 133 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 134 [2021-04-24 04:23:20 +1000] Verifying : screen-4.0.3-19.el6.x86_64 11/14 135 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 136 [2021-04-24 04:23:20 +1000] Verifying : python-2.6.6-68.el6_10.x86_64 12/14 137 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 138 [2021-04-24 04:23:20 +1000] Verifying : tkinter-2.6.6-68.el6_10.x86_64 13/14 139 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 140 [2021-04-24 04:23:20 +1000] Verifying : python-tools-2.6.6-68.el6_10.x86_64 14/14 141 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] 142 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Complete! 143 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] checkyum version 22.3 (excludes: bind-chroot) 144 [2021-04-24 04:23:20 +1000] - Finished command `/usr/local/cpanel/scripts/rpmup` in 133.717 seconds 145 [2021-04-24 04:23:20 +1000] 26% complete 146 [2021-04-24 04:23:20 +1000] - Finished in 133.717 seconds The packages updated successfully and the installs are confirmed in the logs. Packages Updated: python-tools-2.6.6-69.el6.cloudlinux.els6.x86_64 python-libs-2.6.6-69.el6.cloudlinux.els6.x86_64 python-2.6.6-69.el6.cloudlinux.els6.x86_64 python-devel-2.6.6-69.el6.cloudlinux.els6.x86_64 cloudlinux-release-6-6.10.3.el6.x86_64 Wind forward 24 hours and there are obvious comm's errors as shown in the next upcp run. [2021-04-25 04:21:06 +1000] - Processing command `/usr/local/cpanel/scripts/find_and_fix_rpm_issues` 40 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] Checking that the RPM DB is OK... 41 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] glibc-2.12-1.212.el6_10.3.x86_64 42 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] OK: RPM DB is responding to queries 43 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] Testing if rpm_is_working RPM is installed 44 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] package rpm_is_working is not installed 45 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] Testing if it's possible to install a simple RPM 46 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] Preparing... ################################################## 47 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] rpm_is_working ################################################## 48 [2021-04-25 04:21:26 +1000] 25% complete 49 [2021-04-25 04:21:26 +1000] - Finished in 19.973 seconds 50 [2021-04-25 04:21:26 +1000] Processing: Running rpmup (yum -y update) 51 [2021-04-25 04:21:26 +1000] - Processing command `/usr/local/cpanel/scripts/rpmup` 52 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] checkyum version 22.3 (excludes: bind-chroot kernel) 53 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] Loaded plugins: fastestmirror, rhnplugin, universal-hooks 54 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] An error has occurred: 55 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] There was an error communicating with CLN. 56 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] Setting up Update Process 57 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] Loading mirror speeds from cached hostfile 58 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] * EA4: 27.50.85.8 59 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] * cpanel-addons-production-feed: 27.50.85.8 60 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] * cpanel-plugins: 27.50.85.8 61 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] No Packages marked for Update 62 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] checkyum version 22.3 (excludes: bind-chroot) 63 [2021-04-25 04:34:53 +1000] - Finished command `/usr/local/cpanel/scripts/rpmup` in 807.006 seconds This error was not reported and I only came across it when examining logs following an attempt to check a kernel update. I have attempted to run several yum related commands and each hang with no progress.

• yum update kernel
• yum clean all
• yum list all

I never get past Loaded plugins: fastestmirror, rhnplugin, universal-hooks in each case. The python related calls are timing out and can be seen in /var/log/up2date [Sun Apr 25 05:13:58 2021] up2date Updating package profile 97 [Sun Apr 25 09:12:41 2021] up2date A socket error occurred: Timeout Exception, attempt #1 98 [Sun Apr 25 09:12:41 2021] up2date Error communicating with server. The message was: 99 Connection timed out on readline 100 [Sun Apr 25 09:12:41 2021] up2date 101 Traceback (most recent call last): 102 File "/usr/share/rhn/up2date_client/rhnreg.py", line 212, in prepareGradualRolloutCertsIfNeeded 103 result = s.up2date.getCLNCertificate(systemId) 104 File "/usr/share/rhn/up2date_client/rhnserver.py", line 50, in __call__ 105 return rpcServer.doCall(method, *args, **kwargs) 106 File "/usr/share/rhn/up2date_client/rpcServer.py", line 288, in doCall 107 raise up2dateErrors.CommunicationError(e.args[0]) 108 up2date_client.up2dateErrors.CommunicationError: Error communicating with server. The message was: 109 Connection timed out on readline Needs Restarting /usr/bin/needs-restarting 3746 : /usr/bin/python/usr/sbin/tuned-d-c/etc/tuned.conf Any ideas on this please. Thanks!