Question
What to do if the "/scripts/find_and_fix_rpm_issues" command timeouts or doesn't respond?
Answer
If you run the following script by command-line on the server, you should receive some output, like so:
# /scripts/find_and_fix_rpm_issues
Checking that the RPM DB is OK...
glibc-2.17-317.el7.x86_64
OK: RPM DB is responding to queries
Testing if rpm_is_working RPM is installed
package rpm_is_working is not installed
Testing if it's possible to install a simple RPM
Preparing... ########################################
Updating / installing...
rpm_is_working-1.0-0 ########################################
However, if you do not get output for a long period of time (like over a minute), the output will look like this:
# /scripts/find_and_fix_rpm_issues
Checking that the RPM DB is OK...
There won't be any extra output.
We would recommend looking at the process list and seeing multiple copies of 'rpm' running simultaneously.
Example from test server:
# ps awux | grep rpm
root 7890 0.0 0.0 70848 3904 ? S 10:15 0:00 /usr/bin/rpm -q --nodigest --nosignature --queryformat %{NAME} %{VERSION}-%{RELEASE}\n -a kernelcare
root 12529 0.0 0.0 70848 1824 ? S Nov28 0:00 /usr/bin/rpm -q --nodigest --nosignature --queryformat %{NAME} %{VERSION}-%{RELEASE}\n -a kernelcare
root 13645 0.0 0.0 70848 652 ? S Oct19 0:00 /usr/bin/rpm -q --nodigest --nosignature --queryformat %{NAME} %{VERSION}-%{RELEASE}\n -a kernelcare
root 13926 0.0 0.0 70852 3920 pts/0 S 11:33 0:00 /bin/rpm -q --nosignature --nodigest glibc
root 14314 0.0 0.0 70848 944 ? S Nov14 0:00 /usr/bin/rpm -q --nodigest --nosignature --queryformat %{NAME} %{VERSION}-%{RELEASE}\n -a kernelcare
root 14434 0.0 0.0 70852 3928 pts/0 S 11:36 0:00 /bin/rpm -q --nosignature --nodigest glibc
root 14560 0.0 0.0 70852 3916 ? SN 11:38 0:00 /bin/rpm -qa --nodigest --nosignature --queryformat %{NAME}\t%{VERSION}\t%{RELEASE}\t%{VENDOR}\t%{INSTALLTIME}\t%{RSAHEADER:pgpsig}\n
root 14561 0.0 0.0 70852 3912 ? SN 11:38 0:00 /bin/rpm -qa --nodigest --nosignature --queryformat %{NAME}\t%{VERSION}\t%{RELEASE}\t%{VENDOR}\t%{INSTALLTIME}\t%{RSAHEADER:pgpsig}\n
root 14562 0.0 0.0 70852 3920 ? SN 11:38 0:00 /bin/rpm -qa --nodigest --nosignature --queryformat %{NAME}\t%{VERSION}\t%{RELEASE}\t%{VENDOR}\t%{INSTALLTIME}\t%{RSAHEADER:pgpsig}\n
root 15732 0.0 0.0 70852 3920 pts/0 S 11:48 0:00 /bin/rpm -q --nosignature --nodigest glibc
root 16080 0.0 0.0 70852 3924 pts/0 S 11:51 0:00 /bin/rpm -q --nosignature --nodigest glibc
root 16102 0.0 0.0 174572 4024 pts/0 S+ 11:51 0:00 rpm -qa
root 16625 0.0 0.0 73476 680 ? S Sep29
This indicates that 'rpm', which depends on locking the RPM database, competes with other copies of itself. All of these 'rpm' commands will not be complete as they will not timeout if there are too many.
We would recommend finding and killing those 'rpm' processes until they are all gone.
If you use the following command to 'kill' the process, please use caution when doing so.
Command:
kill -9 `ps awux | grep -v grep rpm | awk ' { print $2 } ' | xargs`
Once those 'rpm' are dead, you should be able to run the '/scripts/find_and_fix_rpm_issues' script successfully.
Command:
/scripts/find_and_fix_rpm_issues