Stale repeating log entries in /var/log/secure
Server has unexplained entries in /var/log/secure.
It looks like on a weekly basis, PAST entries for pam_unix and unix_chkpwd get dumped into the /var/log/secure file.
See after the first 2 proper entries, some old items are dumped in
[root@to jon9n7]# cat /var/log/secure
Nov 13 05:21:14 to atd[30306]: pam_unix(atd:session): session opened for user root by (uid=0)
Nov 14 02:35:23 to sshd[1308]: pam_unix(sshd:session): session closed for user jon9n7
Sep 23 02:43:18 to su: pam_unix(su:session): session opened for user root by jon9n7(uid=1001)
Sep 23 06:51:16 to su: pam_unix(su:session): session closed for user root
Sep 23 15:57:18 to su: pam_unix(su:session): session opened for user root by jon9n7(uid=1001)
Sep 24 01:06:20 to su: pam_unix(su:session): session closed for user root
Sep 24 10:40:41 to su: pam_unix(su:auth): authentication failure; logname=jon9n7 uid=1001 euid=0 tty=pts/0 ruser=jon9n7 rhost= user=root
Sep 24 10:40:53 to su: pam_unix(su:session): session opened for user root by jon9n7(uid=1001)
Sep 27 13:48:36 to su: pam_unix(su:session): session opened for user root by jon9n7(uid=1001)
Sep 28 00:16:01 to unix_chkpwd[28]: check pass; user unknown
Sep 28 00:16:09 to unix_chkpwd[29]: check pass; user unknown
[[ BUNCH MORE REMOVED ]]
Nov 7 23:29:08 to su: pam_unix(su:session): session closed for user root
Nov 8 20:07:24 to su: pam_unix(su:auth): authentication failure; logname=jon9n7 uid=1001 euid=0 tty=pts/0 ruser=jon9n7 rhost= user=root
Nov 8 20:14:15 to su: pam_unix(su:session): session opened for user root by jon9n7(uid=1001)
Nov 9 03:11:21 to su: pam_unix(su:session): session closed for user root
Nov 12 16:24:33 to su: pam_unix(su:session): session opened for user root by jon9n7(uid=1001)
Nov 14 02:35:23 to su: pam_unix(su:session): session closed for user root
Nov 14 05:21:16 to atd[17367]: pam_unix(atd:session): session opened for user root by (uid=0)
Nov 14 12:57:12 to sshd[1050]: Accepted password for jon9n7 from xx.xx.xx.xx port 9999 ssh2
Nov 14 12:57:12 to sshd[1050]: pam_unix(sshd:session): session opened for user jon9n7 by (uid=0)
A chunk of entries beginning with "Sep 23" were repeatedly inserted on Oct 18, Oct 19, Oct 19, Oct21,Oct 21,Oct 27, Nov 4, Nov 14. The "chunk" is growing as more entries accumulate in whatever log they originally came from. The dates and times are not consistent so they don't appear to be related to any cron. At this point, we know it was sometime after 2:35am and before 4:00am lfd detects these entries when it runs and sends a "su login failed" email for each auth failure in the chunk, though they aren't "new" activity. The question is how/why are these past entries being randomly copied to the /var/log/secure?
A chunk of entries beginning with "Sep 23" were repeatedly inserted on Oct 18, Oct 19, Oct 19, Oct21,Oct 21,Oct 27, Nov 4, Nov 14. The "chunk" is growing as more entries accumulate in whatever log they originally came from. The dates and times are not consistent so they don't appear to be related to any cron. At this point, we know it was sometime after 2:35am and before 4:00am lfd detects these entries when it runs and sends a "su login failed" email for each auth failure in the chunk, though they aren't "new" activity. The question is how/why are these past entries being randomly copied to the /var/log/secure?
-
I've been seeing the same thing for more than a month. I first noticed it happening when I did a graceful server reboot after a WHM update, but lately it's been happening at random times. I opened a ticket with my server provider, who opened a ticket with cPanel, who said "That seems to be an issue with syslog and not one that would be caused by cPanel or the basic configuration of the cPanel-bundled software." The server provider then updated the system kernel (two days ago), and I haven't seen any stale pam_unix entries since then, but a little while ago I got a chunk of stale ssh "refused connect from" entries instead, along with a bunch of corresponding lfd reports. FWIW I have two other cPanel servers at a different provider but haven't seen this problem there. All three run CentOS Linux release 7.4.1708 and cPanel v66.0.29 or v66.0.30. At this point everybody seems mystified. 0 -
Hi @ottdev, Could you open a support ticket using the link in my signature so we can take a closer look and rule out any issues with the cPanel software itself? Thank you. 0 -
It's a bug with some RHEL Bug 1216957 " rsyslog restart pulls lots of older log entries again, runs into rate-limiting fixed with RHEA-2016:2401 Red Hat Customer Portal and must affect GNU Linux too since we have the same issue... 0 -
Hello, It looks like CentOS has already published the updated rsyslog RPM: # rpm -q --changelog rsyslog-8.24.0-12.el7.x86_64|grep 1216957 resolves: rhbz#1216957
Thank you.0
Please sign in to leave a comment.
Comments
4 comments