Tracking down compromised files that are sending mail.

Introduction

This is a brief overview of how you can track down an account that has been compromised and sending spam on your server. 

Files will often be created in base64 when accounts are compromised in order to mask what they are doing. You can use a variety of tools to check if the file is in use. In our case, we'll use a script written by our cPanel staff and stat to check the modification time. By using the "modify" time, you can then check the access log in hopes of finding out what IP accessed the server and what specific file was affected. 

Procedure

While you may still wish to consult with your server administrator to investigate the mail abuse and potential security issues, here are some tips to help track this down.

First, navigate to the WHM, Main >> Service Configuration >> Exim Configuration Editor >> Advanced Editor tab >>  Combined Exim Configuration > Section: CONFIG > log_selector

For log_selector, change existing option:

+incoming_port +smtp_connection +subject +arguments +received_recipients -retry_defer

With one of the following two:

# Short version:

+all -ident_timeout -pid

or  # Long version:

+address_rewrite +all_parents +arguments +connection_reject +delay_delivery +delivery_size +dnslist_defer +incoming_interface +incoming_port +lost_incoming_connection +queue_run +received_sender +received_recipients +retry_defer +sender_on_delivery +size_reject +skip_delivery +smtp_confirmation +smtp_connection +smtp_protocol_error +smtp_syntax_error +subject +tls_cipher +tls_peerdn

Then, you'll be able to print the most common directories that direct Exim:

awk '$3 ~ /^cwd/{print $3}' /var/log/exim_mainlog | sort | uniq -c | sed "s|^ *||g" | sort -nr

You can usually ignore the main general directories, like from WHM or root. But if a user is sending large amounts of email from a script somewhere in their home folder, this will show it.  Another great command for tracking down the directories sending through Exim:

grep cwd /var/log/exim_mainlog | grep -v /var/spool | awk -F"cwd=" '{print $2}' | awk '{print $1}' | sort | uniq -c | sort -n

Directories mail is originating from:

908136 /home/abusiveuser/public_html/abusivemailfolder
 12636 /home/notasbad/public_html
 12476 /home/user2/public_html/iasset_old
 126 /home/user1/public_html

Next, using grep, search the folders for anything encoded for base64.

/home/abusiveuser/public_html/abusivemailfolder]cPs# grep -i base *
badmailfile.php:PD9waHAgaWYoaXNzZXQoJF9GSUxFU1sidXNlcmZpbGUi
..
..
..
IjsgZWNobyAiU3BpZGVyIFByb2plY3QiOyBleGl0OyA/Pg base64 --decode

Once you find a base that's been encoded, use stat to check the modified time.

/home/abusiveuser/public_html/abusivemailfolder]cPs# stat badmailfile.php
  File: `badmailfile.php'
  Size: 1271          Blocks: 8          IO Block: 4096  regular file
Device: fd02h/64770d    Inode: 33423367    Links: 1
Access: (0644/-rw-r--r--)  Uid: (  527/abusiveuser)  Gid: (  525/abusiveuser)
Access: 2020-06-25 17:39:02.539000002 -0600
Modify: 2020-05-25 13:31:30.276000041 -0600
Change: 2020-05-25 13:31:30.276000041 -0600

There may be other base64 files, so you'll want to check those as well and run stat on each one noting the "Modify" time. 

Modify: 2020-05-25 13:31:30.276000041 -0600

If the date is older than one month, you will need to look through the archived logs.

/home/abusiveuser/archived-logs

If it's within the month, you can look at the access logs under 

/home/abusiveuser/access-logs 

You'll be looking for a POST command in the log. Once you find the entry that accessed the domain, you'll know the IP and file that was used to exploit the account

Once you discover how the account was compromised, and you have backups, you'll want to restore the account and make the necessary changes to prevent the site from being compromised again.