Finding a spammer (or spam script) sometimes takes time and patience

Written by Claudio Kuenzler - 0 comments

Published on May 13th 2014 - Listed in Linux Hacks Internet Mail

---

For a couple of weeks now, a shared web server was continuously listed in the Spamhaus blacklist. After checking the mail logs on that server, no particular spamming could be identified.

I've already seen spamming scripts in the past, which created their own smtp connection (with fsockopen or whatever function), therefore bypassing the local mailserver. That was probably what was happening here too, but the difficult thing was to catch the spammer in the act! After multiple re-listings in the blacklist, I decided to launch a small script which writes the current smtp connections seen with lsof to a logfile. And this is the goal of this script:

#!/bin/sh
##########################
# lsofsmtp.sh
# purpose: output lsof in a logfile to verify it later

echo $(date) >> /var/log/netstatout.log
/usr/sbin/lsof -i -n | grep smtp >> /var/log/netstatout.log
echo "---------------------" >> /var/log/netstatout.log

exit 0

I started a tail -f on the logfile and of course during the whole day nothing appeared.

cat /var/log/netstatout.log

Mon May 5 17:29:01 CEST 2014
master     3213   root   11u  IPv4     8863      0t0  TCP 127.0.0.1:smtp (LISTEN)
---------------------
Mon May 5 17:30:01 CEST 2014
master     3213   root   11u  IPv4     8863      0t0  TCP 127.0.0.1:smtp (LISTEN)
---------------------
Mon May 5 17:31:01 CEST 2014
master     3213   root   11u  IPv4     8863      0t0  TCP 127.0.0.1:smtp (LISTEN)
---------------------
[...]
---------------------
Sat May 10 05:04:01 CEST 2014
master     3213   root   11u  IPv4     8863      0t0  TCP 127.0.0.1:smtp (LISTEN)
[...]

This went on and on, so I eventually forgot about the running script... until the server was blacklisted again.
I immediately verified the logfile and finally, on May 12th, the trap worked:

Mon May 12 15:38:01 CEST 2014
master     3213   root   11u  IPv4     8863      0t0  TCP 127.0.0.1:smtp (LISTEN)
perl      15208  httpd  510u  IPv4 60129149      0t0  TCP 192.168.55.55:56861->63.250.192.45:smtp (ESTABLISHED)
perl      15209  httpd  510u  IPv4 60129254      0t0  TCP 192.168.55.55:52481->63.250.192.45:smtp (ESTABLISHED)
perl      15211  httpd  510u  IPv4 60129038      0t0  TCP 192.168.55.55:49858->17.172.34.71:smtp (ESTABLISHED)
perl      15212  httpd  510u  IPv4 60129390      0t0  TCP 192.168.55.55:33054->98.138.112.35:smtp (SYN_SENT)
perl      15213  httpd  510u  IPv4 60129313      0t0  TCP 192.168.55.55:44641->65.54.188.110:smtp (ESTABLISHED)
perl      15214  httpd  510u  IPv4 60129331      0t0  TCP 192.168.55.55:41362->74.125.136.27:smtp (ESTABLISHED)
perl      15215  httpd  510u  IPv4 60129326      0t0  TCP 192.168.55.55:46223->64.12.88.131:smtp (ESTABLISHED)
perl      15216  httpd  510u  IPv4 60129206      0t0  TCP 192.168.55.55:35967->106.10.166.52:smtp (ESTABLISHED)
perl      15217  httpd  510u  IPv4 60129375      0t0  TCP 192.168.55.55:42439->65.54.188.110:smtp (ESTABLISHED)
perl      15218  httpd  510u  IPv4 60129282      0t0  TCP 192.168.55.55:49330->200.94.181.6:smtp (ESTABLISHED)

No big surprise - a perl script was doing the direct smtp connections, as I expected. As I couldn't use the mail logs to identify the source, I needed to analyze all web server access logs for this particular minute (more or less).
First I created a list of all log files being changed in the last 120 minutes:

find /var/www/*/logs/ -mmin -120 > /tmp/loglist

Then I grepped through all the found logs for one 15:37 to see what has caused the smtp connections:

for logfile in $(cat /tmp/loglist | grep -v referrer | grep -v error | grep -v agent); do echo $logfile; grep "12/May/2014:15:37" $logfile; echo "------------------------------------------------------"; done

Of course there were a few hundred entries, but each virtual host separated by a long line of dashes (making it optically easier to go through). It took me a few seconds until I found the following entries:

46.165.204.8 - - [12/May/2014:15:37:26 +0200] "GET /images.php HTTP/1.1" 200 22 "-" "Mozilla/5.0 (compatible; Goooglebot/2.1; +http://www.google.com/bot.html)"
46.165.204.8 - - [12/May/2014:15:37:26 +0200] "GET /images.php HTTP/1.1" 200 3 "-" "Mozilla/5.0 (compatible; Goooglebot/2.1; +http://www.google.com/bot.html)"
46.165.204.8 - - [12/May/2014:15:37:26 +0200] "GET /images.php HTTP/1.1" 200 3 "-" "Mozilla/5.0 (compatible; Goooglebot/2.1; +http://www.google.com/bot.html)"
46.165.204.8 - - [12/May/2014:15:37:27 +0200] "GET /images.php HTTP/1.1" 200 3 "-" "Mozilla/5.0 (compatible; Goooglebot/2.1; +http://www.google.com/bot.html)"
46.165.204.8 - - [12/May/2014:15:37:28 +0200] "GET /images.php HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Goooglebot/2.1; +http://www.google.com/bot.html)"

Additionally to the name "images.php" another information immediately told me that this must be it: Googlebot's IP starts with 66.249... and not with a 46 ipv4 address...

I checked out images.php, a file already uploaded months ago, and I was surprised. Very surprised. It was completely scrambled with nonsense and started like this:

<?php for($o=0,$e='&\'()*+,-.:]^_`{|,,,|-((.(*,|)`)&(_(*,+)`(

.. and ended like this:

$d='';@ord($e[$o]);$o++){if($o<16){$h[$e[$o]]=$o;}else{$d.=@chr(($h[$e[$o]]<<4)+($h[$e[++$o]]));}}eval($d); ?>

After some investigation it turned out, that this uploaded file was the same as this one: https://gist.github.com/laacz/5387180 

It would've been interesting to find out how this file was uploaded but unfortunately this web server only kept a few weeks of rotated logs.
The affected web site was using a non-updated CMS (WebSite X5 Evolution) which most likely had a vulnerability and therefore an invitation for a hacker.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.