Mail Server Administrators: Need your help

[Freejack2]

My employer has 2 outbound mail servers and several times a year they get blacklisted for spam. We have taken several steps and gotten it down immensely but in order to get it to near 0 has us stumped.

From what I understand if we get blocked because spamcop got a spam mail on their secret accounts we will get no information from spamcop why we are blocked. So this means the only way for us to find out the spammer is to summarize our logs.

One thought has been to try and track what users are sending to whom and keep a tally of who is sending to whom. The idea is to have an idea of who is sending massive amounts of email a day and then look at those customers to see if they are spamming people. The problem is the logs the mail server generates are huge. Our daily logs range from 40 to 100megabytes. My boss wrote a query that would summarize these logs but because of the way the logs are it's a multiple step query and we estimated it would take a box costing 5 figures just to process the logs into a summary format.

I know there is no budget for a 5 figure cost system so we have to find another solution. Has anyone else gone through this or have any suggestions on how to do this within our limited budget?

 

[bwnv]

Usually you get blocked for being an "open relay". Don't have any experience myself, but you should be able to get lots of info from google? Also, the fastest AMD 64 bit chips are faster than any 5 figure computer that you could currently get. You could even build a 4 way server by using 2 dual core procs. ( and probably for around $2000 - $3000 bucks)

 

[Rookie]

bmnv is correct. Usually this is due to being an open relay. This means people can use your mail server to connect to and send mail (IIRC)

Implement a firewall that only allows specified SMTP relays and denies the rest.

 

[wfay]

Originally posted by: Freejack2
The problem is the logs the mail server generates are huge. Our daily logs range from 40 to 100megabytes. My boss wrote a query that would summarize these logs but because of the way the logs are it's a multiple step query and we estimated it would take a box costing 5 figures just to process the logs into a summary format.

No offense, but Java, Perl, etc should be able to "process" the 100mb log into a summary format in a matter of minutes. 100mb log files are tiny. Perl was DESIGNED from the start to do nothing but process text.

I would consider writing code to break up each line in the file into a series of columns and dump them into a Mysql database, then write some SQL queries to generate the summary for you. I'm sure you have a Mysql or MS SQL database around somewhere.

 

[Freejack2]

Thanks wfay, I'll run this by my boss.

As for the relays, we don't have any security issues like that. The spam is most likely coming from either a business customer deliberately doing it or home customers on dsl or dialup have comprimised machines that spammers are using as mailing machines. We block port 25 so if it is comprimised systems the spammers have to use our outgoing mail servers.

We're hoping that by having some kind of statistics compiled out of the logs we can attempt to find out who is doing the spamming.

 

[sourceninja]

what mail server software are you running? What logger are you running? If this linux, you can get some very detailed reporting by using a different logger such as metalog or syslog-ng to parse up your logs based on a lot of criteria. Then you could write a few quick scripts to search them or even use perl regular expressions directly inside metalog to create a second file of things that match your search. Hell, I've even setup syslog-ng for a client to pipe its output directly into a postgres database for even easier searching.

A 100 meg log file should be very easy to parse. But with a good logger like metalog or syslog-ng you can break that log up into multiple files based on your criteria very easy. Then parse them out. You could also have a script that monitors the log file and parses each line as it comes in looking for patterns you think are linked to spam.

Things to look for:

Massive amount of email from a single address
people faking return addresses
huge BCC lists
people faking header information


Those are the 3 easiest things you can look for.

If you do not have the resources in house to setup scripts to parse these logs, perhaps bringing in someone on contract from the outside might be cheaper. But the tools are there. At least if you guys are running linux/unix.

 

[Genx87]

After finding out our domain was being blocked by a certain blacklist I did some digging and found how easily it is to put somebody into one of these lists. It is kind of a wild wild west where you are guilty until proven innocent.

In our case we send out a mass mailing about once every 3 months to customers letting them know what we are upto. Somebody on the list forwarded our email to the blacklisting site and blamo we are on it and anybody using that blacklist wont recieve emails from us unless they make a rule.

 

[Freejack2]

I looked and the boxes are running linux. Judging by the core version I'd say it's probably Centos 4.1. It might be a Fedora core but I tend to doubt it.
Turns out my boss didn't come in today so I'll have to ask him these things tomorrow.
Thanks for all the help so far.