# Spamassassin statistics

## Vermyndax

Hey all... was just wondering if anyone had implemented any scripts or log utilities to generate a report about spamd (Spamassassin) activity.  If so, what did you use?  I tried sa-stats.pl but it returns all 0's for every stat and I'm tired of trying to figure out why (Googling didn't help... several people asked about it, no one answered).

So... any of you use anything to analyze this and make a nice report?

----------

## Jaxom

I'll chime in here and add a "yes, that would be great"   :Smile: 

I don't know of anything that will do it though, but I sure would like to see how much stuff SA is actually stopping.  I know I can see it in MY mail, but that doesn't tell me anything about how much it's helping my users.

----------

## Vermyndax

It would be even better if it was a script that generated an html page as output so I can show it off to the bosses  :Smile: 

----------

## Vermyndax

I did find a solution to this with a sa-stats.pl script.  However, now I've switched to amavisd-new with SA and ClamAV, so it kills the sa-stats.pl's abilities to read the log file (since amavisd doesn't allow spamd to write to the log file anymore).

Anyone know of a quickie way to get amavisd to allow spamd to write to the syslog as usual?

----------

## Torg

I wrote my own in sed/awk/bash  it's a mess, and someone else could prolly write cleaner code.... but in the effort of open source, here it is:

```

#!/bin/bash

# Todd's SpamAssassin Stats Script v0.27t

# Much help was gained from O'Reilly's Sed & Awk book, and the Advanced Bash Scripting Guide.  VERY good reads for this sort of task.

# File Location Vars:

templog=/spare/logtemp/maillogtrimmed.log

quickstats=/home/httpd/htdocs/logs/quickstats.txt

sourcecode=/home/httpd/htdocs/logs/source.txt

scores=/home/httpd/htdocs/logs/scores.txt

# Main Code:

if [ -e $templog ]; then

rm $templog

fi

cat /var/log/mail.log | sed -e 's/  / /g' | grep "identified" > $templog

cat /var/log/mail.log | sed -e 's/  / /g' | grep "clean" >> $templog

if [ -e $quickstats ]; then

rm $quickstats

fi

echo "Quickstats for Torgbox generated at `date | awk '{print $4 " - " $1 ", " $2 " " $3 " " $6}'`" > $quickstats

echo >> $quickstats

echo "Today's filtered spam:" >> $quickstats

cat $templog | grep "`date | awk '{print $2 " " $3}'`" | grep -c "identified" >> $quickstats

echo >> $quickstats

echo "Today's unfiltered mail:" >> $quickstats

cat $templog | grep "`date | awk '{print $2 " " $3}'`" | grep -c "clean" >> $quickstats

echo >> $quickstats

cat $templog | grep "`date | awk '{print $2 " " $3}'`" |grep "clean" | awk '{print $8}' | sed -e "s/\// /g ; s/(//g; s/)//g" |awk '{print $1}' | grep "-" | sed -e "s/-//g" | awk '{if ($1 > 50) totalbr ++};END {printf ("%s""%d\n\n","Whitelisted mail so far today: ",totalbr)}' >> $quickstats

echo "Today's spams by username:" >> $quickstats

cat $templog | grep identified | grep "`date | awk '{print $2 " " $3}'`" | awk '{print $10}' | sed -e 's/:/ /g' | awk '{print $1}' | sort | uniq -c | sort -r >> $quickstats

echo >> $quickstats

cat $templog | grep "identified" | grep "`date | awk '{print $2 " " $3}'`" | awk '{ sum += $14 };{size = sum / 1048576}; END {printf ("%24s" "%.2f""%2s\n","Spam size so far today: ",size,"Mb")}' >> $quickstats

echo >> $quickstats

echo "Logfile total spams filtered:" >> $quickstats

cat $templog | grep -c "identified" >> $quickstats

echo >> $quickstats

echo "Logfile total unfiltered mail:" >> $quickstats

cat $templog | grep -c "clean" >> $quickstats

echo >> $quickstats

cat $templog | grep "clean" | awk '{print $8}' | sed -e "s/\// /g ; s/(//g; s/)//g" |awk '{print $1}' | grep "-" | sed -e "s/-//g" | awk '{if ($1 > 50) totalbr ++};END {printf ("%s""%d\n\n","Logfile total whitelisted mail: ",totalbr)}' >> $quickstats

echo "Logfile total spams by username:" >> $quickstats

cat $templog | grep identified | awk '{print $10}' | sed -e 's/:/ /g' | awk '{print $1}' | sort | uniq -c | sort -r >> $quickstats

echo >> $quickstats

cat $templog | grep "identified" | awk '{sum += $14};{size = sum / 1048576}; END {printf ("%17s""%.2f""%2s\n","Total spam size: ",size,"Mb")}' >> $quickstats

cat $templog | grep "identified" | awk '{print $8}' | sed -e "s/\// /g ; s/(//g; s/)//g" | awk '{print $1}'| awk '{sum += $1};{total ++};{avg = sum / total};END{printf("%20s""%.1f\n","Average spam score: ",avg)}' >> $quickstats

if [ -e $scores ]; then

rm $scores

fi

echo "Spams sorted by score:" > $scores

cat $templog  | grep "identified" | awk '{print $8}' | sed -e 's/\/5.0)//g' | sed -e 's/(//g' | sort | uniq -c | sort -r >> $scores

echo >> $quickstats

echo >> $quickstats

echo "Percentages of spam and non-spam:" >> $quickstats

cat $templog  | awk '{ print $6 }' | awk ' /^identified$/ {spam++ total++} /^clean$/ {notspam++ total++} {percentspam = spam / total * 100;percentnot = notspam / total * 100 };END {print percentspam "% SPAM \n" percentnot "% NON-SPAM \n" spam " Spams / " notspam " Non-spams / " total " Total Emails"}' >> $quickstats

if [ -e $sourcecode ]; then

rm $sourcecode

fi

cat /root/makesalog > $sourcecode

```

If you need help making heads or tails of it, just say so... I wrote the whole thing, so I know how it all works.

-Torg

----------

## Torg

output looks like this:

```

Quickstats for Torgbox generated at 14:27:14 - Tue, Feb 10 2004

Today's filtered spam:

0

Today's unfiltered mail:

0

Whitelisted mail so far today: 0

Today's spams by username:

Spam size so far today: 0.00Mb

Logfile total spams filtered:

4550

Logfile total unfiltered mail:

18225

Logfile total whitelisted mail: 1565

Logfile total spams by username:

   4298   safrax

    252   torg

Total spam size: 25.99Mb

Average spam score: 29.5

Percentages of spam and non-spam:

19.978% SPAM 

80.022% NON-SPAM 

4550 Spams / 18225 Non-spams / 22775 Total Emails

```

This is from a smaller install of this script.... I've got it running on a 124 user mailserver at work also. (Little more interesting to read, but has too many usernames in it for me to print it here =P)

Unfortunatly, there isn't much traffic on this example, but you'll get the idea.

-Torg

----------

## Skywacker

works great, ty  :Smile: 

any of have ideas or know of a program that does the same with some kind of html output over time?

_Skywacker

----------

## deurk

I'll hop in here and bump it once again...

This script is nice but messy and no html output, anyone got new ideas?

Thanks!

----------

## FutureGuru

Have you tried to implement options to the script? So that for example -html option generates new html file and puts all messages to it. 

```

if [ $# == 0]

then

(copy-paste your current script)

else if [ $# == 1 ]

then

if [ $1 == '-html' ]

then

(create file and put all writings to it)

fi

else

echo "Usage: xxxxxxx"

fi

```

I know this is not correct form as I don't have change to check all that, but I hope that this example shows my point.

----------

## georwell

http://www.while.homeunix.net/mailstats/

Will this help?

----------

## Dr_Stein

No, that won't help - it requires mailscanner. 

The OP said he's using amavisd-new

mailgraph might work for you..

http://mail.pbp.net/cgi-bin/mailgraph.cgi <-- my installation

It's pretty easy to get going. No ebuild for it, but it only consists of like 2 files anyway.  :Wink:  I also have it running on my mail gateway box, but HTTP isn't allowed to it from the outside.  :Smile: 

----------

## alterself.com

mailgraph and amavis-stats are rrdtool based nice boss pleasing html based pretty graphs web pages

----------

