# Seeking advice on a spamfiltering solution

## Del Pede

I've decided to add spamfiltering to my postfix server, since the amount of spam is increasing. I run a postfix server, and courier-imap, and it only sorts mail for one domain, and it's all based on users .maildir.

I've been looking at both dspam and spamassasin, but really don't know what to chose, though i'd prefer not to end up with a solution, that isn't dependent of mysql

Thanks in advanced

Del Pede

----------

## col

1 : greylisting

2 : blacklists = spamhaus, dsbl, spamcop njabl 

3 : spamassassin & clamav

should get rid of most spam

----------

## Enverex

I'm using Spamassassin with Razor, spamhaus, dsbl, spamcop njabl, etc and to be honest, it's not stopping a lot of the spam these days as it's simply not finding anything wrong with them (only minor things like "Message in HTML" or "40k picture with only 80 characters of text" etc which you can't base blocks on). So it's quite worrying how ineffective the best methods have become.

----------

## VanDan

I just finished setting up a mail system for our home network, and it works so well I'm upgrading the work server as I type  :Smile: 

I've got postfix ==> dspam ==> dbmail

It's a real pain in the arse to set up, but once it's set up, it's very, very nice. I have it so there are 2 global addresses for training: spam@<domain> and ham@<domain>. I also have a global access account so I can view / clear the quarantined messages. I can post config files and stuff if you like. I was using sendmail ==> canit ==> courier-imap, but each piece of the system was starting to piss me off in various ways ( sendmail a nightmare to maintain, canit starting to let through more spam, courier-imap not scaling well ).

I swear by this new combo though  :Smile: 

----------

## Del Pede

 *VanDan wrote:*   

> I just finished setting up a mail system for our home network, and it works so well I'm upgrading the work server as I type 
> 
> I've got postfix ==> dspam ==> dbmail
> 
> It's a real pain in the arse to set up, but once it's set up, it's very, very nice. I have it so there are 2 global addresses for training: spam@<domain> and ham@<domain>. I also have a global access account so I can view / clear the quarantined messages. I can post config files and stuff if you like. I was using sendmail ==> canit ==> courier-imap, but each piece of the system was starting to piss me off in various ways ( sendmail a nightmare to maintain, canit starting to let through more spam, courier-imap not scaling well ).
> ...

 

I'd love too glance at your config files, that would be much appreciated  :Very Happy: 

I've actually been thinking about migrating from courier-imapd to dovecot, but unfortunately i don't have a test server to crah run on.

Are you setup dependent on a running mysql?

----------

## magic919

I run Postfix with DSPAM and serve up mail with Dovecot.  I use DSPAM with MySQL but that's not the only storage option.  Needs to be trained but quickly adapts to new spam trends.  I don't get tons of messages at home and I'm running about 96% accuracy after 7000 messages.  It has captured 4 messages in error.  That's not bad.

----------

## VanDan

Here come the config files ... minus comments and whitespace.

/etc/postfix/main.conf:

```
queue_directory = /var/spool/postfix

command_directory = /usr/sbin

daemon_directory = /usr/lib/postfix

mail_owner = postfix

mydomain = entropy.homelinux.org

myorigin = $mydomain

inet_interfaces = $myhostname, localhost

mydestination = $myhostname, entropy.homelinux.org

local_recipient_maps = mysql:/etc/postfix/sql-recipients.cf

unknown_local_recipient_reject_code = 550

mynetworks = 192.168.1.0/24 entropy.homelinux.org

relay_domains = *.entropy.homelinux.org

transport_maps = hash:/etc/postfix/transport

mailbox_transport = dbmail-lmtp:127.0.0.1:24

debug_peer_level = 2

debugger_command =

         PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin

         xxgdb $daemon_directory/$process_name $process_id & sleep 5

sendmail_path = /usr/sbin/sendmail

newaliases_path = /usr/bin/newaliases

mailq_path = /usr/bin/mailq

setgid_group = postdrop

html_directory = /usr/share/doc/postfix-2.3.4/html

manpage_directory = /usr/share/man

sample_directory = /etc/postfix

readme_directory = /usr/share/doc/postfix-2.3.4/readme

home_mailbox = .maildir/

dspam_destination_recipient_limit = 1

dspam-spam_destination_recipient_limit = 1

dspam-ham_destination_recipient_limit = 1

```

For /etc/postfix/master.cf I added the following stuff down the bottom of the config file, and left everything else as-is:

```

# DSpam

dspam           unix    -       n       n       -       -       pipe

  flags=Rhq user=dspam:dspam argv=/usr/bin/dspam --deliver=innocent --user ${recipient}

dspam-spam      unix    -       n       n       -       -       pipe

  flags=Rhq user=dspam:dspam argv=/usr/bin/dspam --user ${recipient} --class=spam --source=error

dspam-ham       unix    -       n       n       -       -       pipe

  flags=Rhq user=dspam:dspam argv=/usr/bin/dspam --user ${recipient} --class=innocent --source=error

# DBMail transport - for delivering to storage

dbmail-lmtp     unix    -       -       n       -       -       lmtp

#SMTP daemon listening on port 10025 for filtered mail from dspam

127.0.0.1:10025 inet    n       -       n       -       -       smtpd

  -o smtpd_authorized_xforward_hosts=127.0.0.0/8

  -o smtpd_client_restrictions=

  -o smtpd_helo_restrictions=

  -o smtpd_sender_restrictions=

  -o smtpd_recipient_restrictions=permit_mynetworks,reject

  -o mynetworks=127.0.0.0/8

  -o receive_override_options=no_unknown_recipient_checks

```

/etc/postfix/sql-recipients.cf:

```
user = dbmail_user_name_replace_me

password = dbmail_password_replace_me

host = 127.0.0.1

dbname = dbmail

table = dbmail_aliases

select_field = alias

where_field = alias

```

/etc/postfix/transport:

```
spam@entropy.homelinux.org      dspam-spam:

ham@entropy.homelinux.org       dspam-ham:

entropy.homelinux.org           dspam:
```

I then had to run 'postmap' on the transport file to create the db file that postfix uses.

Important parts ( not all ) of /etc/dbmail/dbmail.conf:

```

driver          = mysql

authdriver      = sql

host            = localhost

sqlport         =                

sqlsocket       = /var/run/mysqld/mysqld.sock

user            = dbmail_username_change_me

pass            = dbmail_password_change_me

db              = dbmail               

table_prefix    = dbmail_   

sendmail        = /usr/sbin/sendmail     

[LMTP]

PORT            = 24                 

[POP]

[IMAP]

PORT            = 143                

TIMEOUT         = 4000            

[SIEVE]

PORT            = 2000  # ****** This was NOT the default, but this setting works with smartsieve
```

/etc/dspam/dspam.conf:

```
Home /var/spool/dspam

StorageDriver /usr/lib/dspam/libmysql_drv.so

TrustedDeliveryAgent "/usr/sbin/dbmail-smtp -d %u"

UntrustedDeliveryAgent "/usr/sbin/dbmail-smtp -d %u"

OnFail error

Trust root

Trust dspam

Trust apache

Trust mail

Trust mailnull 

Trust smmsp

Trust daemon

Trust filter

TrainingMode teft

TestConditionalTraining on

Feature chained

Feature whitelist

Feature tb=3

Algorithm graham burton

PValue graham

SupressWebStats off

ImprobabilityDrive on

Preference "spamAction=quarantine"

Preference "signatureLocation=headers"  # 'message' or 'headers'

Preference "showFactors=off"

AllowOverride trainingMode

AllowOverride spamAction spamSubject

AllowOverride statisticalSedation

AllowOverride enableBNR

AllowOverride enableWhitelist

AllowOverride signatureLocation

AllowOverride showFactors

AllowOverride optIn optOut

AllowOverride whitelistThreshold

MySQLServer     /var/run/mysqld/mysqld.sock

MySQLUser               dspam

MySQLPass               dspam_password_change_me

MySQLDb                 dspam

MySQLCompress           true

MySQLCompress           true

MySQLVirtualTable          dbmail.dbmail_aliases

MySQLVirtualUIDField       deliver_to

MySQLVirtualUsernameField  alias

MySQLUIDInSignature    on

HashRecMax              98317

HashAutoExtend          on  

HashMaxExtents          0

HashExtentSize          49157

HashMaxSeek             100

HashConnectionCache     10

Notifications   off

PurgeSignature  off # Specified in purge.sql

PurgeNeutral   90

PurgeUnused    off # Specified in purge.sql

PurgeHapaxes   off # Specified in purge.sql

PurgeHits1S    off # Specified in purge.sql

PurgeHits1I    off # Specified in purge.sql

LocalMX 127.0.0.1

SystemLog on

UserLog   on

TrainPristine off

Opt out

ParseToHeaders off

ChangeModeOnParse off

ChangeUserOnParse off

ClamAVPort      3310

ClamAVHost      127.0.0.1

ClamAVResponse  reject

ServerPID              /var/run/dspam/dspam.pid

ServerMode auto

ServerDomainSocketPath  "/var/run/dspam/dspam.sock"

ProcessorBias on

```

You have to give the mysql 'dspam' user read access to the dbmail database.

From this point on, you'll have to make your own way  :Smile:  I installed the dspam-web package, which then proceeded to fuck things up considerably. I recommend installing this by hand. I also had to do some dodgy stuff like cd into /var/spool/dspam/data and then make the sym-links:

local -> entropy.homelinux.org

I think I might have had to make other directories as well. I also had to hack the dspam.cgi script to be slightly more descriptive about it's errors when there were path problems, otherwise it just says "Error", without giving any hint as to WTF is wrong.

If you have troubles, post back and I can help you with the finishing touches.

----------

## col

problem with dspam & spamassassin is that they do not work with the new image based spam....this is where blacklists & greylisting is required.

I find greylisting the most simple & elegant solution to spam. I use a very small 5 minute delay which gets rid of 99% of spam. I whitelist a good server for 1 month.

----------

## Enverex

How does Greylisting work then?

----------

## col

 *Enverex wrote:*   

> How does Greylisting work then?

 

http://en.wikipedia.org/wiki/Greylisting

it works very well....I have found that almost all spammers do not use RFC compliant delivery agents....or if they do in combination with blacklists they are blacklisted before they get a chance to resend the spam email.

----------

## VanDan

I just emerged postgrey. It makes a damned nice addition. Thanks for the tip  :Smile: 

----------

## GNUtritious

Been using this setup for almost 2 years: http://freespamfilter.org/  It's based on amavisd-new, spamassassin, etc. and works quite well.

----------

## steveb

 *VanDan wrote:*   

> I just emerged postgrey. It makes a damned nice addition. Thanks for the tip 

 Try SQLGrey. It is more configurable and clever then PostGrey.

cheers

SteveB

----------

## Ateo

 *col wrote:*   

> problem with dspam & spamassassin is that they do not work with the new image based spam....

 

Try FuzzyOcr...

----------

## VanDan

 *Ateo wrote:*   

> Try FuzzyOcr...

 

Are you saying that you've integrated this into a spam filtering system?

----------

## steveb

 *VanDan wrote:*   

>  *Ateo wrote:*   Try FuzzyOcr... 
> 
> Are you saying that you've integrated this into a spam filtering system?

 Yes. SA has interface to FuzzyOCR.

----------

## Ateo

 *VanDan wrote:*   

>  *Ateo wrote:*   Try FuzzyOcr... 
> 
> Are you saying that you've integrated this into a spam filtering system?

 

Yes. I have FuzzyOcr installed and working. For what it's worth, it works as expected. It has caught all email with images (thus far, it's been about 2 months now) that I would consider spam. I suggest using the tesseract. It's faster than gocr (imo)... 

The downside is that it has it's own word database. It doesn't utilize SA filters when scanning.

Here's the ebuild in bugzilla: https://bugs.gentoo.org/show_bug.cgi?id=158445

----------

## VanDan

Interesting. dspam is now starting to get a lot more accurate in identifying this image spam, but OCR in a spam filter still sounds intriguing. I'd like to have it work together with dspam though, so I'll devote some time ( later ) to figuring out how to chain the 2 ( dspam & sa ) together.

----------

## col

dont waste your CPU cycles...use greylisting.

----------

## Ateo

 *col wrote:*   

> dont waste your CPU cycles...use greylisting.

 

greylisting alone doesn't solve everything.

----------

## cmaurand

I've installed fuzzyocr and a thing called scam.sh which uses downloads a database for clam antivirus that allows clam antivirus to deal with pdf content in email.  I really need to send that guy some money.  I've really lowered the 

Curtis

----------

