# [resolved] DSPAM + postfix (as content filter)

## Ateo

Howdy ho...

I read about 3 paragraphs on DSPAM vs SA and, well, dspam seems worthy to try. So, I've tried 2 different approaches in implementing dspam and they both suck, for me. hehe....

I have tried:

1. piped from postfix

2. piped from maildrop

Ideally, I'd like a painless transition from SA to DSPAM and to accomplish this, I think to pass the message to dspam from maildrop seems logical as this is how I currently implement SA, all of which, utilizes SQL for everything.

But, I'm here for input. And maybe discuss some concerns I have about dspam... =P

I know there are several ways to use dspam. Which way is best? When I piped it directly from postfix, it seems I am locked into a 'global' user. Is this good/bad/whatever? This approach seems reasonable and I do believe I am successful with this mode.

But I think I might want individual preferences (tagging subject header is all i care about i think) and I have both system users all with unique user ids (duh) and several other domains which all user share one common unix uid. Is this a good/bad/whatever method?

There's a lot of dspam docs out there. It can be confusing....

I am using the latest and greatest:

Postfix

Courier-IMAP/Auth

Maildrop

DSPAM

PgSQL

Current situtation:

I currently pipe via maildrop:

```
xfilter "$DSPAM --client --deliver=innocent,spam --user $RECIPIENT"
```

But then the Postfix queue eats the message with

```
temporary failure. Command output: maildrop: Timeout quota exceeded.
```

So, I assumed this

```
TrustedDeliveryAgent "/usr/bin/maildrop"
```

But that doesn't seem to pipe it back to maildrop.. Here's my maildrop stuff from master.cf which, according to the documentation *Quote:*   

> # Trusted Delivery Agent: Specifies the local delivery agent DSPAM should call
> 
> # when delivering mail as a trusted user. Use %u to specify the user DSPAM is
> 
> # processing mail for. It is generally a good idea to allow the MTA to specify
> ...

 

I should let Postfix handle my arguments... So, here's master.cf...

```
maildrop  unix  -       n       n       -       -       pipe

 flags=DRhu user=vmail argv=/usr/bin/maildrop -w 90 -d $user@$nexthop 2 $user $nexthop $sender
```

Where is my issue?

Thanks

*hopefully this isn't the wrong place for this*Last edited by Ateo on Thu Jan 11, 2007 7:38 am; edited 4 times in total

----------

## Ateo

ok. so i reverted back using dspam as content filter. And as a test for reporting spam, I created the following:

I have added the following:

```
dspam   unix    -       n       n       -       10      pipe

  flags=Ru user=dspam argv=/usr/bin/dspam --client --deliver=innocent,spam --user $user@$nexthop -i -f $sender -- $user@$nexthop

dspam-train   unix    -       n       n       -       10      pipe

  flags=Ru user=dspam argv=/usr/bin/dspam --client --deliver=innocent,spam --class=innocent --source=corpus --user $user@$nexthop
```

```
smtpd_recipient_restrictions =

 check_recipient_access pcre:/etc/postfix/pipes/dspam

dspam_destination_recipient_limit = 1

dspam-train_destination_recipient_limit = 1
```

```
/^.*@dspam-train.(.*)$/ FILTER dspam-train:${2}

/./                     FILTER dspam:dspam
```

```
shadow postfix # postmap pipes/dspam

shadow postfix # 
```

The problem is when I want to send to dspam-train to train it. It sends it to the correct mailbox that will be collecting these reported mails but does not 'retrain'.

Any ideas?

----------

## UberLord

When piping from procmailrc or maildroprc, the dspam binary needs to be suid root.

As such, only the users group can exec it. Sucky, but I cannot find another workaround :/

----------

## Ateo

 *UberLord wrote:*   

> When piping from procmailrc or maildroprc, the dspam binary needs to be suid root.
> 
> As such, only the users group can exec it. Sucky, but I cannot find another workaround :/

 

i do believe the work around is to using it as a content filter as suid's are not needed. correct me if I'm wrong though.

----------

## Ateo

One thing that concerns me about is using it on a per user basis. This was very easy with spamassassin as it uses the user's email address as the 'uid'. Dspam uses the user's actual UID so my question is... how can a system with virtual users that share a common UID have unique dspam statistics if they all share a common UID?

dspam pulls the correct UID from my database so that part is fine but most of the rows in the database refer to one, single UID... my virtual UID.. Is dspam learning per user or per UID which would cover all my virtual users?

Thanks

----------

## UberLord

 *Ateo wrote:*   

>  *UberLord wrote:*   When piping from procmailrc or maildroprc, the dspam binary needs to be suid root.
> 
> As such, only the users group can exec it. Sucky, but I cannot find another workaround :/ 
> 
> i do believe the work around is to using it as a content filter as suid's are not needed. correct me if I'm wrong though.

 

I think you are wrong - sadly.

When I pipe to dspam, dspam should just be tagging it as it delivers to stdout.

----------

## Ateo

Ok. fair enough. So it acts the same as when you pipe it to maildrop. Makes sense...

So, what about adding users to 'Trust' in dspam.conf? I've added all the relevant Trust users, especially the one doing the retraining (which happens to be a system user, not that it should matter) but it doesn't retrain. personally, I think it's my regexp for my check_recipient_access postfix parameter that is the issue...

----------

## magic919

My 2p worth would be use DSPAM as a content filter.  As far as training is concerned I only ever re-train and I use the webUI or a specific IMAP folder for each user.  If you want to forward spam it has missed then you should be using source error rather than source corpus.

I use DSPAM on one of my systems with virtual users.  It looks like it keeps a table and allocates them UIDs for its own purposes.  I havent mixed system and virtual users on any of my systems.

----------

## steveb

 *Ateo wrote:*   

> I know there are several ways to use dspam. Which way is best? When I piped it directly from postfix, it seems I am locked into a 'global' user. Is this good/bad/whatever? This approach seems reasonable and I do believe I am successful with this mode.

 

I would suggest you to use DSPAM in Postfix with a pipe service. However... when using it with pipe, you don't need a global user. This is really not needed. It only helps you if you to have a global user if you want to offer your users a "out of the box" accuracy. But DSPAM does not need it at all.

 *Ateo wrote:*   

> But I think I might want individual preferences (tagging subject header is all i care about i think) and I have both system users all with unique user ids (duh) and several other domains which all user share one common unix uid. Is this a good/bad/whatever method?

 

Uhh... this get's complicated. In such an environment I would treat all users as virtual users in DSPAM. Then the different UID's/GID's don't disturb DSPAM.

 *Ateo wrote:*   

> I should let Postfix handle my arguments... So, here's master.cf...
> 
> ```
> maildrop  unix  -       n       n       -       -       pipe
> 
> ...

 

What is this "2" in the command? For what do you need that?

----------

## steveb

 *Ateo wrote:*   

> 
> 
> ```
> dspam   unix    -       n       n       -       10      pipe
> 
> ...

 

For training I would not set class to innocent and source to corpus. If it is for retraining then source should be error and class could be innocent or spam. And I would at least add --stdout to the parameter list. Without it, DSPAM will deliver the message again to the box. 

Why don't you use in DSPAM the build in function for reporting FP/FN:

```
#

# ParseToHeaders: In lieu of setting up individual aliases for each user,

# DSPAM can be configured to automatically parse the To: address for spam and

# false positive forwards. From there, it can be configured to either set the

# DSPAM user based on the username specified in the header and/or change the

# training class and source accordingly. The options below can be used to

# customize most common types of header parsing behavior to avoid the need for

# multiple aliases, or if using LMTP, aliases entirely..

#

# ParseToHeader: Parse the To: headers of an incoming message. This must be

#                set to 'on' to use either of the following features.

#

# ChangeModeOnParse: Automatically change the class (to spam or innocent)

#   depending on whether spam- or notspam- was specified, and change the source

#   to 'error'. This is convenient if you're not using aliases at all, but

#   are delivering via LMTP.

#

# ChangeUserOnParse: Automatically change the username to match that specified

#   in the To: header. For example, spam-bob@domain.tld will set the username

#   to bob, ignoring any --user passed in. This may not always be desirable if

#   you are using virtual email addresses as usernames. Options:

#     on or user        take the portion before the @ sign only

#     full              take everything after the initial {spam,notspam}-.

#

ParseToHeaders on

ChangeModeOnParse on

ChangeUserOnParse full
```

 *Ateo wrote:*   

> 
> 
> ```
> /^.*@dspam-train.(.*)$/ FILTER dspam-train:${2}
> 
> ...

 This will not work. At least not the first entry. It should be "dspam-train:${1}"

----------

## Ateo

This is where my issue is currently at. Mail to 'spam[ham]-train@mydomain' does not trigger dspam-retrain:spam[dspam-retrain:innocent] but dspam:dspam is always triggered. What's wrong with my regexp?

```
/^.*@spam-train.(.*)$/ FILTER dspam-retrain:spam

/^.*@ham-train.(.*)$/ FILTER dspam-retrain:innocent

/./     FILTER dspam:dspam
```

//edit:: I edited the topic to reflect that I now choose using it as a content_filter...Last edited by Ateo on Tue Jan 09, 2007 5:57 am; edited 1 time in total

----------

## Ateo

 *steveb wrote:*   

>  *Ateo wrote:*   I should let Postfix handle my arguments... So, here's master.cf...
> 
> ```
> maildrop  unix  -       n       n       -       -       pipe
> 
> ...

 

For my maildroprc script. In main.cf I also have this:

```
mailbox_command = /usr/bin/maildrop -d $USER 1 $USER $DOMAIN $SENDER
```

The '1' and '2' tell my maildroprc file whether is system or virtual. I have had issues in the past where these parameters mysteriously do not pass to maildrop so I sort of 'force' it here..

----------

## Ateo

dspam continues to fail.

```
Jan  8 22:22:29 shadow dspam[29589]: Client exited with error -5

Jan  8 22:22:29 shadow postfix/pipe[29467]: 784724BBBC: to=<dracco@domain.net>, orig_to=<juan@domain.net>, relay=dspam, delay=0.98, delays=0.87/0/0/0.11, dsn=5.3.0, status=bounced (Command died with status 251: "/usr/bin/dspam")

```

I upped my connectioncache

```
PgSQLConnectionCache    25
```

 but that didn't seem to help. Any parameters that I should be looking at? Can't run dspam when it bounces mail... =(

/edit: I can send 2 messages which are processed normal. After the 2nd, I get the above error.

----------

## Ateo

Ok.. blah. I've deleted my last post as it was irrelevant and... ugh.. i've reopened it.. 

All new mail invokes dspam via the dspam service as expected.

What is happening now is mail forwarded to either retraining alias does not get sent to the dspam-retrain service. Everything is sent to the dspam service.

I have 2 aliases for retraining:

spam-train@domain.com

ham-train@domain.com

```
/^spam-train@(.*)$/     FILTER dspam-retrain:spam

/^ham-train@(.*)$/      FILTER dspam-retrain:innocent

/./     FILTER dspam:dspam
```

```
$ postmap /etc/postfix/maps/dspam.pipe
```

```
smtpd_recipient_restrictions =

 permit_mynetworks,

 permit_sasl_authenticated,

 reject_unknown_sender_domain,

 ...,

 check_client_access pcre:/etc/postfix/maps/dspam.pipe

dspam_destination_recipient_limit = 1

dspam-retrain_destination_recipient_limit = 1
```

```
###################################################################################################################################

# DSPAM services

dspam   unix    -       n       n       -       10      pipe

 flags=Ru user=dspam argv=/usr/bin/dspam --client --deliver=innocent,spam --user $recipient -i -f $sender -- $recipient

dspam-retrain   unix    -       n       n       -       10      pipe

 flags=Ru user=dspam argv=/usr/bin/dspam --client --deliver=innocent,spam --class=${nexthop} --source=error --user $sender -i -f $sender -- $recipient
```

I am pretty certain my patterns are correct. I am not sure how to fix it when it looks correct to me and looks correct when comparing to the oodles of examples available online. And even more interesting is this:

```
$ postmap -fq "ham-train@domain.tld" pcre:/etc/postfix/maps/dspam.pipe

FILTER dspam-retrain:innocent

$ postmap -fq "spam-train@domain.tld" pcre:/etc/postfix/maps/dspam.pipe

FILTER dspam-retrain:spam

$ postmap -fq "nobody@domain.tld" pcre:/etc/postfix/maps/dspam.pipe

FILTER dspam:dspam

```

Any input would be appreciated.

Thanks

----------

## Ateo

Ok. Making some progress here. When I send to a retraining alias from outside my network, the correct service (dspam-trainer) is triggered but obviously dies with this error for obvious reasons (no signature):

```
dspam[1016]: Unable to find a valid signature. Aborting.

dspam[1016]: process_message returned error -5.  dropping message.
```

So, i guess my question is, which parameter makes it so mail sent from the same server [localhost] skip smtpd_recipient_restrictions and pipe directly to the LDA? I have gone as far as creating a pcre file with just the aliases and placed it above ALL permit_mynetworks without success..

In dspam.conf, I have set LocalMX to both 192.168.4.240 and 127.0.0.1 (not sure if this has any affect but I've tried both, not at the same time).

----------

## Ateo

```
smtpd_recipient_restrictions =

 reject_non_fqdn_sender,

 reject_non_fqdn_recipient,

 reject_unknown_sender_domain,

 reject_unknown_recipient_domain,

 check_recipient_access pcre:/etc/postfix/maps/dspam-train,

 permit_mynetworks,

 permit_sasl_authenticated,

 ...,

 check_client_access pcre:/etc/postfix/maps/dspam-catch
```

```
###################################################################################################################################

# DSPAM services

dspam-catch   unix    -       n       n       -       10      pipe

 flags=Ru user=dspam argv=/usr/bin/dspam

 --client

 --deliver=innocent,spam

 --user $recipient -i -f $sender -- $recipient

dspam-train   unix    -       n       n       -       10      pipe

 flags=Ru user=dspam argv=/usr/bin/dspam

 --client

 --class=$nexthop

 --source=error

 --deliver=innocent,spam

 --user $user -i -f $sender -- $recipient
```

```
/./     FILTER dspam-catch:
```

```
/^(.*)-train\@(.*)$/ FILTER dspam-train:$1
```

The above works well with the only catch being: in order for my local users to be able to send mail to the training aliases, I had to put the check before permit_mynetworks. With this setup, the 2 training aliases can be abused by outsiders. Looking for a way to prevent this....

----------

