# qmail, spamassasin and vpopmail user prefs

## pachanga

Guys i've read LOTS of information on this forum regarding proper installation of qmail+vpopmail+qmail-scanner+spamassasin+clamav+etc. I read lengthy "HOWTO: qmail vpopmail courier-imap qmail-scanner etc (NEW)" and many other threads on this way tooooooo complicated topic and i finally managed to get it all working. However i don't know which is the best way to let vpopmail users customize spamassasin settings: mark letters as spam(autolearn), delete spam without getting it marked as spam, etc. 

I also found a lot of useful tips at http://spamassassin.apache.org but there's nothing said(well i couldn't find) about integration with vpopmail.... In their tips they're assuming that all users have valid system accounts but in case with vpopmail, as you know, it's not like that.

Any help would be highly appreciated!

----------

## nixnut

Moved from Documentation, Tips & Tricks to Networking & Security because it's not documentation but a support question.

----------

## pachanga

Any tips? Anybody? I really don't want to be annoying with this but i can't believe i'm the only one with such a problem... Maybe i should have tried different MTA not qmail?

----------

## pachanga

After pretty much googling around i came up with my own pretty simple solution. 

I wanted to provide 2 ways for users to report about spam/ham: 

1) via redirecting mail to spam@my.domain.com/ham@my.domain.com if IMAP is not supported

2) via placing spam/ham to SPAM/HAM IMAP folder using its mail client

Here's comes the cron script which scans vpopmail domains and calls sa-learn --spam or --ham for directories where spam or ham may reside:

1) all spam@my.domain.com accounts are considered to contain spam

2) all ham@my.domain.com accounts are considered to contain ham

3) .SPAM IMAP directory of each vpopmail user is considered to contain spam

4) .HAM IMAP directory of each vpopmail user is considered to contain ham

```

#/bin/sh

#vpopmail domains directory, no trailing slash!!!

VDOMAINS="/var/vpopmail/domains"

for domain in `ls $VDOMAINS`

do

  for account in `ls $VDOMAINS/$domain`

  do

    full_account_path="$VDOMAINS/$domain/$account"

    cur="$full_account_path/.maildir/cur"

    new="$full_account_path/.maildir/new"

    #check for global spam and ham accounts in every domain

    if [ $account = "spam" ]; then

      sa-learn --spam --dir $cur >/dev/null 2>&1 && \

      rm $cur/* >/dev/null 2>&1

      sa-learn --spam --dir $new >/dev/null 2>&1 && \

      rm $new/* >/dev/null 2>&1

    elif [ $account = "ham" ]; then

      sa-learn --ham --dir $cur >/dev/null 2>&1 && \

      rm $cur/*  >/dev/null 2>&1

      sa-learn --ham --dir $new >/dev/null 2>&1 && \

      rm $new/*  >/dev/null 2>&1

    fi

    #check for user IMPAP folders

    for dir in `ls -a $full_account_path/.maildir/`

    do

      cur="$full_account_path/.maildir/$dir/cur"

      new="$full_account_path/.maildir/$dir/new"

      if [ $dir = ".SPAM" ]; then

        sa-learn --spam --dir $cur >/dev/null 2>&1 && \

        rm $cur/* >/dev/null 2>&1

        sa-learn --spam --dir $new >/dev/null 2>&1 && \

        rm $new/* >/dev/null 2>&1

      elif [ $dir = ".HAM" ]; then

        sa-learn --ham --dir $cur >/dev/null 2>&1 && \

        rm $cur/*  >/dev/null 2>&1

        sa-learn --ham --dir $new >/dev/null 2>&1 && \

        rm $new/*  >/dev/null 2>&1

      fi

    done

  done

done

```

I placed it into /etc/cron.daily and seems to be working. However i'm not a *nix guru so any improvements and comments are welcomed!

As for deleting spam i edited /var/qmail/bin/qmail-scanner-queue.pl and set my $sa_delete='1.0'; 

All spam with the score higher than (required_score + $sa_delete) should be deleted on the server.

----------

## eltech

This is all very nice.. looks great .. what does you local.cf look like? and your conf.d/spamassassin file look like?

very nice work youve done here.. 

is that the only line in the queue.pl that you edited and adjusted to move spam?

How is the redirect setup? spam goes to spam@yourdomain.com

thanks in advance..

----------

## pachanga

 *eltech wrote:*   

> This is all very nice.. looks great .. what does you local.cf look like? and your conf.d/spamassassin file look like?
> 
> 

 

conf.d/smapd

```

#SPAMD_OPTS="-m 5 -c -H"

#SPAMD_OPTS="-d -u vpopmail"

SPAMD_OPTS="-c -L --siteconfigpath=/etc/spamassassin/local.cf"

# spamd stores its pid in this file. If you use the -u option to

# run spamd under another user, you might need to adjust it.

PIDFILE="/var/run/spamd.pid"

```

/etc/spamassassin/local.cf

```

rewrite_header Subject *****SPAM*****

report safe 1

required_score 5.0

skip_rbl_checks 0

```

 *Quote:*   

> 
> 
> is that the only line in the queue.pl that you edited and adjusted to move spam?
> 
> 

 

Yes, but still i'm not sure if it's working as expected(users still receive mail marked as spam): qmail-scanner is a black box for me which requires lots of voodoo magic  :Sad: 

 *Quote:*   

> 
> 
> How is the redirect setup? spam goes to spam@yourdomain.com
> 
> 

 

Yes, so you have to make spam/ham accounts foreach domain.

----------

## magic919

I can't say I've read every bit of your script but I have some experience of SpamAssassin.  The bit I've found tricky in the past is making sure which user runs SA and making sure the same user is applied when you invoke SA-learn.  If you don't you end up with you're live spam and learned-spam in two different SA databases.  Would be worth looking at.  And do watch that users bounce the mail to your spam and ham addresses and not send/forward.

----------

## pachanga

 *magic919 wrote:*   

> I can't say I've read every bit of your script but I have some experience of SpamAssassin.  The bit I've found tricky in the past is making sure which user runs SA and making sure the same user is applied when you invoke SA-learn.

 

My script simply calls "sa-learn --spam --dir $dir", "sa-learn --ham --dir $dir" - which user would SpamAssassin think invokes it in this case?

----------

## magic919

Whomever the script runs as.  In your case it would be the owner of the crontab I'd expect.

----------

## pachanga

 *magic919 wrote:*   

> Whomever the script runs as.  In your case it would be the owner of the crontab I'd expect.

 

Hm...crontab(SA-learn) is run by root and SA check is run by qmail user, i suppose - will it conflict?

----------

## magic919

Just add -u qmail to the sa-learn script and you're done.

----------

## pachanga

 *magic919 wrote:*   

> Just add -u qmail to the sa-learn script and you're done.

 

Thanks, i'll try it right now...

----------

## eltech

I just tried changing the user, but it doesnt actually make a difference in the learned messages ..

pachanga what do your mail headers look like? how do you know this ham/spam learning is working?

----------

## pachanga

 *eltech wrote:*   

> 
> 
> pachanga what do your mail headers look like? how do you know this ham/spam learning is working?

 

I'm still not 100% sure if ham/spam my learning working. I just read spamassin manual and made this simple script. I can only judge if it's working by looking at mail headers. Here's how look header X-Spam-Status entry of the ham mail:

 *Quote:*   

> 
> 
> X-Spam-Status: No, hits=-0.8 required=5.0
> 
> 

 

And here's the spam one:

 *Quote:*   

> 
> 
> X-Spam-Status: Yes, hits=5.4 required=5.0
> 
> 

 

But for some reason spam is not deleted and not even marked in the title. And i did change the /var/qmail/bin/qmail-scanner-queue.pl and set all the neccessary settings.

Is there a good way to know what's actually happening? I'm watching my /var/log/messages, spamd is definitely working, here's the real log entry: 

 *Quote:*   

> 
> 
> Nov 25 16:37:12 foo.com spamd[23819]: result: Y 11 - BAYES_50,FORGED_MUA_OUTLOOK,FORGED_OUTLOOK_HTML,FORGED_OUTLOOK_TAGS,FROM_ENDS_IN_NUMS,HTML_40_50,HTML_MESSAGE,HTML_MIME_NO_HTML_TAG,MIME_HTML_ONLY,MIME_QP_LONG_LINE,MISSING_DATE,MISSING_MIMEOLE,MSGID_SPAM_CAPS,NO_REAL_NAME scantime=0.2,size=6901,mid=<WUIQHPHSYFYQXSSUTXKSDL@KOJUO>,bayes=0.4985385966171,autolearn=no
> 
> 

 

Still spam mail is not removed  :Sad: 

----------

## eltech

 *pachanga wrote:*   

>  *eltech wrote:*   
> 
> pachanga what do your mail headers look like? how do you know this ham/spam learning is working? 
> 
> I'm still not 100% sure if ham/spam my learning working. I just read spamassin manual and made this simple script. I can only judge if it's working by looking at mail headers. Here's how look header X-Spam-Status entry of the ham mail:
> ...

 hey, from the looks of it we are in the same boat  :Smile:  .. my SA works flawlessely .. it tags mail and all, but i dont have the bayes info in there.. hmm

Will look at the qmail-scanner-queue.pl file now..

thanks again ..

----------

## eltech

Well, i dont have the bayes working.. but SA is doing a great job after editing the *.pl file..

```

Fri, 25 Nov 2005 10:38:38 EST:14924: +++ starting debugging for process 14924 (ppid=14909) by uid=201

Fri, 25 Nov 2005 10:38:38 EST:14924: w_c: elapsed time from start 0.002788 secs

Fri, 25 Nov 2005 10:38:38 EST:14924: return-path='spamman@gmail.com', recips='eltech@mydomain.com'

Fri, 25 Nov 2005 10:38:38 EST:14924: from='<spamman@gmail.com>', subj='Cialis! GREAT SEX GREAT SEX GREAT SEX !!!!!!!!!!!!!!!!!!', via SMTP from 207.90.40.22

Fri, 25 Nov 2005 10:38:39 EST:14924: clamdscan: finished scan in 0.755241 secs

Fri, 25 Nov 2005 10:38:41 EST:14924: SA: REPORT hits = 7.2/3.0

  0.2 NO_REAL_NAME           From: does not include a real name

  1.9 SUBJECT_DRUG_GAP_C     Subject contains a gappy version of 'cialis'

  1.5 DRUG_ED_CAPS           BODY: Mentions an E.D. drug

  0.5 BIZ_TLD                URI: Contains an URL in the BIZ top-level domain

  1.5 RAZOR2_CF_RANGE_51_100 BODY: Razor2 gives confidence level above 50%

                             [cf: 100]

  0.1 RAZOR2_CHECK           Listed in Razor2 (http://razor.sf.net/)

  0.0 DRUGS_ERECTILE         Refers to an erectile drug

  0.9 UPPERCASE_50_75        message body is 50-75% uppercase

  0.4 PLING_PLING            Subject has lots of exclamation marks

  0.0 MANY_EXCLAMATIONS      Subject has many exclamations

Fri, 25 Nov 2005 10:38:41 EST:14924: SA: yup, this smells like SPAM - hits=7.2/3.0/4 - rejecting message...

Fri, 25 Nov 2005 10:38:41 EST:14924: SA: finished scan in 2.050656 secs - hits=7.2/3.0

Fri, 25 Nov 2005 10:38:41 EST:14924: r_e: X-Qmail-Scanner-1.25st: We have reasons to believe this mail is SPAM

Fri, 25 Nov 2005 10:38:41 EST:14924: ------ Process 14924 finished. Total of 2.819056 secs

```

So, the question is when does bayes learn and show up in the headers

here are the smtpd, logs

```
@400000004387313533a438f4 tcpserver: ok 15133 0:::ffff:192.168.2.100:25 :::ffff:201.0.103.171::3219

@400000004387313f2afa9dec X-Qmail-Scanner-1.25st: We have reasons to believe this mail is SPAM

```

----------

## magic919

Bayes wants 200 messages seen to start.  su to the user you use for SA.  Then run

spamassassin -D --lint

will not work if you don't run as correct user.

Bayes not in use will read like this

debug: bayes: Not available for scanning, only xx spam(s) in Bayes DB < 200

----------

## eltech

 *magic919 wrote:*   

> Bayes wants 200 messages seen to start.  su to the user you use for SA.  Then run
> 
> spamassassin -D --lint
> 
> will not work if you don't run as correct user.
> ...

 ok .. i see the output.. lots of info, but non relates to my /var/vpopmail directories.. so what must i do to force it to look at these directories? currently it wants /home/user/.spamassassin/ and more..

----------

## magic919

I don't know quite what you are expecting, but if I understand correctly then you are never going to see it.

What is the reason for want to 'force SA to look at these directories'?

----------

## eltech

 *magic919 wrote:*   

> I don't know quite what you are expecting, but if I understand correctly then you are never going to see it.
> 
> What is the reason for want to 'force SA to look at these directories'?

 We. if im using vpopmail (virtual user and mail accounts) the /home/user/.maildir structure is useless; unless i missed the idea of this all ..

----------

## magic919

I don't think you've completely missed the point although you might currently be barking up the adjacent tree.

I understand these are 'virtual' users but they makes no difference.

Two questions:-

1. Is vpopmail storing the mail in maildirs?

2. Which user do you run SA as?

----------

## eltech

 *magic919 wrote:*   

> I don't think you've completely missed the point although you might currently be barking up the adjacent tree.
> 
> I understand these are 'virtual' users but they makes no difference.
> 
> Two questions:-
> ...

 

Thanks for the response..

I guess im trying to get the whitelist, the bayes learning and the other goodies for all my vpopmail users ..

Yes, mail is in .maildir located at /var/vpopmail/domains/domain.com/user/.maildir

root       9   0 20052 1464 1052 S  0.0  0.3   0:00.77 spamd

----------

## magic919

You'll basically be doing site-wide SA.  Means that you don't have control down to user level.  So the whitelist will be for the whole server.  No individual user prefs.

You are, naturally, running SA (spamd) as root.  However you can still invoke SA as another user.  Mine is a user called filter.  It is vital that you know who you invoke SA as.  This is because the database is kept in a .spamassassin directory in the $HOME of the user invoking SA.

Sa-learn is interesting.  You need to run sa-learn as the same SA user as you use to invoke SA.

There is a very important snag with sa-learn.  I've tested it and find that it takes no notice of the -u option I mentioned above.  All the learnt spam goes into the database of the user running sa-learn regardless.  This means you have to open permissions on any folders you want sa-learn to study and then su - to your SA user before you run sa-learn.

I remember having this problem a couple of years back when I started using SA and that was one of the reasons I stopped bothering with sa-learn.  I'd hoped they had sorted it by now.

----------

## eltech

just seems so 'all over the place' .. my main issue is i'd love to be able to manage some sort of white list and blacklist and unfortunately the ability to do this is so unclear and pretty fustrating and dissappointing. I mean SA does a great job right now catching spam; but itd be great to have a more controlled preference application. the confusion and unsurity around 'HOW' is pretty upsetting   :Rolling Eyes: 

Hey, but thats for all of your help ..   :Wink: 

----------

## magic919

Do a quick find for the file user_prefs.  You can whitelist and blacklist in there.  Check output of spamassassin -D --lint and you'll see just which user_prefs files it looks at.

----------

## eltech

 *magic919 wrote:*   

> Do a quick find for the file user_prefs.  You can whitelist and blacklist in there.  Check output of spamassassin -D --lint and you'll see just which user_prefs files it looks at.

 well since spamd is running as root, i guess i should be running this command as root.

i have looked at this file user_prefs and it looks like my local.cf which is specified in /etc/conf.d/spamd, but not configured

```

SPAMD_OPTS="-i -v --siteconfigpath=/etc/mail/spamassassin/local.cf"
```

so my spamassassin running is defined as whats in that file at that location.

by default the spamd runs as root. this is indicated in this same file

```

# -u USER to run as a user other than root
```

wit that said, how can i use this file to setup whitelisting?

EDIT:

i just realized running the command you suggested..

```
dbg: bayes: not available for scanning, only 130 spam(s) in bayes DB < 200
```

```
pcns .spamassassin # spamassassin -D --lint                     

[9564] dbg: logger: adding facilities: all

[9564] dbg: logger: logging level is DBG

[9564] dbg: generic: SpamAssassin version 3.1.0

[9564] dbg: config: score set 0 chosen.

[9564] dbg: util: running in taint mode? no

[9564] dbg: dns: is Net::DNS::Resolver available? yes

[9564] dbg: dns: Net::DNS version: 0.49

[9564] dbg: dns: name server: 192.168.2.100, family: 2, ipv6: 0

[9564] dbg: diag: perl platform: 5.008006 linux

[9564] dbg: diag: module installed: Digest::SHA1, version 2.10

[9564] dbg: diag: module installed: Razor2::Client::Agent, version 2.77

[9564] dbg: diag: module not installed: Net::Ident ('require' failed)

[9564] dbg: diag: module not installed: IO::Socket::INET6 ('require' failed)

[9564] dbg: diag: module installed: IO::Socket::SSL, version 0.96

[9564] dbg: diag: module installed: Time::HiRes, version 1.66

[9564] dbg: diag: module installed: DBI, version 1.46

[9564] dbg: diag: module installed: Getopt::Long, version 2.34

[9564] dbg: diag: module installed: LWP::UserAgent, version 2.032

[9564] dbg: diag: module installed: HTTP::Date, version 1.46

[9564] dbg: diag: module installed: Archive::Tar, version 1.26

[9564] dbg: diag: module installed: IO::Zlib, version 1.01

[9564] dbg: diag: module installed: MIME::Base64, version 3.05

[9564] dbg: diag: module installed: HTML::Parser, version 3.46

[9564] dbg: diag: module installed: DB_File, version 1.811

[9564] dbg: diag: module installed: Net::DNS, version 0.49

[9564] dbg: diag: module installed: Net::SMTP, version 2.29

[9564] dbg: diag: module not installed: Mail::SPF::Query ('require' failed)

[9564] dbg: diag: module not installed: IP::Country::Fast ('require' failed)

[9564] dbg: ignore: using a test message to lint rules

[9564] dbg: config: using "/etc/mail/spamassassin" for site rules pre files

[9564] dbg: config: read file /etc/mail/spamassassin/init.pre

[9564] dbg: config: read file /etc/mail/spamassassin/v310.pre

[9564] dbg: config: using "/usr/share/spamassassin" for sys rules pre files

[9564] dbg: config: using "/usr/share/spamassassin" for default rules dir

[9564] dbg: config: read file /usr/share/spamassassin/10_misc.cf

[9564] dbg: config: read file /usr/share/spamassassin/11_gentoo.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_advance_fee.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_anti_ratware.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_body_tests.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_compensate.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_dnsbl_tests.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_drugs.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_fake_helo_tests.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_head_tests.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_html_tests.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_meta_tests.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_net_tests.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_phrases.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_porn.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_ratware.cf

[9564] dbg: config: read file /usr/share/spamassassin/20_uri_tests.cf

[9564] dbg: config: read file /usr/share/spamassassin/23_bayes.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_accessdb.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_antivirus.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_body_tests_es.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_body_tests_pl.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_dcc.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_domainkeys.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_hashcash.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_pyzor.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_razor2.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_replace.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_spf.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_textcat.cf

[9564] dbg: config: read file /usr/share/spamassassin/25_uribl.cf

[9564] dbg: config: read file /usr/share/spamassassin/30_text_de.cf

[9564] dbg: config: read file /usr/share/spamassassin/30_text_fr.cf

[9564] dbg: config: read file /usr/share/spamassassin/30_text_it.cf

[9564] dbg: config: read file /usr/share/spamassassin/30_text_nl.cf

[9564] dbg: config: read file /usr/share/spamassassin/30_text_pl.cf

[9564] dbg: config: read file /usr/share/spamassassin/30_text_pt_br.cf

[9564] dbg: config: read file /usr/share/spamassassin/50_scores.cf

[9564] dbg: config: read file /usr/share/spamassassin/60_awl.cf

[9564] dbg: config: read file /usr/share/spamassassin/60_whitelist.cf

[9564] dbg: config: read file /usr/share/spamassassin/60_whitelist_spf.cf

[9564] dbg: config: read file /usr/share/spamassassin/60_whitelist_subject.cf

[9564] dbg: config: using "/etc/mail/spamassassin" for site rules dir

[9564] dbg: config: read file /etc/mail/spamassassin/local.cf

[9564] dbg: config: read file /etc/mail/spamassassin/secrets.cf

[9564] dbg: config: using "/root/.spamassassin" for user state dir

[9564] dbg: config: using "/root/.spamassassin/user_prefs" for user prefs file

[9564] dbg: config: read file /root/.spamassassin/user_prefs

[9564] dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC

[9564] dbg: plugin: registered Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x8fb8d70)

[9564] dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC

[9564] dbg: plugin: registered Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8fd253c)

[9564] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC

[9564] dbg: plugin: registered Mail::SpamAssassin::Plugin::SPF=HASH(0x8ff1738)

[9564] dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC

[9564] dbg: pyzor: network tests on, attempting Pyzor

[9564] dbg: plugin: registered Mail::SpamAssassin::Plugin::Pyzor=HASH(0x8fff668)

[9564] dbg: plugin: loading Mail::SpamAssassin::Plugin::SpamCop from @INC

[9564] dbg: reporter: network tests on, attempting SpamCop

[9564] dbg: plugin: registered Mail::SpamAssassin::Plugin::SpamCop=HASH(0x8fe060c)

[9564] dbg: plugin: loading Mail::SpamAssassin::Plugin::AWL from @INC

[9564] dbg: plugin: registered Mail::SpamAssassin::Plugin::AWL=HASH(0x8fe3138)

[9564] dbg: plugin: loading Mail::SpamAssassin::Plugin::AutoLearnThreshold from @INC

[9564] dbg: plugin: registered Mail::SpamAssassin::Plugin::AutoLearnThreshold=HASH(0x9026848)

[9564] dbg: plugin: loading Mail::SpamAssassin::Plugin::WhiteListSubject from @INC

[9564] dbg: plugin: registered Mail::SpamAssassin::Plugin::WhiteListSubject=HASH(0x903aa9c)

[9564] dbg: plugin: loading Mail::SpamAssassin::Plugin::MIMEHeader from @INC

[9564] dbg: plugin: registered Mail::SpamAssassin::Plugin::MIMEHeader=HASH(0x9041518)

[9564] dbg: plugin: loading Mail::SpamAssassin::Plugin::ReplaceTags from @INC

[9564] dbg: plugin: registered Mail::SpamAssassin::Plugin::ReplaceTags=HASH(0x904ac68)

[9564] dbg: config: adding redirector regex: /^http:\/\/chkpt\.zdnet\.com\/chkpt\/\w+\/(.*)$/i

[9564] dbg: config: adding redirector regex: /^http:\/\/www(?:\d+)?\.nate\.com\/r\/\w+\/(.*)$/i

[9564] dbg: config: adding redirector regex: /^http:\/\/.+\.gov\/(?:.*\/)?externalLink\.jhtml\?.*url=(.*?)(?:&.*)?$/i

[9564] dbg: config: adding redirector regex: /^http:\/\/redir\.internet\.com\/.+?\/.+?\/(.*)$/i

[9564] dbg: config: adding redirector regex: /^http:\/\/(?:.*?\.)?adtech\.de\/.*(?:;|\|)link=(.*?)(?:;|$)/i

[9564] dbg: config: adding redirector regex: m'^http.*?/redirect\.php\?.*(?<=[?&])goto=(.*?)(?:$|[&\#])'i

[9564] dbg: config: adding redirector regex: m'^https?:/*(?:[^/]+\.)?emf\d\.com/r\.cfm.*?&r=(.*)'i

[9564] warn: config: failed to parse, now a plugin, skipping: ok_languages__all

[9564] warn: config: failed to parse line, skipping: use_dcc___1

[9564] warn: config: failed to parse line, skipping: use_razor2__1

[9564] dbg: config: allowing user rules!

[9564] warn: config: failed to parse line, skipping: rewrite_subject         1

[9564] warn: config: failed to parse line, skipping: subject_tag             *****SPAM*****

[9564] warn: config: failed to parse line, skipping: use_terse_report        1

[9564] warn: config: failed to parse line, skipping: defang_mime 1

[9564] warn: config: failed to parse line, skipping: report_header 1

[9564] warn: config: failed to parse, now a plugin, skipping: ok_languages            dk

[9564] warn: config: failed to parse line, skipping: dcc_add_header 1

[9564] warn: config: failed to parse line, skipping: use_dcc 1

[9564] dbg: plugin: Mail::SpamAssassin::Plugin::ReplaceTags=HASH(0x904ac68) implements 'finish_parsing_end'

[9564] dbg: replacetags: replacing tags

[9564] dbg: replacetags: done replacing tags

[9564] dbg: config: using "/root/.spamassassin" for user state dir

[9564] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_toks

[9564] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_seen

[9564] dbg: bayes: found bayes db version 3

[9564] dbg: bayes: DB journal sync: last sync: 0

[9564] dbg: config: using "/root/.spamassassin" for user state dir

[b][9564] dbg: bayes: not available for scanning, only 130 spam(s) in bayes DB < 200[/b]

[9564] dbg: bayes: untie-ing

[9564] dbg: bayes: untie-ing db_toks

[9564] dbg: bayes: untie-ing db_seen

[9564] dbg: config: score set 1 chosen.

[9564] dbg: message: ---- MIME PARSER START ----

[9564] dbg: message: main message type: text/plain

[9564] dbg: message: parsing normal part

[9564] dbg: message: added part, type: text/plain

[9564] dbg: message: ---- MIME PARSER END ----

[9564] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_toks

[9564] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_seen

[9564] dbg: bayes: found bayes db version 3

[9564] dbg: bayes: DB journal sync: last sync: 0

[b][9564] dbg: bayes: not available for scanning, only 130 spam(s) in bayes DB < 200[/b]

[9564] dbg: bayes: untie-ing

[9564] dbg: bayes: untie-ing db_toks

[9564] dbg: bayes: untie-ing db_seen

[9564] dbg: dns: dns_available set to no in config file, skipping test

[9564] dbg: metadata: X-Spam-Relays-Trusted: 

[9564] dbg: metadata: X-Spam-Relays-Untrusted: 

[9564] dbg: message: no encoding detected

[9564] dbg: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x8fb8d70) implements 'parsed_metadata'

[9564] dbg: check: running tests for priority: 0

[9564] dbg: rules: running header regexp tests; score so far=0

[9564] dbg: rules: ran header rule __HAS_MSGID ======> got hit: "<"

[9564] dbg: rules: ran header rule __MSGID_OK_DIGITS ======> got hit: "1133024143"

[9564] dbg: rules: ran header rule __SANE_MSGID ======> got hit: "<1133024143@lint_rules>

[9564] dbg: rules: "

[9564] dbg: rules: ran header rule NO_REAL_NAME ======> got hit: "ignore@compiling.spamassassin.taint.org

[9564] dbg: rules: "

[9564] dbg: rules: ran header rule __MSGID_OK_HOST ======> got hit: "@lint_rules>"

[9564] dbg: plugin: registering glue method for check_hashcash_double_spend (Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8fd253c))

[9564] dbg: plugin: registering glue method for check_for_spf_helo_pass (Mail::SpamAssassin::Plugin::SPF=HASH(0x8ff1738))

[9564] dbg: eval: all '*From' addrs: ignore@compiling.spamassassin.taint.org

[9564] dbg: plugin: registering glue method for check_subject_in_blacklist (Mail::SpamAssassin::Plugin::WhiteListSubject=HASH(0x903aa9c))

[9564] dbg: plugin: registering glue method for check_hashcash_value (Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8fd253c))

[9564] dbg: eval: all '*To' addrs: 

[9564] dbg: plugin: registering glue method for check_for_spf_neutral (Mail::SpamAssassin::Plugin::SPF=HASH(0x8ff1738))

[9564] dbg: plugin: registering glue method for check_for_spf_softfail (Mail::SpamAssassin::Plugin::SPF=HASH(0x8ff1738))

[9564] dbg: rules: ran eval rule NO_RELAYS ======> got hit

[9564] dbg: plugin: registering glue method for check_for_spf_pass (Mail::SpamAssassin::Plugin::SPF=HASH(0x8ff1738))

[9564] dbg: plugin: registering glue method for check_for_spf_helo_softfail (Mail::SpamAssassin::Plugin::SPF=HASH(0x8ff1738))

[9564] dbg: plugin: registering glue method for check_for_def_spf_whitelist_from (Mail::SpamAssassin::Plugin::SPF=HASH(0x8ff1738))

[9564] dbg: plugin: registering glue method for check_for_spf_fail (Mail::SpamAssassin::Plugin::SPF=HASH(0x8ff1738))

[9564] dbg: rules: ran eval rule __UNUSABLE_MSGID ======> got hit

[9564] dbg: plugin: registering glue method for check_subject_in_whitelist (Mail::SpamAssassin::Plugin::WhiteListSubject=HASH(0x903aa9c))

[9564] dbg: rules: ran eval rule MISSING_HEADERS ======> got hit

[9564] dbg: plugin: registering glue method for check_for_spf_whitelist_from (Mail::SpamAssassin::Plugin::SPF=HASH(0x8ff1738))

[9564] dbg: rules: running body-text per-line regexp tests; score so far=0.738

[9564] dbg: rules: ran body rule __NONEMPTY_BODY ======> got hit: "I"

[9564] dbg: uri: running uri tests; score so far=0.738

[b][9564] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_toks

[9564] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_seen

[9564] dbg: bayes: found bayes db version 3

[9564] dbg: bayes: DB journal sync: last sync: 0

[9564] dbg: bayes: not available for scanning, only 130 spam(s) in bayes DB < 200[/b]

[9564] dbg: bayes: not scoring message, returning undef

[9564] dbg: bayes: DB expiry: tokens in DB: 128465, Expiry max size: 150000, Oldest atime: 1124139834, Newest atime: 1132989474, Last expire: 1132904170, Current time: 1133024144

[9564] dbg: bayes: DB journal sync: last sync: 0

[9564] dbg: bayes: untie-ing

[9564] dbg: bayes: untie-ing db_toks

[9564] dbg: bayes: untie-ing db_seen

[9564] dbg: plugin: registering glue method for check_uridnsbl (Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x8fb8d70))

[9564] dbg: rules: running raw-body-text per-line regexp tests; score so far=0.738

[9564] dbg: rules: running full-text regexp tests; score so far=0.738

[9564] dbg: plugin: registering glue method for check_pyzor (Mail::SpamAssassin::Plugin::Pyzor=HASH(0x8fff668))

[9564] dbg: util: current PATH is: /sbin:/bin:/usr/sbin:/usr/bin

[9564] dbg: util: executable for pyzor was found at /usr/bin/pyzor

[9564] dbg: pyzor: pyzor is available: /usr/bin/pyzor

[9564] dbg: info: entering helper-app run mode

[9564] dbg: pyzor: opening pipe: /usr/bin/pyzor  check < /tmp/.spamassassin9564jTTDN3tmp

[9565] dbg: util: setuid: ruid=0 euid=0

[9564] dbg: pyzor: [9565] finished:  exit=0x0100

[9564] dbg: pyzor: got response: downloading servers from http://pyzor.sourceforge.net/cgi-bin/inform-servers-0-3-x\n66.250.40.33:24441_(200, 'OK')_0_0

[9564] dbg: info: leaving helper-app run mode

[9564] dbg: pyzor: failure to parse response "downloading servers from http://pyzor.sourceforge.net/cgi-bin/inform-servers-0-3-x"

[9564] dbg: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x8fb8d70) implements 'check_tick'

[9564] dbg: check: running tests for priority: 500

[9564] dbg: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x8fb8d70) implements 'check_post_dnsbl'

[9564] dbg: rules: running meta tests; score so far=0.738

[9564] dbg: rules: running header regexp tests; score so far=2.216

[9564] dbg: rules: running body-text per-line regexp tests; score so far=2.216

[9564] dbg: uri: running uri tests; score so far=2.216

[9564] dbg: rules: running raw-body-text per-line regexp tests; score so far=2.216

[9564] dbg: rules: running full-text regexp tests; score so far=2.216

[9564] dbg: check: running tests for priority: 1000

[9564] dbg: rules: running meta tests; score so far=2.216

[9564] dbg: rules: running header regexp tests; score so far=2.216

[9564] dbg: plugin: registering glue method for check_from_in_auto_whitelist (Mail::SpamAssassin::Plugin::AWL=HASH(0x8fe3138))

[9564] dbg: config: using "/root/.spamassassin" for user state dir

[9564] dbg: locker: safe_lock: created /root/.spamassassin/auto-whitelist.lock.pcns.mydomain.com.9564

[9564] dbg: locker: safe_lock: trying to get lock on /root/.spamassassin/auto-whitelist with 0 retries

[9564] dbg: locker: safe_lock: link to /root/.spamassassin/auto-whitelist.lock: link ok

[9564] dbg: auto-whitelist: tie-ing to DB file of type DB_File R/W in /root/.spamassassin/auto-whitelist

[9564] dbg: auto-whitelist: db-based ignore@compiling.spamassassin.taint.org|ip=none scores 0/0

[9564] dbg: auto-whitelist: AWL active, pre-score: 2.216, autolearn score: 2.216, mean: undef, IP: undef

[9564] dbg: auto-whitelist: DB addr list: untie-ing and unlocking

[9564] dbg: auto-whitelist: DB addr list: file locked, breaking lock

[9564] dbg: locker: safe_unlock: unlink /root/.spamassassin/auto-whitelist.lock

[9564] dbg: auto-whitelist: post auto-whitelist score: 2.216

[9564] dbg: rules: running body-text per-line regexp tests; score so far=2.216

[9564] dbg: uri: running uri tests; score so far=2.216

[9564] dbg: rules: running raw-body-text per-line regexp tests; score so far=2.216

[9564] dbg: rules: running full-text regexp tests; score so far=2.216

[9564] dbg: check: is spam? score=2.216 required=2

[9564] dbg: check: tests=MISSING_HEADERS,MISSING_SUBJECT,NO_REAL_NAME,NO_RECEIVED,NO_RELAYS,TO_CC_NONE

[9564] dbg: check: subtests=__HAS_MSGID,__MSGID_OK_DIGITS,__MSGID_OK_HOST,__NONEMPTY_BODY,__SANE_MSGID,__UNUSABLE_MSGID

[9564] warn: lint: 9 issues detected, please rerun with debug enabled for more information

pcns .spamassassin # 

```

what it failed to read or apply is the user_prefs file in the /root/.spamassassin/user_prefs file which is ok..

----------

## magic919

The file you'll want to edit is /etc/mail/spamassassin/user_prefs

----------

## eltech

 *magic919 wrote:*   

> The file you'll want to edit is /etc/mail/spamassassin/user_prefs

 no such file.. hmm

So.. i just copied the user_prefs file there and i have fed enough email to sa-learn, and no bayes..   :Mad: Last edited by eltech on Sat Nov 26, 2005 5:51 pm; edited 1 time in total

----------

## magic919

File is /root/.spamassassin/user_prefs if you invoke SA as root.

Run pyzor discover first as you have errors there.  There are other errors too and you ought to try to resolve those as well.

So how do you invoke SA?

----------

## eltech

 *magic919 wrote:*   

> File is /root/.spamassassin/user_prefs if you invoke SA as root.
> 
> Run pyzor discover first as you have errors there.  There are other errors too and you ought to try to resolve those as well.
> 
> So how do you invoke SA?

 by default, not suggesting a user in the conf.d/spamd opts defaults to root.

```
pyzor discover

downloading servers from http://pyzor.sourceforge.net/cgi-bin/inform-servers-0-3-x

```

----------

## magic919

Yes, I'm aware how spamd works.  What I'm asking is how you invoke spamd.  I use Postfix to invoke it.  Some use procmail.  Some use another MTA or MDA.

----------

## eltech

Well .. im giving up on this.. its not that big of a deal .. but it doesnt seem that the ability is there to use the whitelist .. atleast with my setup .. ive fed enough mail, cant figure where to next ..

thanks you though

----------

## eltech

 *magic919 wrote:*   

> Yes, I'm aware how spamd works.  What I'm asking is how you invoke spamd.  I use Postfix to invoke it.  Some use procmail.  Some use another MTA or MDA.

 oh, sorry .. that would be qmail-scanner, qmail?

i just ran a top and then sent a message and qmaild invokes the scan of spamd with qmailscand

----------

## magic919

Ok.  I don't know qmail like I know Postfix and SA but I suspect qmail is not running SA as root.  It's likely to be as qmail I think.  Good to work this out or your tests running spamassassin -D --lint do not relate to the actual use of SA.  Likewise pyzor discover ought to be run as the correct user.

----------

## eltech

Silly silly me..

just restarted spamd and guess what!?

```

3.0 BAYES_95               BODY: Bayesian spam probability is 95 to 99%

                             [score: 0.9573]
```

So its looking great .. but im getting some odd errors now on start up that i cant figure.. they dont mess with the performance of SA, but.. id like to get them corrected

```

spamd[15940]: [config] failed to parse line, skipping: use_razor2_1_

spamd[15940]: [config] failed to parse, now a plugin, skipping: ok_languages_all_

spamd[15940]: [config] failed to parse line, skipping: use_dcc_1_

spamd[15940]: [config] failed to parse line, skipping: use_pyzor_1_

spamd[15940]: [config] failed to parse line, skipping: use_razor2_1_

spamd[15940]: [spamd] server started on port 783/tcp (running version 3.1.0-gr0)_

spamd[15940]: [spamd] server pid: 15940_

spamd[15940]: [spamd] server successfully spawned child process, pid 15955_

spamd[15940]: [spamd] server successfully spawned child process, pid 15957_

spamd[15940]: [prefork] child states: II_

```

Also when i do have an incoming mail.. i do get this error..

```

spamd[15955]: [spamd] connection from localhost.localdomain [127.0.0.1] at port 46869_

spamd[15955]: [spamd] cannot use vpopmail without -u_

spamd[15955]: [spamd] checking message <438C77E2.6020208@agr.unicamp.br> for qmaild:0_

spamd[15955]: [spamd] clean message (-2.1/3.0) for qmaild:0 in 0.3 seconds, 9814 bytes._

spamd[15955]: [spamd] result: . -2 - BAYES_00,HTML_40_50,HTML_MESSAGE scantime=0.3,size=9814,user=qmaild,uid=0,required_score=3.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=46869,mid=<438C77E2.6020208@agr.unicamp.br>,bayes=3.33066907387547e-16,autolearn=disabled_

spamd[15940]: [prefork] child states: II_

```

so im not sure.. what i need to do here..   :Rolling Eyes: 

any ideas? How is it i have  some minor errors? but its all working swell? 

I also had to change the spmad file to read

```
#

  # If vpopmail config enabled then set $dir to virtual homedir

  #

  if ( $opt{'vpopmail'} ) {

    my $vpopdir = $dir;

    $dir = `/var/vpopmail/bin/vuserinfo -d $username`;

    if ($? != 0) {  

      #      

      # If vuserinfo failed $username could be an alias

      #

      $dir = `/var/vpopmail/bin/valias $username`;

      if ($? == 0 && $dir !~ /.+ -> &/) {

        $dir =~ s,.+ -> (/.+)/Maildir/,$1,;

      } else {

        undef($dir);

```

----------

## magic919

What an arduous one this is.

Spamd is still not happy.  At least from some of the output indicates it is running as qmaild.  There are errors that show it may not be running all the tests.

Step 1 must be su - qmaild and then run spamassassin -D --lint

We want to see where it grabs the configs, what errors, what does and doesn't run.  You've done this as root but you must do it as the user than invokes spamd.  The output is huge but it's the only way forward with this.

----------

## pachanga

BTW, where does spamassassin store its spam patterns databases? I think the most straight-forward way to know whether it auto-learns anything is by simply checking the size of that database before and after learning...

----------

## magic919

It can vary but you'd expect to find it in the .spamassassin directory on the home of the user invoking SA.

----------

## eltech

 *magic919 wrote:*   

> What an arduous one this is.
> 
> Spamd is still not happy.  At least from some of the output indicates it is running as qmaild.  There are errors that show it may not be running all the tests.
> 
> Step 1 must be su - qmaild and then run spamassassin -D --lint
> ...

 running the 

```
spamassassin -D --lint
```

 as qmaild i dont get any errors.. but if I use 

```
SPAMD_OPTS="-m 5 -u qmaild -v --siteconfigpath=/etc/mail/spamassassin/local.cf"
```

mail is scanned, identified with things like

```
Thu, 21 Sep 2006 12:15:18 EDT:8360: SA: REPORT hits = 0.0/3.5

  0.0 HTML_MESSAGE           BODY: HTML included in message

  0.0 UPPERCASE_25_50        message body is 25-50% uppercase

```

but its not scored or tagged as you can see ..

Any suggestions?

Thanks

----------

## magic919

I've given up using SA in favour of DSPAM these days.  But here goes.

Same as before really.  You need to look at the config files influencing SA and this come from the output mentioned above (-D lint).

There are a few other places than just the /etc/mail/spamassassin/local.cf.

Then look at report_safe options and set as appropriate for tagging.  You'll be able to look at docs for all these options.

----------

## eltech

 *magic919 wrote:*   

> I've given up using SA in favour of DSPAM these days.  But here goes.
> 
> Same as before really.  You need to look at the config files influencing SA and this come from the output mentioned above (-D lint).
> 
> There are a few other places than just the /etc/mail/spamassassin/local.cf.
> ...

 Naw ... even before the date of this post.. its tagging, scoring and moving.. but when i run spamd as qmaild it doesnt tag, score or move.

----------

