# amavisd-new/spamassassin problem (RESOLVED)

## Fred Krogh

My mail is not getting through since sometime today.  "mailq" shows a bunch of stuff, but its not getting picked up.  I've tried restarting different related things.  I can't start spamd or amavisd, although I don't think spamd is usually started.  The error messages generate are not informative.  I've tried re-emerging amavisd-new and spamassassin and both fail.  equery list shows  mail-filter/spamassassin-3.2.5-r2:0, and mail-filter/amavisd-new-2.6.4:0, so both were installed successfully at one point.  When installing amavisd-new, I get the following error *Quote:*   

> 
> 
> Amavisd-new no longer requires SpamAssassin, but no anti-spam checking
> 
> will be performed without it. Since you do not have SpamAssassin installed,
> ...

 When installing spamassassin, I get a bunch of errors" *Quote:*   

> ***************************************************************************
> 
> ERROR: the required HTML::Parser (version 3.43) module is not installed. at lib/Mail/SpamAssassin/Util/DependencyInfo.pm line 294.
> 
>   HTML is used for an ever-increasing amount of email so this dependency
> ...

 When I tried this the first time, the error seemed to make sense as there was no /lib/Mail directory.  But after doing " ln -s /usr/lib64/perl5/vendor_perl/5.8.8/Mail Mail", /lib/Mail points to the files that seemed to be needed.

emerge -pv spamassassin amavisd-new shows use flags as follows *Quote:*   

> ebuild   R   ] mail-filter/spamassassin-3.2.5-r2  USE="berkdb ipv6 mysql ssl -doc -ldap -postgres -qmail -sqlite -tools" 0 kB
> 
> [ebuild   R   ] mail-filter/amavisd-new-2.6.4  USE="mysql spamassassin -courier -dkim -ldap -milter -postgres -qmail -razor" 0 kB
> 
> 

 

I'd appreciate any help even if it just means I can see the mail.  Thanks,

FredLast edited by Fred Krogh on Mon Feb 15, 2010 6:18 pm; edited 1 time in total

----------

## cach0rr0

try emerging dev-perl/HTML-Parser and dev-perl/Net-DNS

then see if you can build spamassassin

You are missing absolute tonnes of perl modules, some more critical than others. Not positive that's why your startup is failing, but that looks to be why your merge is failing. 

Note that you can fairly easily bypass amavis for the time being if getting mail through is more critical than having it scanned.

----------

## Fred Krogh

Your comments were helpful, but not quite enough to do the job.  I emerged what you suggested, and after there was no trouble emerging spamassassin and amavisd-new.  I then tried to emerge all the missing modules flagged by spamassasin.  On the last emerge of spamassassin, it says at the end *Quote:*   

> * A note from the SA developers:
> 
>  * Perl 5.8 now uses Unicode internally by default, which causes trouble for
> 
>  * SpamAssassin (and almost all other reasonably complex pieces of perl
> ...

 

Just prior to this emerge I had emerged all the ones listed above with no problems.

I'm using perl-5.10.1.  Might that be the problem?

Also, I commented out all references to amavis in postfix/main.cf and postfix/master.cf, but when restarting postfix I see *Quote:*   

> # /etc/init.d/postfix restart
> 
>  * Stopping postfix (/etc/postfix) ...                                                                        [ ok ]
> 
>  * Starting amavisd-new ...                                                                                   [ !! ]
> ...

 

It seems I don't know how to get postfix to start without amavis.  More help would be much appreciated.  Thanks,

Fred

----------

## cach0rr0

 *Fred Krogh wrote:*   

>  Your comments were helpful, but not quite enough to do the job.  I emerged what you suggested, and after there was no trouble emerging spamassassin and amavisd-new.  I then tried to emerge all the missing modules flagged by spamassasin.  On the last emerge of spamassassin, it says at the end *Quote:*   * A note from the SA developers:
> 
>  * Perl 5.8 now uses Unicode internally by default, which causes trouble for
> 
>  * SpamAssassin (and almost all other reasonably complex pieces of perl
> ...

 

So that all looks good - just build messages, nothing to worry about there. 

 *Fred Krogh wrote:*   

> 
> 
> Just prior to this emerge I had emerged all the ones listed above with no problems.
> 
> I'm using perl-5.10.1.  Might that be the problem?
> ...

 

Distinctly possible it's a version issue, or some other miscellaneous issue related to your build of perl. 

I've not tried amavisd-new on perl 5.10, couldn't say - I will say trying to run perl with USE="threads" is known to cause problems with amavis, and I have hit that one myself. 

 *Fred Krogh wrote:*   

> 
> 
> Also, I commented out all references to amavis in postfix/main.cf and postfix/master.cf, but when restarting postfix I see *Quote:*   # /etc/init.d/postfix restart
> 
>  * Stopping postfix (/etc/postfix) ...                                                                        [ ok ]
> ...

 

You don't even need to touch master.cf - just comment it out in main.cf. All leaving it in master.cf does is leave behind a vestigial instance of Postfix running on :10025 - unless your main.cf tells Postfix to ship a message off to this instance, it will never be used (as would be the case if you comment it out)

The fact that amavisd-new tries to start/restart whenever you do the same to Postfix would hint at the init script for Postfix having a dependency set for amavis. 

```

grep amavis /etc/init.d/postfix

```

amavis doesn't need to be functional for Postfix to start, but if the init script has this as a hard dependency, you'll get no joy trying to segregate the two. 

I would say make sure you dont have USE="threads" set for perl. If you don't, I would think a perl downgrade is in order. 

If you really really wan't to get to the heart of the problem, do this as root:

```

usermod -s /bin/bash amavis

su amavis -

cd

/usr/sbin/amavisd (you may want to fire this off in strace)

```

(and to clean up)

```

exit

usermod -s /bin/false amavis

```

Anyway, before you go through that trouble check the threads USE. I would say only try and go through that bit of troubleshooting if you have an absolutely huge aversion to downgrading perl.

If it helps, this is the entirety of my configuration - https://whitehathouston.com/topics/index.php/WHHMail (ignore the cert error)

I used this as a guide, deviating slightly as my needs were different - http://www.gentoo.org/doc/en/mailfilter-guide.xml

----------

## Fred Krogh

Hi cach0rr0 --

  I am extremely impressed at all that you suggested to help.  I can now get mail, but the mailq has lots and lots of messages that say (mail transport unavailable).  It is almost all spam, but there may be something in there that I'd want to see.  Is there a way to get it flushed?

The rest of what is written here is more of a rant.  I have amavisd started, but the path to getting there was painful, and the need for a soft link for "Convert" makes me uneasy.

I've tried your suggestion of running /usr/sbin/amavisd.  I've gotten rid of some of its complaints, but then got *Quote:*   

> ERROR: MISSING REQUIRED BASIC MODULES:
> 
>   MIME::Words
> 
>   MIME::Head
> ...

 I finally figured out that I needed to emerge MIME-tools.  Instead of the above I got *Quote:*   

> fetch_modules: error loading required module MIME/Head.pm:
> 
>   Can't locate Mail/Header.pm in @INC (@INC contains: /usr/lib64/perl5/site_perl/5.10.1/x86_64-linux /usr/lib64/perl5/site_perl/5.10.1 /usr/lib64/perl5/site_perl /usr/lib64/perl5/vendor_perl/5.10.1/x86_64-linux /usr/lib64/perl5/vendor_perl/5.10.1 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.10.1/x86_64-linux /usr/lib64/perl5/5.10.1) at /usr/lib64/perl5/vendor_perl/5.10.1/MIME/Head.pm line 120.
> 
>   BEGIN failed--compilation aborted at /usr/lib64/perl5/vendor_perl/5.10.1/MIME/Head.pm line 120.
> ...

 Note that Head, Entity Parser are still there.  "equery files MIME-tools" shows Head.pm, Parser.pm and Entity.pm.

Focusing on the first error, I finally discovered that I needed to emerge MailTools.  The next result was *Quote:*   

> fetch_modules: error loading optional module MIME/Decoder/BinHex.pm:
> 
>   Can't locate Convert/BinHex.pm in @INC (@INC contains: /usr/lib64/perl5/site_perl/5.10.1/x86_64-linux /usr/lib64/perl5/site_perl/5.10.1 /usr/lib64/perl5/site_perl /usr/lib64/perl5/vendor_perl/5.10.1/x86_64-linux /usr/lib64/perl5/vendor_perl/5.10.1 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.10.1/x86_64-linux /usr/lib64/perl5/5.10.1) at /usr/lib64/perl5/vendor_perl/5.10.1/MIME/Decoder/BinHex.pm line 43.
> 
>   BEGIN failed--compilation aborted at /usr/lib64/perl5/vendor_perl/5.10.1/MIME/Decoder/BinHex.pm line 43.
> ...

   Since /usr/lib64/perl5/vendor_perl/5.10.1 had no Convert directory I tried

"ln -s /usr/lib64/perl5/vendor_perl/5.8.8/Convert Convert" in that directory.  This led to  *Quote:*   

> Error reading config file "/etc/amavisd.conf": Permission denied at /usr/sbin/amavisd line 1804.

  I tried running /usr/sbin/amavisd as root and got *Quote:*   

> Problem in Amavis:B or Amavis:B::SNMP code: Can't locate BerkeleyDB.pm in @INC (@INC contains: /usr/lib64/perl5/site_perl/5.10.1/x86_64-linux /usr/lib64/perl5/site_perl/5.10.1 /usr/lib64/perl5/site_perl /usr/lib64/perl5/vendor_perl/5.10.1/x86_64-linux /usr/lib64/perl5/vendor_perl/5.10.1 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.10.1/x86_64-linux /usr/lib64/perl5/5.10.1) at (eval 95) line 19.
> 
> BEGIN failed--compilation aborted at (eval 95) line 19.
> 
> 

  Then an emerge of BerkeleyDB, gave *Quote:*   

> Problem in Amavis::Unpackers code: Can't locate Convert/UUlib.pm in @INC (@INC contains: /usr/lib64/perl5/site_perl/5.10.1/x86_64-linux /usr/lib64/perl5/site_perl/5.10.1 /usr/lib64/perl5/site_perl /usr/lib64/perl5/vendor_perl/5.10.1/x86_64-linux /usr/lib64/perl5/vendor_perl/5.10.1 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.10.1/x86_64-linux /usr/lib64/perl5/5.10.1) at (eval 105) line 33.
> 
> BEGIN failed--compilation aborted at (eval 105) line 33.
> 
> 

 An install of Covert-UUlib resulted in *Quote:*   

> Problem in Amavis::Unpackers code: Can't locate Archive/Zip.pm in @INC (@INC contains: /usr/lib64/perl5/site_perl/5.10.1/x86_64-linux /usr/lib64/perl5/site_perl/5.10.1 /usr/lib64/perl5/site_perl /usr/lib64/perl5/vendor_perl/5.10.1/x86_64-linux /usr/lib64/perl5/vendor_perl/5.10.1 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.10.1/x86_64-linux /usr/lib64/perl5/5.10.1) at (eval 105) line 35.
> 
> BEGIN failed--compilation aborted at (eval 105) line 35

 An emerge of Archive-Zip gave us a much longer error *Quote:*   

> fetch_modules: error loading optional module IO/Socket/INET6.pm:
> 
>   Attempt to reload IO/Socket/INET6.pm aborted.
> 
>   Compilation failed in require at /usr/sbin/amavisd line 197.
> ...

 But at least at this point I'm getting the hang of finding things.  So I emerged IO-Socket-INET6, and then when trying to run /usr/sbin/amavisd I got a message that it was already running.  I killed everything that amavis in the name, and /etc/init.d/amavisd actually started.  In my humble opinion it should not be this difficult.

----------

## Fred Krogh

Sorry -- postqueue -f is now flushing the mailq.  All is working.

----------

## cach0rr0

it shouldn't be that difficult normally. in theory if you pull amavis through portage, all of those perl modules should have been pulled as dependencies (see below - snip from the ebuild)

I'm a touch surprised they weren't. I wonder if they weren't clandestinely nuked at some point by one of the usual tidying up tasks? 

```

DEPEND=">=sys-apps/sed-4

        >=dev-lang/perl-5.8.2

        milter? ( || ( mail-filter/libmilter >=mail-mta/sendmail-8.12 ) )"

RDEPEND="${DEPEND}

        >=sys-apps/coreutils-5.0-r3

        app-arch/gzip

        app-arch/bzip2

        app-arch/arc

        app-arch/cabextract

        app-arch/freeze

        app-arch/lha

        app-arch/unarj

        || ( app-arch/unrar app-arch/rar )

        app-arch/zoo

        >=dev-perl/Archive-Zip-1.14

        >=virtual/perl-IO-Compress-1.35

        dev-perl/Convert-TNEF

        >=dev-perl/Convert-UUlib-1.08

        virtual/perl-MIME-Base64

        >=dev-perl/MIME-tools-5.415

        >=dev-perl/MailTools-1.58

        >=dev-perl/net-server-0.91

        virtual/perl-Digest-MD5

        dev-perl/IO-stringy

        >=virtual/perl-Time-HiRes-1.49

        dev-perl/Unix-Syslog

        >=sys-libs/db-3.2

        dev-perl/BerkeleyDB

        dev-perl/Convert-BinHex

        >=dev-perl/Mail-DKIM-0.31

        virtual/mta

        ldap? ( >=dev-perl/perl-ldap-0.33 )

        mysql? ( dev-perl/DBD-mysql )

        postgres? ( dev-perl/DBD-Pg )

        razor? ( mail-filter/razor )

        spamassassin? ( mail-filter/spamassassin )"

```

----------

## Fred Krogh

Both equery and emerge -s showed them installed so it makes sense that portage would not have brought them in.  Now were they really installed or did they need to be re-installed for some mysterious reason?  I have no idea.  Your help was extremely helpful whatever the reason.

----------

## cach0rr0

funny enough, i just did this same upgrade, and hit this exact issue myself. 

royal pain in the ass, and very un-portage-like the way i had to rebuild shit tonnes of packages to get this working (some just had to be rebuilt against the new perl 5.10)

not sure what of these was the silver bullet, but

```

 696  ACCEPT_KEYWORDS="~amd64" emerge -pv IO-Stringy MIME-Tools MIME-Parser Unix-Syslog

  697  ACCEPT_KEYWORDS="~amd64" emerge -pv MIME-Tools MIME-Parser Unix-Syslog

  698  ACCEPT_KEYWORDS="~amd64" emerge -pv  MIME-Parser Unix-Syslog

  699  emerge MIME-Decoder

  705  emerge LWP-User-Agent-Determined

  706  emerge LWP-UserAgent-Determined

  707  ACCEPT_KEYWORDS="~amd64" emerge LWP-UserAgent-Determined

  710  emerge IO-stringy

  713  emerge Unix-Syslog

  717  emerge -v dev-perl/MIME-tools

  718  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/MIME-tools

  721  emerge MIME-Types

  733  emerge net-server

  739  ACCEPT_KEYWORDS="~amd64" emerge -pv MIME-tools

  740  ACCEPT_KEYWORDS="~amd64" emerge -v MIME-tools

  742  emerge MailTools

  743  ACCEPT_KEYWORDS="~amd64" emerge -pv MailTools

  744  ACCEPT_KEYWORDS="~amd64" emerge -v MailTools

  747  ACCEPT_KEYWORDS="~amd64" emerge -v BerkeleyDB

  750  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/DBI

  755  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/Convert-UUlib

  758  emerge Convert-TNEF

  761  ACCEPT_KEYWORDS="~amd64" emerge -v Archive-Zip

  764  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/DBD-mysql

  766  emerge -v dev-perl/DBD-mysql

  769  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/Mail-DKIM

  771  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/Mail-SPF

  777  ACCEPT_KEYWORDS="~amd64" emerge dev-perl/URI

  778  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/Mail-SPF

  780  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/Error

  781  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/Mail-SPF

  783  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/Mail-DKIM

  786  ACCEPT_KEYWORDS="~amd64" emerge -v dev-perl/Crypt-OpenSSL-Bignum dev-perl/Crypt-OpenSSL-RSA dev-perl/Crypt-OpenSSL-Random

```

had to rebuild Mail-SPF and Mail-DKIM multiple times, because I'd noticed as the text scrolled by 100 miles an hour, it was missing URI and Error. So I merged those two, then remerged Mail-DKIM and Mail-SPF. 

Other than that it was just making educated guesses on what packages had what, and merging the requisite stuff (again, some of which were already installed, but I guess had to be built against perl 5.10 or something)

I'm too tired/lazy to sift through that list and identify what all worked, but that snippet of `history |grep emerge` shows the process I went through.

----------

## cyrius

Just go for a qclist -IC dev-perl | xargs emerge solve this.

I've got the same problem. Seems to be link to an update in dev-perl or perl itself.

----------

## Fred Krogh

You want qlist where you have qclist, but is a nice way to take care of this sort of problem.

----------

## Fred Krogh

Had a similar problem again!!  Ran *Quote:*   

> qlist -IC dev-perl | xargs emerge

 and  *Quote:*   

> postqueue -f

 and the mail is once again working.

----------

## aZZe

Today I upgraded to dev-lang/perl-5.16.3:0/5.16 and I got the same issue. I couldn't start amavisd-new anymore. Trillions of perl modules were broken! WTF? To be honest this is pretty much pain in the ass. 

```

qlist -IC dev-perl | xargs emerge

```

and a emerge spamassassin did fix it for me to get email up and running again.

----------

## Moriah

Same here.  My mail is full of chinese stuff that is cearting great havoc because spamd is not running,    :Evil or Very Mad:   :Evil or Very Mad:   :Evil or Very Mad:   :Evil or Very Mad:   :Evil or Very Mad: 

Seems like a recurrance of the problem this thread started with several years ago.  I believe that someone clobbered the dependencies in the spamassassin embuild, probably because the new version of perl has packages more finely factored, and some stufff that used to get pulle din with other things in perl is now packages separately, but that's just my guess.    :Confused: 

Hopefully this will get fixed soon, as the hoiliday season is known to be bad as far as spam/bots/trojans/malware,etc goes, since a lot of kiddies are plugging in brand new systems that have sat on the shef in a warehouse for 6 months without an update.    :Sad: 

Until them, does anybody know a quick way to learn to read chinese?    :Rolling Eyes:   :Wink: 

----------

## Moriah

After waiting a week for the next weekly update to sort things out, my spamassassin still would not work, so I took matters into my own hands and manually emerged all the missing perl modules, then re-emerged spamassassin, and now it works again.    :Smile: 

The ebuild for spamassassin still needs to be fixed, since this manual horsing around should not need to be done.    :Evil or Very Mad: 

----------

