# clamd crashing - Urgent

## hanj

Hello All

As of 1PM today, clamd has been having a rough time staying up. Looking at the logs, I saw the following:

```
Jun 28 13:09:02 comp freshclam[12068]: freshclam daemon 0.96.1 (OS: linux-gnu, ARCH: i386, CPU: i686)

Jun 28 13:09:02 comp freshclam[12068]: ClamAV update process started at Mon Jun 28 13:09:02 2010

Jun 28 13:09:03 comp freshclam[12068]: main.cvd is up to date (version: 52, sigs: 704727, f-level: 44, builder: sven)

Jun 28 13:09:04 comp freshclam[12068]: Downloading daily-11274.cdiff [100%]

Jun 28 13:09:14 comp freshclam[12068]: daily.cld updated (version: 11274, sigs: 99028, f-level: 53, builder: ccordes)

Jun 28 13:09:14 comp freshclam[12068]: bytecode.cvd is up to date (version: 28, sigs: 6, f-level: 53, builder: nervous)

Jun 28 13:09:14 comp freshclam[12068]: Can't open mirrors.dat for writing

Jun 28 13:09:14 comp freshclam[12068]: Database updated (803761 signatures) from database.clamav.net (IP: 194.186.47.19)

Jun 28 13:09:14 comp freshclam[12068]: Clamd successfully notified about the update.

Jun 28 13:09:14 comp freshclam[12068]: --------------------------------------

Jun 28 13:09:50 comp kernel: PAX: terminating task: /usr/sbin/clamd(clamd):12092, uid/euid: 105/105, PC: 48a716d0, SP: 4820c2ec

Jun 28 13:10:25 comp kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):12255, uid/euid: 105/105, PC: 462946d0, SP: 5cc9f08c

Jun 28 13:26:27 comp kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):14290, uid/euid: 105/105, PC: 3f7eb6d0, SP: 581766cc

Jun 28 13:34:56 comp kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):15237, uid/euid: 105/105, PC: 4a4b26d0, SP: 5b5060ac

Jun 28 13:36:36 comp kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):15481, uid/euid: 105/105, PC: 4cad66d0, SP: 5eab854c

Jun 28 13:36:40 comp kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):15486, uid/euid: 105/105, PC: 468226d0, SP: 587e32cc

Jun 28 13:38:16 comp kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):15631, uid/euid: 105/105, PC: 44fd16d0, SP: 5913079c

Jun 28 13:38:47 comp freshclam[12068]: Update process interrupted
```

Restarts would work, but then eventually fail. The mailq was quite loaded, so I tried to flush those out (exclude virus scan) and rebuild clamav, thinking it was related to one of my updates.

Today I updated the following:

```
     Sun Jun 27 17:51:37 2010 >>> sys-libs/glibc-2.11.1

     Sun Jun 27 17:52:38 2010 >>> dev-libs/libassuan-2.0.0

     Sun Jun 27 17:53:02 2010 >>> sys-apps/help2man-1.37.1
```

I rebuilt clamav.. and thought I was good, then several hours later, I received the same problem..

```
Jun 28 15:35:54 comp freshclam[26277]: freshclam daemon 0.96.1 (OS: linux-gnu, ARCH: i386, CPU: i686)

Jun 28 15:35:54 comp freshclam[26277]: ClamAV update process started at Mon Jun 28 15:35:54 2010

Jun 28 15:35:54 comp freshclam[26277]: main.cvd is up to date (version: 52, sigs: 704727, f-level: 44, builder: sven)

Jun 28 15:36:24 comp freshclam[26277]: getfile: daily-11044.cdiff not found on remote server (IP: 209.209.47.66)

Jun 28 15:36:24 comp freshclam[26277]: getpatch: Can't download daily-11044.cdiff from database.clamav.net

Jun 28 15:36:42 comp freshclam[26277]: getfile: daily-11044.cdiff not found on remote server (IP: 199.184.215.2)

Jun 28 15:36:42 comp freshclam[26277]: getpatch: Can't download daily-11044.cdiff from database.clamav.net

Jun 28 15:36:42 comp freshclam[26277]: Trying host database.clamav.net (64.142.100.50)...

Jun 28 15:36:42 comp freshclam[26277]: getfile: daily-11044.cdiff not found on remote server (IP: 64.142.100.50)

Jun 28 15:36:42 comp freshclam[26277]: getpatch: Can't download daily-11044.cdiff from database.clamav.net

Jun 28 15:36:42 comp freshclam[26277]: Incremental update failed, trying to download daily.cvd

Jun 28 15:36:42 comp freshclam[26277]: Trying host database.clamav.net (150.214.142.197)...

Jun 28 15:37:07 comp freshclam[26277]: Downloading daily.cvd [100%]

Jun 28 15:37:42 comp freshclam[26277]: daily.cvd updated (version: 11274, sigs: 99028, f-level: 53, builder: ccordes)

Jun 28 15:37:42 comp freshclam[26277]: bytecode.cvd is up to date (version: 28, sigs: 6, f-level: 53, builder: nervous)

Jun 28 15:37:42 comp freshclam[26277]: Can't open mirrors.dat for writing

Jun 28 15:37:42 comp freshclam[26277]: Database updated (803761 signatures) from database.clamav.net (IP: 150.214.142.197)

Jun 28 15:37:42 comp freshclam[26277]: Clamd successfully notified about the update.

Jun 28 15:37:42 comp freshclam[26277]: --------------------------------------

Jun 28 15:51:26 comp kernel: PAX: terminating task: /usr/sbin/clamd(clamd):27911, uid/euid: 105/105, PC: 406946d0, SP: 406922ec

Jun 28 15:51:54 comp kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):29592, uid/euid: 105/105, PC: 4a2206d0, SP: 5deab04c

Jun 28 16:08:25 comp freshclam[26277]: Update process interrupted
```

I pushed the queue out again, and I seem to be up for the moment. I thought this was odd.. then on a entirely different server, I saw the same thing:

```
Jun 28 15:02:38s comp2 kernel: PAX: terminating task: /usr/sbin/clamd(clamd):8276, uid/euid: 106/106, PC: 4a24d6d0, SP: 499e82ec

Jun 28 15:02:53 scomp2 kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):8282, uid/euid: 106/106, PC: 455516d0, SP: 59d59bac

Jun 28 15:02:53 scomp2 kernel: grsec: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /usr/bin/clamscan[clamscan:8282] uid/euid:106/106 gid/egid:1056/1056, parent /usr/sbin/amavisd[amavisd:5501] uid/euid:106/106 gid/egid:1056/1056

Jun 28 15:09:16 scomp2 kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):8517, uid/euid: 106/106, PC: 472336d0, SP: 5cb962ac

Jun 28 15:09:16 scomp2 kernel: grsec: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /usr/bin/clamscan[clamscan:8517] uid/euid:106/106 gid/egid:1056/1056, parent /usr/sbin/amavisd[amavisd:31886] uid/euid:106/106 gid/egid:1056/1056

Jun 28 15:19:15 scomp2 kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):8994, uid/euid: 106/106, PC: 450c56d0, SP: 5d9d831c

Jun 28 15:19:15 scomp2 kernel: grsec: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /usr/bin/clamscan[clamscan:8994] uid/euid:106/106 gid/egid:1056/1056, parent /usr/sbin/amavisd[amavisd:31886] uid/euid:106/106 gid/egid:1056/1056

Jun 28 15:39:15 scomp2 kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):9799, uid/euid: 106/106, PC: 4040e6d0, SP: 584e62cc

Jun 28 15:39:15 scomp2 kernel: grsec: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /usr/bin/clamscan[clamscan:9799] uid/euid:106/106 gid/egid:1056/1056, parent /usr/sbin/amavisd[amavisd:9605] uid/euid:106/106 gid/egid:1056/1056

Jun 28 16:00:07 scomp2 freshclam[5900]: Update process interrupted
```

The other server had similar updates, but for the sake if this discussion, I'll just focus on the first server. Here is the emerge --info output of the first server:

```

Portage 2.1.8.3 (hardened/linux/x86/10.0, gcc-4.3.4, glibc-2.11.1-r0, 2.6.28-hardened-r9 i686)

=================================================================

System uname: Linux-2.6.28-hardened-r9-i686-AMD_Duron-TM-with-gentoo-1.12.13

Timestamp of tree: Mon, 28 Jun 2010 08:30:01 +0000

app-shells/bash:     4.0_p37

dev-java/java-config: 2.1.10

dev-lang/python:     2.5.4-r3, 2.6.5-r2, 3.1.2-r3

sys-apps/baselayout: 1.12.13

sys-apps/sandbox:    1.6-r2

sys-devel/autoconf:  2.13, 2.65

sys-devel/automake:  1.7.9-r1, 1.9.6-r2, 1.10.3, 1.11.1

sys-devel/binutils:  2.20.1-r1

sys-devel/gcc:       4.1.2, 4.3.4

sys-devel/gcc-config: 1.4.1

sys-devel/libtool:   2.2.6b

virtual/os-headers:  2.6.30-r1

ACCEPT_KEYWORDS="x86"

ACCEPT_LICENSE="* -@EULA"

CBUILD="i686-pc-linux-gnu"

CFLAGS="-O3 -march=i686 -funroll-loops -pipe "

CHOST="i686-pc-linux-gnu"

CONFIG_PROTECT="/etc"

CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"

CXXFLAGS="-O3 -march=i686 -funroll-loops -pipe "

DISTDIR="/usr/portage/distfiles"

FEATURES="assume-digests distlocks fixpackages news parallel-fetch protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"

GENTOO_MIRRORS="http://distfiles.gentoo.org"

LDFLAGS="-Wl,-O1"

LINGUAS="en"

MAKEOPTS="-j2"

PKGDIR="/usr/portage/packages"

PORTAGE_CONFIGROOT="/"

PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"

PORTAGE_TMPDIR="/var/tmp"

PORTDIR="/usr/portage"

PORTDIR_OVERLAY="/usr/local/portage"

SYNC="rsync://rsync.gentoo.org/gentoo-portage"

USE="apache2 berkdb bzip2 cli cracklib crypt cxx dri gdbm gpm hardened iconv innodb maildir modules mudflap mysql ncurses nptl nptlonly openmp openssh pam pcre perl php pic pppd pwdb python readline reflection sasl session snortsam spl ssl sysfs tcpd urandom x86 xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1   emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m       maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" APACHE2_MPMS="prefork" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i740 intel      mach64 mga neomagic nsc nv r128 radeon rendition s3 s3virge savage  siliconmotion sis sisusb tdfx tga trident tseng v4l vesa via vmware     voodoo" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"

Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
```

Here are USE flags for clamav:

```
[ebuild   R   ] app-antivirus/clamav-0.96.1  USE="bzip2 iconv -clamdtop -ipv6 -milter (-selinux)" 0 kB
```

I ran revdep-rebuild, and nothing clam related came up. I also notice that it continually says 'Update process interrupted', which makes me wonder if it's a specific update from the mirror causing the problem. If so.. you'd think there would be tons of chatter on this problem.

Also, you can see it's PAX and grsec related, but there have been no changes with the kernel for a bit on either server. No changes with clamav prior, but could be a dependency/library associated.. or as I said, something from clam directly.

Anyone have any ideas what the problem could be.. or where I should poke further?

Thanks!

hanji

----------

## Hu

 *hanj wrote:*   

> Also, you can see it's PAX and grsec related, but there have been no changes with the kernel for a bit on either server.

 Not exactly.  PaX may be terminating the task in some cases, but that would usually indicate that the program was misbehaving.  The grsecurity messages inform you that the core dump was blocked.

 *hanj wrote:*   

> Anyone have any ideas what the problem could be.. or where I should poke further?

 Get a core dump and find out why it died.

----------

## hanj

 *Hu wrote:*   

> Get a core dump and find out why it died.

 

What do you suggest for getting the core dump? I was able to strace it when it died, but didn't seem very useful. 

```

read(5, "", 4096)                       = 0

read(5, "", 8192)                       = 0

access("/var/lib/clamav/bytecode.cld", R_OK) = -1 ENOENT (No such file or directory)

lseek(5, 512, SEEK_SET)                 = 512

read(5, "\37\213\10\0\0\0\0", 7)        = 7

lseek(5, 512, SEEK_SET)                 = 512

dup(5)                                  = 6

fcntl64(6, F_GETFL)                     = 0 (flags O_RDONLY)

fstat64(6, {st_mode=S_IFREG|0644, st_size=14725, ...}) = 0

mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x54c3d000

_llseek(6, 0, [512], SEEK_CUR)          = 0

read(6, "\37\213\10\0\0\0\0\0\0\3\355\275kw\0337\2220\274_\267\317\376\210:\371\360\304>\303hu"..., 16384) = 14213

read(6, "", 4096)                       = 0

close(6)                                = 0

munmap(0x54c3d000, 4096)                = 0

lseek(5, 512, SEEK_SET)                 = 512

read(5, "\37\213\10\0\0\0\0", 7)        = 7

lseek(5, 512, SEEK_SET)                 = 512

dup(5)                                  = 6

fcntl64(6, F_GETFL)                     = 0 (flags O_RDONLY)

fstat64(6, {st_mode=S_IFREG|0644, st_size=14725, ...}) = 0

mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x54c3d000

_llseek(6, 0, [512], SEEK_CUR)          = 0

read(6, "\37\213\10\0\0\0\0\0\0\3\355\275kw\0337\2220\274_\267\317\376\210:\371\360\304>\303hu"..., 16384) = 14213

read(6, "", 4096)                       = 0

read(6, "", 16384)                      = 0

close(6)                                = 0

munmap(0x54c3d000, 4096)                = 0

close(5)                                = 0

munmap(0x54c3e000, 4096)                = 0

getdents(4, /* 0 entries */, 32768)     = 0

close(4)                                = 0

stat64("/var/log/clamav/clamd.log", {st_mode=S_IFREG|0640, st_size=33871, ...}) = 0

time(NULL)                              = 1277771778

write(3, "Mon Jun 28 18:36:18 2010 -> Load"..., 54) = 54

mmap2(NULL, 159744, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4ccf4000

munmap(0x4ccf4000, 159744)              = 0

munmap(0x547fc000, 266240)              = 0

munmap(0x4cd21000, 237568)              = 0

mmap2(NULL, 65536, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x5482d000

mmap2(0x5483d000, 65536, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x5481d000

mmap2(0x5482d000, 524288, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4cc9b000

brk(0x80f6000)                          = 0x80f6000

gettimeofday({1277771780, 520646}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 520836}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 520969}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 521100}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 521100}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 521232}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 521363}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 521526}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 521662}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 521792}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 521921}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 522050}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 522179}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 522307}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 522436}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 522565}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 522713}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 522844}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 522974}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 523104}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 523234}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 523364}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 523494}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 523624}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 523785}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 523916}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 524046}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 524176}, NULL) = 0

times({tms_utime=908, tms_stime=43, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 524337}, NULL) = 0

times({tms_utime=908, tms_stime=44, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 524470}, NULL) = 0

times({tms_utime=908, tms_stime=44, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 524470}, NULL) = 0

times({tms_utime=908, tms_stime=44, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 524599}, NULL) = 0

times({tms_utime=908, tms_stime=44, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 524728}, NULL) = 0

times({tms_utime=908, tms_stime=44, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 524857}, NULL) = 0

times({tms_utime=908, tms_stime=44, tms_cutime=0, tms_cstime=0}) = 632182636

gettimeofday({1277771780, 525003}, NULL) = 0

times({tms_utime=908, tms_stime=44, tms_cutime=0, tms_cstime=0}) = 632182636

brk(0x8117000)                          = 0x8117000

brk(0x8138000)                          = 0x8138000

umask(0777)                             = 022

socket(PF_FILE, SOCK_STREAM, 0)         = 4

bind(4, {sa_family=AF_FILE, path="/var/amavis/tmp/clamd"}, 110) = 0

stat64("/var/log/clamav/clamd.log", {st_mode=S_IFREG|0640, st_size=33925, ...}) = 0

time(NULL)                              = 1277771782

write(3, "Mon Jun 28 18:36:22 2010 -> LOCA"..., 74) = 74

stat64("/var/log/clamav/clamd.log", {st_mode=S_IFREG|0640, st_size=33999, ...}) = 0

time(NULL)                              = 1277771782

write(3, "Mon Jun 28 18:36:22 2010 -> LOCA"..., 73) = 73

listen(4, 15)                           = 0

umask(022)                              = 0777

chmod("/var/amavis/tmp/clamd", 0666)    = 0

open("/dev/null", O_RDONLY)             = 5

open("/dev/null", O_WRONLY)             = 6

open("/dev/null", O_WRONLY)             = 7

dup2(5, 0)                              = 0

dup2(6, 1)                              = 1

dup2(7, 2)                              = 2

close(5)                                = 0

close(6)                                = 0

close(7)                                = 0

clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x54f93738) = 32710

munmap(0x4cc9b000, 524288)              = 0

munmap(0x5481d000, 65536)               = 0

munmap(0x5482d000, 65536)               = 0

munmap(0x53bdb000, 135168)              = 0

munmap(0x4cd1b000, 24576)               = 0

munmap(0x4cd5b000, 262144)              = 0

munmap(0x4cd9b000, 262144)              = 0

munmap(0x4cddb000, 262144)              = 0

munmap(0x4ce1b000, 262144)              = 0

munmap(0x4ce5b000, 262144)              = 0

munmap(0x4ce9b000, 262144)              = 0

munmap(0x4cedb000, 262144)              = 0

munmap(0x4cf1b000, 262144)              = 0

munmap(0x4cf5b000, 262144)              = 0

munmap(0x4cf9b000, 262144)              = 0

munmap(0x4cfdb000, 262144)              = 0

munmap(0x4d01b000, 262144)              = 0

munmap(0x4d05b000, 262144)              = 0

munmap(0x4d09b000, 262144)              = 0

... tons of these...

exit_group(0)                           = ?

```

I'm thinking I need to get the reason why PAX/grsec is terminating.

Thanks!

hanji

----------

## hanj

Ok.. Looks like I was missing some of the logs. PAX is complaining about anonymous mapping??

```
Jun 28 20:17:14 comp2 freshclam[14917]: freshclam daemon 0.96.1 (OS: linux-gnu, ARCH: i386, CPU: i686)

Jun 28 20:17:14 comp2 freshclam[14917]: ClamAV update process started at Mon Jun 28 20:17:14 2010

Jun 28 20:17:14 comp2 freshclam[14917]: main.cvd is up to date (version: 52, sigs: 704727, f-level: 44, builder: sven)

Jun 28 20:17:27 comp2 kernel: PAX: From xxx.xxx.xxx.xxx: execution attempt in: <anonymous mapping>, 444ec000-44d72000 444ec000

Jun 28 20:17:27 comp2 kernel: PAX: terminating task: /usr/sbin/clamd(clamd):14936, uid/euid: 106/106, PC: 44cec6d0, SP: 444e92ec

Jun 28 20:17:27 comp2 kernel: PAX: bytes at PC: 83 ec 04 8b 4c 24 08 e8 d4 fe ff ff 83 c4 04 c3 b3 04 00 00

Jun 28 20:17:27 comp2 kernel: PAX: bytes at SP-4:

Jun 28 20:17:39 comp2 freshclam[14917]: Update process interrupted

Jun 28 20:17:42 comp2 kernel: PAX: execution attempt in: <anonymous mapping>, 456c7000-45f4d000 456c7000

Jun 28 20:17:42 comp2 kernel: PAX: terminating task: /usr/bin/clamscan(clamscan):14939, uid/euid: 106/106, PC: 45ec76d0, SP: 596155fc

Jun 28 20:17:42 comp2 kernel: PAX: bytes at PC: 83 ec 04 8b 4c 24 08 e8 d4 fe ff ff 83 c4 04 c3 b3 04 00 00

Jun 28 20:17:42 comp2 kernel: PAX: bytes at SP-4:

Jun 28 20:17:42 comp2 kernel: grsec: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /usr/bin/clamscan[clamscan:14939] uid/euid:106/106 gid/egid:1056/1056, parent /usr/sbin/amavisd[amavisd:5193] uid/euid:106/106 gid/egid:1056/1056
```

I also tried to set the ulimit -c unlimited, but I might have to adjust that via grsec?

Thanks!

hanji

----------

## cach0rr0

what's the emerge --info for the other server? 

Was reading through this bug - https://bugs.gentoo.org/show_bug.cgi?id=275928

wondering if toning down the cflags a bit and rebuilding might improve the behaviour here (e.g. just change to -march=i686 -pipe, and remove -funroll-loops as well remove -O3 )

----------

## hanj

Here is emerge --info on the other server..

```

Portage 2.1.8.3 (hardened/linux/x86/10.0, gcc-4.3.4, glibc-2.11.2-r0, 2.6.28-hardened-r9 i686)

=================================================================

System uname: Linux-2.6.28-hardened-r9-i686-AMD_Sempron-tm-_2600+-with-gentoo-1.12.13

Timestamp of tree: Mon, 28 Jun 2010 09:00:01 +0000

app-shells/bash:     4.0_p37

dev-lang/python:     2.5.4-r3, 2.6.5-r2, 3.1.2-r3

sys-apps/baselayout: 1.12.13

sys-apps/sandbox:    1.6-r2

sys-devel/autoconf:  2.13, 2.65

sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.3, 1.11.1

sys-devel/binutils:  2.20.1-r1

sys-devel/gcc:       3.4.6-r2, 4.1.2, 4.3.4

sys-devel/gcc-config: 1.4.1

sys-devel/libtool:   2.2.6b

virtual/os-headers:  2.6.30-r1

ACCEPT_KEYWORDS="x86"

ACCEPT_LICENSE="* -@EULA"

CBUILD="i686-pc-linux-gnu"

CFLAGS="-march=athlon-xp -O3 -pipe"

CHOST="i686-pc-linux-gnu"

CONFIG_PROTECT="/etc /var/bind"

CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"

CXXFLAGS="-march=i686 -O2 -pipe"

DISTDIR="/usr/portage/distfiles"

FEATURES="assume-digests distlocks fixpackages news parallel-fetch protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"

GENTOO_MIRRORS="http://distfiles.gentoo.org"

LDFLAGS="-Wl,-O1"

LINGUAS="en"

PKGDIR="/usr/portage/packages"

PORTAGE_CONFIGROOT="/"

PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"

PORTAGE_TMPDIR="/var/tmp"

PORTDIR="/usr/portage"

PORTDIR_OVERLAY="/usr/local/portage"

SYNC="rsync://rsync.gentoo.org/gentoo-portage"

USE="apache2 berkdb bzip2 cli cracklib crypt cxx dri gdbm gpm hardened iconv innodb maildir modules mudflap mysql ncurses nptl nptlonly openmp openssh pam pcre perl php pic pppd pwdb python readline reflection sasl session snmp snortsam spl ssl sysfs tcpd urandom x86 xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1  emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m   maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" APACHE2_MPMS="prefork" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i740 intel  mach64 mga neomagic nsc nv r128 radeon rendition s3 s3virge savage  siliconmotion sis sisusb tdfx tga trident tseng v4l vesa via vmware     voodoo" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"

Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

```

What I don't get... why this is happening on two servers.. out of the blue.

hanji

----------

## cach0rr0

 *hanj wrote:*   

> Here is emerge --info on the other server..
> 
> 

 

I would try plucking out -O3 from that server, and pluck out both -funroll-loops and -O3 on the other, rebuild, see if it persists. 

 *hanj wrote:*   

> 
> 
> What I don't get... why this is happening on two servers.. out of the blue.
> 
> 

 

Not all that uncommon  with AV scanners to be honest. A rogue DAT update comes out that causes the scanner to crash on certain files or file types, utter chaos for a while, dats or engine are updated, all is well. 

Or, the same old dats/engines, and a new type of binary comes out (e.g. new virus) that when scanned causes the scanners to upchuck. 

Seen it all too many times with commercial AV vendors. 

From reading through the bug, it seems this can be exacerbated/avoided based on compile settings in some cases. I can say, however, that if indeed this is logging-a-bug-worthy, one of the first things the assignee will have you do is likely taper back the -O3 and -funroll-loops (on the one box).

----------

## hanj

so.. do you think I should roll with these CFLAGS?

```
CFLAGS="-O2 -march=i686 -pipe "
```

Thanks much for the replies!!

hanji

----------

## cach0rr0

 *hanj wrote:*   

> so.. do you think I should roll with these CFLAGS?
> 
> ```
> CFLAGS="-O2 -march=i686 -pipe "
> ```
> ...

 

At least for this package, yes. 

Actually, for this package,remove the -O2 as well. You can set it back later once you've tested to confirm whether or not this has any effect on the issue. 

(also, if your profile is set to use that GCC 4.3 you have installed, you could do -march=native. The usual gcc-config -l should show what you're using)

----------

## hanj

 *cach0rr0 wrote:*   

> (also, if your profile is set to use that GCC 4.3 you have installed, you could do -march=native. The usual gcc-config -l should show what you're using)

 

I'm using i686-pc-linux-gnu-4.3.4.

Related note.. I posted a bug at clam...

https://wwws.clamav.net/bugzilla/show_bug.cgi?id=2092

Thanks!

hanji

----------

## agoston7777

Have you solved the problem? Because 2 of my servers also doing this. 

My solution was to paxctl -m /usr/sbin/clamd, but i dont really like it

----------

## hanj

 *agoston7777 wrote:*   

> Have you solved the problem? Because 2 of my servers also doing this. 
> 
> My solution was to paxctl -m /usr/sbin/clamd, but i dont really like it

 

I'm still working on it. I didn't want to go down the paxctl route. If you can.. might be helpful to join in on the bug at : https://wwws.clamav.net/bugzilla/show_bug.cgi?id=2092

Thanks!

hanji

----------

## hanj

Does anyone know how to get the output of 'make check' to happen? I see the following in my build log:

```
Test phase [not enabled]: app-antivirus/clamav-0.96.1
```

I want to activate this test phase, but I'm not sure how to do this. The ebuild shows RESTRICT="test". Is there way to override this without doing a new ebuild in local portage? I don't see a USE flag option.

Thanks!

hanji

----------

## agoston7777

```
You are not authorized to access bug #2092.
```

I can't believe, that only us hit the bug...

Any solution?

----------

## hanj

 *agoston7777 wrote:*   

> 
> 
> ```
> You are not authorized to access bug #2092.
> ```
> ...

 

I just requested that the bug be 'public'. Looks to be restricted to 'security group'. Not sure why.

They said they new what the problem is after going through various rebuilds/test of clamav on my system. He's working on a way to make this work on PaX systems now. I guess I'll post this in Gentoo bugs as well.

hanji

----------

## hanj

Ok.. I created the following bug:

https://bugs.gentoo.org/show_bug.cgi?id=326199

yeah... agoston7777.. I guess we're the lucky ones. I went ahead and ran paxctl -m on clam. No problems after that? I would imagine that it'll be a little bit before clamav has a fix.. then a little bit for it to be introduced in portage.

hanji

----------

## hanj

 *agoston7777 wrote:*   

> 
> 
> ```
> You are not authorized to access bug #2092.
> ```
> ...

 

Ok.. the bug is public now.

https://wwws.clamav.net/bugzilla/show_bug.cgi?id=2092

hanjij

----------

## hanj

 *agoston7777 wrote:*   

> Have you solved the problem? Because 2 of my servers also doing this. 
> 
> My solution was to paxctl -m /usr/sbin/clamd, but i dont really like it

 

Did you have to do anything else? I'm still having problems with this set.

Here is my output showing it's disabled...

```
paxctl -v /usr/sbin/clamd

PaX control v0.5

Copyright 2004,2005,2006,2007 PaX Team <pageexec@freemail.hu>

- PaX flags: -----m-x-e-- [/usr/sbin/clamd]

      MPROTECT is disabled

      RANDEXEC is disabled

      EMUTRAMP is disabled
```

I'm wondering f I need to do that to freshclam?

Update: I just added -m on clamscan, but I'm thinking I need -p (disable PAGEEXEC)? We'll see what happens.

Update: So it's still getting stopped by PAX, I added -p flag

Update: So it's still having problems after -m and -p flags have been applied via paxctl!.. wtf????

Thanks!

hanji

----------

## agoston7777

So far the paxctl -p solved my problems. Have you recompiled the clamav as suggested before? Becuase one of my server was on -O1 CFLAGS, and the clamd even didn't start. When I changed to -O2 than it was started just crashed after some little time. Maybe the -O2 compilation makes the least problem.

I am going to clamav bugtracker to post this info.

----------

## hanj

Thanks for your post on the bug! I'm glad it's starting to get some activity. I ended up using 'Bytecode off' in freshclam.conf. We'll see if it stays up. Been a very frustrating couple of days.

hanji

----------

