# clamd killed by OOM

## Ringtail

Hi all.  I have recently set up an inexpensive home router/server box with Gentoo Hardened (amd64 stable), running quite a bit of software.  In particular, I've configured squid to scan http traffic via squidclamav/clamd (there are two Windows PCs behind the router).  It works beautifully (or, well, at least it detects EICAR test file correctly   :Very Happy: ), but yesterday when logging on that machine via ssh I noticed that all swap space and most of RAM was occupied, and that clamd had a RSS of 350+ MB.  Later (at about 3 AM) clamd was terminated by oom-killer, and it took me another eighteen hours to log onto that box one more time and notice that.  (I'm not quite finished setting up postfix/courier-imap/amavis-new there at the time, that's why I keep logging on there every evening.)

My question that, is that supposed to happen?  The home server box has 512 MB of memory (which I assumed was plenty for that purpose), and about 700 MB of swap space.  Freshly-restarted, clamd had a 80 MB memory footprint; after two hours after today's restart, it has already grown to 140 MB.  I did have some big downloads yesterday, including a multipart RAR archive in 100 MB portions, but I assume that shouldn't be a problem; as far as I understand, clamd by default doesn't scan files larger than 25 MB at all.

Here is emerge --info output for the reference.  Nothing in clamd or squidclamav logs seems out of order (before oom killing, that is).

(Oh, and also, is it mandatory to restart squid after clamd?  When I manually restarted clamd today, the scanning still didn't work and squidclamav reported "ERROR Clamd daemon not ready for stream scanning." in its log file.  After restarting squid, everything went back to normal.)

Edit: Forgot to mention, clamav version is 0.93.3, the latest stable version in portage.

----------

## steveb

Do you think it would be possible for you to post your squid, clamd and squidclamav configuration?

// SteveB

----------

## Ringtail

Sure:

/etc/clamd.conf

/etc/squidclamav.conf

/etc/squid/squid.conf

It's all pretty basic stuff, nothing fancy.  clamd.conf is particular uses nearly all defaults.

----------

## steveb

I start top down. In your clamd.conf you don't have any limitation for ClamAV. Why is that? How about limiting some stuff. For example:

```
# clamd.conf for facebones.mordhaus.darktech.org

LogFile /var/log/clamav/clamd.log

LogTime yes

PidFile /var/run/clamav/clamd.pid

LocalSocket /var/run/clamav/clamd.sock

MaxConnectionQueueLength 30

StreamMaxLength 20M

MaxThreads 20

ReadTimeout 300

User clamav

AllowSupplementaryGroups yes

DetectPUA yes

ScanPDF yes

MaxScanSize 150M

MaxFileSize 30M
```

Could you post the output of:

```
clamconf
```

And could you post the output of:

```
ps -ylC clamd,freshclam --sort:rss
```

// SteveB

----------

## Ringtail

 *steveb wrote:*   

> I start top down. In your clamd.conf you don't have any limitation for ClamAV. Why is that?

 

Um, because clamd.conf(5) says that all limiting options are set to quite sane limits by default?  It says that the defaults for MaxConnectionQueueLength, StreamMaxLength, MaxThreads, ReadTimeout, MaxScanSize, and MaxFileSize are 15, 10M, 10, 120, 100M, and 25M, respectively.

 *Quote:*   

> Could you post the output of: clamconf

 

Here it is, although I'm pretty sure it somehow got screwed up.  It shows both clamd.conf and freshclam.conf as "freshclam directives", and main/daily.cld are definitely present in /var/lib/clamav.

 *Quote:*   

> And could you post the output of: ps -ylC clamd,freshclam --sort:rss

 

```
S   UID    PID   PPID  C PRI  NI   RSS    SZ WCHAN  TTY          TIME CMD

S   108  69993      1  0  99  19  1028  7980 -      ?        00:00:00 freshclam

S   108  69984      1  0  83   3 156452 77028 -     ?        00:00:17 clamd
```

----------

## steveb

 *Ringtail wrote:*   

> Here it is, although I'm pretty sure it somehow got screwed up.  It shows both clamd.conf and freshclam.conf as "freshclam directives", and main/daily.cld are definitely present in /var/lib/clamav.

 The config file looks strange to me. I have 0.94 over here but my clamconf output looks much different then yours:

```
/etc/clamd.conf: clamd directives

------------------------------

LogFile = "/var/log/clamav/clamd.log"

LogFileUnlock = no

LogFileMaxSize = 2097152

LogTime = yes

LogClean = no

LogVerbose = no

LogSyslog = yes

LogFacility = "LOG_LOCAL6"

PidFile = "/var/run/clamav/clamd.pid"

TemporaryDirectory = "/var/tmp"

ScanPE = yes

ScanELF = yes

DetectBrokenExecutables = no

ScanMail = yes

MailFollowURLs = no

ScanPartialMessages = no

PhishingSignatures = yes

PhishingScanURLs = yes

PhishingAlwaysBlockCloak = no

PhishingAlwaysBlockSSLMismatch = no

HeuristicScanPrecedence = no

DetectPUA = yes

ExcludePUA not set

IncludePUA not set

StructuredDataDetection = no

StructuredMinCreditCardCount = 3

StructuredMinSSNCount = 3

StructuredSSNFormatNormal = yes

StructuredSSNFormatStripped = no

AlgorithmicDetection = yes

ScanHTML = yes

ScanOLE2 = yes

ScanPDF = yes

ScanArchive = yes

MaxScanSize = 157286400

MaxFileSize = 31457280

MaxRecursion = 16

MaxFiles = 10000

ArchiveLimitMemoryUsage = no

ArchiveBlockEncrypted = no

DatabaseDirectory = "/var/lib/clamav"

TCPAddr = "127.0.0.1"

TCPSocket = 3310

LocalSocket not set

MaxConnectionQueueLength = 30

StreamMaxLength = 20971520

StreamMinPort = 1024

StreamMaxPort = 2048

MaxThreads = 20

ReadTimeout = 300

IdleTimeout = 30

MaxDirectoryRecursion = 15

ExcludePath not set

FollowDirectorySymlinks = no

FollowFileSymlinks = no

ExitOnOOM = no

Foreground = no

Debug = no

LeaveTemporaryFiles = no

FixStaleSocket = yes

User = "clamav"

AllowSupplementaryGroups = yes

SelfCheck = 1800

VirusEvent not set

ClamukoScanOnAccess not set

ClamukoScanOnOpen not set

ClamukoScanOnClose not set

ClamukoScanOnExec not set

ClamukoIncludePath not set

ClamukoExcludePath not set

ClamukoMaxFileSize = 5242880

DevACOnly not set

DevACDepth not set

/etc/freshclam.conf: freshclam directives

------------------------------

LogFileMaxSize = 1048576

LogTime = no

LogVerbose = no

LogSyslog = no

LogFacility = "LOG_LOCAL6"

PidFile = "/var/run/clamav/freshclam.pid"

DatabaseDirectory = "/var/lib/clamav"

Foreground = no

Debug = no

AllowSupplementaryGroups = yes

DatabaseOwner = "clamav"

Checks = 12

UpdateLogFile = "/var/log/clamav/freshclam.log"

DNSDatabaseInfo = "current.cvd.clamav.net"

DatabaseMirror = "database.clamav.net"

MaxAttempts = 3

ScriptedUpdates = yes

CompressLocalDatabase = no

HTTPProxyServer not set

HTTPProxyPort not set

HTTPProxyUsername not set

HTTPProxyPassword not set

HTTPUserAgent not set

NotifyClamd = "/etc/clamd.conf"

OnUpdateExecute not set

OnErrorExecute not set

OnOutdatedExecute not set

LocalIPAddress not set

ConnectTimeout = 30

ReceiveTimeout = 30

Engine and signature databases

------------------------------

Engine version: 0.94

Database directory: /var/lib/clamav

main db: Format: .cld, Version: 48, Build time: Thu Sep  4 20:51:34 2008

daily db: Format: .cld, Version: 8290, Build time: Sat Sep 20 03:23:09 2008
```

Reloading clamd results in the following entries added to the log:

```
Sep 20 08:07:40 mail clamd[2774]: Pid file removed.

Sep 20 08:07:40 mail clamd[2774]: --- Stopped at Sat Sep 20 08:07:40 2008

Sep 20 08:07:41 mail clamd[30793]: clamd daemon 0.94 (OS: linux-gnu, ARCH: i386, CPU: i686)

Sep 20 08:07:41 mail clamd[30793]: Running as user clamav (UID 101, GID 407)

Sep 20 08:07:41 mail clamd[30793]: Log file size limited to 2097152 bytes.

Sep 20 08:07:41 mail clamd[30793]: Reading databases from /var/lib/clamav

Sep 20 08:07:44 mail clamd[30793]: Loaded 442881 signatures.

Sep 20 08:07:44 mail clamd[30793]: TCP: Bound to address 127.0.0.1 on port 3310

Sep 20 08:07:44 mail clamd[30793]: TCP: Setting connection queue length to 30

Sep 20 08:07:44 mail clamd[30795]: Limits: Global size limit set to 157286400 bytes.

Sep 20 08:07:44 mail clamd[30795]: Limits: File size limit set to 31457280 bytes.

Sep 20 08:07:44 mail clamd[30795]: Limits: Recursion level limit set to 16.

Sep 20 08:07:44 mail clamd[30795]: Limits: Files limit set to 10000.

Sep 20 08:07:44 mail clamd[30795]: Archive support enabled.

Sep 20 08:07:44 mail clamd[30795]: Algorithmic detection enabled.

Sep 20 08:07:44 mail clamd[30795]: Portable Executable support enabled.

Sep 20 08:07:44 mail clamd[30795]: ELF support enabled.

Sep 20 08:07:44 mail clamd[30795]: Mail files support enabled.

Sep 20 08:07:44 mail clamd[30795]: OLE2 support enabled.

Sep 20 08:07:44 mail clamd[30795]: PDF support enabled.

Sep 20 08:07:44 mail clamd[30795]: HTML support enabled.

Sep 20 08:07:44 mail clamd[30795]: Self checking every 1800 seconds.
```

The 442881 signatures are coming from the fact that I have additional signatures then the stock ClamAV. You will probably have less. Without those additional signatures I would have 428864 signatures:

```
mail ~ # /usr/bin/sigtool --info=/var/lib/clamav/main.cld

File: /var/lib/clamav/main.cld

Build time: 04 Sep 2008 18:51 +0000

Version: 48

Signatures: 399264

Functionality level: 35

Builder: sven

mail ~ # /usr/bin/sigtool --info=/var/lib/clamav/daily.cld

File: /var/lib/clamav/daily.cld

Build time: 20 Sep 2008 01:23 +0000

Version: 8290

Signatures: 29600

Functionality level: 35

Builder: guitar

mail ~ #
```

Anyway... something seems fishy with your installation. clamconf should recognize your singnatures but according to your output of clamconf it does not recognize any of them. Could you try to run once freshclam and force a download of all the signatures? Maybe look in /var/lib/clamav/ and look that your ClamAV user has at least read access to the signature database.

// SteveB

----------

## Ringtail

But apart from clamconf output and memory consumption, clamd behaves perfectly normally.  Restarting:

```
Sun Sep 21 11:36:56 2008 -> --- Stopped at Sun Sep 21 11:36:56 2008

Sun Sep 21 11:36:57 2008 -> +++ Started at Sun Sep 21 11:36:57 2008

Sun Sep 21 11:36:57 2008 -> clamd daemon 0.93.3 (OS: linux-gnu, ARCH: x86_64, CPU: x86_64)

Sun Sep 21 11:36:57 2008 -> Running as user clamav (UID 108, GID 415)

Sun Sep 21 11:36:57 2008 -> Log file size limited to 1048576 bytes.

Sun Sep 21 11:36:57 2008 -> Reading databases from /var/lib/clamav

Sun Sep 21 11:36:57 2008 -> Not loading PUA signatures.

Sun Sep 21 11:37:02 2008 -> Loaded 428542 signatures.

Sun Sep 21 11:37:02 2008 -> LOCAL: Unix socket file /var/run/clamav/clamd.sock

Sun Sep 21 11:37:02 2008 -> LOCAL: Setting connection queue length to 15

Sun Sep 21 11:37:02 2008 -> Limits: Global size limit set to 104857600 bytes.

Sun Sep 21 11:37:02 2008 -> Limits: File size limit set to 26214400 bytes.

Sun Sep 21 11:37:02 2008 -> Limits: Recursion level limit set to 16.

Sun Sep 21 11:37:02 2008 -> Limits: Files limit set to 10000.

Sun Sep 21 11:37:02 2008 -> Archive support enabled.

Sun Sep 21 11:37:02 2008 -> Algorithmic detection enabled.

Sun Sep 21 11:37:02 2008 -> Portable Executable support enabled.

Sun Sep 21 11:37:02 2008 -> ELF support enabled.

Sun Sep 21 11:37:02 2008 -> Mail files support enabled.

Sun Sep 21 11:37:02 2008 -> OLE2 support enabled.

Sun Sep 21 11:37:02 2008 -> PDF support disabled.

Sun Sep 21 11:37:02 2008 -> HTML support enabled.

Sun Sep 21 11:37:02 2008 -> Self checking every 1800 seconds.
```

Signatures:

```
# sigtool --info=/var/lib/clamav/main.cld

File: /var/lib/clamav/main.cld

Build time: 04 Sep 2008 18:51 +0000

Version: 48

Signatures: 399264

Functionality level: 35

Builder: sven

# sigtool --info=/var/lib/clamav/daily.cld

File: /var/lib/clamav/daily.cld

Build time: 21 Sep 2008 03:04 +0000

Version: 8296

Signatures: 29618

Functionality level: 35

Builder: guitar
```

Permissions:

```
# ls -la /var/lib/clamav

total 39247

drwxrwxr-x  2 clamav clamav      136 Sep 21 10:58 ./

drwxr-xr-x 22 root   root        656 Sep 11 11:17 ../

-rw-r--r--  1 clamav clamav  1865216 Sep 21 10:58 daily.cld

-rw-r--r--  1 clamav clamav 38275584 Sep 11 16:20 main.cld

-rw-------  1 clamav clamav      260 Sep 21 11:37 mirrors.dat
```

Freshclam reports that the signatures are up to date, and seems to be working without any issues.

----------

## steveb

Looks okay to me. What is amazing me is the fact, that your clamav consumes so much memory. I have a heavy used mail server where I use clamav and after starting it uses around 120MB of memory and then goes down to around 80MB of memory usage. My clamav is however compiled with Intel C/C++ (icc) because I want it to be as fast as possible. I don't think that using icc is the reason for the 80MB memory usage (compared to yours 3 digit memory usage).

One reason for the high memory usage could be the 15 url_rewrite_children you are using in squid. Do you really need that much?

// SteveB

----------

