# What tools do you use to analyze logs?

## jkcunningham

I'm looking for some decent tools to analyze /var/log/messages on a gateway firewall. What do you use?

-Jeff

----------

## andrewy

Analyze the logs how? I use Logwatch and Logcheck to email me logs daily, but they might not be what you're looking for.

----------

## jkcunningham

stats on port scans and other hack attempts, stats on traffic from machines on the LAN to/from the outside world. Maybe stats on url's visited, ssh's between machines on the LAN. Stuff like that. I'm just starting to think about it. I figured I could learn a lot from seeing what the tools do.

----------

## andrewy

Logwatch and Logcheck do some of that, I'll give examples of what they show below.

Logwatch:

```

################### LogWatch 4.3.2 (02/18/03) #################### 

       Processing Initiated: Sat May 15 04:02:03 2004

       Date Range Processed: yesterday

     Detail Level of Output: 0

          Logfiles for Host: hydra.andrewyates.net

 ################################################################ 

 --------------------- pam_unix Begin ------------------------ 

su:

   Sessions Opened:

      andrew(uid=0) -> root: 1 Time(s)

      andrew(uid=500) -> root: 3 Time(s)

 ---------------------- pam_unix End ------------------------- 

 --------------------- Connections (secure-log) Begin ------------------------ 

Connections:

   Service imap:

      127.0.0.1: 308 Time(s)

 ---------------------- Connections (secure-log) End ------------------------- 

 --------------------- SSHD Begin ------------------------ 

Users logging in through sshd:

   andrew logged in from zeus.andrewyates.net (192.168.1.101) using password: 3 Time(s)

 ---------------------- SSHD End ------------------------- 

------------------ Disk Space --------------------

Filesystem            Size  Used Avail Use% Mounted on

/dev/hda2              11G  6.1G  4.1G  60% /

/dev/hda1              99M   49M   46M  52% /boot

/dev/hdc1              19G   15G  3.5G  81% /home

none                  252M     0  252M   0% /dev/shm

 ###################### LogWatch End ######################### 

```

Notice the detail level is set to 0, I could get much more information emailed, but I like to have Logwatch just do the more basic stuff, and have Logcheck take care of the rest.

Here's an example of what Logcheck sends me:

```

Security Violations

=-=-=-=-=-=-=-=-=-=

May 14 20:09:54 hydra httpd.new: httpd startup failed

May 14 20:10:14 hydra httpd: httpd shutdown succeeded

May 14 20:12:52 hydra httpd.new: httpd shutdown succeeded

May 14 20:12:52 hydra httpd.new: httpd startup failed

May 14 20:13:06 hydra httpd.new: httpd shutdown failed

May 14 20:13:06 hydra httpd.new: httpd startup failed

May 14 20:13:26 hydra httpd.new: httpd startup failed

May 14 20:46:01 hydra httpd: httpd shutdown succeeded

May 14 20:47:04 hydra httpd: httpd shutdown succeeded

May 14 20:47:04 hydra httpd: httpd startup failed

May 14 20:48:33 hydra httpd: httpd shutdown succeeded

May 14 20:49:02 hydra httpd: httpd shutdown succeeded

May 14 20:56:01 hydra httpd: httpd shutdown failed

May 14 20:56:01 hydra httpd: httpd startup failed

May 14 21:05:05 hydra httpd: httpd startup failed

May 14 21:05:25 hydra httpd: httpd startup failed

May 14 21:05:28 hydra httpd: httpd shutdown succeeded

May 14 21:07:10 hydra httpd: httpd shutdown succeeded

May 14 21:09:36 hydra httpd: httpd shutdown succeeded

localhost.localdomain[127.0.0.1] in MAIL command: <Tracie Devine@yahoo.com>

May 14 01:13:22 hydra postfix/smtpd[25586]: warning: Illegal address syntax from

localhost.localdomain[127.0.0.1] in RCPT command: <Tracie Devine@yahoo.com>

May 14 01:13:24 hydra postfix/smtpd[25584]: warning: Illegal address syntax from

localhost.localdomain[127.0.0.1] in MAIL command: <Sondra Wyatt@uswest.net>

May 14 01:13:25 hydra postfix/smtpd[25586]: warning: Illegal address syntax from

localhost.localdomain[127.0.0.1] in RCPT command: <Sondra Wyatt@uswest.net>

May 14 03:13:31 hydra postfix/smtpd[25684]: warning: Illegal address syntax from

localhost.localdomain[127.0.0.1] in MAIL command: <Tracie Devine@yahoo.com>

```

As you can see, I was restarting Apache alot to test some settings. Logcheck has a few files that it uses to decide what to display and what to hide, you can also use it to place different log messages in different categories.

----------

## jkcunningham

I just installed logwatch but couldn't find logcheck in portage - only on some debian site. Logwatch is pretty sparce on documentation. I don't see that it runs as a daemon. Do you set it up to run from a cron or something?

----------

## andyknownasabu

 *jkcunningham wrote:*   

> I just installed logwatch but couldn't find logcheck in portage - only on some debian site.

 

The ebuild you are looking for is called "logsentry" in "app-admin"

 *jkcunningham wrote:*   

> Logwatch is pretty sparce on documentation. I don't see that it runs as a daemon. Do you set it up to run from a cron or something?

 

Yes ;)

```

 * you have to manually add logwatch to cron...

 * 0 0 * * * /usr/sbin/logwatch.pl 2>&1 > /dev/null

```

----------

## jkcunningham

Hmmm. I've installed both logwatch and logsentry. 

I'm having trouble getting logwatch.pl off the ground. I read through the /etc/log.d/conf/logwatch.conf file and the defaults looked reasonable. I added my email address on another machine and tried running it from the command line. It worked away for several minutes (big messages file and slow machine) but in the end produced nothing whatever. I tried using the --print > localfile option but same result - empty file. What's the trick to configuring this thing? There's nothing unusual about my log name or location: /var/log/messages. 

And logsentry installed w/o errors but isn't evident anywhere, at least there are no executables named logsentry in any of root's paths. How do you set it up and run it? I notice that there is now an /etc/logcheck directory with a script and some other files. None seems to be a configuration file.

----------

## andrewy

/etc/logcheck/logcheck.sh is the logsentry program, I believe you just edit that file to configure it.

logwatch.pl runs fine for me, maybe try having it send the results to a local user?

----------

## etnoy

I use the excellent Prelude Hybrid Intrusion Detection System together with it's log-watcher and the nice web interface. 

Check out the Prelude docs on gentoo.org.

----------

## Hayl

 *etnoy wrote:*   

> Check out the Prelude docs on gentoo.org.

 

where?  i don't see any.

----------

## jkcunningham

I just installed fwanalog. It looks promising.

----------

## Chris W

 *Hayl wrote:*   

>  *etnoy wrote:*   Check out the Prelude docs on gentoo.org. 
> 
> where?  i don't see any.

 

I hadn't seen these either - Google had  :Smile:  http://www.gentoo.org/proj/en/hardened/prelude-ids.xml

----------

## andrewy

cool. I had considered trying out Prelude in the past, but it sounded too complex. I'll have to check out the Gentoo docs.

----------

