# I've got a hard disk getting filled and I don't know why ?

## dcreatorx

Hi guys, I've got a problem. I have built a monitoring and logging server with gentoo that runs : Nagios with PNP ( performance data being stored ),syslog-ng (it's a logging servers and like 14 devices are sending the logs to a directory )  and few services more that don't do a thing on disk terms.

Now, the logs are stored at /var/log/HOSTS/. Inside directories are created dinamically by syslog when any of the devices that are logging send any log.

I have erased all the contents by now, but they get re-created so anyway.- 

The disk is at 93% and I don't have any big files, I have executed many scripts to detect larger files and directories . Nagios was executing Nagvis, that is database-oriented but I have stopped nagvis and mysql for a better debugging. I'm out of ideas, what can I do now ?

Thanks.

----------

## bbgermany

Hi,

can you show the output of "df -i" pls?

bb

----------

## dcreatorx

Yep, here you are :

```
Filesystem            Inodes   IUsed   IFree IUse% Mounted on

/dev/sda1            29212672  103625 29109047    1% /

tmpfs                 506195       2  506193    1% /lib/init/rw

udev                  506195     457  505738    1% /dev

tmpfs                 506195       1  506194    1% /dev/shm

```

----------

## d2_racing

Hi, can you post this :

```

# du -sh /var

# du -sh /usr

# du -sh /home

```

----------

## dcreatorx

```
fisheye:~# du -sh /var

709M    /var

fisheye:~# du -sh /usr

        2.1G    /usr

fisheye:~# du -sh /home

32K     /home

fisheye:~#

```

This is the strange stuff . . . . none of the places I use to keep the data contain more than 1 Gb . . .

----------

## d2_racing

Weird in fact.

----------

## dcreatorx

That's why I posted . . .  this is the most busy server I have in terms of writes to disk and network traffic and CPU load so I had it very controlled. Now I's wild and I don't know why !

----------

## d2_racing

Maybe you are running out of Inodes :

```

# df -i

```

----------

## dcreatorx

```
Filesystem            Inodes   IUsed   IFree IUse% Mounted on

/dev/sda1            29212672  103625 29109047    1% /

tmpfs                 506195       2  506193    1% /lib/init/rw

udev                  506195     457  505738    1% /dev

tmpfs                 506195       1  506194    1% /dev/shm

```

The sistem is swaping 58ks. It's possible that before, mysql have been using virtual memory ? Maybe a reboot will cleanse that ficticious disk space ? 

I don't know . .

----------

## bbgermany

hi,

which filesystem is running out of space? where do you place your mysql databases? i had a similiar issue as mysql wrote some logfiles that were huge.

bb

edit: you could check which directory uses most of the space by the following command:

```

du -smh /*

```

but this can take a while.

----------

## ccp

I guess you just did

```
# rm /var/log/HOSTS/*
```

if this is the case then this is because syslog-ng did not close the log file. simply remove the log files does not release file space.

use command,

```
# kill -HUP $(cat /var/run/syslog-ng.pid)
```

will release the file space used by the old log.

----------

