# Disappearing disk space?

## odioworks_com

I ran out of space on one of my drives, so I removed some files.  However, that didn't seem to free up any space:

After deleting 12 gigs of data:

```
$df -h --sync

Filesystem            Size  Used Avail Use% Mounted on

/dev/hdd1             230G  217G  662M 100% /var/ftp

```

Any ideas?

----------

## frostschutz

Maybe you deleted sparse files which look like they take 12G space, but actually don't? Maybe the files were still in use somehow, so they get deleted only after they are not in use anymore? Filesystems may also have a root reserve, that can only be filled by root, as to prevent the system becoming unusable because some user took all available diskspace. Also check your filesystem for corruption of any kind (umount, fsck in read-only mode).

----------

## odioworks_com

Frostschutz,

Thanks for your feedback, that's really helpful.  However I don't think any of those suggestions are the solution 

The drive is a secondary device, so there's no system files (or root reserve) on it.

The files deleted were media files (movies, emails, etc).  There's no way I only deleted 662MB of data, but that's all that is reported as free.  The emails themselves were 5 GB.  So, I don't think sparse files are the issue.

I've tried a ext3.fsck -f to no avail.  I'll try again with the -n flag, which according to the man page opens the filesystem in read only mode.  I'll let you know if that does it.Last edited by odioworks_com on Sat Apr 21, 2007 5:24 pm; edited 3 times in total

----------

## odioworks_com

fsck.ext3 -fn didn't work.  Neither did a reboot  :Smile:  Although the amount used did change (see below).

However Used + Avail still doesn't equal size.

Any other ideas?  I'm stumped.

```
$df -h --sync

Filesystem            Size  Used Avail Use% Mounted on

/dev/hdd1             230G  226G     0 100% /var/ftp

```

----------

## frostschutz

Well, you could try and see what is supposed to be taking all the space; either with 'du' as a command line tool, or 'xdiskusage' for a graphical representation of occupied space of your directory tree. Maybe it's really full because some process or other writes stuff on it?

The case where the file system is displayed as full when it really isn't and the file system is not corrupt in some way is unknown to me. So either it's really full or something's really wrong with your fs.

I'm not sure wether Size and Used actually have to be the same number; every file system has some overhead. If this overhead is included in the 'Used' statistics of df, it should be the same, otherwise it will be lower; depending on size/type of files and overhead for that specific filesystem (may be considerably huge with many small files) the difference may be normal.

----------

## odioworks_com

I just don't get it though.

I can unmount the drive, which would indicate there aren't any processes writing to it.

When I delete more files on the drive, the space doesn't appear as available.

Before deleting 9GB of data:

```
$df -h --sync

Filesystem            Size  Used Avail Use% Mounted on

/dev/hdd1             230G  226G     0 100% /var/ftp 
```

After deleting 9GB of data:

```
$ df -h --sync

Filesystem            Size  Used Avail Use% Mounted on

/dev/hdd1             230G  217G  767M 100% /var/ftp
```

The drive is STILL FULL  :Question:   :Exclamation:   :Question:   :Exclamation: 

Ugh, I'm going crazy.

----------

## sonicbhoc

emerge filelight. That will show you where the problems lay!

----------

## odioworks_com

It looks like firelight isn't in portage.

Regardless, according to the README it requires kdelibs.  To install that, I would need to emerge 20 dependences and resolve 2 blocks.  Not ideal  :Sad: 

Any other ideas?  How safe would it be for me to emerge kdelibs?

```
$ emerge -pv kdelibs

These are the packages that I would merge, in order:

Calculating dependencies ...done!

[blocks B     ] kde-base/kde-env (is blocking kde-base/kdelibs-3.5.5-r9)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-misc/util-macros-1.1.0)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/kbproto-1.0.3)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xextproto-7.0.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xf86bigfontproto-1.1.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/inputproto-1.3.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libX11-1.0.3)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xproto-7.0.7)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXau-1.0.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/bigreqsproto-1.0.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXdmcp-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xcmiscproto-1.1.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/xtrans-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libICE-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libSM-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXt-1.0.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xf86vidmodeproto-2.2.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXext-1.0.1)

[ebuild     U ] sys-apps/man-1.6d [1.6-r1] +nls 263 kB 

[ebuild  N    ] x11-misc/util-macros-1.1.0  -debug 42 kB 

[ebuild     U ] sys-devel/binutils-2.16.1-r3 [2.16.1] -multislot -multitarget +nls -test -vanilla 12,367 kB 

[ebuild  N    ] x11-proto/kbproto-1.0.3  -debug 56 kB 

[ebuild  N    ] x11-proto/xextproto-7.0.2  -debug 66 kB 

[ebuild  N    ] x11-proto/xf86bigfontproto-1.1.2  -debug 36 kB 

[ebuild  N    ] x11-proto/inputproto-1.3.2  -debug 45 kB 

[ebuild  N    ] x11-proto/xproto-7.0.7  -debug 130 kB 

[ebuild  N    ] x11-libs/libXau-1.0.2  -debug 214 kB 

[ebuild  N    ] x11-proto/bigreqsproto-1.0.2  -debug 35 kB 

[ebuild  N    ] x11-libs/libXdmcp-1.0.1  -debug 226 kB 

[ebuild  N    ] x11-proto/xcmiscproto-1.1.2  -debug 35 kB 

[ebuild  N    ] x11-libs/xtrans-1.0.1  -debug 89 kB 

[ebuild  N    ] x11-libs/libX11-1.0.3  -debug +ipv6 1,415 kB 

[ebuild  N    ] x11-libs/libICE-1.0.1  -debug +ipv6 248 kB 

[ebuild  N    ] x11-libs/libSM-1.0.1  -debug +ipv6 234 kB 

[ebuild  N    ] x11-libs/libXt-1.0.2  -debug 505 kB 

[ebuild  N    ] x11-proto/xf86vidmodeproto-2.2.2  -debug 38 kB 

[ebuild  NS   ] kde-base/arts-3.5.5  +alsa -artswrappersuid -debug (-elibc_FreeBSD) -esd -jack -kdeenablefinal +mp3 -nas +vorbis -xinerama 944 kB 

[ebuild  N    ] app-dicts/aspell-en-0.51.1  168 kB 

[ebuild  N    ] net-dns/libidn-0.6.5-r1  -doc (-elibc_FreeBSD) -emacs -java -mono +nls 2,078 kB 

[ebuild     U ] dev-libs/libxml2-2.6.27 [2.6.23] -debug -doc +ipv6 +python +readline -test 4,483 kB 

[ebuild  N    ] dev-libs/libgpg-error-1.0-r1  +nls 316 kB 

[ebuild  N    ] dev-libs/libgcrypt-1.2.2-r1  +nls 939 kB 

[ebuild  N    ] dev-libs/libxslt-1.1.17  +crypt -debug +python 1,865 kB 

[ebuild  N    ] media-libs/libart_lgpl-2.3.17  -debug 282 kB 

[ebuild  N    ] x11-libs/libXext-1.0.1  -debug 254 kB 

[ebuild  N    ] media-fonts/gnu-gs-fonts-std-8.11  3,664 kB 

[ebuild  N    ] app-text/ghostscript-gpl-8.54  +X -cjk +cups -djvu -emacs +gtk -jpeg2k 12,082 kB 

[ebuild  N    ] virtual/ghostscript-0  0 kB 

[ebuild  N    ] kde-base/kdelibs-3.5.5-r9  -acl +alsa +arts -avahi +cups -debug -doc (-elibc_FreeBSD) -fam -jpeg2k -kdeenablefinal -kdehiddenvisibility -kerberos +kernel_linux -legacyssl -linguas_he -lua -openexr +spell +ssl +tiff -utempter -xinerama -zeroconf 15,164 kB 

```

----------

## terminal9

lsof is your friend.

----------

## sonicbhoc

FILELIGHT not FIRELIGHT. close though.   :Wink: 

emerge filelight

----------

## odioworks_com

 *Quote:*   

> FILELIGHT not FIRELIGHT. close though.  

 

Oops   :Embarassed: 

It looks life filelight isn't much better, though:

```
$ emerge -pv filelight

These are the packages that I would merge, in order:

Calculating dependencies ...done!

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-misc/util-macros-1.1.0)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/kbproto-1.0.3)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xextproto-7.0.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xf86bigfontproto-1.1.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/inputproto-1.3.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libX11-1.0.3)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xproto-7.0.7)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXau-1.0.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/bigreqsproto-1.0.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXdmcp-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xcmiscproto-1.1.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/xtrans-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libICE-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libSM-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXt-1.0.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xf86vidmodeproto-2.2.2)

[blocks B     ] kde-base/kde-env (is blocking kde-base/kdelibs-3.5.5-r9)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXext-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xf86miscproto-0.9.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXxf86misc-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libdmx-1.0.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/dmxproto-2.2.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/printproto-1.0.3)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXp-1.0.0)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/renderproto-0.9.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXrender-0.9.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xf86dgaproto-2.0.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-apps/xdpyinfo-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/xineramaproto-1.1.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXinerama-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXxf86vm-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXxf86dga-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXi-1.0.1)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-proto/recordproto-1.13.2)

[blocks B     ] <=x11-base/xorg-x11-6.9 (is blocking x11-libs/libXtst-1.0.1)

[ebuild     U ] sys-apps/man-1.6d [1.6-r1] +nls 263 kB 

[ebuild  N    ] x11-misc/util-macros-1.1.0  -debug 42 kB 

[ebuild     U ] sys-devel/binutils-2.16.1-r3 [2.16.1] -multislot -multitarget +nls -test -vanilla 12,367 kB 

[ebuild  N    ] x11-proto/kbproto-1.0.3  -debug 56 kB 

[ebuild  N    ] x11-proto/xextproto-7.0.2  -debug 66 kB 

[ebuild  N    ] x11-proto/xf86bigfontproto-1.1.2  -debug 36 kB 

[ebuild  N    ] x11-proto/inputproto-1.3.2  -debug 45 kB 

[ebuild  N    ] x11-proto/xproto-7.0.7  -debug 130 kB 

[ebuild  N    ] x11-libs/libXau-1.0.2  -debug 214 kB 

[ebuild  N    ] x11-proto/bigreqsproto-1.0.2  -debug 35 kB 

[ebuild  N    ] x11-libs/libXdmcp-1.0.1  -debug 226 kB 

[ebuild  N    ] x11-proto/xcmiscproto-1.1.2  -debug 35 kB 

[ebuild  N    ] x11-libs/xtrans-1.0.1  -debug 89 kB 

[ebuild  N    ] x11-libs/libX11-1.0.3  -debug +ipv6 1,415 kB 

[ebuild  N    ] x11-libs/libICE-1.0.1  -debug +ipv6 248 kB 

[ebuild  N    ] x11-libs/libSM-1.0.1  -debug +ipv6 234 kB 

[ebuild  N    ] x11-libs/libXt-1.0.2  -debug 505 kB 

[ebuild  N    ] x11-proto/xf86vidmodeproto-2.2.2  -debug 38 kB 

[ebuild  NS   ] kde-base/arts-3.5.5  +alsa -artswrappersuid -debug (-elibc_FreeBSD) -esd -jack -kdeenablefinal +mp3 -nas +vorbis -xinerama 944 kB 

[ebuild  N    ] app-dicts/aspell-en-0.51.1  168 kB 

[ebuild  N    ] net-dns/libidn-0.6.5-r1  -doc (-elibc_FreeBSD) -emacs -java -mono +nls 2,078 kB 

[ebuild     U ] dev-libs/libxml2-2.6.27 [2.6.23] -debug -doc +ipv6 +python +readline -test 4,483 kB 

[ebuild  N    ] dev-libs/libgpg-error-1.0-r1  +nls 316 kB 

[ebuild  N    ] dev-libs/libgcrypt-1.2.2-r1  +nls 939 kB 

[ebuild  N    ] dev-libs/libxslt-1.1.17  +crypt -debug +python 1,865 kB 

[ebuild  N    ] media-libs/libart_lgpl-2.3.17  -debug 282 kB 

[ebuild  N    ] x11-libs/libXext-1.0.1  -debug 254 kB 

[ebuild  N    ] media-fonts/gnu-gs-fonts-std-8.11  3,664 kB 

[ebuild  N    ] app-text/ghostscript-gpl-8.54  +X -cjk +cups -djvu -emacs +gtk -jpeg2k 12,082 kB 

[ebuild  N    ] virtual/ghostscript-0  0 kB 

[ebuild  N    ] kde-base/kdelibs-3.5.5-r9  -acl +alsa +arts -avahi +cups -debug -doc (-elibc_FreeBSD) -fam -jpeg2k -kdeenablefinal -kdehiddenvisibility -kerberos +kernel_linux -legacyssl -linguas_he -lua -openexr +spell +ssl +tiff -utempter -xinerama -zeroconf 15,164 kB 

[ebuild  N    ] x11-proto/xf86miscproto-0.9.2  -debug 37 kB 

[ebuild  N    ] x11-libs/libXxf86misc-1.0.1  -debug 217 kB 

[ebuild  N    ] x11-proto/dmxproto-2.2.2  -debug 38 kB 

[ebuild  N    ] x11-libs/libdmx-1.0.2  -debug 225 kB 

[ebuild  N    ] x11-proto/printproto-1.0.3  -debug 42 kB 

[ebuild  N    ] x11-libs/libXp-1.0.0  -debug 238 kB 

[ebuild  N    ] x11-proto/renderproto-0.9.2  -debug 39 kB 

[ebuild  N    ] x11-libs/libXrender-0.9.1  -debug 234 kB 

[ebuild  N    ] x11-proto/xf86dgaproto-2.0.2  -debug 39 kB 

[ebuild  N    ] x11-proto/xineramaproto-1.1.2  -debug 37 kB 

[ebuild  N    ] x11-libs/libXinerama-1.0.1  -debug 201 kB 

[ebuild  N    ] x11-libs/libXxf86vm-1.0.1  -debug 222 kB 

[ebuild  N    ] x11-libs/libXxf86dga-1.0.1  -debug 226 kB 

[ebuild  N    ] x11-libs/libXi-1.0.1  -debug 240 kB 

[ebuild  N    ] x11-proto/recordproto-1.13.2  -debug 38 kB 

[ebuild  N    ] x11-libs/libXtst-1.0.1  -debug 207 kB 

[ebuild  N    ] x11-apps/xdpyinfo-1.0.1  -debug 85 kB 

[ebuild  N    ] kde-misc/filelight-1.0_rc2  +arts -debug (-elibc_FreeBSD) -xinerama 518 kB 
```

I'll try lsof.

----------

## sonicbhoc

You're still running an old version of X? Why not upgrade to modular?

----------

## odioworks_com

/dev/hdd1 is mounted on /var/ftp.

$lsof | grep "/var/ftp" returns nothing.

I haven't bothered with X because it's a server, and I only ssh into it.  I didn't want to break anything by trying to upgrade it.

----------

## sonicbhoc

try unmounting the drive and doing a file system check on it. If that doesn't work... well, I'm out of ideas.

----------

## odioworks_com

Yah, tried that several times, fsck.ext3 was a no go:

 *Quote:*   

> 
> 
> fsck.ext3 -fn didn't work. Neither did a reboot  Although the amount used did change (see below).
> 
> However Used + Avail still doesn't equal size.
> ...

 

The problem persists: If I delete 10 gigs of data using "rm -rf" - the space doesn't become "Available."  I can't figure this out.  I guess I might have to try wiping the drive.

----------

## terminal9

I'll propose a number of dumb ideas, sometimes simplicity resolves:

1. Make sure the folders you claim to have deleted gigabytes of data from are REALLY on hdd1 and not some symlinks to other drive(s).

2. I assume you have an FTP server running on this machine (hint: /var/ftp) ? Could somebody be flooding incoming folder in /var/ftp? Try stopping ftp server. Actually, try to stop all other servers, if any, and observe the usage for sometime.

3. And -deviating from dumb mode- use lsof in repeat mode, e.g.

```
lsof -r3 +D /var/ftp
```

That will print open files on /var/ftp and all its subdirs every 3 seconds, until you interrupt with Ctrl-C.

----------

## odioworks_com

 *Quote:*   

> I'll propose a number of dumb ideas, sometimes simplicity resolves:
> 
> 1. Make sure the folders you claim to have deleted gigabytes of data from are REALLY on hdd1 and not some symlinks to other drive(s).
> 
> 2. I assume you have an FTP server running on this machine (hint: /var/ftp) ? Could somebody be flooding incoming folder in /var/ftp? Try stopping ftp server. Actually, try to stop all other servers, if any, and observe the usage for sometime.
> ...

 

No symlinks, this is a 300GB drive, the stuff couldn't fit anywhere else.  The only other drive on the system is 60GB - and it's is only for the OS.  

FTP isn't running, and I'm the only one who has file-level access to the machine.

lsof -r3 +D /var/ftp returns nothing  :Sad: 

Any other ideas?

----------

## alex6z

It looks like someone filled the drive almost all the way up, and then changed the root reserve to a large %age.  The drive can be 60% full but show 0 free space in df if the root reserve is set to 40%.

Also check that you are not out of inodes.

Try du -c /foo  to see if things add up.  Try using find to find large files too.

----------

## terminal9

Check for bad blocks:

```
dumpe2fs -b /dev/hdd1
```

If you get a badblock report, use badblocks command to get detailed analysis. 

If no bad block, check the output of

```
dumpe2fs -h /dev/hdd1
```

 and observe the size of journal (or post it here, we can have a look together)

----------

