# 1GB of difference between usage report from 'df' and du

## devsk

I know about the hanging 'inodes' issue where by df and du can differ in their reported values. But around 1GB is a lot of difference:

```
root@localhost /

$ df -k

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/root             10321208   4833440   5068340  49% /

udev                    387984       384    387600   1% /dev

/dev/hda4              5160576    232800   4665632   5% /ccache-and-var

none                    387984         0    387984   0% /dev/shm

//192.168.0.6/win-share

                      94093312  43538432  50554880  47% /win-share

root@localhost /

$ total=0

root@localhost /

$ for i in `du -sb *|awk '{print $1}'`;do total=$((total+i));done

root@localhost /

$ echo $total

49103241904

root@localhost /

$ du -sb *|sort -n

4096    eth0

4097    boot

16384   lost+found

24576   mnt

112708  dev

4862914 sbin

6279168 sys

8799787 bin

19995149        root

20570018        lib

21237491        clients

64235030        etc

111908529       var

165377908       tmp

180453243       ccache-and-var

268986904       opt

464700179       home

805452142       proc

2797554893      usr

44162668862     win-share

root@localhost /

$ bc

bc 1.06

Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.

This is free software with ABSOLUTELY NO WARRANTY.

For details type `warranty'.

49103241904-44162668862-805452078-180453243

3954667721

4833440*1024

4949442560

4949442560-3954667721

994774839

```

So, where are my 994774839 bytes of disk?

----------

## bunder

ext3?  if so, the system stores and hides 5% of filesystem space in case you do exceed your "imaginary" limit.  i think tune2fs can change that.

----------

## devsk

yes, it is ext3. it seems more like 10%. I did the same test on another system with reiserfs and difference came to be around 70MB which is probably fine for metadata/fragmentation on a 18GB partition. So, the tools are not the culprit. Neither is dangling inodes.

----------

## bunder

try `tune2fs -m 2 /dev/drive`   :Laughing: 

----------

## devsk

Reserved block count:     104857

That's around 429494272 bytes. Where is the rest (around 500MB)? Is it possible that my FS is corrupt? let me force a check on next boot and post.

----------

## devsk

well, well, well!

tune2fs, reboot and forced check. clean.

```
Reserved block count:     26214 (about 100MB).

$ df -k

Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/hda1             10321208   4834196   5382156  48% /

udev                    387984       384    387600   1% /dev

/dev/hda4              5160576    232800   4665632   5% /ccache-and-var

none                    387984         0    387984   0% /dev/shm
```

almost the same. where is my 1GB now?

----------

## BitJam

At least 400 Meg of it is in /usr/portage.  The rest is probably spread out elsewhere.  Seriously.  I put /usr/portage on its own partition with a smaller block size and the space used (via df) dropped by 400 Meg to 223 Meg.

The du program reports the exact number of bytes used by each file and directory.  If you have a file that contains a single byte, du will say it contains one byte.  But the smallest amount of disk space used up for that one byte file is one block, typically 4 kilobytes.  All those 4 kilobytes will no longer be available for other stuff so df adds 4k to the used column and subtracts 4k from the available column.

Thus the difference between the du output and the df output is all those ends of blocks that aren't filled.

The Reiser file systems have the ability to use that extra space, but people often disable that feature by specifying "notail" or something like that.  This might account for the differences you see with ReiserFS.  It can potentially use that "wasted" block space thus it reported as available by df but this is only a guess on my part.

----------

## devsk

You are saying that I am wasting 10% space with a 4K block size? I don't waste that much with 32K cluster size in the much hated windows.

----------

## BitJam

I also said that almost half of that wastage is in the 223 Meg /usr/portage directory because it is filled with little files.

The problem in not in the file system (IMO) but in the ugly, ugly Portage kludge of using the /usr/portage file system instead of a database.

I installed my system fresh onto a new hard drive about 2 months ago.  About 2 weeks ago, I timed "du -sh /usr/portage" and it took over 3 minutes.  After I put that directory on its own small block partition (instructions here) the time went down to 20 seconds, a gain of almost a factor of ten.   It still only takes 21 seconds.  I mounted /usr/portage as a file so it didn't require an actual spare partition on the drive.

If you want to squeeze out every last bit of disk space (which I think is very silly considering the current price of disks) you could use Reiser with the tail option set, or you could move your system to a new partition that has a smaller block size but if you do either of these things, your disk performance will suffer.

----------

