# EXT4 does not give full drive?

## FizzyWidget

I will try and elaborate on this

I have in my system a 1TB (930GB) hard drive, when formatted with XFS it gives me the full 930GB, when i use EXT4 it gives me 917GB, so iam losing 16GB right off the bat, even if i remove the reserved blocks and the 5% reserved space for root, I still don't get the full amount.

Is there some option i need to do when formatting the drive? Would a smaller block size help or would that just give me more inodes?

I only ask as i am thinking of moving away from XFS due to the fact it has started to play up on my home partition and a few times i have had the machine lock up on me, when i have rebooted XFS has always said it needs looking at as it couldn't write metadata to the disc and denied access to prevent further damage.

I have been lucky and not lost anything but I dont wish to temp fate and feel using a different filesystem might be the best course for me

The drive contains a mixture of little to large files, the usual txt picture music and video files, so need a system suited to store those size of files, ranging fome a few ytes to 20GB+ (bluray)

----------

## titanofold

I forget which specific thing it is, but Ext2/3/4 reserves some space. This is configurable.

----------

## titanofold

 *Quote:*   

> -m reserved-blocks-percentage
> 
>     Specify the percentage of the filesystem blocks reserved for the super-user. This avoids fragmentation, and allows root-owned daemons, such as syslogd(, to continue to function correctly after non-privileged processes are prevented from writing to the filesystem. The default percentage is 5%.

 

----------

## FizzyWidget

 *Dark Foo wrote:*   

> even if i remove the reserved blocks and the 5% reserved space for root, I still don't get the full amount.

 

i know this, my issue is with it not giving me the full space when i format, if i remove the reserved blocks my available space gets higher, but it still says i only have 917GB as opposed to the 930GB it should be

----------

## titanofold

 *Dark Foo wrote:*   

>  *Dark Foo wrote:*   even if i remove the reserved blocks and the 5% reserved space for root, I still don't get the full amount. 
> 
> i know this, my issue is with it not giving me the full space when i format, if i remove the reserved blocks my available space gets higher, but it still says i only have 917GB as opposed to the 930GB it should be

 

Yeah, I noticed that after I submitted. Here's the best explanation I can find: http://whrl.pl/Rcdk0L

----------

## FizzyWidget

Thanks, guess its the inodes then, if i use a smaller blocksize i will get more of them and less space? Do i bite the bullet i wonder and lose 16GB or take the risk of XFS taking a big dump :/

----------

## mv

ext4 needs space for the inode tables. You can decrease the inode ratio by using mkfs.ext4 with e.g. -T largefile or -T largefile4 (see /etc/mke2fs.conf for details).

However, you should be aware that if you run out of inodes you will get a "disk full" error even if there should still be plenty of space.

This shouldn't be an issue if you really have lots of large files on the disk.Last edited by mv on Sat Dec 24, 2011 6:00 pm; edited 1 time in total

----------

## FizzyWidget

will be loads of little files and loads of big files, guess i will just have to take the loss, bit annoying though, guess someone on the ext4 dev team didnt think that one through

Now to format and move over 500GB again  :Sad: 

----------

## mv

 *Dark Foo wrote:*   

> will be loads of little files and loads of big files

 

If you have loads of big files, the little files will not hurt too much (if it is a reasonable amount like e.g. a typical / of a linux system). Tthink about what a ratio means. It is only a problem if you really have millions of files <4k like for a news server. In such a case, you should consider splitting that partition.

----------

## FizzyWidget

not many will be that small, normal size jpgs/png/psd mp3 movies, will see what is what after i have moved everything across, I dont think it should be that much of an issue

----------

## Ant P.

256 bytes per inode, 1 inode per 16KB, that's where your 1.56% of disk space has gone.

If you need an extra 12GB of that and wasting up to 64KB per file isn't an issue then use `mkfs.ext4 -T huge`. XFS/reiserfs/btrfs/* aren't inode-based but they have their own unique ways of losing space, sometimes with existing data in it.

----------

## FizzyWidget

I dont want to waste anything, was curious as to were the space went, and what would be the best options when formating the drive for an ext4 system

----------

## Ant P.

Alternatively you could make one filesystem for the large files and put a loopback partition inside it for the other stuff.

----------

## NeddySeagoon

Dark Foo,

All filesystems need some space for themselves to describe where your data is stored and which space is free.

Some filesystems. like FAT and extX allocate it when the filesystem is made, others assign it as files are added/deleted.

You should not assume that because you can see empty space, you can use it for your data.

----------

## FizzyWidget

 *NeddySeagoon wrote:*   

> Dark Foo,
> 
> All filesystems need some space for themselves to describe where your data is stored and which space is free.
> 
> Some filesystems. like FAT and extX allocate it when the filesystem is made, others assign it as files are added/deleted.
> ...

 

I know this now, if you dont ask you'll never learn, so i asked, and now i have learnt something  :Smile: 

----------

## NeddySeagoon

Dark Foo,

If you use LVM2 to divide the disk, you can move empty space around as you need to.

You could make say two partitions, one for big files and one for small files with differing block sizes and inode ratios.

----------

## FizzyWidget

i am going to see what the inode count is like after i have added the 500G or so, wont take long on a GB network, i could leave it over night and see what it was in the morning.

----------

## Hu

You can anticipate with a fairly high level of accuracy how many inodes are needed.  You need one inode for each file, directory, and so on.  Thus, the output of find . | wc -l at the top of your source hierarchy should tell you how many inodes are used in the source, which will tell you also how many inodes you will use in the destination.  You can tune the find expression if there are areas of the source hierarchy which are not copied into the destination.

----------

## FizzyWidget

 *Hu wrote:*   

> You can anticipate with a fairly high level of accuracy how many inodes are needed.  You need one inode for each file, directory, and so on.  Thus, the output of find . | wc -l at the top of your source hierarchy should tell you how many inodes are used in the source, which will tell you also how many inodes you will use in the destination.  You can tune the find expression if there are areas of the source hierarchy which are not copied into the destination.

 

bit hard to do that on a windows system, although it will help me for when i move everything back once i convert the windows machine to linux, doing a badblock check on it atm, i am anticipating that on a raid5 930gb drive its going to take a while, so will be able to find out the inode use from that

----------

## FizzyWidget

Well after transfering near 172,000 files here is what the inode situation is like

```
/dev/mapper/storage 61054976 191347 60863629    1% /storage

/dev/mapper/storage ext4      917G  402G  469G  47% /storage
```

and this is it with only 1% reserved for root and reserved blocks (although i think for a storage drive i could remove them completely, what do people think?)

```
tune2fs -r 1 /dev/mapper/storage

tune2fs 1.42 (29-Nov-2011)

Setting reserved blocks count to 1

tune2fs -m 1 /dev/mapper/storage

tune2fs 1.42 (29-Nov-2011)

Setting reserved blocks percentage to 1% (2441916 blocks)

/dev/mapper/storage 61054976 191347 60863629    1% /storage

/dev/mapper/storage ext4      917G  402G  507G  45% /storage
```

As you can see no change at all on the indoes but i have got back 2% of my drive, a healthy 38GB  :Smile: 

----------

## NeddySeagoon

Dark Foo,

The reserved space for root is intended to prevent normal users from crashing the box by filling up /.

When / is full, nothing can open new files.  That means lock files and logs.  When that happens, nobody can log in either - not even root.

To regain control, you need to boot with a liveCD and make some space.

The reserved space stops that happening.  The 5% was set many years ago when drives were much smaller than they are now.

You only need reserved space on / and /var if thats separate.

----------

## FizzyWidget

yes / is about 1GB and that only has about a few hundred meg on it if that, and /var is 10GB as is /usr, so your saying its safe to fully remove the reserved blocks from /home and /storage, well any mount point other than / and /var ?

----------

## NeddySeagoon

Dark Foo,

Its safe to fully remove the reserved blocks from /home, /storage and /usr.

For other file systems it depends on your setup.

Keep some reserved space for root on any filesystem that the system depends on root being able to write on for normal operation that is also writable by (or on behalf of) ordinary users.

----------

## FizzyWidget

my set up is

/

/opt

/usr

/var

/home

/storage

same on laptop except /storage - don't have that on there, but nice to now i can claw some space back on the other mount points  :Smile: 

More I learn about Linux the more I think I will like it  :Smile: 

----------

## NeddySeagoon

Dark Foo,

/

/opt

/usr 

Should only be writable by root anyway, so having the reserved space there makes no difference.

By definition, root can use it and users can't write there at all. 

/var is where your logs and lock files go, so you need the reserved space here, in case a user does something silly.

/home and /storage are not used for system operations, so you don't need to reserve space for root here either.

roots home dir is /root.

Where is your /tmp ?

If /tmp is on /, you must reserve space for root on / since users can write to /tmp, potentially filling /, which is a very bad thing.

----------

## FizzyWidget

tmp is in a tmpfs gave it about 2GB - seems to be handling it ok

```
tmpfs               tmpfs     2.0G   20K  2.0G   1% /tmp
```

----------

