# Where is trim in ext4? How do I know if it's enabled?

## dman777

I have a ssd and I am using ext4. Where is the trim feature? Is it implemented in ext4? If trim is a feature of ext4 and not the kernel, how do I know if it's enabled in ext4?

----------

## cach0rr0

Do you already have TRIM enabled in your firmware? 

Might have a read through this - http://www.ocztechnologyforum.com/forum/showthread.php?54379-Linux-Tips-tweaks-and-alignment - for the long-winded set of checks.

----------

## poOoch

From Documentation/filesystems/ext4.txt in 2.6.33

```

discard         Controls whether ext4 should issue discard/TRIM

nodiscard(*)            commands to the underlying block device when

                        blocks are freed.  This is useful for SSD devices

                        and sparse/thinly-provisioned LUNs, but it is off

                        by default until sufficient testing has been done.

```

So to enable it now, use discard mount option for ext4.

----------

## tnt

I've found a test for TRIM on OCZ forum:

 *Quote:*   

> if it really work you can test it:
> 
> get the used sectors for a file
> 
> hdparm --fibmap filename
> ...

 

and that should look something like this:

```
# dd if=/dev/urandom of=tempfile count=100 bs=512k oflag=direct

100+0 records in

100+0 records out

52428800 bytes (52 MB) copied, 6.47137 s, 8.1 MB/s

# hdparm --fibmap tempfile

tempfile:

 filesystem blocksize 4096, begins at LBA 0; assuming 512 byte sectors.

 byte_offset  begin_LBA    end_LBA    sectors

           0   37094400   37196799     102400

# hdparm --read-sector 37094400 /dev/sdb

/dev/sdb:

reading sector 37094400: succeeded

b0e8 3ad7 d080 84e8 b4b2 7e60 21f1 eff3

0ef9 fa10 b172 89f8 186f 0194 4cb1 e190

d6b5 b2fe 2577 5dba e6f2 5ad7 34a0 f09f

ca5c 07ef 6e86 c3a8 9e77 77f3 78ff 672f

af71 dea7 ac23 a55d e31e ff83 164e bb76

8ea4 416d 343a 9f5e b41f b1d0 b6e9 6ed8

90c0 3cba ec07 1d96 fdd6 3940 1290 7cd2

c506 c3ee c120 3732 17eb 6e68 11aa 721c...

# rm tempfile

# sync

# hdparm --read-sector 37094400 /dev/sdb

/dev/sdb:

reading sector 37094400: succeeded

b0e8 3ad7 d080 84e8 b4b2 7e60 21f1 eff3

0ef9 fa10 b172 89f8 186f 0194 4cb1 e190

d6b5 b2fe 2577 5dba e6f2 5ad7 34a0 f09f

ca5c 07ef 6e86 c3a8 9e77 77f3 78ff 672f

af71 dea7 ac23 a55d e31e ff83 164e bb76

8ea4 416d 343a 9f5e b41f b1d0 b6e9 6ed8

90c0 3cba ec07 1d96 fdd6 3940 1290 7cd2...

(wait a few minutes)

# hdparm --read-sector 37094400 /dev/sdb

/dev/sdb:

reading sector 37094400: succeeded

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000
```

unfortunately, this test shows that there is no trim enabled for me, although I have:

```
uname -a

Linux tnt 2.6.33-gentoo #3 SMP Fri Mar 26 12:35:12 CET 2010 x86_64 Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz GenuineIntel GNU/Linux
```

```
rootfs / rootfs rw 0 0

/dev/root / ext4 rw,noatime,commit=100,barrier=0,nobh,stripe=128,data=writeback,discard 0 0

proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0

rc-svcdir /lib64/rc/init.d tmpfs rw,nosuid,nodev,noexec,relatime,size=1024k,mode=755 0 0

sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0

udev /dev tmpfs rw,nosuid,relatime,size=10240k,mode=755 0 0

devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620 0 0

shm /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime 0 0

none /tmp tmpfs rw,relatime,size=2097152k 0 0

none /var/tmp tmpfs rw,relatime,size=2097152k,nr_inodes=1048576 0 0

usbfs /proc/bus/usb usbfs rw,nosuid,noexec,relatime,devgid=85,devmode=664 0 0
```

manual TRIM of that sector using hdparm works (I get zeroes) and that means SSD is TRIM enabled, but waiting after rm and sync gives no results.

where could I go wrong?  :Rolling Eyes: 

some kernel configuration option missing?

----------

## tnt

removing data=writeback from mount and rootflag grub options resulted in working ext4 TRIM support.

is data=writeback and TRIM hate bug or a feature?

----------

## mbar

 *tnt wrote:*   

> is data=writeback and TRIM hate bug or a feature?

 

Is this confirmed?

----------

## tnt

 *mbar wrote:*   

>  *tnt wrote:*   is data=writeback and TRIM hate bug or a feature? 
> 
> Is this confirmed?

 

http://thread.gmane.org/gmane.comp.file-systems.ext4/18431/focus=18448

----------

## Smeagel

Just wanted to write and say thanks tnt, this post proved helpful for me when setting my ssd up.

----------

## tnt

I'm very happy if I could help anyone adopting this great old technology called Linux and this great new technology called SSD, and mixing them together.  :Wink: 

----------

## dmitryilyin

It doesn't work(

I don't get zeros when I hdparm --read-sector after file delete.

Even hdparm --trim-sector-ranges doesn't help, but returns success.

Does trim realy zeros sector or just tells drive to use this sector for wear leveling when needed?

Why some drives have only

"Data Set Management TRIM supported"

But other also have

"Deterministic read ZEROs after TRIM"

Does second mean that drive zeroes trimed secotrs instead of only marking them as unused?

I tried Intel, Kingston and OCZ ssds with same result.

Mounted ext4 with discard,noatime,data=ordered

Kernel 2.6.35-23-generic

Maybe I need AHCI enabled controller instead of my Promise and NForce like in FreeBSD?

By the way to enable trim in FreeBSD 8.2+ 

newfs -tU /dev/disk or tunefs -t enable /dev/disk

     -t      Turn on the TRIM enable flag.  If enabled, and if the underlying

             device supports the BIO_DELETE command, the file system will send

             a delete request to the underlying device for each freed block.

             The trim enable flag is typically set when the underlying device

             uses flash-memory as the device can use the delete command to

             pre-zero or at least avoid copying blocks that have been deleted.

It only works with new ada disks using mvs module for marvel controller or modern Intel controller.

If you try mount ad disk kernel will complain that trim is enabled but not supported.

Looks like I need to try newer kernel and better hardware.

Also I cannot get more then 100Mb/s from ssd on Linux box but I can get 220 and more on FreeBSD with modern controller.

----------

## tnt

you need an AHCI enabled controller and driver to get TRIM command passed to the SSD.

----------

## dmitryilyin

Tried with 39 kernel and Marvel controller (no shure is it AHCI) with same result.

I also noticed that hdparm returns "Deterministic read data after TRIM" for my OCZ and only trim supported for Kingston and Intel.

But some drives return "Deterministic read ZEROs after TRIM".

Maybe first means that drive doesnt zero trimed sectors and second means that it does zero and only supported means that behavior is unknown/unreported?

----------

## tnt

nowdays, all OCZ and Intel drives should support TRIM. Kingston is different story (they use many different controllers, including JMicrons...  :Sad:  ).

when drive receive TRIM command, it does not have to fill up zeros instantly. 

go to BIOS (of the motherboard or controller itself) and try to put Marvell into the AHCI mode.

after that, try booting with "ahci.marvell_enable=1" kernel option.

then, try looking for something similar to this:

```
grep AHCI /var/log/dmesg           

ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x39 impl SATA mode
```

----------

## dmitryilyin

How do I find out is trim working or not if trim doesnt determenistic zero trimed sectors?

Perhaps test that returned zeros you quoted was with drive that "Deterministic return zeros"?

Or should it zero after "wait few minutes" or after some random writes?

At least FreeBSD writes that "trim is enabled, but not supported" if i use AHCI-less controller/driver, but Linux does say nothing, just mounted ext4 fs with discard option.

I also found that hdparm --security-erase should zero and set as unused entire drive and it does it almost instantly!

----------

## Moriah

I have been running an OCZ Vertex 256 GB SSD on my laptop for almost 2 years now.  It died night before last.  OCZ said to reflash the firmware to fix it, which I have not had a chance to try yet, but when I do, I will have firmware version 1.6 with trim and garbage collection.

I use my laptop for my business, and it contains confidential and proprietary information for both my company and my clients.

I am running luks to encrypt the entire disk.  I boot from a USB stick and enter a passphrase to decrypt the drive.  I have lvm running on the decrypted device manager device, then xfs on a logical volume under lvm.  My SSD firmware has only had garbage collection up till now, as when I bought the drive, trim was still very much bleeding edge, and there was only flakey support under windows7, and none under linux.

Given that garbage collection needs some way to tell whether a sector is being used or not, it must have some knowledge of the filesystem being used on the drive.  I would be very surprised if it could make any sense of a fully encrypted disk, since the ssd controller can only see the raw bits on the drive, and has no knowledge of the encryption.  

I would even be suprised if the garbage collection firmware could even make sense of an xfs file system anyway.  Add to that the fact that there are no partition tables on the drive, since it is running lvm on the decrypted device manager device, and I do not see how garbage collection could possibly work.  

It would seem to me the only way the drive could know that a sector was being freed was that a write command had just been issued to that sector.  Given that the ssd controller maintains a pool of free chunks of flash above and beyond what is visible to the sata controller on the motherboard of the computer, this allows the drive to still perform reasonably well, but nowhere near what it could do if trim was working properly.

So I see that the ext4 filesystem can now support trim.  I also read that the kernel 2.6.37 permits lvm to pass trim commands thru to the ssd, but not with snapshot or crypt targets.  That's too bad, because the main reason I use lvm is for snapshots, as they are needed to do a clean backup, which my backup server does every night automatically.  

Since my nightly backup uses a snapshot, but does not write to the snapshot logical device, does this mean I am ok to use trim with my read-only snapshots?

How about luks for whole drive encryption?  Will that also pass trim commands? 

Until this is all resolved, I guess I am stuck with xfs and garbage collection.  I really do want trim.     :Crying or Very sad: 

----------

## cach0rr0

 *Moriah wrote:*   

> 
> 
> How about luks for whole drive encryption?  Will that also pass trim commands? 
> 
> 

 

ive been reading about this recently, as a nice Thinkpad with SSD just hit craigslist, and I too insist on full disk crypto

googling around, looks like dmcrypt does not yet support TRIM, and while they're planning on adding it (there's a patch already i think?) there are some concerns in doing so as it leaves visible information on which blocks are used/unused. 

something like that. I'm too lazy to google again and find the long discussion (heaps of discussions, only one i found that discussed it in detail), but was enough to convince me to keep my rotational disk for the time being.

----------

## Moriah

Well I have been running almost 2 years with an OCZ Vertex 256 GB drive using luks/dmcrypt and lvm with an xfs filesystem, and I've been loving it -- until 2 days ago when the SSD died.  It actually started getting flakey 2 days before that, but I just thought it was a filesystem problem because I had to force a power down on it a few days earlier.  

I ran xfs_check and it showed problems, so I ran xfs_repair.  I then booted and everything seemed happy again, but about 2 days later, I got a segfault during a routine emerge, so I immeadiatly shut down in an orderly fashion and ran xfs_check again.  It was sick again.  I ran xfs_repair, then xfs_check right afterwards.  It still showed one problem area -- I don't recall the details.  This was enough for me to say it was time to run mkfs.xfs and reload the filesystem from backup.  After I did that, it would not boot.  It was my fault.  I messed up the perms during the restore.  

Being lazy, I then did the hdparm trick to do a data security erase on the drive to reset its free pool.  I then wanted to restore a known good backup image -- yes, I ran xfs_check on the image first to be sure -- and when I booted the system to a live usb, I got bios errors saying the sata controller had problems.  So I shut down and tried the drive in the other sata slot on my laptop.  Same thing.  OK, I tried pulling the drive and hot-plugging it after a clean boot without the drive installed.  When I hot plugged the drive, I got ugly ATA errors about the drive not being good from gentoo.

I contacted OCZ and they said to reflash the firmware, which I am about to do now.  Supposedly, they have seen this before, and a re-flash fixes it.  If not, then I will have to RMA the drive.  Meanwhile, I am running my laptop on a 7200 RPM HDD, and it sure seems slow after the SSD.    :Evil or Very Mad: 

I will post back here after the re-flash and restore of the disk image to say how it went.

----------

## Moriah

It went lousy.  In fact, it didn't go at all.  The re-flash program, which runs under windoze, detected a data corruption error, and left a pair of files, named good.bin and bad10.bin, in the current directory.  I hexdumped them and diffed them with kdiff3, and the errors all look like they occur on 512 byte boundries, like an address line to the memory was stuck or something.

So I emailed OCZ with the screen output from the flashing program and the 2 .bin files.  I am waiting to hear from them about it, but with it being Labor Day Weekend, I don'r expect to hear anything before Tuesday.

----------

## Stufo76

 *tnt wrote:*   

> I've found a test for TRIM on OCZ forum:
> 
>  *Quote:*   if it really work you can test it:
> 
> get the used sectors for a file
> ...

 

Hi, I've done this test, but:

- immediately before sync I have random

- immediately after sync I have all ffff istead of zeroes, and this remains after some minutes

What's up?Last edited by Stufo76 on Wed Dec 14, 2011 8:35 am; edited 1 time in total

----------

## Moriah

Because I am runnung both LVM and dmcrypt on my OCZ Vertex-I 256 GB SSD, I have decided not to use TRIM after all.  Because I need to run some additional Vmware Workstation vm's, I purchased an OCZ Vertex-II 480 GB SSD.  Turns out the 256 GB drive was really dead, and OCZ replaced it, but I needed more space, so I got the 480 GB drive, which I just installed last Saturday evening.  My first use of the additional space was to install w7-32bit for compatibility with my current client.

I wanted to put all the vmdk vmware virtual disks on a LVM logical device with dedup, but I could not find any dedup filesystems that I felt comfortable with.  For vmware vmdk virtual disks, a 4 KB chunk size is recommended for a block level dedup scheme.  ZFS is massive overkill, trying to be all things to all people.  Lessfs and btrfs still do not appear ready for me to trust actual income generating work to.  So, I just made another logical device under LVM and formatted it for XFS, and the vmdk's are going there.

----------

