# Extremely slow USB writes [solved]

## cdstealer

Hi Guys/Gals,  For a while now I've been struggling with very poor USB write speeds.  It's borderline unusable  :Sad: 

My system on has XHCI:

```
# lspci -v | grep HCI

00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller (rev 31) (prog-if 30 [XHCI])

   Subsystem: Micro-Star International Co., Ltd. [MSI] Sunrise Point-H USB 3.0 xHCI Controller

01:00.0 USB controller: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller (prog-if 30 [XHCI])
```

I tested a large transfer to a USB3 & a USB2 flashdrive with the following results.

USB3 drive (write):

```
$ rsync -rhv --progress "/NAS/Videos/Mr Robot" /run/media/cdstealer/STUFF/

sending incremental file list

Mr Robot/Mr.Robot.S01E01.Hello.Friend.mkv

          2.56G 100%  500.86MB/s    0:00:04 (xfr#1, to-chk=9/11)

Mr Robot/Mr.Robot.S01E02.Ones.and.Zer0es.mkv

          1.92G 100%    3.91MB/s    0:07:48 (xfr#2, to-chk=8/11)

Mr Robot/Mr.Robot.S01E03.d3bug.mkv

          1.77G 100%    4.99MB/s    0:05:39 (xfr#3, to-chk=7/11)

Mr Robot/Mr.Robot.S01E04.da3m0ns.mkv

          1.80G 100%    5.29MB/s    0:05:24 (xfr#4, to-chk=6/11)

Mr Robot/Mr.Robot.S01E05.3xpl0its.mkv

          1.76G 100%    5.56MB/s    0:05:01 (xfr#5, to-chk=5/11)

Mr Robot/Mr.Robot.S01E06.Br4ve.Trave1er.mkv

          1.86G 100%    5.78MB/s    0:05:06 (xfr#6, to-chk=4/11)

Mr Robot/Mr.Robot.S01E07.v1ew.s0urce.mkv

          1.81G 100%    5.22MB/s    0:05:31 (xfr#7, to-chk=3/11)

Mr Robot/Mr.Robot.S01E08.wh1ter0se.mkv

          1.80G 100%    5.79MB/s    0:04:56 (xfr#8, to-chk=2/11)

Mr Robot/Mr.Robot.S01E09.m1rr0r1ng.mkv

          2.03G 100%    5.92MB/s    0:05:26 (xfr#9, to-chk=1/11)

Mr Robot/Mr.Robot.S01E10.zer0.daY.mkv

          2.26G 100%    5.75MB/s    0:06:14 (xfr#10, to-chk=0/11)

sent 19.57G bytes  received 207 bytes  5.67M bytes/sec

total size is 19.56G  speedup is 1.00
```

USB3 drive (read):

```
$ rsync -rhv --progress "/run/media/cdstealer/STUFF/Mr Robot" ~

sending incremental file list

Mr Robot/

Mr Robot/Mr.Robot.S01E01.Hello.Friend.mkv

          2.56G 100%   52.57MB/s    0:00:46 (xfr#1, to-chk=9/11)

Mr Robot/Mr.Robot.S01E02.Ones.and.Zer0es.mkv

          1.92G 100%   54.39MB/s    0:00:33 (xfr#2, to-chk=8/11)

Mr Robot/Mr.Robot.S01E03.d3bug.mkv

          1.77G 100%   53.92MB/s    0:00:31 (xfr#3, to-chk=7/11)

Mr Robot/Mr.Robot.S01E04.da3m0ns.mkv

          1.80G 100%   54.20MB/s    0:00:31 (xfr#4, to-chk=6/11)

Mr Robot/Mr.Robot.S01E05.3xpl0its.mkv

          1.76G 100%   54.51MB/s    0:00:30 (xfr#5, to-chk=5/11)

Mr Robot/Mr.Robot.S01E06.Br4ve.Trave1er.mkv

          1.86G 100%   54.58MB/s    0:00:32 (xfr#6, to-chk=4/11)

Mr Robot/Mr.Robot.S01E07.v1ew.s0urce.mkv

          1.81G 100%   55.61MB/s    0:00:31 (xfr#7, to-chk=3/11)

Mr Robot/Mr.Robot.S01E08.wh1ter0se.mkv

          1.80G 100%   54.66MB/s    0:00:31 (xfr#8, to-chk=2/11)

Mr Robot/Mr.Robot.S01E09.m1rr0r1ng.mkv

          2.03G 100%   54.05MB/s    0:00:35 (xfr#9, to-chk=1/11)

Mr Robot/Mr.Robot.S01E10.zer0.daY.mkv

          2.26G 100%   48.40MB/s    0:00:44 (xfr#10, to-chk=0/11)

sent 19.57G bytes  received 210 bytes  56.64M bytes/sec

total size is 19.56G  speedup is 1.00
```

USB2 drive (write):

```
$ rsync -rhv --progress "/NAS/Videos/Mr Robot" /run/media/cdstealer/Tiny/

sending incremental file list

Mr Robot/Mr.Robot.S01E01.Hello.Friend.mkv

          2.56G 100%   15.25MB/s    0:02:40 (xfr#1, to-chk=9/11)

Mr Robot/Mr.Robot.S01E02.Ones.and.Zer0es.mkv

          1.92G 100%    1.44MB/s    0:21:13 (xfr#2, to-chk=8/11)

Mr Robot/Mr.Robot.S01E03.d3bug.mkv

          1.77G 100%    1.86MB/s    0:15:09 (xfr#3, to-chk=7/11)

Mr Robot/Mr.Robot.S01E04.da3m0ns.mkv

          1.80G 100%    1.93MB/s    0:14:47 (xfr#4, to-chk=6/11)

Mr Robot/Mr.Robot.S01E05.3xpl0its.mkv

          1.76G 100%    1.88MB/s    0:14:52 (xfr#5, to-chk=5/11)

Mr Robot/Mr.Robot.S01E06.Br4ve.Trave1er.mkv

          1.86G 100%    2.12MB/s    0:13:54 (xfr#6, to-chk=4/11)

Mr Robot/Mr.Robot.S01E07.v1ew.s0urce.mkv

          1.81G 100%    1.89MB/s    0:15:14 (xfr#7, to-chk=3/11)

Mr Robot/Mr.Robot.S01E08.wh1ter0se.mkv

          1.80G 100%    2.06MB/s    0:13:52 (xfr#8, to-chk=2/11)

Mr Robot/Mr.Robot.S01E09.m1rr0r1ng.mkv

          2.03G 100%    2.36MB/s    0:13:38 (xfr#9, to-chk=1/11)

Mr Robot/Mr.Robot.S01E10.zer0.daY.mkv

          2.26G 100%    2.13MB/s    0:16:51 (xfr#10, to-chk=0/11)

sent 19.57G bytes  received 207 bytes  2.04M bytes/sec

total size is 19.56G  speedup is 1.00

```

USB2 drive (read):

```
$ rsync -rhv --progress "/run/media/cdstealer/Tiny/Mr Robot" ~

sending incremental file list

Mr Robot/

Mr Robot/Mr.Robot.S01E01.Hello.Friend.mkv

          2.56G 100%   20.71MB/s    0:01:57 (xfr#1, to-chk=9/11)

Mr Robot/Mr.Robot.S01E02.Ones.and.Zer0es.mkv

          1.92G 100%   20.47MB/s    0:01:29 (xfr#2, to-chk=8/11)

Mr Robot/Mr.Robot.S01E03.d3bug.mkv

          1.77G 100%   20.58MB/s    0:01:22 (xfr#3, to-chk=7/11)

Mr Robot/Mr.Robot.S01E04.da3m0ns.mkv

          1.80G 100%   20.32MB/s    0:01:24 (xfr#4, to-chk=6/11)

Mr Robot/Mr.Robot.S01E05.3xpl0its.mkv

          1.76G 100%   20.51MB/s    0:01:21 (xfr#5, to-chk=5/11)

Mr Robot/Mr.Robot.S01E06.Br4ve.Trave1er.mkv

          1.86G 100%   20.43MB/s    0:01:26 (xfr#6, to-chk=4/11)

Mr Robot/Mr.Robot.S01E07.v1ew.s0urce.mkv

          1.81G 100%   20.38MB/s    0:01:24 (xfr#7, to-chk=3/11)

Mr Robot/Mr.Robot.S01E08.wh1ter0se.mkv

          1.80G 100%   20.43MB/s    0:01:23 (xfr#8, to-chk=2/11)

Mr Robot/Mr.Robot.S01E09.m1rr0r1ng.mkv

          2.03G 100%   20.41MB/s    0:01:34 (xfr#9, to-chk=1/11)

Mr Robot/Mr.Robot.S01E10.zer0.daY.mkv

          2.26G 100%   20.45MB/s    0:01:45 (xfr#10, to-chk=0/11)

sent 19.57G bytes  received 210 bytes  21.54M bytes/sec

total size is 19.56G  speedup is 1.00
```

I'm running kernel 4.4.2 (gentoo-sources).

dmesg gives the following info about each drive:

USB2 drive:

```
[ +14.812474] usb 1-8: new high-speed USB device number 3 using xhci_hcd

[  +0.164846] usb 1-8: New USB device found, idVendor=0781, idProduct=5571

[  +0.000010] usb 1-8: New USB device strings: Mfr=1, Product=2, SerialNumber=3

[  +0.000004] usb 1-8: Product: Cruzer Fit

[  +0.000004] usb 1-8: Manufacturer: SanDisk

[  +0.000003] usb 1-8: SerialNumber: 4C532000061023112472

[  +0.000633] usb-storage 1-8:1.0: USB Mass Storage device detected

[  +0.000613] scsi host6: usb-storage 1-8:1.0

[  +1.001301] scsi 6:0:0:0: Direct-Access     SanDisk  Cruzer Fit       1.26 PQ: 0 ANSI: 5

[  +0.000360] sd 6:0:0:0: Attached scsi generic sg6 type 0

[  +0.001082] sd 6:0:0:0: [sdf] 62530624 512-byte logical blocks: (32.0 GB/29.8 GiB)

[  +0.000875] sd 6:0:0:0: [sdf] Write Protect is off

[  +0.000010] sd 6:0:0:0: [sdf] Mode Sense: 43 00 00 00

[  +0.000297] sd 6:0:0:0: [sdf] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA

[  +0.007708]  sdf: sdf1

[  +0.001747] sd 6:0:0:0: [sdf] Attached SCSI removable disk
```

USB3 drive:

```
[Feb21 07:46] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd

[  +0.113671] usb 2-2: New USB device found, idVendor=1b1c, idProduct=1a14

[  +0.000009] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3

[  +0.000005] usb 2-2: Product: Voyager VEGA

[  +0.000004] usb 2-2: Manufacturer: Corsair

[  +0.000003] usb 2-2: SerialNumber: 0708496B277D2780

[  +0.001741] usb-storage 2-2:1.0: USB Mass Storage device detected

[  +0.000225] scsi host7: usb-storage 2-2:1.0

[  +1.067718] scsi 7:0:0:0: Direct-Access     Corsair  Voyager VEGA     000B PQ: 0 ANSI: 6

[  +0.000360] sd 7:0:0:0: Attached scsi generic sg7 type 0

[Feb21 07:47] sd 7:0:0:0: [sdg] 120999936 512-byte logical blocks: (61.9 GB/57.6 GiB)

[  +0.000556] sd 7:0:0:0: [sdg] Write Protect is off

[  +0.000008] sd 7:0:0:0: [sdg] Mode Sense: 23 00 00 00

[  +0.000503] sd 7:0:0:0: [sdg] No Caching mode page found

[  +0.000004] sd 7:0:0:0: [sdg] Assuming drive cache: write through

[  +0.027561]  sdg: sdg1 sdg2

[  +0.003923] sd 7:0:0:0: [sdg] Attached SCSI removable disk
```

LSUSB give:

```
# lsusb 

Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 002 Device 002: ID 1b1c:1a14 Corsair 

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 001 Device 003: ID 0781:5571 SanDisk Corp. Cruzer Fit

Bus 001 Device 002: ID 045e:07b2 Microsoft Corp. 

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
```

Does anyone have any ideas?

TIA

----------

## NeddySeagoon

cdstealer,

Some USB3 root hubs don't like to mix USB2 and USB3 devices.

If you connect a USB2 device first, it will not operate at USB3 speeds when you connect a USB3 device later. 

Your lsusb suggests that you only have USB3 root hubs.

Check with lspci.  If thats true, do not build the USB1 and USB2 drivers for your kernel, as the XHCI driver will take care of everything.

----------

## cdstealer

Hi Neddy,  I only have the XHCI driver compiled as that is the only USB chipset on the board.  I'll keep digging as there are no errors reported anywhere.

ta

----------

## NeddySeagoon

cdstealer,

Boot with no USB devices connected.

Connect your USB3 pen drive and speed test that.

Then add in your USB2 pen drive and speed test the USB3 again then the USB2.

Can you format the USB3 device?

You may be running into slowdowns caused by waiting for blocks to erase.

Formatting will offer to trim the whole device, so that there is no erase wait, providing the device supports trim.

----------

## cdstealer

Hi Neddy,

Yep, already done all that.  I even left the USB3 drive doing a full format which took several hours with no improvement  :Sad:   Although I've just tried something else.  I had a USB3 2.5" enclosure buried away.  So I put a mechanical HDD in and that doesn't appear to suffer.  So you maybe correct in that the issue is with the flash drives themselves.  Time for some more fault finding.

Thank you very much for your help so far, it's very appreciated  :Smile: 

```
sending incremental file list

Mr Robot/

Mr Robot/Mr.Robot.S01E01.Hello.Friend.mkv

          2.56G 100%  435.19MB/s    0:00:05 (xfr#1, to-chk=9/11)

Mr Robot/Mr.Robot.S01E02.Ones.and.Zer0es.mkv

          1.92G 100%  513.28MB/s    0:00:03 (xfr#2, to-chk=8/11)

Mr Robot/Mr.Robot.S01E03.d3bug.mkv

          1.77G 100%  179.48MB/s    0:00:09 (xfr#3, to-chk=7/11)

Mr Robot/Mr.Robot.S01E04.da3m0ns.mkv

          1.80G 100%   80.64MB/s    0:00:21 (xfr#4, to-chk=6/11)

Mr Robot/Mr.Robot.S01E05.3xpl0its.mkv

          1.76G 100%   77.92MB/s    0:00:21 (xfr#5, to-chk=5/11)

Mr Robot/Mr.Robot.S01E06.Br4ve.Trave1er.mkv

          1.86G 100%   76.12MB/s    0:00:23 (xfr#6, to-chk=4/11)

Mr Robot/Mr.Robot.S01E07.v1ew.s0urce.mkv

          1.81G 100%   76.86MB/s    0:00:22 (xfr#7, to-chk=3/11)

Mr Robot/Mr.Robot.S01E08.wh1ter0se.mkv

          1.80G 100%   75.52MB/s    0:00:22 (xfr#8, to-chk=2/11)

Mr Robot/Mr.Robot.S01E09.m1rr0r1ng.mkv

          2.03G 100%   74.91MB/s    0:00:25 (xfr#9, to-chk=1/11)

Mr Robot/Mr.Robot.S01E10.zer0.daY.mkv

          2.26G 100%   74.10MB/s    0:00:29 (xfr#10, to-chk=0/11)

sent 19.57G bytes  received 210 bytes  107.23M bytes/sec

total size is 19.56G  speedup is 1.00
```

----------

## spectromas

Does the target filesystem make any difference? I have a ongoing issue writing to ntfs taking forever on usb2 and causing very bad performance while the writing is taking place, with xfs or ext3/4 it is fine.

----------

## cdstealer

Hi spectromas,  I tried both VFAT and EXT4.  EXT4 it appears in terminal to be transferring at the correct speed, but actually isn't and with VFAT, the terminal output stalls after each file until it has been written to the flash drive.

It's entirely possible that I've missed an option out of the kernel as I've just transferred a 144Mb file on my work laptop which transferred @22Mbs.  I'll compare the configs when I get home and report back.

----------

## spectromas

Please do post the results here if you find anything that makes a difference, maybe I've missed something too.

----------

## NeddySeagoon

spectromas,

ntfs on Linux is a FUSE filesysem.  There is a speed penalty for that.

----------

## spectromas

NeddySeagoon, that is what I have come to accept but I have a feeling something isn't right somewhere, I don't think the system should be come unusable when a file is being transferred. I've been through lots of troubleshooting and could never narrow it down much. However, tonight I tried out manjaro (as much as I dislike it) on another hard drive because I know it comes with a kernel with bfs and bfq enabled. Sure enough file transfer to ntfs was smooth as silk, drastically different from what I have at the moment. Now the different factors I can think of include, kernel config, bfs, bfq, different hard drive, ext3 instead of xfs, systemd. I've tried bfs and bfq together and each separately and hadn't previously noticed much difference but it could be that I suppose. The others could all be contenders as far as I can see, thinking it could be my drive though as not many other people seem to share the same problem. Transfer speed was still quite slow but still faster. Ejecting/unmounting was also much faster, sometimes it takes my system 10mins or so after a file of a few gbs, it took about 1min in manjaro. 

Not intending to hijack the thread but this is certainly food for thought for me, just being able to do stuff while transferring files and it not taking 30mins to transfer and unmount a few gb sized file is revelatory right now.

----------

## cdstealer

Had time to test properly on my work laptop today.  It only has USB2, but it would prove a point.

```
Read speed

$ rsync -hv --progress "Mr.Robot.S01E10.zer0.daY.mkv" ~/

Mr.Robot.S01E10.zer0.daY.mkv

          2.26G 100%   28.25MB/s    0:01:16 (xfr#1, to-chk=0/1)

sent 2.26G bytes  received 35 bytes  29.54M bytes/sec

total size is 2.26G  speedup is 1.00

Write speed

$ rsync -hv --progress "/home/me/Mr.Robot.S01E10.zer0.daY.mkv" /run/media/me/STUFF/

Mr.Robot.S01E10.zer0.daY.mkv

          2.26G 100%  238.72MB/s    0:00:09 (xfr#1, to-chk=0/1)

sent 2.26G bytes  received 35 bytes  9.85M bytes/sec

total size is 2.26G  speedup is 1.00
```

So in conclusion, either my USB3 drive is only running at USB2 speed, or the ASMedia ASM1142 USB3.1 controller isn't working as desired (yet)

```
01:00.0 USB controller: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller (prog-if 30 [XHCI])

        Subsystem: Micro-Star International Co., Ltd. [MSI] ASM1142 USB 3.1 Host Controller

        Flags: bus master, fast devsel, latency 0

        Memory at df200000 (64-bit, non-prefetchable) [size=32K]

        Capabilities: [50] MSI: Enable- Count=1/8 Maskable- 64bit+

        Capabilities: [68] MSI-X: Enable+ Count=8 Masked-

        Capabilities: [78] Power Management version 3

        Capabilities: [80] Express Endpoint, MSI 00

        Capabilities: [100] Virtual Channel

        Capabilities: [200] Advanced Error Reporting

        Capabilities: [280] #19

        Capabilities: [300] Latency Tolerance Reporting

        Kernel driver in use: xhci_hcd
```

I'll keep digging, but I think I've exhausted the kernel options that I can think of that could effect this.

----------

## cdstealer

Hi Guys, I'm wrapping this one up.  After many kernel compiles and testing different OSes and USB drives.  I can categorically confirm that my 64Gb Corsair USB3.0 drive is not fit for purpose.  Time to go research a full speed replacement.   :Rolling Eyes: 

Here are my benchmark results.

http://cdstealer.com/images/usb_benchmark.jpg

Thanks again.

----------

## P.Kosunen

https://www.sandisk.com/home/usb-flash/extreme-usb

SanDisk Extreme USB has good write performance, even random write.

----------

## cdstealer

Cheers P.Kosunen I'll go check it out  :Smile: 

----------

