# Memory usage, usb removable drive, slow system?

## breakerfall

I posted this in other things gentoo, but I'm not getting any responses and I don't think I will either. Maybe I posted it in the wrong place to begin with... if a mod sees this, could you please merge the two into this forum? Thanks  :Mr. Green: 

https://forums.gentoo.org/viewtopic-t-543310.html

 *Quote:*   

> Ok, I have a basic understanding on how linux memory management works, with the caching and the like. But I'm having a daft, annoying experience. It's nothing major as I don't do it often, but I just copied two 350 MiB files from my HD to a USB key-drive and nearing half way through the second file, I notice the transfer speed starts to GRIND. My computer starts to become less responsive and when I check free -m, I see that a tonne is cached, nearly all of it. In fact, the "used" columb shows all but a few MBs being used. Normal linux behaviour? Maybe. But should my computer start dying when transferring files to an external removable drive?
> 
> Granted, it was easy to fix the problem. The transfer was grinding to a near halt, so I cancelled it, cleared my cache with echo 3 > /proc/sys/vm/drop_caches and the transfer went nice and speedy. It quickly sapped up the memory again during the transfer though.
> 
> So my question: What's going on?
> ...

 

----------

## NeddySeagoon

breakerfall,

Your USB key drive will have a maximum write speed of about 8MB/sec. Thats a feature of FLASH memory.

You can read data from a single had drive at 50Mb/sec, so the data is cached in RAM, waiting to be flushed to the USB key.

As these buffers are 'dirty' (waiting to be written), they cannot be swapped [1], so memory fills with dirty buffers.

As a result, your system becomes unresponsive because other things get swapped.

[1] Think about that. It would not be sensible to write to (the wrong) disk, buffers that were waiting to be written to disk.

You may be able to limit the amount of RAM used for caching this way but I don't know how.

----------

## breakerfall

Thanks for the response...

I've not seen this kind of behaviour before though. What you're saying doesn't sound completely implausible, but I just find it strange that I haven't come across it before. The first 350 megs being transfered went pretty quickly. But then the whole thing came to a grind as it continued. Surely the system should flush/clear the contents of the cache that have been written to the specified destination successfully, no?

Has anyone else at least come across this kind of behaviour? Has anyone tried copying 700. 800+ MBytes to a USB key drive or something similar and witnessed the same level of slowdown, in both transfer speed and system responsiveness?

Thanks

----------

## breakerfall

BUMP... please, can someone confirm they experience the same or not!?

I'm even having the same problem when transferring from my HD to another over a network share. Surely, somebody has an idea. The transfer is really grinding! I haven't come across this problem before and I've been using gentoo for years. It's a 100mbit network, yet the connection isn't even nearing that of a 10mbit connection. I can download files from others on the network at great speeds, but transferring outbound has the same problem as my original post dictates. The memory is chewed up. I can run watch free -m and watch it just disappearing into the cache, never to come back unless I clear it manually.

----------

## Lawrence Gold

 *breakerfall wrote:*   

> BUMP... please, can someone confirm they experience the same or not!?
> 
> I'm even having the same problem when transferring from my HD to another over a network share. Surely, somebody has an idea. The transfer is really grinding! I haven't come across this problem before and I've been using gentoo for years. It's a 100mbit network, yet the connection isn't even nearing that of a 10mbit connection. I can download files from others on the network at great speeds, but transferring outbound has the same problem as my original post dictates. The memory is chewed up. I can run watch free -m and watch it just disappearing into the cache, never to come back unless I clear it manually.

 

The same thing happens to me. Copying to the USB flash drive is pathetically slow -- much, much slower than from OS X or Windows. I also see unresponsiveness during heavy I/O.

The first time I saw it was after I upgraded to a Gigabyte DS3 motherboard with a  Core2 Duo (E6400). I'm running with ~amd64, and don't recall seeing this problem on my previous ~x86 system. I'll have to try an x86 liveCD, such as Knoppix, to see if the problem is possibly limited to amd64.

By the way, this thread may be relevant:

https://forums.gentoo.org/viewtopic-t-482731-highlight-unresponsive.html

----------

## bigBonsai

I had a similar problem. I always used -o sync for my vfat usb stick and went fine with it. But since a kernel update to 2.6.19 usb writes became really slow.

This helped me: https://forums.gentoo.org/viewtopic-t-537871-start-0-postdays-0-postorder-asc-highlight-usb+flush.html

It says you should use the option flush instead of sync for mount or in your fstab.

----------

## breakerfall

 *bigBonsai wrote:*   

> I had a similar problem. I always used -o sync for my vfat usb stick and went fine with it. But since a kernel update to 2.6.19 usb writes became really slow.
> 
> This helped me: https://forums.gentoo.org/viewtopic-t-537871-start-0-postdays-0-postorder-asc-highlight-usb+flush.html
> 
> It says you should use the option flush instead of sync for mount or in your fstab.

 

But what if people don't use fstab? Does udev automounting use the sync option? I think ivman did by default (didn't find a way to change that), but now there's an automounting manager with all the major DEs, fun.

----------

## DirtyHairy

I've never encountered this problem myself, but as I have 1GB RAM, this is not surprising (although CPU consumption goes up to 100% while waiting for the writes to finish, but that causes no real performance issue). However, there is an option in 2.6 kernel called swappiness which controls the inclination of the system to sacrifice buffers in order to avoid swapping. It is controlled via /proc/sys/vm/swappiness; the maximum value 100 means "always prefer swapping", the minimum 0 the opposite. Since the default 60 is on the "swap!" side, you might consider reducing this setting and see if it helps...

----------

## breakerfall

 *DirtyHairy wrote:*   

> I've never encountered this problem myself, but as I have 1GB RAM, this is not surprising (although CPU consumption goes up to 100% while waiting for the writes to finish, but that causes no real performance issue). However, there is an option in 2.6 kernel called swappiness which controls the inclination of the system to sacrifice buffers in order to avoid swapping. It is controlled via /proc/sys/vm/swappiness; the maximum value 100 means "always prefer swapping", the minimum 0 the opposite. Since the default 60 is on the "swap!" side, you might consider reducing this setting and see if it helps...

 

The thing is, I also have a 1 Gig of RAM and even on a fresh boot, using minimal memory, transferring a 350MiB file causes this issue. My swap is seemingly untouched, but the memory is chomped anyway. It's bizarre.

----------

## DirtyHairy

OK, this is strange, because if nothing is has to be swapped, then the system shouldn't be slowed down by heavy memory usage. Have you checked CPU usage during the transfers? What exactly do you mean my "grinding to a halt" --- have you measured the write speed to the USB device in this situation?

----------

## drescherjm

 *Quote:*   

> My swap is seemingly untouched, 

 

Swap is never used unless it is absolutely needed. 

 *Quote:*   

> but the memory is chomped anyway.

 

It should be. Linux will use nearly all of your ram for disk cache so after reading 1GB of data from your drives you should have almost no memory free. There is no problem with that at all as when an application needs it linux will shrink the disk cache and give your app the memory.

BTW, Have you selected a preemptable kernel? As I have seen this help in a few cases.

----------

## drescherjm

 *Quote:*   

> Has anyone tried copying 700. 800+ MBytes to a USB key drive or something similar and witnessed the same level of slowdown, in both transfer speed and system responsiveness?
> 
> 

 

I have a 2GB cosair flash drive that is supposed to have a write speed of around 19MB/s which I can test today at work if I get time.

----------

## breakerfall

 *drescherjm wrote:*   

>  *Quote:*   My swap is seemingly untouched,  
> 
> Swap is never used unless it is absolutely needed. 
> 
>  *Quote:*   but the memory is chomped anyway. 
> ...

 

As I mentioned in my original post, I understand how linux memory management works. 

That's why I find this a bizarre problem. The actual data transfer grinds to a halt, meaning it will be mid transfer and the transfer will become so slow, that it's transferring mere KiB at a time. The computer also starts to become slightly unresponsive, but I rarely let it get to that point, I clear the cache manually myself. Now considering I've been using gentoo for a good chunk of time and have never run across this problem, I'm going to say the problem isn't me and my kernel options (I am indeed using a preemptive kernel by the way). I really need to sit down and compare the differences between my laptop (where the problem exists) and my desktop machine, which is running an older kernel (2.6.14 or lower I would guess).

----------

## DirtyHairy

Have you tried different USB devices? My girlfriends mp3-player e.g. seems to have some internal cache which makes transfers to be fast at first and then slow down considerably. If you have enough buffers, then you wouldn't notice, but if you are low on them, then it will start to show up...Last edited by DirtyHairy on Fri Mar 16, 2007 3:37 pm; edited 1 time in total

----------

## drescherjm

 *Quote:*   

> As I mentioned in my original post, I understand how linux memory management works. 

 

Sorry, I did not go to the other thread.

----------

## breakerfall

 *drescherjm wrote:*   

>  *Quote:*   As I mentioned in my original post, I understand how linux memory management works.  
> 
> Sorry, I did not go to the other thread.

 

I meant the original post of this thread. :p

No matter though, I'm just hoping it's fixable. I really wonder if it's kernel related.

----------

## eccerr0r

I don't recall seeing way too much problems with USB devices but the other thing that could take effect here is the queueing system.  After 350 or so MB gets dumped into the write queue, the machine has to actually write it... Now the device is slow, and the machine really can only write one chunk at a time, so if you submit another request to the hard disk while the usb write is taking place and there's a _lot_ of crap in the queue/buffer - the hard drive request needs to wait just because it's in the queue and thus the whole system slows down.  It's an "in-order/out of order" issue - to make sure everything is coherent, in-order is preferred, but for throughput, out of order is king.

Does the system return to normal after all the data gets written out to the USB device?

I'm not that familar with the queueing and how it handles queueing with slow devices, though I would imagine there should be some way to make the two requests go out of order here (if it doesn't already... I thought they should be on separate queues).  Might be a master queue too that handles requests for all disks on the system?  Anyway, try reducing the maximum sectors it will handle in a chunk to the flash disk, it might help too... try some sysfs tuning, reduce maximum chunk size like /sys/block/sda/queue/max_sectors_kb and see if it's any better, hopefully it will take less time to write a single block at a cost of reducing overall throughput to that disk?

----------

## drescherjm

I tested my corsair flash voyager 2gb stick formatted with fat32 with 3 ~600MB files with cp and dd on an athlon64 3200 with 2 GB of memory and I did not see any slowdown at all. Actually dd returned before the drive stopped writing with benchmark of around 50MB/s a few seconds later the drive stopped flashing and I was able to unmount it after a second or two activity. This is on a 2.6.19 vserver-sources kernel. I am sorry this is no help.

----------

