# harddrive usage eats up memory

## kortec

I just finished installing Gentoo on a box, and it seems like everything is going just fine -- got X looking great, sound working, and all that junk -- but there's an odd lingering bug. 

Whenever there's a relatively large disk read, the memory utilization goes through the roof, and everything slows to a crawl. This could be as simple as loading the firefox binary if it isn't in RAM already, downloading a file that's bigger than about 1 MB, starting acroread, editing id3 tags on mp3s -- whatever. As long as the hard drive is spinning, this happens. (When I say 'memory utilization goes through the roof', I mean that I have half a gig of ram in here, and after disk IO starts, only about 9MB is free according to 'free -m').

Any ideas what might be causing this? I did check to make sure that DMA support was enabled in the kernel, and it is.

----------

## tobr

That sounds normal.

a) Hard drive access is (relatively) slow. If there’s a large read things can slow down.

b) because of a) Linux caches hard disk in RAM (because empty RAM is useless). This will be freed if an application needs it. free shows the cache (surprisingly) under “cached”, which you need to subtract from the used RAM if you want to know what your apps use.

----------

## gentoo_ram

Yes, repeat after me:

"Free memory is wasted memory."

The kernel will use as much RAM as possible to cache disk accesses.  This is a good thing.  If the memory is needed by real programs, the kernel will free the disk cache.  Don't worry about it unless you're running out of swap space too.

----------

## kortec

So lots of RAM usage is normal, and I do not pretend to understand the kernel's vm decisions, but the severe slow down isn't normal. Moving a 80 MB tar file from one directory to another shouldn't take 5 mins and make it impossible to move the mouse. The hardware isn't that old (athlon xp, 7200 RPM 250 GB hd, etc.). This isn't just in X either -- if I'm doing similar things just in a terminal, I can't switch between ttys or, if I background the process with a lot of disk usage, the tty takes too long to update from keyboard input to make anything possible.

----------

## eccerr0r

The problem is likely not memory usage, but rather DMA issues.  Is DMA turned on for your hard drive?

Did you compile your disk controller's native driver into the kernel?

hdparm -t /dev/harddrive?

----------

## fenrisulfr

Is the drive on an addon controller card? SATA or IDE?  What is the result of hdparm -t /dev/device?  That choking on the drive is definitly not right.  My little 233 laptop does just fine with read writes to the drive.

----------

## kortec

I get this from hdparm:

```

tilysi ~ # hdparm -tT /dev/hda

/dev/hda:

 Timing cached reads:   588 MB in  2.00 seconds = 293.51 MB/sec

 Timing buffered disk reads:    8 MB in  3.91 seconds =   2.05 MB/sec

tilysi ~ # 

```

Giving hdparm no args to get an info dump yields

```

tilysi ~ # hdparm /dev/hda

/dev/hda:

 multcount     = 16 (on)

 IO_support    =  0 (default 16-bit)

 unmaskirq     =  0 (off)

 using_dma     =  0 (off)

 keepsettings  =  0 (off)

 readonly      =  0 (off)

 readahead     = 256 (on)

 geometry      = 30401/255/63, sectors = 488397168, start = 0

tilysi ~ # 

```

I tried to turn DMA on, but got this

```

tilysi ~ # hdparm -d1 /dev/hda

/dev/hda:

 setting using_dma to 1 (on)

 HDIO_SET_DMA failed: Operation not permitted

 using_dma     =  0 (off)

tilysi ~ # 

```

even though it's enabled in the kernel. Thoughts?

----------

## kortec

Found the bug, I think. I recompiled my kernel with more specific/correct IDE chipset drivers, and got this output.

```

tilysi ~ # hdparm -tT /dev/hda

/dev/hda:

 Timing cached reads:   464 MB in  2.01 seconds = 231.42 MB/sec

 Timing buffered disk reads:  172 MB in  3.02 seconds =  56.98 MB/sec

tilysi ~ # hdparm /dev/hda

/dev/hda:

 multcount     = 16 (on)

 IO_support    =  1 (32-bit)

 unmaskirq     =  1 (on)

 using_dma     =  1 (on)

 keepsettings  =  0 (off)

 readonly      =  0 (off)

 readahead     = 256 (on)

 geometry      = 30401/255/63, sectors = 488397168, start = 0

tilysi ~ # 

```

which is heartening, and makes me feel slightly dumb. Ah well.

----------

## eccerr0r

glad you got it working.

The weirdest symptoms can mislead the proper solution eh?

----------

## kortec

Seriously. The weirder thing is that I had tried to set the DMA on the drive earlier and got that very error and I barely noticed it. And then I had to talk through the really silly things I was noticing happening to get to any sort of a solution at all. Ah well.

----------

## bunder

 *kortec wrote:*   

> So lots of RAM usage is normal.

 

bingo.

```
$ free -m

             total       used       free     shared    buffers     cached

Mem:          1010        973         37          0         76        546

-/+ buffers/cache:        349        661 <--- this is the line you want to look at

Swap:         1019          0       1019

```

----------

