# Excessive caching kernel

## dE_logics

100% of all my main memory has been utilized... system seems to hang for a few seconds periodically.

There's no single process consuming such memory, DVD encoding going on in background using data from an nfs FS

I tried - 

sync; echo 3 > /proc/sys/vm/drop_caches

And the memory usage drops all of a sudden to something feasible, but after a few seconds it's up again.

e.g. - 

```
free -m;sync; echo 3 > /proc/sys/vm/drop_caches;free -m;sleep 60;free -m

             total       used       free     shared    buffers     cached

Mem:           967        958          9          0          4        582

-/+ buffers/cache:        371        596

Swap:         1027        164        863

             total       used       free     shared    buffers     cached

Mem:           967        559        407          0          2        193

-/+ buffers/cache:        364        602

Swap:         1027        164        863

             total       used       free     shared    buffers     cached

Mem:           967        958          9          0          2        588

-/+ buffers/cache:        367        600

Swap:         1027        163        864

```

This is cause of NFS (problem resolved after DVD encoding). Mine is not working as a server, just client.

----------

## gentoo_ram

Unused memory is wasted memory.  The kernel is caching data so it doesn't have to be read off the disk again (slow!).  Why are you fighting it?

----------

## dE_logics

As said - 

 *Quote:*   

> system seems to hang for a few seconds periodically. 

 

----------

## kernelOfTruth

 *dE_logics wrote:*   

> As said - 
> 
>  *Quote:*   system seems to hang for a few seconds periodically.  

 

consider adding the ck-patchset to your kernel this should improve swap-handling

in addition to that:

try the following compilation:

 *Quote:*   

> echo "10" > /proc/sys/vm/page-cluster
> 
> # default: 3
> 
> # for swapping (?) and page-faults, higher value might speed up linear readings significantly (!) - something like readahead
> ...

 

if you prefer higher throughput and higher performance the dirty_*_ratio  settings of course should be set higher

also add the following patch:

https://forums.gentoo.org/viewtopic-p-6437248.html#6437248

----------

## dE_logics

I did set the dirt ratio in my zen kernel. Thanks for the tips... now I can change them in realtime.

----------

## dE_logics

I didnt apply the patch, but this is a general problem with appeared when I switched to the Zen kernel. The above tweaking did not work.

Its appears that whenever I do copy paste operation, the data is being extensively written to the disk for caching... this should have been solved by lowering dirty_background_ratio and dirty_ratio, but it did not. In the mean time, I've selected the desktop profile in Zen, which means - 

DIRTY_RATIO: 50

BACKGROUND_DIRTY_RATIO: 20

Which appears to be high... I'll reduce them.

----------

## dE_logics

Apparently it didin't work.

----------

## energyman76b

 *dE_logics wrote:*   

> As said - 
> 
>  *Quote:*   system seems to hang for a few seconds periodically.  

 

caused by disk access. Not memory. Turn down swappiness. Use vanilla kernel. Don't enable stupid stuff.

----------

## dE_logics

 *energyman76b wrote:*   

>  *dE_logics wrote:*   As said - 
> 
>  *Quote:*   system seems to hang for a few seconds periodically.   
> 
> caused by disk access. Not memory. Turn down swappiness. Use vanilla kernel. Don't enable stupid stuff.

 

Well it appears to be deadline behavior. Swappiness is always under my control, and it is cause of memory. Any data transfer operation and memory starts filling, and when ~9 MB is free, the hanging starts.

----------

## energyman76b

then use cfq.

----------

## dE_logics

I don't think that will solve the memory problems... and with this usage, using CFQ IMO will hang the system completely.

Trying it out.

----------

## dE_logics

Yup, there's no change in the memory usage, but that DID solve the issue with lower leaner performance.

Can I switch between the 2?... I head I can.

One more question, suppose I've set my dirty_ratio and dirty_background_ratio to 100%... and it happens that memory is full to 95% mostly cause of the dirty pages, and I wanna allocate 100 MB for a new program... then will the dirty pages be written to reduce memory usage, or will it start using more swap?

----------

## kernelOfTruth

 *dE_logics wrote:*   

> Yup, there's no change in the memory usage, but that DID solve the issue with lower leaner performance.
> 
> Can I switch between the 2?... I head I can.
> 
> One more question, suppose I've set my dirty_ratio and dirty_background_ratio to 100%... and it happens that memory is full to 95% mostly cause of the dirty pages, and I wanna allocate 100 MB for a new program... then will the dirty pages be written to reduce memory usage, or will it start using more swap?

 

I suppose that depends on how high you set vfs_cache_pressure

if it's more than 100 those are more likely reclaimed - if you set it lower things will start to swap up provided you set swappiness high

----------

## Ant P.

 *dE_logics wrote:*   

> Yup, there's no change in the memory usage, but that DID solve the issue with lower leaner performance.
> 
> Can I switch between the 2?... I head I can.

 

/sys/block/*/queue/scheduler

----------

## dE_logics

Thanks

----------

