# linux memory management

## ravalox

Okay, I have a question to posit to all us linux users out there about memory management.  The kernel does one thing that I can't wrap my mind around.  I recently got a copy of neverwinter nights.  I feel this runs better on linux than I ever saw it run on windows, however linux has a strange disability.  When I'm playing a game, when it consumes past a certain amount of memory, the kernel will kill the process.  So I go from game to desktop.  I've seen the kernel do this on a number of occassions, gdesklet memory leakage used to consume most of my RAM, so often UT2k4 would be aborted right in front of me.  I am positive this is what linux is doing.  Now, what I don't get is that I can play neverwinter on windows all day and never have to worry about this.  So what's the deal?  I don't understand how linux can just do that when a process exceeds some kind of unclear threshold?  This hardly seems like an "elegant solution" to memory management problems, when side-by-side windows seems to chug on through.

----------

## bollucks

Linux memory management does not do what you say unless your programs have completely exhausted all the ram and swap. More likely the linux versions are more memory hungry and also more unstable since they're usually 3rd class citizen ports of the originals.

----------

## luisfelipe

Dude, how much memory/swap you have ? Cuz I've never seen that ?

Perhaps you have something else running that is leaking memory or something.

----------

## ravalox

I'm running off of 512 megs of RAM, I can't recall the swapspace off the top of my head for that machine.  I have another gentoo system that runs off of a gig of RAM and I think has a 512 swap setting that does the same thing to me.  I don't have any other processes eating RAM though, I top'ed to look for one.

----------

## quickshiftin

have you tried looking at any system logs at all?  maybe your system can provide an explanation.

you said the program gets shutdown once a certain amount of memroy is consumed, how did you arrive at this conclusion; and what amount of memory consumption make the program halt execution?

----------

## luisfelipe

Well, you got another problem then. Cuz I played neverwinter for 3 days straight with only 256 ram and 

256 swap. 

Try closing every single program you have, start X witouth any window manager (or with a really simple one

such as evilwm) and try running nwn to see what happens.

----------

## ravalox

I am running this after a fresh boot usually with nothing extraneous outside of gnome running.  With the increase in memory on my other computer (the one with a gig) the same things happens but it takes MUCH longer for it to.  I don't believe this is the client crashing, this has all the symptoms of memory conservation.  On my desktop it did the exact same thing with Ut2k4.

----------

## luisfelipe

Really weird. Which kernel are you using ? Vanilla, gentoo-sources, or other ?

----------

## ravalox

the gentoo sources for 2.6.13, but honestly I have an Ubuntu install that does the exact same thing to me.

----------

## quickshiftin

can you post the output of

```
$tail dmesg
```

directly after a crash?  i think this will shed some light.

i run vmware 5 and it takes over 512MB of ram by itself.  i have never had it suddenly crash, there must be a way to get to the bottom of this.  i think kernel logging is a good place to start.

----------

## luisfelipe

Another couple of questions :

1) Is your RAM working ok (memtest and stuff) ?

2) What is your video card and driver version ?

----------

## curtis119

This is an extremely odd error your having. If the kernel uses up all the RAM+swap it will do one or all of three things:

Reduce the size of the buffer and page caches

Swap out System V shared memory pages

Swap out and discard the oldest pages

One thing it NEVER does is kill an application outright, if it runs out of physical RAM+swap your system will simply crawl to a halt and require a hard reboot. There are daemons around, like verynice, that will kill a runaway proccess for you and I *think* ubuntu may install one by default (don't quote me on that though) but on a Gentoo box you would have to explicitly install and configure it.

One thing that *would* kill the app is openGL errors. It wouldn't kill it implicitly but if a fatal error occured the app would die. If I were you I would look into the video card (driver) and your openGL implementations.

----------

## luisfelipe

Actually, I run Gentoo and I've already seen apps that use all the RAM+swap cause other applications to be killed.

Lots of times actually. I can even reproduce it (it's been happening with me for the last two gcc 4.1 beta ebuilds, 

during the installation phase).

----------

## martin.k

Under emergency memory situations - when Linux runs out of memory (both physical and swap combined) the kernel starts killing processes. It uses an algorithm to work out which process to kill first - it tries to kill offending memory hogs that have been running for a short amount of time first before less used processes that have been running for a long time, which are most likely important system services. 

This functionality is known as the out of memory (OOM) killer.

So it's probably your crappy game, app or whatever that causes the above issue. If your game is RAM + swap hog and OOM kills it - that's perfect! If the game is not the cause - then nobody's perfect  :Smile: 

----------

## frenkel

Are you sure the kernel killed it? I have a lot of friends who experience segmentation faults with neverwinter nights. Then you also drop back to the desktop. Try to start it from a terminal and past the errors on the terminal after the game was killed.

----------

## curtis119

 *martin.k wrote:*   

> Under emergency memory situations - when Linux runs out of memory (both physical and swap combined) the kernel starts killing processes. It uses an algorithm to work out which process to kill first - it tries to kill offending memory hogs that have been running for a short amount of time first before less used processes that have been running for a long time, which are most likely important system services. 
> 
> This functionality is known as the out of memory (OOM) killer.
> 
> So it's probably your crappy game, app or whatever that causes the above issue. If your game is RAM + swap hog and OOM kills it - that's perfect! If the game is not the cause - then nobody's perfect 

 

Holy CRAP! I cannot believe they actually implemented that god awful thing! I remember the flamewar at the end of the 90's about it and I thought it was decided not to include it. Fuck, I need to read more. All this time I had no idea that it existed.

I'm off to read about it and figure out how to turn it off.

OK, all you have to do is echo 2 >> /proc/sys/vm/overcommit_memory to turn it off. What were they thinking when they implemented this? I jsut read about it and it SUCKS! It can turn off ANY process at random, including things like screen lockers and other security features. STUPID!!!!!!!!!!!!!!!!!!

----------

## ravalox

Nwn has only seg faulted on me one time, every other instance of this application behaving unhelpfully it has simply dropped me back to my desktop with an error along the lines of "stopped on line 312".  I have learned that gdesklets seems to have a voracious memory leaks, so restarting it periodically helps matters a lot, but it still shouldn't do this.

----------

## widan

 *curtis119 wrote:*   

> What were they thinking when they implemented this? I jsut read about it and it SUCKS! It can turn off ANY process at random, including things like screen lockers and other security features. STUPID!!!!!!!!!!!!!!!!!!

 

It's not random. It will select processes with heavy memory usage and short run time. It should not kill a lock program (as it uses very little memory). Look at mm/oom_kill.c in the kernel source if you want to know the algorithm is uses to select programs to kill (particularly the badness() function, at the beginning, and the comments in it).

```
/*

 * oom_badness - calculate a numeric value for how bad this task has been

 * @p: task struct of which task we should calculate

 * @uptime: current uptime in seconds

 *

 * The formula used is relatively simple and documented inline in the

 * function. The main rationale is that we want to select a good task

 * to kill when we run out of memory.

 *

 * Good in this context means that:

 * 1) we lose the minimum amount of work done

 * 2) we recover a large amount of memory

 * 3) we don't kill anything innocent of eating tons of memory

 * 4) we want to kill the minimum amount of processes (one)

 * 5) we try to kill the process the user expects us to kill, this

 *    algorithm has been meticulously tuned to meet the principle

 *    of least surprise ... (be careful when you change it)

 */
```

And usually it is better to kill the one process that is hogging a lot of memory (and probably slowing the machine to a crawl, because the kernel is constantly swapping things in and out) rather than render the machine unusable or causing it to crash. A crash could cause filesystem corruption and data loss.

----------

## AllenJB

An OOM Killer is stupid? What do you really expect to happen when an app consumes all your available memory? I'd rather have the offending app killed as opposed to my system becoming totally useless.

Allen

----------

## codergeek42

 *curtis119 wrote:*   

>  What were they thinking when they implemented this? I jsut read about it and it SUCKS! It can turn off ANY process at random, including things like screen lockers and other security features. STUPID!!!!!!!!!!!!!!!!!!

 Well it does choose trhe process(es) to kill based on much memory they are using, so if you're screen-locking application is using up all the memory, you may want to think of replacing it.  :Wink: 

I had this happen with Nexuiz a few times, but it's since stopped now that I've added another gig of RAM in there (4x512MB now   :Cool:  ).

----------

## curtis119

Apparently the docs I found on this OOM killer were out of date and highlighted the faults of the BETA code. It seems that it is a good thing after all now that they have the bugs worked out of it. All this time I have been carefully keeping tabs on memory usage on my servers in order to avoid OOM lockups. Not that I will stop doing this since it is still a good idea to keep track of this sort of stuff for utilization tracking but now I won't be so paranoid about it. Thanks for bringing this up. I learned something new today and that's always a Good Thing(tm).

----------

