# how to protect from oom-killer [solved]

## asdasdasdasdasd

Hello dear gentoo users,

I am a vmware gsx user. I ran my virtualmachines since begginning of this year on win2003-x64. 

Everything ran just fine - I cannot remember any minor/big issues-it just ran. 

Then I completly changed to linux. Honestly, this was a more religious decision. 

So used my virtuel machines on Suse 9.2 . I think, it was running with kernel 2.6.8 - I am not sure. 

But doing heavy io load and staring randomly some vms - simply copying 130GB from one disk to another any of my virtual machines get randomly killed, 

and also worth: my sshd also somestimes get killed...but they restart...but my screensaver on kde logged in with root does not restart. 

This document of vmware describes also my situation well:

http://www.vmware.com/support/kb/enduser/std_adp.php?p_sid=Rrxvr1Th&p_lva=&p_faqid=1669&p_created=1113855235&p_sp=cF9zcmNoPTEmcF9ncmlkc29ydD0mcF9yb3dfY250PTEmcF9zZWFyY2hfdGV4dD1vb20mcF9zZWFyY2hfdHlwZT03JnBfcHJvZF9sdmwxPX5hbnl_JnBfcHJvZF9sdmwyPX5hbnl_JnBfc29ydF9ieT1kZmx0JnBfcGFnZT0x&p_li=

After searching the web, I did not found, how to change oom behavior -except patching the kernel, 

maybe this is just a speicifc issue on this specific kernel revision. 

So I changed to Gentoo - because of nice emerge,packages.gentoo.org, excellent documents and of course this forum - so that future kernel upgrades/changes will be more painless.

gsx server on my gentoo host is running fine. But my problem exists, when copying again about 130gb files (with cp)  from one disk to another.

this system has  6GB Ram ("Dell PowerEdge 2800 System") all os are x64 (am64). 

the virtuelserver gets 2GB RAM and choose "Allow most virtual machine memory to be swapped". 

I also tried now these:

echo "-15" > proc/[pids of vmware-vmx]/oom_adj

or echo 2 > /proc/sys/vm/overcommit_memory, but this also didn't work, so I restartet changed to echo 1 > /proc/sys/vm/overcommit_memory, also failed, and keeping to 0 failes also.

But I am REALLY afraid of the oom, it just kills randomly(this seems so to me as a normal user)  serious process. 

why is this so , it's terrible!? It has now killed my sshd and is not restarting (but connecting to gsx console works) .

I cannot believe that this is the normal behavior of 2.6.x kernels -  with win2003 installed, 

any user can log in to the gsx console and can do any heavy io operation or even start another vms with silly memory settings 

(most of them will of course fail, when doing that on my gentoo with kernel 2.6.x, almost everything get killed), 

but my serios virtual server has never been killed! A friend of my uses linux with linux 2.4 (Redhat), he does not experience such oom-killer situations - when performing similar actions. 

But I do not want to use kernel 2.4.x systems, because they may be unsupported in future.

How can I change the behavior of the oom-killer not to kill "randomly" processes ? Is  the kernel 2.6.x generally recommened for memory intensiv usage ?

 (I just only give  the vm 2GB, then when starting a single cp process (as guest user!!), there is no guarentee, that my serious virtual machine will survive  :Sad:  ) 

There were some patches in kernel-mailinglist with things like /proc/sys/vm/oom_protect and similiar....may be this helps somehow ?

Best regards

steveLast edited by asdasdasdasdasd on Mon Feb 06, 2006 10:25 pm; edited 1 time in total

----------

## asdasdasdasdasd

the server is currenlty running with 2.6.13-r3

----------

## asdasdasdasdasd

hello gentoo users,

I solved the memory problem.  It is a fast hack to prevent "random" kills - it prevents killing any process.

Therefore I added this " if (1) {

 return;

}" in __oom_kill_task (at the beginning)

oom_kill.c:

And now, no vmware process is killed anymore!! When other user try to start more virtualmaschines, vmware gives out the message that no memory is left any more and my production virtual server is still alive! That's the behavior I wanted and which is as same as it is in kernel 2.4 of my friend and win2003 x64!

Steve

----------------

/*

 *  linux/mm/oom_kill.c

 *

 *  Copyright (C)  1998,2000  Rik van Riel

 *      Thanks go out to Claus Fischer for some serious inspiration and

 *      for goading me into coding this file...

 *

 *  The routines in this file are used to kill a process when

 *  we're seriously out of memory. This gets called from kswapd()

 *  in linux/mm/vmscan.c when we really run out of memory.

 *

 *  Since we won't call these routines often (on a well-configured

 *  machine) this file will double as a 'coding guide' and a signpost

 *  for newbie kernel hackers. It features several pointers to major

 *  kernel subsystems and hints as to where to find out what things do.

 */

#include <linux/mm.h>

#include <linux/sched.h>

#include <linux/swap.h>

#include <linux/timex.h>

#include <linux/jiffies.h>

/* #define DEBUG */

/**

 * oom_badness - calculate a numeric value for how bad this task has been

 * @p: task struct of which task we should calculate

 * @p: current uptime in seconds

 *

 * The formula used is relatively simple and documented inline in the

 * function. The main rationale is that we want to select a good task

 * to kill when we run out of memory.

 *

 * Good in this context means that:

 * 1) we lose the minimum amount of work done

 * 2) we recover a large amount of memory

 * 3) we don't kill anything innocent of eating tons of memory

 * 4) we want to kill the minimum amount of processes (one)

 * 5) we try to kill the process the user expects us to kill, this

 *    algorithm has been meticulously tuned to meet the principle

 *    of least surprise ... (be careful when you change it)

 */

unsigned long badness(struct task_struct *p, unsigned long uptime)

{

        unsigned long points, cpu_time, run_time, s;

        struct list_head *tsk;

        if (!p->mm)

                return 0;

        /*

         * The memory size of the process is the basis for the badness.

         */

        points = p->mm->total_vm;

        /*

         * Processes which fork a lot of child processes are likely

         * a good choice. We add the vmsize of the childs if they

         * have an own mm. This prevents forking servers to flood the

         * machine with an endless amount of childs

         */

        list_for_each(tsk, &p->children) {

                struct task_struct *chld;

                chld = list_entry(tsk, struct task_struct, sibling);

                if (chld->mm != p->mm && chld->mm)

                        points += chld->mm->total_vm;

        }

        /*

         * CPU time is in tens of seconds and run time is in thousands

         * of seconds. There is no particular reason for this other than

         * that it turned out to work very well in practice.

         */

        cpu_time = (cputime_to_jiffies(p->utime) + cputime_to_jiffies(p->stime))

                >> (SHIFT_HZ + 3);

        if (uptime >= p->start_time.tv_sec)

                run_time = (uptime - p->start_time.tv_sec) >> 10;

        else

                run_time = 0;

        s = int_sqrt(cpu_time);

        if (s)

                points /= s;

        s = int_sqrt(int_sqrt(run_time));

        if (s)

                points /= s;

        /*

         * Niced processes are most likely less important, so double

         * their badness points.

         */

        if (task_nice(p) > 0)

                points *= 2;

        /*

         * Superuser processes are usually more important, so we make it

         * less likely that we kill those.

         */

        if (cap_t(p->cap_effective) & CAP_TO_MASK(CAP_SYS_ADMIN) ||

                                p->uid == 0 || p->euid == 0)

                points /= 4;

        /*

         * We don't want to kill a process with direct hardware access.

         * Not only could that mess up the hardware, but usually users

         * tend to only have this flag set on applications they think

         * of as important.

         */

        if (cap_t(p->cap_effective) & CAP_TO_MASK(CAP_SYS_RAWIO))

                points /= 4;

        /*

         * Adjust the score by oomkilladj.

         */

        if (p->oomkilladj) {

                if (p->oomkilladj > 0)

                        points <<= p->oomkilladj;

                else

                        points >>= -(p->oomkilladj);

        }

#ifdef DEBUG

        printk(KERN_DEBUG "OOMkill: task %d (%s) got %d points\n",

        p->pid, p->comm, points);

#endif

        return points;

}

/*

 * Simple selection loop. We chose the process with the highest

 * number of 'points'. We expect the caller will lock the tasklist.

 *

 * (not docbooked, we don't want this one cluttering up the manual)

 */

static struct task_struct * select_bad_process(void)

{

        unsigned long maxpoints = 0;

        struct task_struct *g, *p;

        struct task_struct *chosen = NULL;

        struct timespec uptime;

        do_posix_clock_monotonic_gettime(&uptime);

        do_each_thread(g, p)

                /* skip the init task with pid == 1 */

                if (p->pid > 1 && p->oomkilladj != OOM_DISABLE) {

                        unsigned long points;

                        /*

                         * This is in the process of releasing memory so wait it

                         * to finish before killing some other task by mistake.

                         */

                        if ((unlikely(test_tsk_thread_flag(p, TIF_MEMDIE)) || (p->flags & PF_EXITING)) &&

                            !(p->flags & PF_DEAD))

                                return ERR_PTR(-1UL);

                        if (p->flags & PF_SWAPOFF)

                                return p;

                        points = badness(p, uptime.tv_sec);

                        if (points > maxpoints || !chosen) {

                                chosen = p;

                                maxpoints = points;

                        }

                }

        while_each_thread(g, p);

        return chosen;

}

/**

 * We must be careful though to never send SIGKILL a process with

 * CAP_SYS_RAW_IO set, send SIGTERM instead (but it's unlikely that

 * we select a process with CAP_SYS_RAW_IO set).

 */

static void __oom_kill_task(task_t *p)

{

 if (1) {

 return;

}

        if (p->pid == 1) {

                WARN_ON(1);

                printk(KERN_WARNING "tried to kill init!\n");

                return;

        }

        task_lock(p);

        if (!p->mm || p->mm == &init_mm) {

                WARN_ON(1);

                printk(KERN_WARNING "tried to kill an mm-less task!\n");

                task_unlock(p);

                return;

        }

        task_unlock(p);

        printk(KERN_ERR "Out of Memory: Killed process %d (%s).\n", p->pid, p->comm);

        /*

         * We give our sacrificial lamb high priority and access to

         * all the memory it needs. That way it should be able to

         * exit() and clear out its resources quickly...

         */

        p->time_slice = HZ;

        set_tsk_thread_flag(p, TIF_MEMDIE);

        force_sig(SIGKILL, p);

}

static struct mm_struct *oom_kill_task(task_t *p)

{

        struct mm_struct *mm = get_task_mm(p);

        task_t * g, * q;

        if (!mm)

                return NULL;

        if (mm == &init_mm) {

                mmput(mm);

                return NULL;

        }

        __oom_kill_task(p);

        /*

         * kill all processes that share the ->mm (i.e. all threads),

         * but are in a different thread group

         */

        do_each_thread(g, q)

                if (q->mm == mm && q->tgid != p->tgid)

                        __oom_kill_task(q);

        while_each_thread(g, q);

        return mm;

}

static struct mm_struct *oom_kill_process(struct task_struct *p)

{

        struct mm_struct *mm;

        struct task_struct *c;

        struct list_head *tsk;

        /* Try to kill a child first */

        list_for_each(tsk, &p->children) {

                c = list_entry(tsk, struct task_struct, sibling);

                if (c->mm == p->mm)

                        continue;

                mm = oom_kill_task(c);

                if (mm)

                        return mm;

        }

        return oom_kill_task(p);

}

/**

 * oom_kill - kill the "best" process when we run out of memory

 *

 * If we run out of memory, we have the choice between either

 * killing a random task (bad), letting the system crash (worse)

 * OR try to be smart about which process to kill. Note that we

 * don't have to be perfect here, we just have to be good.

 */

void out_of_memory(unsigned int __nocast gfp_mask, int order)

{

        struct mm_struct *mm = NULL;

        task_t * p;

        if (printk_ratelimit()) {

                printk("oom-killer: gfp_mask=0x%x, order=%d\n",

                        gfp_mask, order);

                show_mem();

        }

        read_lock(&tasklist_lock);

retry:

        p = select_bad_process();

        if (PTR_ERR(p) == -1UL)

                goto out;

        /* Found nothing?!?! Either we hang forever, or we panic. */

        if (!p) {

                read_unlock(&tasklist_lock);

                panic("Out of memory and no killable processes...\n");

        }

        mm = oom_kill_process(p);

        if (!mm)

                goto retry;

 out:

        read_unlock(&tasklist_lock);

        if (mm)

                mmput(mm);

        /*

         * Give "p" a good chance of killing itself before we

         * retry to allocate memory.

         */

        __set_current_state(TASK_INTERRUPTIBLE);

        schedule_timeout(1);

}

----------------

----------

