# [Memory] Cannot allocate memory with lot of available memory

## pums974

Helo,

The title summarize it, I got frequently error messages like "cannot allocate memory" during compilation of large program (like chromium or libreoffice for example). 

It has been a while and i thought it was due to hardware problem but, I recently change everything (CPU RAM and mother board) and i still have the same problem.

Moreover I remark that :

- it rarely use more than a half of my 8G of RAM (without cache)

- it doesn't use SWAP (200M of 5G)

One other thing which can be unrelated, I can't deactivate my swap, even if i got lot of free place in RAM.

Now for example, a free -h gives me : 

```

                  total       used       free     shared    buffers     cached 

Mem:          7,8G       7,2G       599M         0B       1,0G       3,4G 

-/+ buffers/cache:       2,8G       5,0G 

Swap:         5,0G       174M       4,8G 

```

And a swapoff -av :

```

swapoff /dev/mapper/vg-Swap

swapoff: /dev/mapper/vg-Swap : échec de swapoff: Ne peut allouer de la mémoire

```

Am I the only one with those kind of problem ?

Does anyone has a clue of the origin ?

----------

## NeddySeagoon

pums974,

Please post your 

```
emerge --info
```

As a WAG, you might have a 32 bit install with PXE enabled but you still only get a 4G address space per process, which will be split 1G/3G for kernel/user.

In fact, depending on your hardware, your per process useful address space may be as little as 3.2G, as memory mapped I/O cannot be mapped out.

----------

## pums974

Thanks for your answer.

But i do have a 64bit install, I'm sorry I should have precised it sooner.

Here is my emerge --info

```

Portage 2.1.11.23 (default/linux/amd64/10.0/desktop, gcc-4.6.3, glibc-2.15-r3, 3.5.4-gentooPerso x86_64)

=================================================================

System uname: Linux-3.5.4-gentooPerso-x86_64-Intel-R-_Core-TM-_i7-2600_CPU_@_3.40GHz-with-gentoo-2.2

Timestamp of tree: Sun, 30 Sep 2012 11:45:01 +0000

ccache version 3.1.8 [disabled]

app-shells/bash:          4.2_p37

dev-java/java-config:     2.1.12

dev-lang/python:          2.7.3-r2, 3.2.3-r1

dev-util/ccache:          3.1.8

dev-util/cmake:           2.8.9

dev-util/pkgconfig:       0.27.1

sys-apps/baselayout:      2.2

sys-apps/openrc:          0.10.5

sys-apps/sandbox:         2.6

sys-devel/autoconf:       2.13, 2.69

sys-devel/automake:       1.9.6-r3, 1.10.3, 1.11.6, 1.12.4

sys-devel/binutils:       2.22.90

sys-devel/gcc:            4.6.3

sys-devel/gcc-config:     1.7.3

sys-devel/libtool:        2.4.2

sys-devel/make:           3.82-r4::gnome

sys-kernel/linux-headers: 3.5 (virtual/os-headers)

sys-libs/glibc:           2.15-r3

Repositories: gentoo desktop-effects eva gamerlay-stable games kde laurentb multimedia portato retroshare-overlay science systemd ubuntu voyageur x11 vmware virtualization qt mozilla kvm kde-sunset gentoo-guis gnome Local

ACCEPT_KEYWORDS="amd64 ~amd64"

ACCEPT_LICENSE="*"

CBUILD="x86_64-pc-linux-gnu"

CFLAGS="-march=native -O2 -pipe "

CHOST="x86_64-pc-linux-gnu"

CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa"

CONFIG_PROTECT_MASK="${EPREFIX}/etc/gconf /etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"

CPPFLAGS="-march=native -O2 -pipe "

CXXFLAGS="-march=native -O2 -pipe "

DISTDIR="/usr/portage/tree/../distfiles"

EMERGE_DEFAULT_OPTS="--complete-graph y --with-bdeps y --autounmask y --misspell-suggestions y -D"

FCFLAGS="-O2 -pipe"

FEATURES="assume-digests binpkg-logs candy config-protect-if-modified distlocks ebuild-locks fixlafiles metadata-transfer news parallel-fetch parallel-install protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch usersandbox xattr"

FFLAGS="-O2 -pipe"

GENTOO_MIRRORS="ftp://ftp.free.fr/mirrors/ftp.gentoo.org/ http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"

LANG="fr_FR.UTF-8"

LDFLAGS="-Wl,-O1 -Wl,--as-needed "

LINGUAS="fr en"

PKGDIR="/usr/portage/tree/../packages"

PORTAGE_COMPRESS="bzip2"

PORTAGE_COMPRESS_FLAGS="-9"

PORTAGE_CONFIGROOT="/"

PORTAGE_RSYNC_EXTRA_OPTS=""

PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"

PORTAGE_TMPDIR="/var/tmp"

PORTDIR="/usr/portage/tree"

PORTDIR_OVERLAY="/var/lib/layman/desktop-effects /var/lib/layman/eva /var/lib/layman/gamerlay /var/lib/layman/games /var/lib/layman/kde /var/lib/layman/laurentb /var/lib/layman/multimedia /var/lib/layman/portato /var/lib/layman/retroshare-overlay /var/lib/layman/science /var/lib/layman/systemd /var/lib/layman/ubuntu /var/lib/layman/voyageur /var/lib/layman/x11 /var/lib/layman/vmware /var/lib/layman/virtualization /var/lib/layman/qt /var/lib/layman/mozilla /var/lib/layman/kvm /var/lib/layman/kde-sunset /var/lib/layman/gentoo-guis /var/lib/layman/gnome /usr/portage/local"

SYNC="rsync://rsync.fr.gentoo.org/gentoo-portage"

USE="32bit 64bit X a52 aac acl acpi alsa amd64 apache2 apm avahi bash-completion berkdb branding btrfs bzip2 cairo cli colord consolekit cxx dbus directfb dri emboss encode exif fam fat fbcon fbcondecor ffmpeg fftw firefox flac fontconfig fortran g3dvl gdbm gif git glew gnome gnome-keyring gpm gsl gstreamer gtk gtk3 iconv icu introspection ipv6 java joystick jpeg kde lcms libnatspec libnotify lm_sensors lto lzma mad minimal mmx mmxext mng modules mp3 mp4 mpeg mpi mpi-threads mudflap multilib mysql nautilus ncurses networkmanager nls nptl nsplugin nspluginwrapper ntfs nvidia offensive ogg opencl opengl openmp openvg pam pango pcre pdf pdo phonon php plasma pmu png policykit ppds pppd pulseaudio python python3 qt3support qt4 readline sdl semantic-desktop session smp spell sql sqlite sqlite3 sse sse2 sse3 sse4 sse4_1 sse4_2 ssl ssse3 startup-notification svg system-sqlite systemd tcpd theora threads tiff truetype udev udisks unicode upnp upower usb vdpau vorbis wayland webgl webkit wxwidgets x264 xattr xcb xinerama xml xulrunner xv xvid xvmc zeitgeist zeroconf zlib zsh-completion" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="flow karbon krita sheets stage words" ELIBC="glibc" GRUB_PLATFORMS="pc efi-64" INPUT_DEVICES="evdev mouse keyboard" KERNEL="linux" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="fr en" PHP_TARGETS="php5-3 php5-4" PYTHON_TARGETS="python3_2 python2_7" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby19" SANE_BACKENDS="lexmark" USERLAND="GNU" VIDEO_CARDS="nvidia fglrx"

Unset:  CTARGET, INSTALL_MASK, LC_ALL, MAKEOPTS, PORTAGE_BUNZIP2_COMMAND, USE_PYTHON

```

----------

## NeddySeagoon

pums974,

Please post your df output.  Your emerge --info looks good.

----------

## pums974

here is my df :

```
Sys. fich.                                            1K-blocks     Util. Disponible Uti% Monté sur

rootfs                                                103212320  31267056   66702384  32% /

udev                                                      10240         8      10232   1% /dev

/dev/mapper/vg-Root                                   103212320  31267056   66702384  32% /

tmpfs                                                   4102952       872    4102080   1% /run

cgroup_root                                               10240         4      10236   1% /sys/fs/cgroup

shm                                                     4102952      2492    4100460   1% /dev/shm

/dev/mapper/vg-Boot                                     1032088    103376     876284  11% /boot

/dev/sdc1                                                201633         1     201633   1% /boot/efi

/dev/mapper/vg-Portage                                 20642428   6328972   13264880  33% /usr/portage

/dev/mapper/vg-Home                                   103212320  14460964   83508476  15% /home

/dev/mapper/vg-Temp                                    41284928  32987244    6200532  85% /temp

/dev/mapper/vg-Data                                   686486944 436544172  215071272  67% /Data

curlftpfs#ftp://freebox:freebox@mafreebox.freebox.fr/ 999999999         0  999999999   0% /media/freebox

/dev/sr0                                                7923158   7923158          0 100% /media/cdrom

```

Last edited by pums974 on Mon Oct 01, 2012 6:26 am; edited 1 time in total

----------

## NeddySeagoon

pums974,

I was wondering about /dev/shm being full as its allocated half your RAM and allowed to use swap space too.

Its not that.

Does top show anything hogging RAM.  That might point to an application with a 'memory leak'.  That would be a bug.

----------

## pums974

Not particularly no.

In general, with gnome shell and chromium, around 3G are used wich should leave enough for compilling libreoffice or chromium.

----------

## BitJam

This is a strange problem.  It is theoretically possible that your RAM becomes highly fragmented which would make it impossible to allocate large blocks of RAM even though most of the RAM is free.  The /proc/buddyinfo file will let you know if this is the problem.  Use "getconf PAGESIZE" to find the page size.  It's 4096 on my system.  My /proc/buddyinfo tells me I have 93 4-meg chunks available and this is the majority of my free RAM.

More realistically: is it possible your kernel is somehow mis-configured?   I can't even imagine exactly how a kernel mis-configuration could cause such a problem but after you swapped out hardware this seems to be one of the few common denominators that's left.

----------

## pums974

 *Quote:*   

> This is a strange problem.

 

I agree

I have the same PAGESIZE as you, but doesn't understand buddyinfo.

```
tux ~ # getconf PAGESIZE

4096

tux ~ # cat /proc/buddyinfo

Node 0, zone      DMA      3      2      0      1      3      2      0      0      1      1      3 

Node 0, zone    DMA32  60727  50075  26540   9815   2291    550    158     32      3      1      0 

Node 0, zone   Normal 109563  41269   2590     62      0      0      0      0      0      0      1 
```

 *Quote:*   

> is it possible your kernel is somehow mis-configured?

 

That's also what I have in mind, but I can't find what could cause this.

----------

## BitJam

 *pums974 wrote:*   

> I have the same PAGESIZE as you, but doesn't understand buddyinfo.
> 
> ```
> tux ~ # getconf PAGESIZE
> 
> ...

 

Imagine there is a "size" row like this:

```
SIZE -->                   1      2      4      8     16     32     64    128    256    512   1024

                      ----------------------------------------------------------------------------

Node 0, zone      DMA      3      2      0      1      3      2      0      0      1      1      3 

Node 0, zone    DMA32  60727  50075  26540   9815   2291    550    158     32      3      1      0 

Node 0, zone   Normal 109563  41269   2590     62      0      0      0      0      0      0      1 
```

Look at the columns on the far right.  You only have a total of 4 (3 + 0 + 1) blocks that have size 1024 * PAGESIZE and only 2 (1 + 1 + 0) with size 512 * PAGESIZE.  So it does appear that you may be running out of the larger blocks of memory.   Here are a couple of snapshots from my system:

```
Node 0, zone      DMA      1      1      1      0      2      1      1      0      1      1      3

Node 0, zone    DMA32   6896   6395   2148    693    397    100     67     32     34      3     88

Node 0, zone   Normal    167    201    136     88     39     20      7      7      3      2      2
```

and then later (after watching HD movie via Adobe Flash):

```
Node 0, zone      DMA      1      1      1      0      2      1      1      0      1      1      3 

Node 0, zone    DMA32  50302  34545  11103   2964   1141    549    266    112     85     45     12 

Node 0, zone   Normal    402    263    140     76     35     13      5      4      9      2      1 
```

It does seems that your "Normal" memory is almost completely fragmented with only one block available that is larger than 8 * PAGESIZE.   Processes that need blocks of normal memory  64K or larger will be simply out of luck and will probably give you the "cannot allocate memory" error that you are getting.

I don't know if the problem stems from your kernel or user-space programs.  I'm not an expert on how to track this down further.  One approach would be to kill off programs while keeping an eye on /proc/buddyinfo and see if the situation improves.  You should probably try to free your caches as you go along so blocks of memory can be reclaimed (I'm not absolutely certain this is required).

Running "watch cat /proc/buddyinfo" in one terminal and "watch free -m" in another may be useful.

----------

## pums974

At the moment I killed everything (gdm chromium...) and I access my computer via ssh.

I have freed my caches (echo 3 > /proc/sys/vm/drop_caches) but I wasn't able to put it back at 0 ("-bash: echo: erreur d'écriture : Argument invalide" wich mean "bash: echo: write error: Invalid argument ")

So my memory is completely free (230M in RAM and 26M in swap).

Then I tried to emerge libreoffice. As usual, it failed claiming that he doesnt have enough memory ("Could not reserve enough space for object heap").

The result of buddyinfo I gave you was just after this failure.

Now, few hours later (not doing anything on this machine), my buddyinfo looks just fine :

```
 Node 0, zone      DMA      3      2      0      1      3      2      0      0      1      1      3 

Node 0, zone    DMA32   1272   1298   1320   1274   1196    993    896    750    543    403    325 

Node 0, zone   Normal  10974  10544   9174   7984   6575   4913   3342   1778    650    126     16 
```

Si finally, seems to me that fragmented memory is not the cause of my problem, just a residue of the compilation, not the source of the problem.

----------

## NeddySeagoon

pums974,

Pastebin your kernel .config please.

You could also try one of Pappys seeds.  See kernel-seeds.org and the sticky in unsupported software.

If your system still does this with a seed, I'm sure Pappy will be pleased to hear from you.

----------

## pums974

No problemo : http://pastebin.com/sg8auaGG

I already took a look in seeds kernel, In fact I even started my config from one of him (3years ago).

I may have to restart from a blank kernel soon, but I would prefer to avoid that and understand what cause this problem.

----------

## NeddySeagoon

pums974,

If building from a kernel seed fixes the problem we can compare yur seed kernel with the one you just posted. Do keep a copy though as pastebins don't live very long.

Regard building a kernel from a seed as a part of the investigation, not a fix.

Did you ever at any time edit the kernel .config with a text editor?

Thats a no-no as many menuconfig options flip several CONF flags in the kernel. When you don't flip them all, you get an illegal .config that may build and give odd, hard to diagnose problems.

The only fix is to start with a new .config.

CONFIG_EXPERT=y should be off.  Its only needed on Embedded systems.

CONFIG_PROFILING=y should be off - thats for kernel devs.

CONFIG_SLUB_DEBUG=y should be off

CONFIG_DEBUG_FS=y should be off. Edit.  This is harmless

There may be others.  _DEBUG optios always cause logspam and may interfere with proper operation.

If you are not debugging the kernel, they should always be off.

Anything with Verbose in its name should be off too unless you really feel you can't live without it.

The exception is verbose boot messages, which only cause logspam at boot.

----------

## pums974

 *Quote:*   

> Did you ever at any time edit the kernel .config with a text editor? 

 

Nope. Never.

But in order to be really sure, i restarted my config from the last seed kernel. Reactivating all the options in order to retrieve my previous .config.

I also followed your advice by desactivating all debug and verbose options as well as expert and profiling.

That didn't solve my problem.

I will restart a new config from seed this week end.

----------

## Herring42

Silly question, but one that has caught me out before:

You are running the kernel you think you are running, right? I once got very frustrated that my new hardware wasn't working, before I realised that I was failing to mount /boot before running make install.

unale -a pointed out my error  :Wink: 

----------

## pums974

 *Quote:*   

> Silly question, but one that has caught me out before: You are running the kernel you think you are running, right?

 

There is no silly question, everyone make mistakes. By chance, I didn't made that one.

----------

## NeddySeagoon

pums974,

For a quick and dirty test, boot a liveCD, get into the chroot so you are running your apps on top of the liveCD kernel and support.

Does the problem still exist?

IF so, this narrows it down to the kernel, glibc and supporting services.

----------

## pums974

A friend of mine found a solution.

I had "vm.overcommit_memory = 2" in /etc/sysctl.conf.

I don't remember putting it in, and I don't really understand why this is here.

The fact is that since I commented that out, I am able to compile libreoffice without errors anymore.

Can you explain me what does this options mean ?

Is that normal that I had all those problem with it ?

Anyway, thank you for helping me.

----------

## BitJam

man 5 proc says:

```
/proc/sys/vm/overcommit_memory

       This file contains the kernel virtual memory accounting mode.  Values

       are:

              0: heuristic overcommit (this is the default)

              1: always overcommit, never check

              2: always check, never overcommit

       In mode 0, calls of mmap(2) with MAP_NORESERVE are not checked, and the

       default check is very weak, leading to the risk of getting a process

       "OOM-killed".  Under Linux 2.4 any nonzero value implies mode 1.  In

       mode 2 (available since Linux 2.6), the total virtual address space on

       the system is limited to (SS + RAM*(r/100)), where SS is the size of

       the swap space, and RAM is the size of the physical memory, and r is

       the contents of the file /proc/sys/vm/overcommit_ratio.

/proc/sys/vm/overcommit_ratio

       See the description of /proc/sys/vm/overcommit_memory.
```

----------

## kernelOfTruth

 *pums974 wrote:*   

> A friend of mine found a solution.
> 
> I had "vm.overcommit_memory = 2" in /etc/sysctl.conf.
> 
> I don't remember putting it in, and I don't really understand why this is here.
> ...

 

ok, so that's the cause for my similar error

thanks for posting the solution   :Smile: 

edit:

a really controversial and contradictive topic (at least when following advice from forum & other posts on the net)

here's some documentation after I finally understood it completely ^^

http://www.novell.com/support/kb/doc.php?id=7002775

----------

