# Yet another 4GB memory problem

## boris_qd

I just added 4Gb of memory to my system (4x1GB).  I only seem to see 3.6GB and am wondering what happend to the rest.

```

ladybug abak # free

                                   total         used            free       shared       buffers     cached

Mem:                        3630924     379568    3251356          0        18260     183288

-/+ buffers/cache:                        178020    3452904

Swap:                       2449904          0        2449904

```

```

ladybug abak # cat /proc/mtrr

reg00: base=0x00000000 (   0MB), size=4096MB: write-back, count=1

reg01: base=0xe0000000 (3584MB), size= 512MB: uncachable, count=1

reg02: base=0x100000000 (4096MB), size= 512MB: write-back, count=1

reg03: base=0xdff00000 (3583MB), size=   1MB: write-through, count=1

```

```

ladybug abak # cat /usr/src/linux/.config | grep MEM

CONFIG_SHMEM=y

# CONFIG_TINY_SHMEM is not set

# CONFIG_NOHIGHMEM is not set

CONFIG_HIGHMEM4G=y

# CONFIG_HIGHMEM64G is not set

CONFIG_HIGHMEM=y

CONFIG_ARCH_FLATMEM_ENABLE=y

CONFIG_ARCH_SPARSEMEM_ENABLE=y

CONFIG_ARCH_SELECT_MEMORY_MODEL=y

CONFIG_SELECT_MEMORY_MODEL=y

CONFIG_FLATMEM_MANUAL=y

# CONFIG_DISCONTIGMEM_MANUAL is not set

# CONFIG_SPARSEMEM_MANUAL is not set

CONFIG_FLATMEM=y

CONFIG_FLAT_NODE_MEM_MAP=y

CONFIG_SPARSEMEM_STATIC=y

# CONFIG_SPARSEMEM_VMEMMAP_ENABLE is not set

CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y

CONFIG_BLK_DEV_UMEM=m

CONFIG_INPUT_FF_MEMLESS=m

CONFIG_FIX_EARLYCON_MEM=y

CONFIG_INFINIBAND_USER_MEM=y

# CONFIG_DEBUG_HIGHMEM is not set

CONFIG_HAS_IOMEM=y

```

```

ladybug abak # uname -a

Linux ladybug 2.6.24-gentoo-r3 #2 SMP Fri Mar 14 21:16:21 CET 2008 i686 Intel(R) Core(TM)2 Duo CPU E6750 @ 2.66GHz GenuineIntel GNU/Linux

```

```

ladybug abak # dmesg | grep -i mem

system 00:0c: iomem range 0xd2000-0xd3fff has been reserved

system 00:0c: iomem range 0xf0000-0xf7fff could not be reserved

system 00:0c: iomem range 0xf8000-0xfbfff could not be reserved

system 00:0c: iomem range 0xfc000-0xfffff could not be reserved

system 00:0c: iomem range 0xdfee0000-0xdfefffff could not be reserved

system 00:0c: iomem range 0x0-0x9ffff could not be reserved

system 00:0c: iomem range 0x100000-0xdfedffff could not be reserved

system 00:0c: iomem range 0xfec00000-0xfec00fff could not be reserved

system 00:0c: iomem range 0xfed10000-0xfed1dfff could not be reserved

system 00:0c: iomem range 0xfed20000-0xfed8ffff could not be reserved

system 00:0c: iomem range 0xfee00000-0xfee00fff could not be reserved

system 00:0c: iomem range 0xffb00000-0xffb7ffff could not be reserved

  MEM window: e8000000-efffffff

  MEM window: disabled.

  MEM window: f4000000-f4ffffff

  MEM window: f5000000-f6ffffff

  MEM window: disabled.

Freeing initrd memory: 2424k freed

highmem bounce pool size: 64 pages

Freeing unused kernel memory: 224k freed

ehci_hcd 0000:00:1a.7: irq 18, io mem 0xf7004000

ehci_hcd 0000:00:1d.7: irq 19, io mem 0xf7005000

```

My motherboard is Gigabyte GA-P35-DS3L P35+ICH9 (rev. 2)

----------

## guen

# CONFIG_HIGHMEM64G is not set

set "High Memory support" to 64GB

under "Processor type and features"

in your kernel config.

That should solve your problem.

----------

## NeddySeagoon

boris_qd,

You appear to be running a 32 bit kernel.

You have a 32 bit address space, which can address as most 4Gb. Some of this space is reserved for memory mapped devices and memory provided by devices e.g. your Video Card. Thus you cannot locate real RAM here, so your motherboard maps it out.

You need to use a 64 bit install to see all your RAM and then it will depends on your motherboard

Some map the hidden memory to addresses above 4G, others just throw it away.

A 32 bit to 64 bit change is a re-install.

----------

## boris_qd

So it sounds like i either have to upgrade or leave it like it is.

Question about leaving it:  Am i losing the benefit of having matched sticks of memory in each channel if i just leave it like it is?  

thanks.

----------

## gerard27

Hi boris_qd,

I disagree with Neddy and I agree with guen.

I am running 32 bit on a 64 bit system because lots of programs I need do not

run well on 64 bit.

Your Bios should be set for 4G!If it is not then you'll never have 4G.

I also set config HIGHMEM 64 G.

Top shows me 4G.

Gerard.

----------

## NeddySeagoon

boris_qd,

Selecting 64G Himem forces the use of PAE, which is a paging trick to be able to address more memory than that address space permits. Like expanded (not extended) RAM in early IBM PCs.

It works but its slow.

You still get the benefit of matched RAM, all the way up to the 3.6G that you can see.

Incidently, when you install more than 1Gb RAM for a x86 kernel, you start to suffer from some paging slow downs, the slowdown gets much worse at 4Gb.

If you make a x86_64 kernel, with 32 bit support, so your 32 bit userland still works, you don't have the address space gymnastics, nor the slowdowns associated with the various paging mechanisms.

It still depends on how your motherboard manages the address space clash, how much RAM you see. 

You have a working 42 bit address bus, so you don't need the HIMEM kernel options. Indeed, they are not there in the x86_64  kernel config.

----------

## eccerr0r

I've found it to be highly dependent on motherboard and chipset to determine whether or not you get all 4GB - depends on how things get mapped.  The /proc/mtrr I see looks fairly messed up.  The 4G machine I have (32-bit) looks like (64G PAE enabled, Intel G965 chipset):

```
reg00: base=0x100000000 (4096MB), size= 512MB: write-back, count=1

reg01: base=0x120000000 (4608MB), size= 256MB: write-back, count=1

reg02: base=0x00000000 (   0MB), size=2048MB: write-back, count=1

reg03: base=0x80000000 (2048MB), size=1024MB: write-back, count=1

reg04: base=0xc0000000 (3072MB), size= 256MB: write-back, count=1

reg05: base=0xcf800000 (3320MB), size=   8MB: uncachable, count=1

reg06: base=0xcf700000 (3319MB), size=   1MB: uncachable, count=1

reg07: base=0xd0000000 (3328MB), size= 256MB: write-combining, count=1

```

and its 'free' looks like

```
ouka:~$ uname -a

Linux ouka 2.6.21-test1 #1 SMP PREEMPT Thu Nov 22 12:28:57 MST 2007 i686 Intel(R) Core(TM)2 CPU 6700 @ 2.66GHz GenuineIntel GNU/Linux

ouka:~$ free

             total       used       free     shared    buffers     cached

Mem:       4141684    4034956     106728          0     248560    3470420

-/+ buffers/cache:     315976    3825708

Swap:      1959920        136    1959784

ouka:~$

```

A BIOS upgrade may help.  The fact that you get over 3.3GB probably means you have all the PAE settings enabled, so that may be your only option...

Solutions for other questions:

* Your dual channel will work fine if it's showing up enabled in BIOS even though you see some strange odd amount of RAM in Linux.

* I don't see way too much slowdown but there is a considerable amount when using 64G PAE.  It will cause extra memory juggling to see the full amount of RAM, but it's fairly low and you'll see it most when doing I/O that require the kernel.  Depending on what you're doing, you may not notice any slowdown.

----------

## boris_qd

Just to summarize.  It sounds like i can use all of my ram and suffer from a small slowdown if I use a 64 bit kernel with a 32 bit userspace.  (With the caveat of my bios may or may not support all the ram even in this situation)

If i move to a complete 64 bit system then there is not slowdown and i get to use all my ram (bios caveat again?).

I have no problem per se going to a complete 64 bit system but the last time I checked it's still more work then running a 32 bit system (more problems compiling, some (common) software not supported, more work setting up compatibility environments etc).  What is the latest state of things?

----------

## gerard27

Well as far as I know there still are problems although more software is being ported to 64.

I use cinepaint for instance.Only available in 32.

I have some spare room on my HD's going to try 64 soon.

But for the time being I am quite happy with 32.

Gerard.

----------

## Kwark

I have the same questions and boris_qd, with one addition:

How much of a slowdown does using the 64G PAE kernel option cause? I am thinking of upgrading RAM soon, but am unsure what the best way is to address this extra memory.

----------

## juniper

i am also having a 4G mem problem, but different.

My OS sees the right amount of memory, but my BIOS says that "3 gigs reserved for OS" and "1 gig reserved for devices".  what the hell is that about?

----------

## NeddySeagoon

juniper,

Your system will haver a 32 bit PCI bus and 32 bit PCI devices on the motherboard.

On a 32 bit system, this gives you a memory 'hole' at the top of RAM, where the PCI devices are mapped into memory addresses that would otherwise be used for RAM in your RAM chips.

e.g. Your Video Card will have RAM in the address space there, as will any PCI device that provides a memory mapped buffer.

In a 32 bit system, its at the top of the 4G address space that 32 bits can address (2^32=4G)

On 64 bit systems, the PCI bus is still 32 bits and still sits just under the 4G boundary (it can't go anywhere else).

Unfortunately, this creates a hole in the real RAM address space as you can now have RAM above and below it.

There are several solutions, one involves hiding the RAM that clashes with the PCI space, effectively throwing it away.

Another remaps the RAM to other physical addresses and the MMU just deals with it

Decoding the 3G to 4G memory range is easy - its only 2 bits to be tested so some motherboard vendors hide/remap physical RAM between 3G and 4G.

----------

## juniper

 *NeddySeagoon wrote:*   

> juniper,
> 
> Your system will haver a 32 bit PCI bus and 32 bit PCI devices on the motherboard.
> 
> On a 32 bit system, this gives you a memory 'hole' at the top of RAM, where the PCI devices are mapped into memory addresses that would otherwise be used for RAM in your RAM chips.
> ...

 

Thanks for the reply.  But I have to admit, I haven't the foggiest idea what you are saying about solving this problem ("it's only 2 bits to be tested...").  Also, I have a 64 bit system (processor at least).

----------

## NeddySeagoon

juniper,

The PC wasn't designed the way it is today, things have just been grafted on.

A 64 bit system is an expasnion of a 32 bit system.

The PCI memory mapped area at the top of the 4G maximum address space of a 32 bit bus stays in the same place when you make the address bus bigger. You cannot have two different memory cells at the same location at the same time, so you end up with a hole in the memory map.

----------

## juniper

ok.  I think i get it, but how do I fix it?

ain't like the good ole days, when life was simple.

----------

## eccerr0r

If your OS works fine, likely all you can look for is BIOS and BIOS settings...  Also make sure your chipset supports memory remapping.

gone are the days of simplicity, welcome to the world of new unnecessary complexities...?

----------

## EzInKy

 *Gerard van Vuuren wrote:*   

> Well as far as I know there still are problems although more software is being ported to 64.
> 
> I use cinepaint for instance.Only available in 32.
> 
> I have some spare room on my HD's going to try 64 soon.
> ...

 

This is a never ending source of amazement for me. It's hard to believe that even after two years of running 64bit systems that I still have to keep "~amd64" in my make.conf. Is it a lack of testers that is the problem? If so where do I apply because despite the "unstable" designation my systems run pretty darn good.

----------

## HeXiLeD

I have the same problem but a bit different. it  is kinda like juniper's issue

2 x 2gb and the bios only reports 3.100 GB

Gentoo reports 3.07 GB.

Its a 64bit install. 

Asus board and i read that it is a bios 4gb issue. It is even writen on the board manual.

However  i still have one question. 

While it is reporting only 3100 mb; will it still use the full 4gb ?

My hardware and software specs are on my signature:

My system hardware and software SPECIFICATIONS

----------

## EzInKy

 *HeXiLeD wrote:*   

> I have the same problem but a bit different. it  is kinda like juniper's issue
> 
> 2 x 2gb and the bios only reports 3.100 GB
> 
> Gentoo reports 3.07 GB.
> ...

 

Look for an option in your bios that changes the memory mapping, or memory hole. For my Tyan board with 8gb of memory installed  I chose the "discrete" option, which gives me this:

 *Quote:*   

> 
> 
> #free
> 
>              total       used       free     shared    buffers     cached
> ...

 

----------

