# SPARSEMEM vs. FLATMEM (Solved)

## causality

In my kernel config under "Processor Type and Features" I now have a choice to make for "Memory Model".

For the longest time, "Flat Memory" was the only available option.  Recently, I can also choose "Sparse Memory".  These are the only two options available to me.  Flat memory is the default and it's what I have been using for a long time now, but I am curious about sparse memory and whether I should try it.

I am having a hard time finding a real, solid definition of what sparse memory is.  Near as I can tell, it's like sparse files in that it may be non-contiguous and may have holes in its allocation.  But that leaves some things unanswered.  It doesn't tell me if SPARSEMEM is a drop-in replacement for FLATMEM that won't break anything.  It doesn't tell me what performance differences there may be, if any.  If it helps, this is a standard desktop PC with an AMD64 X2 processor; I have nothing exotic in terms of hardware.

Any attempt to Google this topic results in tons of posts from various Linux Kernel mailing lists where people are talking about how to implement it in C without discussing its merits.  Any other search results just regurgitate the "help" text from the kernel config, something a lot of sites like to repost endlessly without adding new information to it.  That makes it difficult for those who are seaching in the first place because they need more information than provided by the help text they already have.    :Mad:  At any rate, it's a surprisingly difficult and not-straightforward topic to try to research.

The kernel config help text does suggest selecting FLATMEM if you are in doubt, but it doesn't explain why.  

Anyway, I was hoping someone here who is knowledgable about these things might be willing to help me understand the difference.Last edited by causality on Thu Apr 07, 2011 6:32 pm; edited 1 time in total

----------

## aCOSwt

The only measure I know about this topic was at the time sparsemem was challenging discontig :

```
Measurements in cycle counts. 1000 memory allocations are performed and then the

average cycle count are calculated.

Order   FlatMem   Discontig   SparseMem

0     639     665        641

1     567     647        593

2     679     774        692

3     763     967        781

4     961    1501        962

5    1356    2344       1392

6    2224    3982       2336

7    4869    7225       5074

8   12500   14048      12732

9   27926   28223      28165

10   58578   58714      58682 
```

This was showing some significant interest to sparsemem over discontig when flatmem producing a non significant advantage over sparsemem.

For what it's worth... dating from about 5 years ago...

----------

## NeddySeagoon

causality,

Sparse memory is only of use on NUMA systems or systems supporting hot pluggable memory.

A NUMA system supports several CPUs (not CPU cores) each with their own local memory and a way for each CPU to addess the global memory pool. Accessing the non-local memory is usually slower.

When memory is hot plugged, the kernel determines where in the physical address space it goes.

Both features make a big hole in your wallet - you would know if you had either of them.

----------

## causality

You just provided what a great deal of searching did not.  Thank you for giving me a truly helpful answer.   :Very Happy: 

I will stick with FLATMEM since it sounds like SPARSEMEM wouldn't work on my regular PC.

----------

## aCOSwt

 *NeddySeagoon wrote:*   

>  Sparse memory is only of use on NUMA systems or systems supporting hot pluggable memory.

 

 :Shocked: 

Not willing to argue in any manner, just wishing to understand something I am getting confused with since your post.

Are we speaking about CONFIG_SPARSEMEM_MANUAL ?

In which case this is what I get for my Gentoo-Sources 2.6.34-r12 running on a standard (Non NUMA) x86_64 SMP arch :

```

Symbol: SPARSEMEM_MANUAL [=y]

Depends on: <choice> && ARCH_SPARSEMEM_ENABLE [=y]

Symbol: ARCH_SPARSEMEM_ENABLE [=y]

Selects: SPARSEMEM_STATIC [=n] && SPARSEMEM_VMEMMAP_ENABLE [=y]

Symbol: SPARSEMEM_VMEMMAP_ENABLE [=y] 

Selected by: ARCH_SPARSEMEM_ENABLE [=y] && (X86_64 [=y] || NUMA [=n] || EXPERIMENTAL [=y] && X86_32 [=n] || X86_32_NON_STANDARD [=n]) && X86_64[=y]

```

Moreover... it is the only available option because :

```

Symbol: FLATMEM_MANUAL [=n] 

Depends on: <choice> && (!ARCH_DISCONTIGMEM_ENABLE [=n] && !ARCH_SPARSEMEM_ENABLE [=y] || ARCH_FLATMEM_ENABLE [=n]) 

```

CONFIG_SPARSEMEM_MANUAL appears not linked with NUMA or other hot-pluggable memory option, it even appears to be the only option available as soon as ARCH_SPARSEMEM_ENABLE is set. (This behind my back...)

----------

## NeddySeagoon

aCOSwt,

My 2.6.38 is aet up as 

```
CONFIG_ARCH_SPARSEMEM_DEFAULT=y

CONFIG_ARCH_SPARSEMEM_ENABLE=y

CONFIG_SPARSEMEM_MANUAL=y

CONFIG_SPARSEMEM=y

CONFIG_SPARSEMEM_EXTREME=y

CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y

CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER=y

CONFIG_SPARSEMEM_VMEMMAP=y
```

Not all those options are user visible.  Memuconfig shows I have sparse memory select as the only choice too.

Having Spare Memory set (or forced on you) does not mean your RAM cannot be contiguious, only that it need not be.

Some time ago, I had both options available.

----------

## 1clue

 *NeddySeagoon wrote:*   

> causality,
> 
> Sparse memory is only of use on NUMA systems or systems supporting hot pluggable memory.
> 
> A NUMA system supports several CPUs (not CPU cores) each with their own local memory and a way for each CPU to addess the global memory pool. Accessing the non-local memory is usually slower.
> ...

 

Seriously.

This needs to go into the kernel documentation.

----------

## NeddySeagoon

1clue,

I think that's where I found it - but it was a few years ago now.

Edit: See /usr/src/linux/Documentation/memory-hotplug.txt

----------

## 1clue

I KNOW I have clicked the help on that feature nearly every time through, but never got the message as clearly as when you said it.

Maybe they just need to add the part about the hole in your wallet.

----------

