# nvidia-drivers and MTRR [satisfactory answers]

## Lebkoungcity

Hello,

like the topic says: I'm confused by articles on wiki.gentoo.org, namely https://wiki.gentoo.org/wiki/NVidia/nvidia-drivers and https://wiki.gentoo.org/wiki/MTRR_and_PAT.

There is a nvidia-card in my machine and I use nvidia-drivers. Occasionally KDE/Plasma tells me that some part crashed (e.g. kwin) and was restarted or that hardware graphics acceleration was stopped and was replaced by software graphics acceleration. This is a behavior I see for some years now and from time to time I try to solve it - without success to this moment.

One thing I found some time ago was that the output of 

```
cat /proc/mtrr
```

 was not optimal, there were lots of lines saying 'uncachable'.

And this is when my confusion started:

I looked into the BIOS like recommended here: https://wiki.gentoo.org/wiki/NVidia/nvidia-drivers#Getting_2D_acceleration_to_work_on_machines_with_4GB_memory_or_more but found no entry I could set from 'continuous' to 'discrete' (or something similar). So I read https://wiki.gentoo.org/wiki/MTRR_and_PAT and changed the kernel config to this:

 *Quote:*   

> 
> 
> -*- MTRR (Memory Type Range Register) support                                             │ │   
> 
>   │ │                   [*]   MTRR cleanup support                                                                │ │   
> ...

 

Now I have this:

```
 # cat /proc/mtrr 

reg00: base=0x000000000 (    0MB), size= 2048MB, count=1: write-back

reg01: base=0x080000000 ( 2048MB), size= 1024MB, count=1: write-back

reg02: base=0x0bfe00000 ( 3070MB), size=    2MB, count=1: uncachable
```

So now, what to do next?

To complete my infos a bit more, some specs of this machine:

 *Quote:*   

> description: Motherboard
> 
>        product: GA-MA770T-UD3
> 
>        vendor: Gigabyte Technology Co., Ltd.
> ...

 

Thanks a lot for your attention!

Andy

----------

## NeddySeagoon

Lebkoungcity,

```
# cat /proc/mtrr

reg00: base=0x000000000 (    0MB), size= 2048MB, count=1: write-back

reg01: base=0x080000000 ( 2048MB), size= 1024MB, count=1: write-back

reg02: base=0x0bfe00000 ( 3070MB), size=    2MB, count=1: uncachable
```

That all looks good. write-back, is correct. That's most of your RAM.

If you had write-through, you get read caching only. Writes update both the cache and main RAM at the speed of main RAM.

That 2MB uncachable close to the top of the first 4G of address space is probably PCI hardware registers that can be changed by hardware.

Its a really bad idea to try to cache hardware registers. There is usually no way to keep the cache synchronised.

How much RAM do you have?

If all your RAM is covered by write-back cache, you are good. 

Its not always easy to tell, since physical RAM can be mapped around the PCI hardware address space.

If you have 16G RAM fitted, you have a problem. If you have 4G RAM, you might be able to do better. If you have 3G RAM, it looks good.

----------

## Lebkoungcity

 *NeddySeagoon wrote:*   

> (...) If you have 4G RAM, you might be able to do better.(...)

 

Hi NeddySeagoon,

it's always a pleasure to read your posts, thank you  :Smile: 

I have 4G RAM, it looks like this:

```
 # cat /proc/meminfo 

MemTotal:        4041668 kB

MemFree:          761240 kB

MemAvailable:    3575924 kB

Buffers:          621688 kB

Cached:          1788476 kB

SwapCached:           40 kB

Active:          2045752 kB

Inactive:         594852 kB

Active(anon):     117160 kB

Inactive(anon):   120876 kB

Active(file):    1928592 kB

Inactive(file):   473976 kB

Unevictable:        8448 kB

Mlocked:            8448 kB

SwapTotal:       5253248 kB

SwapFree:        5252728 kB

Dirty:                76 kB

Writeback:             0 kB

AnonPages:        238968 kB

Mapped:           149484 kB

Shmem:               672 kB

Slab:             515852 kB

SReclaimable:     468480 kB

SUnreclaim:        47372 kB

KernelStack:        2544 kB

PageTables:         6088 kB

NFS_Unstable:          0 kB

Bounce:                0 kB

WritebackTmp:          0 kB

CommitLimit:     7274080 kB

Committed_AS:     653104 kB

VmallocTotal:   34359738367 kB

VmallocUsed:           0 kB

VmallocChunk:          0 kB

Percpu:             1952 kB

HugePages_Total:       0

HugePages_Free:        0

HugePages_Rsvd:        0

HugePages_Surp:        0

Hugepagesize:       2048 kB

Hugetlb:               0 kB

DirectMap4k:      190400 kB

DirectMap2M:     4001792 kB

DirectMap1G:     2097152 kB
```

How could I do better? To me it's unclear wheat it means with this cached/uncached/write-back/... memory. I do understand the words but the meaning is uncertain to me...

Greetings,

Andy

----------

## NeddySeagoon

Lebkoungcity,

Can you pastebin dmesg please? 

We need it all, especially the BIOS provided memory map at the start.

I say "might be able to do better" because it depends on your hardware and your install.

----------

## Jaglover

Here is some discussion about it.

----------

## Lebkoungcity

 *NeddySeagoon wrote:*   

> 
> 
> Can you pastebin dmesg please? 

 

Sure, here it is:

https://pastebin.com/xKP5J1s3

----------

## Lebkoungcity

 *Jaglover wrote:*   

> Here is some discussion about it.

 

Thank you! Seems I'm not the only one who doesn't understand it fully   :Laughing: 

----------

## Jaglover

Direct MTRR use by drivers on Linux is now completely phased out ...

Go figure, it was written in 2015.

----------

## Ant P.

I have (nearly) the same motherboard, but 6GB RAM and amdgpu. No crash problems here.

```

GA-MA770-UD3            Gigabyte Technology Co., Ltd.   x.x     11/19/2009      Award Software International, Inc.      FGc
```

Neither the MTRR nor PAT ranges fully cover RAM, but I don't see any obvious ill effects:

```
reg00: base=0x000000000 (    0MB), size= 2048MB, count=1: write-back

reg01: base=0x080000000 ( 2048MB), size=  512MB, count=1: write-back

reg02: base=0x0a0000000 ( 2560MB), size=  256MB, count=1: write-back

reg03: base=0x0afe00000 ( 2814MB), size=    2MB, count=1: uncachable

PAT memtype list:

write-back @ 0xafde0000-0xafde1000

write-back @ 0xafde3000-0xafdeb000

write-back @ 0xafde3000-0xafde4000

write-back @ 0xafdea000-0xafdec000

write-combining @ 0xb0000000-0xc0000000

write-combining @ 0xb0000000-0xc0000000

uncached-minus @ 0xcfe00000-0xcfe01000

uncached-minus @ 0xe0000000-0xf0000000

uncached-minus @ 0xe00a0000-0xe00a1000

uncached-minus @ 0xfde80000-0xfdec0000

uncached-minus @ 0xfdefc000-0xfdf00000

uncached-minus @ 0xfdfe0000-0xfdfe1000

uncached-minus @ 0xfdfff000-0xfe000000

uncached-minus @ 0xfe028000-0xfe029000

uncached-minus @ 0xfe029000-0xfe02a000

uncached-minus @ 0xfe02a000-0xfe02b000

uncached-minus @ 0xfe02b000-0xfe02c000

uncached-minus @ 0xfe02c000-0xfe02d000

uncached-minus @ 0xfe02d000-0xfe02e000

uncached-minus @ 0xfe02e000-0xfe02f000

uncached-minus @ 0xfe02f000-0xfe030000

uncached-minus @ 0xfed00000-0xfed01000
```

```
        0x0      0x983ff  System RAM

    0x98400      0x9ffff  Reserved

    0xf0000      0xfffff  Reserved

   0x100000   0xafd80fff  System RAM

 0xafde0000   0xafde2fff  ACPI Non-volatile Storage

 0xafde3000   0xafdeffff  ACPI Tables

 0xafdf0000   0xafdfffff  Reserved

 0xe0000000   0xefffffff  Reserved

 0xfec00000   0xffffffff  Reserved

0x100000000  0x1cfffffff  System RAM
```

----------

## NeddySeagoon

Lebkoungcity,

Lets start with 

```
[    0.000000] BIOS-provided physical RAM map:

[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009f7ff] usable

[    0.000000] BIOS-e820: [mem 0x000000000009f800-0x000000000009ffff] reserved

[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved

[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bfdeffff] usable

[    0.000000] BIOS-e820: [mem 0x00000000bfdf0000-0x00000000bfdf2fff] ACPI NVS

[    0.000000] BIOS-e820: [mem 0x00000000bfdf3000-0x00000000bfdfffff] ACPI data

[    0.000000] BIOS-e820: [mem 0x00000000bfe00000-0x00000000bfefffff] reserved

[    0.000000] BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved

[    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000ffffffff] reserved

[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000013fffffff] usable
```

Even today, your CPU starts up in 16 bit real mode and fetches its start address from 0xfff0, just like the original 8086. Keep that in mind.

```
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009f7ff] usable

[    0.000000] BIOS-e820: [mem 0x000000000009f800-0x000000000009ffff] reserved

[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
```

is the real mode address space.

The first line is the bottom 640kB, of which Bill Gates famously said, 640k of RAM is enough for anybody.

The reserved regions following are for BIOS expansion ROMs.  You can't write there, but read caching works. Some expansion cards, like video cards expose the pixel buffer there too, so it might not all be ROM.  

Once you get out of real mode into protected mode, the above still applies and you get

```
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bfdeffff] usable

[    0.000000] BIOS-e820: [mem 0x00000000bfdf0000-0x00000000bfdf2fff] ACPI NVS

[    0.000000] BIOS-e820: [mem 0x00000000bfdf3000-0x00000000bfdfffff] ACPI data

[    0.000000] BIOS-e820: [mem 0x00000000bfe00000-0x00000000bfefffff] reserved

[    0.000000] BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved

[    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000ffffffff] reserved
```

The first is real RAM above 1MiB, you have 640k below 1MiB too, so its not contiguous. There is a hole from 640k to 1M for peripherals to do what they want. That's for ISA cards.

You don't actually have any plug in ISA cards but some are still there.

The rest are for ROM of one sort or another, PCI hardware address space, the BIOS that's also at 0xf0000.  The PC is really quite a house of cards.

[mem 0x0000000000100000-0x00000000bfdeffff]  is about 3G of RAM. 4G would be 0x0 to 0xffffffff. But then there is nowhere for the BIOS, video card, expansion ROM and the other memory mapped devices.

This is the problem with 4G of RAM and a 32 bit install. You don't get all 4G mapped into the address space at the same time.

The last entry is 

```
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000013fffffff] usable
```

That's your entire 4G RAM mapped into the address space at the 4G boundary.  A 32 bit install or 32 bit hardware can't do that.

I would expect all that 4G to be write-back or write-combining but it looks like its the 3G starting at 1M that's write-back cached.

Its the same physical RAM that appears in the memory map several times.

-- edit --

Here's your PCI memory mapped region.

```
[    0.043790] [mem 0xbff00000-0xdfffffff] available for PCI devices
```

You have 64Mb of RAM hidden under hardware.

```
[    0.162934] AGP: Your BIOS doesn't leave an aperture memory hole

[    0.162934] AGP: Please enable the IOMMU option in the BIOS setup

[    0.162935] AGP: This costs you 64MB of RAM

[    0.162937] AGP: Mapping aperture over RAM [mem 0xb4000000-0xb7ffffff] (65536KB)
```

----------

## Lebkoungcity

NeddySeagoon,

wow, thank you for your detailed post! It's amazing how you put these things in easy-to-understand-words even for non-native-speakers like me!

So, there's not really something for me to do regarding memory? OK, as long as it works it's fine for me  :Smile: 

Does it also apply to this?

 *NeddySeagoon wrote:*   

> 
> 
> You have 64Mb of RAM hidden under hardware.
> 
> ```
> ...

 

In the kernel's config everything concerning AGP is disabled and in the BIOS I can't find an entry saying IOMMU... Does this mean it's OK the way it is?

Cheers,

Andy

----------

## Jaglover

Do you need IOMMU? To my knowledge it shouldn't be turned on in vain as it may add a little overhead.

----------

## Lebkoungcity

 *Ant P. wrote:*   

> I have (nearly) the same motherboard, but 6GB RAM and amdgpu. No crash problems here.
> 
> (...)

 

Ant P.,

thank you for your input! On my machine it looks similar:

```
 # paste /sys/class/dmi/id/{board,bios}_*

GA-MA770T-UD3           Gigabyte Technology Co., Ltd.   x.x     07/09/2010      Award Software International, Inc.     F7
```

```
 # cat /proc/mtrr /sys/kernel/debug/x86/pat_memtype_list

reg00: base=0x000000000 (    0MB), size= 2048MB, count=1: write-back

reg01: base=0x080000000 ( 2048MB), size= 1024MB, count=1: write-back

reg02: base=0x0bfe00000 ( 3070MB), size=    2MB, count=1: uncachable

PAT memtype list:

write-back @ 0xbfdf0000-0xbfdf1000

write-back @ 0xbfdf3000-0xbfdfb000

write-back @ 0xbfdf3000-0xbfdf4000

write-back @ 0xbfdfa000-0xbfdfc000

write-combining @ 0xc0000000-0xc0005000

write-combining @ 0xc0020000-0xc0025000

write-combining @ 0xc0040000-0xc0045000

write-combining @ 0xc0060000-0xc0065000

write-combining @ 0xc0080000-0xc0085000

write-combining @ 0xc00a0000-0xc00a5000

write-combining @ 0xc00c0000-0xc00c5000

write-combining @ 0xc00e0000-0xc00e5000

write-combining @ 0xc0100000-0xc0105000

write-combining @ 0xc0120000-0xc0125000

write-combining @ 0xc0140000-0xc0145000

write-combining @ 0xc0160000-0xc0165000

write-combining @ 0xc0180000-0xc0181000

write-combining @ 0xc01a0000-0xc01a1000

write-combining @ 0xc01c0000-0xc01c1000

write-combining @ 0xc01e0000-0xc0220000

write-combining @ 0xc0220000-0xc0221000

write-combining @ 0xc0240000-0xc0241000

write-combining @ 0xc0260000-0xc0262000

write-combining @ 0xc0280000-0xc0281000

write-combining @ 0xc02a0000-0xc02c0000

write-combining @ 0xc02c0000-0xc02e0000

write-combining @ 0xc02e0000-0xc0300000

write-combining @ 0xc0300000-0xc0320000

uncached-minus @ 0xcfee0000-0xcfee1000

uncached-minus @ 0xcfee0000-0xcfee1000

uncached-minus @ 0xcfee0000-0xcffe0000

uncached-minus @ 0xcfee1000-0xcfee2000

uncached-minus @ 0xcfee1000-0xcfee2000

uncached-minus @ 0xcfee2000-0xcfee3000

uncached-minus @ 0xcfee2000-0xcfee3000

uncached-minus @ 0xcfee2000-0xcfee3000

uncached-minus @ 0xcfee2000-0xcfee3000

write-combining @ 0xde000000-0xdf000000

uncached-minus @ 0xe0000000-0xf0000000

uncached-minus @ 0xe0010000-0xe0011000

uncached-minus @ 0xe00a0000-0xe00a1000

uncached-minus @ 0xe0100000-0xe0101000

uncached-minus @ 0xfb000000-0xfc000000

uncached-minus @ 0xfcffc000-0xfd000000

uncached-minus @ 0xfdafe000-0xfdb00000

uncached-minus @ 0xfdaff000-0xfdb00000

uncached-minus @ 0xfddf8000-0xfddf9000

uncached-minus @ 0xfddff000-0xfde00000

uncached-minus @ 0xfe024000-0xfe028000

uncached-minus @ 0xfe028000-0xfe029000

uncached-minus @ 0xfe029000-0xfe02a000

uncached-minus @ 0xfe02a000-0xfe02b000

uncached-minus @ 0xfe02b000-0xfe02c000

uncached-minus @ 0xfe02c000-0xfe02d000

uncached-minus @ 0xfe02d000-0xfe02e000

uncached-minus @ 0xfe02e000-0xfe02f000

uncached-minus @ 0xfe02f000-0xfe030000

uncached-minus @ 0xfed00000-0xfed01000
```

```
 # for i in /sys/firmware/memmap/*; do paste $i/{start,end,type}; done | column -ts $'\t' -R 1,2        0x0      0x9f7ff  System RAM

    0x9f800      0x9ffff  Reserved

    0xf0000      0xfffff  Reserved

   0x100000   0xbfdeffff  System RAM

 0xbfdf0000   0xbfdf2fff  ACPI Non-volatile Storage

 0xbfdf3000   0xbfdfffff  ACPI Tables

 0xbfe00000   0xbfefffff  Reserved

 0xe0000000   0xefffffff  Reserved

 0xfec00000   0xffffffff  Reserved

0x100000000  0x13fffffff  System RAM
```

So I think these 'hiccups' with Plasma are one of those things I just have to live with. Something slightly annoying but not that big issue.

Cheers,

Andy

----------

## Lebkoungcity

 *Jaglover wrote:*   

> Do you need IOMMU? To my knowledge it shouldn't be turned on in vain as it may add a little overhead.

 

To be honest: I'm not sure   :Wink:   But after this thread I think I don't need it. I was just looking into this memory/MTRR stuff because I thought, it would be worth to know if it was a problem (and if it was to solve it).

Cheers,

Andy

----------

## NeddySeagoon

Lebkoungcity,

I lose 64Mb RAM like that too and I don't have a IOMMU option in my BIOS.

----------

## Lebkoungcity

Now that I know my machine is OK I close this thread. Thank you all for your assistance!

----------

