# NVIDIA GTX1080 Idle Power Usage 50W

## John5788

I just noticed that my graphics card is using about 50W at what I consider idle. I found this through the nvidia-smi utility:

```
$ nvidia-smi

Wed Dec 26 11:57:22 2018       

+-----------------------------------------------------------------------------+

| NVIDIA-SMI 415.25       Driver Version: 415.25       CUDA Version: 10.0     |

|-------------------------------+----------------------+----------------------+

| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |

| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |

|===============================+======================+======================|

|   0  GeForce GTX 1080    Off  | 00000000:01:00.0  On |                  N/A |

| 12%   53C    P0    47W / 215W |   1179MiB /  8111MiB |      7%      Default |

+-------------------------------+----------------------+----------------------+

                                                                               

+-----------------------------------------------------------------------------+

| Processes:                                                       GPU Memory |

|  GPU       PID   Type   Process name                             Usage      |

|=============================================================================|

|    0      4672      G   /usr/bin/X                                   535MiB |

|    0      5123      G   cinnamon                                      80MiB |

|    0      5425      G   ...passed-by-fd --v8-snapshot-passed-by-fd    34MiB |

|    0      5569      G   ...quest-channel-token=2787466181135642490   137MiB |

|    0      6550      G   ...88/.local/share/Steam/ubuntu12_32/steam    29MiB |

|    0      6557      G   ./steamwebhelper                               2MiB |

|    0     10729      G   /usr/lib64/virtualbox/VirtualBox             291MiB |

+-----------------------------------------------------------------------------+

```

It looks like the card is "stuck" in P0 state which is the highest performance state. Is there a way to "encourage" the card to goto a different performance state (P1 or higher?) or something to reduce the power usage? My PowerMizer settings are already set to Adaptive. I just realize this is a huge chunk of my power coming from my system at idle. My tower consumes about 130W of power at idle and having about a third of that being the GPU while I'm not doing anything major on the computer just seems absurd.

```
$ nvidia-smi -q -d PERFORMANCE

==============NVSMI LOG==============

Timestamp                           : Wed Dec 26 12:01:54 2018

Driver Version                      : 415.25

CUDA Version                        : 10.0

Attached GPUs                       : 1

GPU 00000000:01:00.0

    Performance State               : P0

    Clocks Throttle Reasons

        Idle                        : Active

        Applications Clocks Setting : Not Active

        SW Power Cap                : Not Active

        HW Slowdown                 : Not Active

            HW Thermal Slowdown     : Not Active

            HW Power Brake Slowdown : Not Active

        Sync Boost                  : Not Active

        SW Thermal Slowdown         : Not Active

        Display Clock Setting       : Not Active
```

If that's the tax I pay for having high powered PC components, then so be it. But I'd like to know if there's a way to get the card to a different P-state if the clock throttle recognizes that I am at idle as signified by the last command output above.

----------

## John5788

As a reference, I booted up my Windows PC with a GTX1070 (I understand it's a different card, but same family, so the results should be similar) and ran the nvidia-smi utility in there. I got the results I was expecting to ee on my Linux/GTX1080

```
PS C:\Program Files\NVIDIA Corporation\NVSMI> .\nvidia-smi.exe

Wed Dec 26 12:59:16 2018

+-----------------------------------------------------------------------------+

| NVIDIA-SMI 417.35       Driver Version: 417.35       CUDA Version: 10.0     |

|-------------------------------+----------------------+----------------------+

| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |

| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |

|===============================+======================+======================|

|   0  GeForce GTX 1070   WDDM  | 00000000:01:00.0  On |                  N/A |

|  0%   29C    P8    12W / 151W |    277MiB /  8192MiB |      3%      Default |

+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+

| Processes:                                                       GPU Memory |

|  GPU       PID   Type   Process name                             Usage      |

|=============================================================================|

|    0      1240    C+G   Insufficient Permissions                   N/A      |

|    0      3520    C+G   ...hell.Experiences.TextInput.InputApp.exe N/A      |

|    0      4364    C+G   ...mmersiveControlPanel\SystemSettings.exe N/A      |

|    0      8112    C+G   C:\Windows\explorer.exe                    N/A      |

|    0      8568    C+G   ...dows.Cortana_cw5n1h2txyewy\SearchUI.exe N/A      |

|    0      9100    C+G   ...t_cw5n1h2txyewy\ShellExperienceHost.exe N/A      |

|    0      9876    C+G   ...36.52.0_x64__kzf8qxf38zg5c\SkypeApp.exe N/A      |

|    0     10148    C+G   ...osoft.LockApp_cw5n1h2txyewy\LockApp.exe N/A      |

|    0     11212    C+G   Insufficient Permissions                   N/A      |

+-----------------------------------------------------------------------------+
```

The card is idling in P8 state with only 12W power consumption.

The clock speeds are dramatically different between the two systems:

```
$ nvidia-smi -q -d CLOCK

==============NVSMI LOG==============

Timestamp                           : Wed Dec 26 13:03:02 2018

Driver Version                      : 415.25

CUDA Version                        : 10.0

Attached GPUs                       : 1

GPU 00000000:01:00.0

    Clocks

        Graphics                    : 1721 MHz

        SM                          : 1721 MHz

        Memory                      : 5005 MHz

        Video                       : 1544 MHz

    Applications Clocks

        Graphics                    : N/A

        Memory                      : N/A

    Default Applications Clocks

        Graphics                    : N/A

        Memory                      : N/A

    Max Clocks

        Graphics                    : 2025 MHz

        SM                          : 2025 MHz

        Memory                      : 5005 MHz

        Video                       : 1708 MHz

    Max Customer Boost Clocks

        Graphics                    : N/A

    SM Clock Samples

        Duration                    : 49196.68 sec

        Number of Samples           : 100

        Max                         : 1974 MHz

        Min                         : 139 MHz

        Avg                         : 1881 MHz

    Memory Clock Samples

        Duration                    : 49194.33 sec

        Number of Samples           : 100

        Max                         : 5005 MHz

        Min                         : 405 MHz

        Avg                         : 5004 MHz

    Clock Policy

        Auto Boost                  : N/A

        Auto Boost Default          : N/A

```

```
PS C:\Program Files\NVIDIA Corporation\NVSMI> .\nvidia-smi.exe -q -d CLOCK

==============NVSMI LOG==============

Timestamp                           : Wed Dec 26 13:04:44 2018

Driver Version                      : 417.35

CUDA Version                        : 10.0

Attached GPUs                       : 1

GPU 00000000:01:00.0

    Clocks

        Graphics                    : 139 MHz

        SM                          : 139 MHz

        Memory                      : 405 MHz

        Video                       : 544 MHz

    Applications Clocks

        Graphics                    : N/A

        Memory                      : N/A

    Default Applications Clocks

        Graphics                    : N/A

        Memory                      : N/A

    Max Clocks

        Graphics                    : 1999 MHz

        SM                          : 1999 MHz

        Memory                      : 4004 MHz

        Video                       : 1708 MHz

    Max Customer Boost Clocks

        Graphics                    : N/A

    SM Clock Samples

        Duration                    : 64889.82 sec

        Number of Samples           : 100

        Max                         : 1594 MHz

        Min                         : 139 MHz

        Avg                         : 613 MHz

    Memory Clock Samples

        Duration                    : 65002.57 sec

        Number of Samples           : 100

        Max                         : 4006 MHz

        Min                         : 405 MHz

        Avg                         : 3545 MHz

    Clock Policy

        Auto Boost                  : N/A

        Auto Boost Default          : N/A
```

Any reason why on Linux I cannot achieve the same low power idle usage?

----------

## saboya

It's not very useful to compare both cards since they're executing completely different tasks, with different drivers / operating systems.

----------

## John5788

 *saboya wrote:*   

> It's not very useful to compare both cards since they're executing completely different tasks, with different drivers / operating systems.

 

With that being said, the Linux / GTX1080 isn't even hitting a lower power performance state. I wanted to illustrate that the Windows / GTX1070 is achieving P8 while my Linux / GTX1080 is stuck in P0, forget the actual power draw numbers 12W vs 47W. Is the Linux driver unable to get out of a P0 state due to design/bug/feature/that's just the way it is?

----------

## John5788

I did some digging on NVIDIA forums and it looks like a driver issue specific to Linux that hasn't been fixed for over 1 year:

https://devtalk.nvidia.com/default/topic/1002912/linux/very-slow-ramp-down-from-high-to-low-clock-speeds-leading-to-a-significantly-increased-power-consumption/1

The problem is there is a very slow ramp down delay (reported ~36 seconds) before the clocks decrease and enter P8 state for idle. But anything on the desktop, compositing/chrome/etc.. will reset the idle clock and bring the card back into high performance state aggressively. It looks like NVIDIA is finally tracking this issue as of a few months ago, but still no proposed solution yet.

There is a workaround, but it looks like you cannot switch out of a low power state without reloading X since it requires modifying the RegistryDwords option in xorg.conf:

```
Section "Device"

        Identifier      "Videocard0"

        Driver          "nvidia"

        Option          "Coolbits" "28"

        Option          "metamodes" "nvidia-auto-select +0+0 {ForceCompositionPipeline=On, ForceFullCompositionPipeline=On}"

        Option          "RegistryDwords" "PowerMizerEnable=0x1; PerfLevelSrc=0x2222; PowerMizerLevel=0x3; PowerMizerDefault=0x3; PowerMizerDefaultAC=0x3"

EndSection
```

I'll just live with how it is for now or switch to AMD  :Smile: 

----------

## Ant P.

 *John5788 wrote:*   

> I'll just live with how it is for now or switch to AMD :)

 

AMD ain't perfect either (4.18 had a bug causing my RX550 to idle at 12 watts! The horror!)

----------

## Marlo

 *Ant P. wrote:*   

> 
> 
> AMD ain't perfect either (4.18 had a bug causing my RX550 to idle at 12 watts! The horror!)

 

How did you measure this value?

How high should the value be in idle?

Thanks

Ma

----------

## Ant P.

 *Marlo wrote:*   

> How did you measure this value?

 

/usr/bin/sensors

 *Quote:*   

> How high should the value be in idle?

 

Normal idle for the card is around 4W.

----------

## John5788

I played around with some configuration settings. The RegistryDwords option in xorg.conf are not picked up by the driver anymore and are now instead part of the kernel module.

I set this option in /etc/modprobe.d/nvidia.conf

```
options nvidia NVreg_RegistryDwords="OverrideMaxPerf=0x1
```

And this seems to force my card into P8 performance level and yeah, games are impossible in this setting. But hey, 17W power draw.

```
$ nvidia-smi 

Wed Dec 26 16:13:12 2018       

+-----------------------------------------------------------------------------+

| NVIDIA-SMI 415.25       Driver Version: 415.25       CUDA Version: 10.0     |

|-------------------------------+----------------------+----------------------+

| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |

| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |

|===============================+======================+======================|

|   0  GeForce GTX 1080    Off  | 00000000:01:00.0  On |                  N/A |

|  0%   48C    P8    17W / 215W |    642MiB /  8111MiB |     50%      Default |

+-------------------------------+----------------------+----------------------+

                                                                               

+-----------------------------------------------------------------------------+

| Processes:                                                       GPU Memory |

|  GPU       PID   Type   Process name                             Usage      |

|=============================================================================|

|    0      4624      G   /usr/bin/X                                   341MiB |

|    0      5175      G   cinnamon                                      81MiB |

|    0      5383      G   ...passed-by-fd --v8-snapshot-passed-by-fd    43MiB |

|    0      5906      G   ...quest-channel-token=5854525692940951161    68MiB |

|    0      6986      G   ...88/.local/share/Steam/ubuntu12_32/steam    39MiB |

|    0      6995      G   ./steamwebhelper                               2MiB |

+-----------------------------------------------------------------------------+
```

Normal desktop usage seems perfectly fine even when using composite effects. General consensus on the Linux boards over at NVIDIA seem to be that the devs need to get their power management under Linux working properly.

----------

## hhfeuer

Since you're using ForceCompositionPipeline, you've been running into another Nvidia driver bug. The composition pipeline has been changed starting with the 390 driver exibiting gpu load while idle on some or most systems. You can return to the previous behaviour using

```
Option "UseNvKmsCompositionPipeline" "false"
```

in your xorg.conf.

----------

## John5788

 *hhfeuer wrote:*   

> Since you're using ForceCompositionPipeline, you've been running into another Nvidia driver bug. The composition pipeline has been changed starting with the 390 driver exibiting gpu load while idle on some or most systems. You can return to the previous behaviour using
> 
> ```
> Option "UseNvKmsCompositionPipeline" "false"
> ```
> ...

 

Thanks for that, I didn't even know. I found the original post from NVIDIA describing the issue: https://devtalk.nvidia.com/default/topic/1029484/linux/-various-all-distros-numerous-performance-amp-rendering-issues-on-390-25/post/5245514/#5245514

nvidia-smi was reporting 4% usage for me sitting in idle and I didn't even realize it. After adding that line to my xorg.conf screen section, the idle usage went down to 0%. Since I spent some time benchmarking and overclocking my 1080 yesterday, I ran Superposition again today with that change and noticed an increase in my score for 1080P extreme preset.

Stock: 3508

Overclock core +100MHz, memory +1000MHz: 3823

Reverting back to old composition pipeline behavior: 3954

Score increased by 130 points just by freeing up some idle GPU usage  :Smile: 

----------

## John5788

I spent on time on the NVIDIA forums and found out that triple monitors are keeping my GPU from clocking down into a power saving state. I disconnected one of my displays to test this with twm and some xterms, and I found that after 40 seconds, the card does clock itself down to a lower power state.

First off, 40 seconds is a long time to wait for entering a low power state considering the Intel CPU can do it hundreds of times per second going from C0 to C7.

Second, not sure why the limitation of triple monitor is the reason for not entering a lower power state unless there's some written condition in the driver to not throttle down if the screen resolution exceeds a certain amount or if the number of displays active is greater than 2. I was able to use my desktop just fine in P8 state while testing the forced power saving modes by modifying the RegistryDwords option via kernel module.

But now I'm looking at ways to use my i965 onboard GPU to drive my third monitor on the same session of X if that's even possible.

----------

## Ant P.

That's something AMD cards do too; the card can only downclock fully if it has a single CRTC to drive, otherwise there'll be flickering as the clocks adjust and gradually drift.

40 seconds does seem a bit too long though.

I've got a keyboard shortcut for that reason to toggle off my other monitor when I don't need it.

----------

