# Haswell, Intel P State and fiery inferno

## mbar

I have that infamous Sony Vaio Pro 13 ultrabook with i5-4200U Haswell CPU and I'm struggling with power consumption / performance.

To cut long story short, I can't make the gentoo 3.11.1 kernel (intel_pstate.c patched with Haswell ID) or even 3.12-rc1 to behave. It is not downclocking the CPU in idle when Intel P state control driver is enabled. Quite the contrary, this is the situation in idle:

```
uptime

 09:55:06 up 15 min,  3 users,  load average: 0,00, 0,03, 0,05
```

And see i7z:

```
Cpu speed from cpuinfo 2294.00Mhz

cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating via tsc

Linux's inbuilt cpu_khz code emulated now

True Frequency (without accounting Turbo) 2294 MHz

  CPU Multiplier 23x || Bus clock frequency (BCLK) 99.74 MHz

Socket [0] - [physical cores=2, logical cores=4, max online cores ever=2]

  TURBO ENABLED on 2 Cores, Hyper Threading ON

  Max Frequency without considering Turbo 2393.74 MHz (99.74 x [24])

  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  26x/23x/23x/23x

  Real Current Frequency 2296.89 MHz [99.74 x 23.03] (Max of below)

        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %  Temp

        Core 1 [0]:       2292.35 (22.98x)       100       1       0       0    55

        Core 2 [1]:       2296.89 (23.03x)         1     100       0       0    55

C0 = Processor running without halting

C1 = Processor running with halts (States >C0 are power saver)

C3 = Cores running with PLL turned off and core cache turned off

C6 = Everything in C3 + core state saved to last level cache

  Above values in table are in percentage over the last 1 sec

[core-id] refers to core-id number in /proc/cpuinfo

'Garbage Values' message printed when garbage values are read

  Ctrl+C to exit

```

One core is running without halting! And yes, I have full dyntics, and all power management options known to mankind enabled, etc. 

Situation changes somewhat when emerging, it will downclock when all cores are used due to power envelope. 

As you can see, in idle, Intel P state driver keeps this CPU constantly in turbo mode.

If I switch to plain ACPI P-state and ondemand governor, it underclocks to 800 MHz, but then I have no turbo modes available. Only 800 - 1600 MHz range, but during emerge it kinda sticks to 800 MHz, which is pointless.

Any help is greatly appreciated.

EDIT: ACPI ondemand driver:

```
Cpu speed from cpuinfo 2295.00Mhz

cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating via tsc

Linux's inbuilt cpu_khz code emulated now

True Frequency (without accounting Turbo) 2295 MHz

  CPU Multiplier 23x || Bus clock frequency (BCLK) 99.78 MHz

Socket [0] - [physical cores=2, logical cores=4, max online cores ever=2]

  TURBO ENABLED on 2 Cores, Hyper Threading ON

  Max Frequency without considering Turbo 2394.78 MHz (99.78 x [24])

  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  26x/23x/23x/23x

  Real Current Frequency 798.86 MHz [99.78 x 8.01] (Max of below)

        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %  Temp

        Core 1 [0]:       798.26 (8.00x)         100    65.2       0       0    47

        Core 2 [1]:       798.86 (8.01x)           1    98.9       1       0    46

C0 = Processor running without halting

C1 = Processor running with halts (States >C0 are power saver)

C3 = Cores running with PLL turned off and core cache turned off

C6 = Everything in C3 + core state saved to last level cache

  Above values in table are in percentage over the last 1 sec

[core-id] refers to core-id number in /proc/cpuinfo

'Garbage Values' message printed when garbage values are read

  Ctrl+C to exit

```

----------

## PaulBredbury

 *mbar wrote:*   

> all power management options

 

Also undervolt the CPU.

----------

## mbar

Will look into that.

Also please scratch that under ACPI driver turbo is not available. It is. This is during emerge libreoffice, about 30 minutes into it:

```
Cpu speed from cpuinfo 2294.00Mhz

cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating via tsc

Linux's inbuilt cpu_khz code emulated now

True Frequency (without accounting Turbo) 2294 MHz

  CPU Multiplier 23x || Bus clock frequency (BCLK) 99.74 MHz

Socket [0] - [physical cores=2, logical cores=4, max online cores ever=2]

  TURBO ENABLED on 2 Cores, Hyper Threading ON

  Max Frequency without considering Turbo 2393.74 MHz (99.74 x [24])

  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  26x/23x/23x/23x

  Real Current Frequency 1921.87 MHz [99.74 x 19.27] (Max of below)

        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %  Temp

        Core 1 [0]:       1921.77 (19.27x)      99.7    16.5       0       0    64

        Core 2 [1]:       1921.87 (19.27x)      99.3    16.8       0       0    66

C0 = Processor running without halting

C1 = Processor running with halts (States >C0 are power saver)

C3 = Cores running with PLL turned off and core cache turned off

C6 = Everything in C3 + core state saved to last level cache

  Above values in table are in percentage over the last 1 sec

[core-id] refers to core-id number in /proc/cpuinfo

'Garbage Values' message printed when garbage values are read

  Ctrl+C to exit

```

Not too bad, seems I'll settle with ACPI P state and ondemand for now.

----------

## mbar

All those troubles were caused by full dynticks. After I reverted to dyntics idle, everything is OK even with intel_pstate:

```
Cpu speed from cpuinfo 2294.00Mhz

cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating via tsc

Linux's inbuilt cpu_khz code emulated now

True Frequency (without accounting Turbo) 2294 MHz

  CPU Multiplier 23x || Bus clock frequency (BCLK) 99.74 MHz

Socket [0] - [physical cores=2, logical cores=4, max online cores ever=2]

  TURBO ENABLED on 2 Cores, Hyper Threading ON

  Max Frequency without considering Turbo 2393.74 MHz (99.74 x [24])

  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  26x/23x/23x/23x

  Real Current Frequency 1631.04 MHz [99.74 x 16.35] (Max of below)

        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %  Temp

        Core 1 [0]:       1595.34 (16.00x)         1    98.8       1       0    44

        Core 2 [1]:       1631.04 (16.35x)      4.97    96.5       0       0    46

C0 = Processor running without halting

C1 = Processor running with halts (States >C0 are power saver)

C3 = Cores running with PLL turned off and core cache turned off

C6 = Everything in C3 + core state saved to last level cache

  Above values in table are in percentage over the last 1 sec

[core-id] refers to core-id number in /proc/cpuinfo

'Garbage Values' message printed when garbage values are read

  Ctrl+C to exit

```

Yeah, the frequency is still 1600 Mhz, but both cores are halted nearly 100% of the time.

----------

## shad0w_GR

Hi, i have 4570, do you need acpi_cpufreq in order pstates to throttle cpu?

----------

## Gusar

The intel_pstate driver is a replacement for acpi_cpufreq, you load one or the other. It contains internal governors, so whatever acpi governors you choose in the kernel config is irrelevant. And it doesn't actually do much with p-states because switching them constantly is an outdated concept on modern Intel CPUs. In deeper C-states, the frequency is zero anyway! And Haswell will spend most of its time in the C7 state. For more info, read this: https://plus.google.com/114657443111661859546/posts/dLn9T4ehywL

One more thing - if you want to use i7z, you must use i7z-git from here: https://github.com/ajaiantilal/i7z. Any other version is outdated and will not show info correctly on Haswell. For example, the i7z output in mbar's posts is bogus. This is how it should look like (note the additional column for C7):

```
Cpu speed from cpuinfo 2893.00Mhz

cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating via tsc

Linux's inbuilt cpu_khz code emulated now

True Frequency (without accounting Turbo) 2892 MHz

  CPU Multiplier 29x || Bus clock frequency (BCLK) 99.72 MHz

Socket [0] - [physical cores=4, logical cores=4, max online cores ever=4]

  TURBO ENABLED on 4 Cores, Hyper Threading OFF

  Max Frequency without considering Turbo 2991.72 MHz (99.72 x [30])

  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  36x/35x/33x/32x

  Real Current Frequency 3534.81 MHz [99.72 x 35.45] (Max of below)

        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %   C7 %  Temp      VCore

        Core 1 [0]:       3534.81 (35.45x)      1.27    6.54       0       0    91.9    30      0.9780

        Core 2 [1]:       3527.95 (35.38x)         1    0.211      0       0    99.2    27      1.0220

        Core 3 [2]:       3531.66 (35.41x)         1    0.0421     0       0    99.6    26      1.0245

        Core 4 [3]:       3517.45 (35.27x)         1    4.36       0       0    95.1    30      1.0198

C0 = Processor running without halting

C1 = Processor running with halts (States >C0 are power saver modes with cores idling)

C3 = Cores running with PLL turned off and core cache turned off

C6, C7 = Everything in C3 + core state saved to last level cache, C7 is deeper than C6

  Above values in table are in percentage over the last 1 sec

[core-id] refers to core-id number in /proc/cpuinfo

'Garbage Values' message printed when garbage values are read

  Ctrl+C to exit

```

You could also use turbostat, you'll find it in the kernel source, /usr/src/linux/tools/power/x86/turbostat to be exact. Example output:

```
cor CPU    %c0  GHz  TSC SMI    %c1    %c3    %c6    %c7 CTMP PTMP   %pc2   %pc3   %pc6   %pc7  Pkg_W  Cor_W GFX_W

          0.40 3.51 2.89   0   2.69   0.12   0.10  96.69   33   34  92.74   0.00   0.00   0.00   8.01   0.34  0.01

  0   0   0.84 3.52 2.89   0   3.78   0.39   0.20  94.79   33   34  92.74   0.00   0.00   0.00   8.01   0.34  0.01

  1   1   0.75 3.51 2.89   0   5.47   0.09   0.20  93.49   33

  2   2   0.02 3.35 2.89   0   1.49   0.00   0.00  98.49   29

  3   3   0.01 3.45 2.89   0   0.02   0.00   0.00  99.98   32

```

This one also shows Package C states and power consumption. Note how the CPUs are at the max *turbo* freq I have. But that doesn't matter, because they're actually at this freq only in the C0 state.

----------

## shad0w_GR

So having CONFIG_X86_ACPI_CPUFREQ=m isn't bad, as much as CONFIG_X86_INTEL_PSTATE=y is enabled. Right?

I am using cpupower, and cpupower indeed shows that most of the time is in C7 state, but frequency is pretty high. Is cpu throttle in this cpus unnecessary? 

```
    |Nehalem                    || SandyBridge        || Mperf              || Idle_Stats         

CPU | C3   | C6   | PC3  | PC6  || C7   | PC2  | PC7  || C0   | Cx   | Freq || POLL | C1   | C2   

   0|  0.00|  0.00|  0.00|  0.00|| 86.16|  0.00|  0.00||  6.90| 93.10|  1197||  0.00|  5.84| 87.21

   1|  0.00|  0.00|  0.00|  0.00|| 81.42|  0.00|  0.00||  8.97| 91.03|  1366||  0.00|  8.65| 82.30

   2|  0.00|  0.00|  0.00|  0.00|| 92.42|  0.00|  0.00||  4.46| 95.54|  1489||  0.00|  2.49| 93.06

   3|  0.00|  0.00|  0.00|  0.00|| 88.48|  0.00|  0.00||  5.31| 94.69|  1482||  0.00|  5.48| 89.17

```

Using turbostat

```
cor CPU    %c0  GHz  TSC SMI    %c1    %c3    %c6    %c7 CTMP PTMP   %pc2   %pc3   %pc6   %pc7  Pkg_W  Cor_W GFX_W

          8.93 1.83 3.20   0   9.78   0.00   0.00  81.29   29   30   0.00   0.00   0.00   0.00   7.78   1.58  0.12

  0   0  10.72 1.62 3.20   0  13.58   0.00   0.00  75.69   29   30   0.00   0.00   0.00   0.00   7.78   1.58  0.12

  1   1   7.27 1.65 3.20   0   8.77   0.00   0.00  83.96   26

  2   2  10.62 1.94 3.20   0   8.83   0.00   0.00  80.55   27

  3   3   7.11 2.17 3.20   0   7.94   0.00   0.00  84.94   26

```

But in lsmod acpi_cpufreq is loaded. Without it, lowest frequency is never reached. I can understand that in C7 states, frequency doesn't matter, but why?

Thanks

----------

## Gusar

 *shad0w_GR wrote:*   

> frequency doesn't matter, but why?

 

Did you real the link I gave? It explains everything in detail: The freq is *zero* when the CPU is in C3, C6 or C7 states. And at the lowest when it's in the C1 state. It's only at the number you see when it's in the C0 state.

----------

## shad0w_GR

 *Gusar wrote:*   

>  *shad0w_GR wrote:*   frequency doesn't matter, but why? 
> 
> Did you real the link I gave? It explains everything in detail: The freq is *zero* when the CPU is in C3, C6 or C7 states. And at the lowest when it's in the C1 state. It's only at the number you see when it's in the C0 state.

 

Oh i see. But turbo(using turbostat) doesn't seem to work. Based on frequency Ghz column in turbostat reports.

More from i7z:

```
Cpu speed from cpuinfo 3199.00Mhz

cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating via tsc

Linux's inbuilt cpu_khz code emulated now

True Frequency (without accounting Turbo) 3198 MHz

  CPU Multiplier 32x || Bus clock frequency (BCLK) 99.94 MHz

Socket [0] - [physical cores=4, logical cores=4, max online cores ever=4]

  TURBO ENABLED on 4 Cores, Hyper Threading OFF

  Max Frequency without considering Turbo 3297.94 MHz (99.94 x [33])

  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  36x/36x/35x/34x

  Real Current Frequency 3199.69 MHz [99.94 x 32.02] (Max of below)

        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %   C7 %  Temp      VCore

        Core 1 [0]:       3199.08 (32.01x)      85.6    6.69       0       0    7.76    40      0.9509

        Core 2 [1]:       3199.59 (32.02x)      45.7    15.6       0       0    38.8    40      0.9480

        Core 3 [2]:       3199.69 (32.02x)      21.6      20       0       0    58.4    39      0.9484

        Core 4 [3]:       3199.25 (32.01x)      44.1    10.4       0       0    45.5    35      0.9510

C0 = Processor running without halting

C1 = Processor running with halts (States >C0 are power saver modes with cores idling)

C3 = Cores running with PLL turned off and core cache turned off

C6, C7 = Everything in C3 + core state saved to last level cache, C7 is deeper than C6

  Above values in table are in percentage over the last 1 sec

[core-id] refers to core-id number in /proc/cpuinfo

'Garbage Values' message printed when garbage values are read

  Ctrl+C to exit

```

----------

## toralf

 *shad0w_GR wrote:*   

> So having CONFIG_X86_ACPI_CPUFREQ=m isn't bad, as much as CONFIG_X86_INTEL_PSTATE=y is enabled. Right?

 Yes, and You can switch off pstate at the kernel command line via intel_pstate=disabke IIRC. SO compiling ondemand governor as module and you always have a working fall back.

----------

## shad0w_GR

Everything works with acpi_cpufreq module loaded.

Turbo isn't used if acpi_cpufreq is missing.

----------

## Gusar

What exactly does acpi_cpufreq missing mean - that there's no driver handling the CPU or that intel_pstate is running? Turbo works here with intel_pstate on my i5-4570S.

Oh, I just remembered something: Which kernel are you using? If it's not 3.12, you need a patch for intel_pstate to work on haswell, did you apply that patch?

----------

## shad0w_GR

 *Gusar wrote:*   

> What exactly does acpi_cpufreq missing mean - that there's no driver handling the CPU or that intel_pstate is running? Turbo works here with intel_pstate on my i5-4570S.
> 
> Oh, I just remembered something: Which kernel are you using? If it's not 3.12, you need a patch for intel_pstate to work on haswell, did you apply that patch?

 

I mean that if i have blacklist acpi_cpufreq, turbo doesn't work.

I use vanilla 3.11.6. Ok ill give a shot to 3.12. Thanks

----------

## shad0w_GR

 *Gusar wrote:*   

> What exactly does acpi_cpufreq missing mean - that there's no driver handling the CPU or that intel_pstate is running? Turbo works here with intel_pstate on my i5-4570S.
> 
> Oh, I just remembered something: Which kernel are you using? If it's not 3.12, you need a patch for intel_pstate to work on haswell, did you apply that patch?

 

Thanks, using 3.12 turbo works without loading acpi_cpufreq.

----------

## albright

I have a thinkpad T440s, i5-4300u and i7z does not seem to work

properly. I have the git cloned version but the output is

like this:

```
Cpu speed from cpuinfo 2494.00Mhz

cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating via tsc

Linux's inbuilt cpu_khz code emulated now

True Frequency (without accounting Turbo) 2494 MHz

  CPU Multiplier 25x || Bus clock frequency (BCLK) 99.76 MHz

Socket [0] - [physical cores=2, logical cores=4, max online cores ever=2]

  TURBO ENABLED on 2 Cores, Hyper Threading ON

  Max Frequency without considering Turbo 2593.76 MHz (99.76 x [26])

  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  29x/26x/26x/26x

  Real Current Frequency 2456.61 MHz [99.76 x 24.63] (Max of below)

        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %  Temp      VCore

        Core 1 [0]:       2456.61 (24.63x)         1    98.7       1       0    36      0.8752

        Core 2 [2]:       2202.06 (22.07x)         1     100       0       0    35      0.8755

```

Am I right to assume that i7z is not yet ready to handle the i5-4300u?

----------

## Gusar

 *albright wrote:*   

> Am I right to assume that i7z is not yet ready to handle the i5-4300u?

 

It's possible. Observe its output when you quit it (or start it with --nogui), then it'll tell you if it doesn't recognize the CPU.

In the meantime, try turbostat. And make sure you're running a 3.12 kernel.

----------

## albright

Here's some output from turbostat (no c7):

```
~/Docs/insts/linux-v3.1-rc6-1332-g18e42dc $ sudo ./turbostat

Password:                                                                                                                                                                         

 cr CPU    %c0  GHz  TSC    %c1    %c3    %c6  %pc3  %pc6                                                                                                                         

           0.25 2.88 2.49  99.72   0.02   0.01  0.00  0.00

   0   0   0.12 2.88 2.49  99.85   0.03   0.01  0.00  0.00

   0   1   0.85 2.89 2.49  99.12   0.03   0.01  0.00  0.00

   1   2   0.01 2.79 2.49  99.97   0.01   0.01  0.00  0.00

   1   3   0.03 2.79 2.49  99.96   0.01   0.01  0.00  0.00

 cr CPU    %c0  GHz  TSC    %c1    %c3    %c6  %pc3  %pc6

           0.25 2.81 2.49  99.71   0.03   0.01  0.00  0.00

   0   0   0.11 2.80 2.49  99.82   0.05   0.02  0.00  0.00

   0   1   0.87 2.82 2.49  99.06   0.05   0.02  0.00  0.00

   1   2   0.01 2.80 2.49  99.99   0.00   0.00  0.00  0.00

   1   3   0.03 2.70 2.49  99.97   0.00   0.00  0.00  0.00

 cr CPU    %c0  GHz  TSC    %c1    %c3    %c6  %pc3  %pc6

           0.25 2.88 2.49  99.74   0.01   0.00  0.00  0.00

   0   0   0.12 2.88 2.49  99.88   0.01   0.00  0.00  0.00

   0   1   0.85 2.89 2.49  99.15   0.01   0.00  0.00  0.00

   1   2   0.01 2.82 2.49  99.98   0.01   0.00  0.00  0.00

   1   3   0.03 2.75 2.49  99.96   0.01   0.00  0.00  0.00

 cr CPU    %c0  GHz  TSC    %c1    %c3    %c6  %pc3  %pc6

           0.27 2.89 2.49  99.70   0.02   0.01  0.00  0.00

   0   0   0.17 2.89 2.49  99.77   0.03   0.02  0.00  0.00

   0   1   0.87 2.89 2.49  99.08   0.03   0.02  0.00  0.00

   1   2   0.01 2.86 2.49  99.99   0.00   0.00  0.00  0.00

   1   3   0.03 2.77 2.49  99.97   0.00   0.00  0.00  0.00

 cr CPU    %c0  GHz  TSC    %c1    %c3    %c6  %pc3  %pc6

           0.25 2.88 2.49  99.72   0.02   0.01  0.00  0.00

   0   0   0.51 2.89 2.49  99.44   0.04   0.01  0.00  0.00

   0   1   0.45 2.89 2.49  99.50   0.04   0.01  0.00  0.00

   1   2   0.01 2.83 2.49  99.98   0.01   0.00  0.00  0.00

   1   3   0.03 2.76 2.49  99.96   0.01   0.00  0.00  0.00

 cr CPU    %c0  GHz  TSC    %c1    %c3    %c6  %pc3  %pc6

           0.28 2.89 2.49  99.70   0.01   0.00  0.00  0.00                                                                                                                        

   0   0   0.88 2.89 2.49  99.10   0.02   0.00  0.00  0.00                                                                                                                        

   0   1   0.22 2.89 2.49  99.75   0.02   0.00  0.00  0.00                                                                                                                        

   1   2   0.01 2.82 2.49  99.99   0.00   0.00  0.00  0.00                                                                                                                        

   1   3   0.03 2.79 2.49  99.97   0.00   0.00  0.00  0.00                                                                                                                        

  
```

----------

