# [HOWTO] Obtain some safe CPU flags

## frankenputer

Updated the wiki to include the code below which allows the user to obtain some safe CPU flags.

Copy 'n paste ready.

```
gcc '-###' -march=native -x c - 2>&1 | gawk -lfilefuncs '{ gsub(/\"/,""); for (x=1; x < NF; x++) { if (match($x,/march|cache/)) { cflagz[$x]++; } else if (match($x,/mpopcnt|m3dnow|msse|mssse|maes/)) { x86[substr($x,3)]++; } else if (match($x,/mcx16|mabm|mlzcnt|msahf/)) { x86[substr($x,2)]++; } } } function concatMe(arr) { ret=""; for (x in arr) { str=(match(x,/cache/) ? "--param"" "x : x); ret=ret" "str; } return ret; } END { PROCINFO["sorted_in"]="@ind_str_asc"; cpuf="/proc/cpuinfo"; if (0 == stat(cpuf, buf)) { while (0 != (getline cur_line < cpuf)) { if (match(cur_line, "flags")) { split(cur_line, arr, " "); if (0 != arr_len=length(arr)) { for (x=1; x < arr_len; x++) { if (match(arr[x],/mmxext|3dnowext/)) { x86[arr[x]]++; } } } break; } } close(cpuf); } if (0 != length(x86)) { printf("CPU_FLAGS_X86=\"...%s\"\n",concatMe(x86)); } if (0 != length(cflagz)) { printf("CFLAGS=\"...%s\"\n",concatMe(cflagz)); } }'
```

The script used to be ~ 5 lines long at the beginning.

Filename cpuorakle.awk

```
#!/usr/bin/awk -f

@load "filefuncs";

{

  gsub(/\"/,"");

  for (x=1; x < NF; x++) {

    if (match($x,/march|cache/)) {

      cflagz[$x]++;

    } else if (match($x,/mpopcnt|m3dnow|msse|mssse|maes/)) {

      x86[substr($x,3)]++;

    } else if (match($x,/mcx16|mabm|mlzcnt|msahf/)) {

      x86[substr($x,2)]++;

    }

  }

}

function concatMe(arr) {

  ret="";

  for (x in arr) {

    str=(match(x,/cache/) ? "--param"" "x : x);

    ret=ret" "str;

  }

  return ret;

}

END {

  PROCINFO["sorted_in"]="@ind_str_asc";

  cpuf="/proc/cpuinfo";

  if (0 == stat(cpuf, buf)) {

    while (0 != (getline cur_line < cpuf)) {

      if (match(cur_line, "flags")) {

        split(cur_line, arr, " ");

        if (0 != arr_len=length(arr)) {

          for (x=1; x < arr_len; x++) {

            if (match(arr[x],/mmxext|3dnowext/)) {

              x86[arr[x]]++;

            }

          }

        }

        break;

      }

    }

    close(cpuf);

  }

  if (0 != length(x86)) {

    printf("CPU_FLAGS_X86=\"...%s\"\n",concatMe(x86));

  }

  if (0 != length(cflagz)) {

    printf("CFLAGS=\"...%s\"\n",concatMe(cflagz));

  }

}
```

Execute it with:

```
chmod +x cpuorakle.awk

gcc '-###' -march=native -x c - 2>&1 | ./cpuorakle.awk
```

Sample output:

```
CPU_FLAGS_X86="... 3dnow 3dnowext mabm mcx16 mlzcnt mmxext msahf popcnt sse sse2 sse3 sse4a" 

CFLAGS="... -march=amdfam10 --param l1-cache-line-size=64 --param l1-cache-size=64 --param l2-cache-size=512"
```

---

Synced the code with the upstream repo.Last edited by frankenputer on Sun Jul 17, 2016 3:02 am; edited 8 times in total

----------

## Ant P.

I have an ugly script that generates human-readable output for CPU_FLAGS_X86, may be of use to someone:

```
#!/bin/sh

flags_path="$(portageq get_repo_path / gentoo)/profiles/desc/cpu_flags_x86.desc"

flags_regex='('$(sed -n '/^flags/ { s/^.*:\s//; y/ /|/; p; q }' /proc/cpuinfo)')'

awk -F' - ' '!/^#/ { printf "%-10s %s\n", $1, $2 }' "$flags_path" \

    | egrep --color "(^|\[)$flags_regex\W"
```

----------

## frankenputer

 *Ant P. wrote:*   

> I have an ugly script that generates human-readable output for CPU_FLAGS_X86, may be of use to someone:
> 
> ```
> #!/bin/sh
> 
> ...

 

Thank you, added /proc/cpuinfo parsing.

----------

## Randy Andy

Hi frankenputer,

your sample output contains lots of faults regarding the CPU_FLAGS_X86="" line.

It should only contain CPU Flags which also exist as USE-Flags.

But you don't have to fiddle this out by your own, if you would like to read the news message from:

 *Quote:*   

> 2015-01-28-cpu_flags_x86-introduction

  and follow the instruction, except on little detail.  :Wink: 

Since yesterday the package has been renamed to:

 *Quote:*   

> app-portage/cpuid2cpuflags

 

Would be great if someone could release a new message regarding the renaming:

https://packages.gentoo.org/packages/app-portage/cpuid2cpuflags

and also correct the wiki asap:

https://wiki.gentoo.org/wiki/CPU_FLAGS_X86

Regarding dertermine safe CFLAGS:

Inspired by a lecture, I gave at the workshop here (sorry, only german language):

https://forums.gentoo.org/viewtopic-t-1017200-highlight-trolug.html

Sebastian Pipping wrote a nice python-script which does the job quite well.

The package is in the tree since almost a year and is called:

app-misc/resolve-march-native

After installation you can call it e.g. with:

```
resolve-march-native -a
```

and take over the line to you CFLAGS="" line.

A hint to this from the 

https://wiki.gentoo.org/wiki/Safe_CFLAGS would be also a great idea, cause it make things a lot easier for the user.

Best regards,  Andy.

----------

## frankenputer

 *Randy Andy wrote:*   

> Hi frankenputer,
> 
> your sample output contains lots of faults regarding the CPU_FLAGS_X86="" line.
> 
> It should only contain CPU Flags which also exist as USE-Flags.
> ...

 

Hi Andy,

The Mihai's program detects avx512* flags which doesn't exists as USE flags at this point.

Including those existing cpu instructions that does not exists as CPU_FLAGS_X86 USE flags won't break the system.

In case they are added upstream, all users that have avx512* flags in their configs will be able to leverage them.

---

 *Quote:*   

> But you don't have to fiddle this out by your own, if you would like to read the news message from: 

 

Running the Mihai's program:

```
cpuinfo2cpuflags-x86 

CPU_FLAGS_X86="3dnow 3dnowext mmx mmxext popcnt sse sse2 sse3 sse4a"
```

My gawk kung fu:

```
CPU_FLAGS_X86="... 3dnow 3dnowext mabm mcx16 mlzcnt mmxext msahf popcnt sse sse2 sse3 sse4a "
```

Compare both CPU_FLAGS_X86 lines.

mmx is included in vanilla make.conf

---

 *Quote:*   

> Sebastian Pipping wrote a nice python-script which does the job quite well. 
> 
> The package is in the tree since almost a year and is called: 
> 
> app-misc/resolve-march-native 
> ...

 

Running the program:

```
resolve-march-native -a

-march=amdfam10 --param l1-cache-line-size=64 -O2 -pipe
```

My gawk kung fu:

```
CFLAGS="... -march=amdfam10 --param l1-cache-line-size=64 --param l1-cache-size=64 --param l2-cache-size=512 "
```

Compare both CFLAGS lines.

-O2 -pipe is included in vanilla make.conf

---

I'm not sure how stable avx* flags are, since I don't have intel cpu. This along with the following quote is the reason not to include them in first place.

 *Quote:*   

> GCC depresses SSEx instructions when -mavx is used. Instead, it generates new AVX instructions or AVX equivalence for all SSEx instructions when needed.

 

At the end of the day the user has choice, either mine gawk kung fu, Mihai's program or the Sebastian's one or the Ant P. one.

I appreciate negative and positive feedback, but can't stand next to "jump to conclusions" ones.

----------

## Randy Andy

Hi frankenputer,

 *frankenputer wrote:*   

> 
> 
> The Mihai's program detects avx512* flags which doesn't exists as USE flags at this point.
> 
> Including those existing cpu instructions that does not exists as CPU_FLAGS_X86 USE flags won't break the system.
> ...

 

Right, when your prediction once occurs.

---

 *frankenputer wrote:*   

> 
> 
> Running the Mihai's program:
> 
> ```
> ...

 

Only cause mmx is named into the vanilla make.conf, does not guaranty that it would be thrown away by the average user.

You leave it out by your kung fu, which mean that packages which makes uses of it, doesn't compile this functionality into, although you set the superior grade mmext flag.

Much more packages makes usage of mmx compared to mmext, just compare the output of:

```
equery h cpu_flags_x86_mmx
```

 vs. 

```
equery h cpu_flags_x86_mmxext
```

Similar for your mabm mcx16 mlzcnt msahf, these doen't exist as cpu_flags_x86 and probably never would.

Theses flags belongs to the CFLAGS, if there's a need activate (-m) or deactivate (-mno) it there.

---

 *frankenputer wrote:*   

> 
> 
> Running the program:
> 
> ```
> ...

 

Regarding resolve-march-native -a

The option -a is only for:  *Quote:*   

> 
> 
> --add-recommended, -a
> 
>                         add recommended flags 

 

to give the user an idea what makes sense to add to his line.

Your claim here the 2 missing cache-size parameters,  but since I found a package (openmpi), which doesn't compile successfully here on my westmere Xeon cpu if I use 

the  l1-cache-size= parameter, I'm not sure if leaving out the cache parameters is good or bad. How big would be the difference regarding performance? I never tested it.

In the sense of giving safe CFLAGS, it seems to be better to leave it out to me. To present safe CFLAGS is lastly the intention of this little helper script. 

The power user (or worser the ricer) hast to decide by himself, how to optimize the settings stronger, but that's not what the title of such wikis reflect.

---

 *frankenputer wrote:*   

> 
> 
> At the end of the day the user has choice, either mine gawk kung fu, Mihai's program or the Sebastian's one or the Ant P. one.
> 
> I appreciate negative and positive feedback, but can't stand next to "jump to conclusions" ones.

 

Only to ask GCCs CPU recognition fails sometimes, there are lots of bug reports which shows that (references at the bottom).

That's why the Safe_CFLAGS Wiki and Sebastians program tries a different approach, compiling two files with different settings and compare it afterwards, to filter out wrongly recognized flags.

So far my feedback, but I agree, lastly the users decide what to use.

Regards, Andy.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=39851

https://gcc.gnu.org/ml/gcc-help/2009-04/msg00301.html

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=43718

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=39851

https://gcc.gnu.org/ml/gcc-help/2009-04/msg00305.html

----------

## frankenputer

 *Quote:*   

> Only cause mmx is named into the vanilla make.conf, does not guaranty that it would be thrown away by the average user. 
> 
> You leave it out by your kung fu, which mean that packages which makes uses of it, doesn't compile this functionality into, although you set the superior grade mmext flag.

 

Learn what mmxext is.

Use the appropriate page to lookup such cpu instructions: https://packages.gentoo.org/useflags/cpu_flags_x86_mmxext

https://forums.gentoo.org/viewtopic-t-554289-start-0.html for the padlock cpu_flags_x86.

 *Quote:*   

> Similar for your mabm mcx16 mlzcnt msahf, these doen't exist as cpu_flags_x86 and probably never would. 
> 
> Theses flags belongs to the CFLAGS, if there's a need activate (-m) or deactivate (-mno) it there.
> 
> Your claim here the 2 missing cache-size parameters, but since I found a package (openmpi), which doesn't compile successfully here on my westmere Xeon cpu if I use 
> ...

 

Never say never, also google which cpu's are supporting avx512* and when they first appeared. Also it's bad idea to enable such CFLAGS globally, as it's bad idea to use USE flags globally instead per-package USE/CFLAG.

The "big" difference is that you've not used Google, also it tells me that you have not programmed in low level language.

Your two comments contradict with each other.

So you've found a bug regarding the cache size, can you be a nice person and report it to openmpi, so they can fix it ?

 *Quote:*   

> mmx ... 

 

Same could be said for pipe, just because it ships in vanilla make.conf and is hard-coded in Sebastian's program doesn't make it "safe":

It tells the compiler to use pipes instead of temporary files during the different stages of compilation, which uses more memory. On systems with low memory, GCC might get killed. In those cases do not use this flag.

Also, no one needs hard-coded -O2 -pipe just to tell them that those flags have been "detected".

---

I'll stop right here and ask any moderator/administrator to lock this thread, before some **already angry** people start insulting me and embarrass themselves even further than they are currently.

----------

## Randy Andy

 *frankenputer wrote:*   

> 
> 
> Learn what mmxext is.
> 
> The "big" difference is that you've not used Google, also it tells me that you have not programmed in low level language.
> ...

 

Wow,

it seems that you know more about me, than myself. 

Respect, never experienced that before here, maybe because you're relative new to this outstanding forum.

It looks like you're not really interested in technical based criticism and react very emotional when you get one. 

Then you blame on me on a personal basis and would try to lock the thread - not bad guy, but to late. 

No need to, cause I'm not longer willing to discuss/escalate it with you in that way!

Bye.

----------

## josephg

 *frankenputer wrote:*   

> Updated the wiki to include the code below which allows the user to obtain some safe CPU flags.

 

which wiki.. link please?

wiki mentons this tool, which gives me slightly different output

```
$ cpuinfo2cpuflags-x86

CPU_FLAGS_X86="mmx mmxext sse sse2 sse3 ssse3"
```

your "Copy 'n paste ready" script gives me

```
CPU_FLAGS_X86="... mcx16 msahf sse sse2 sse3 ssse3"

CFLAGS="... -march=core2 --param l1-cache-line-size=64 --param l1-cache-size=32 --param l2-cache-size=2048"
```

i see the thread is a bit old, and wonder if it is still correct. can i safely add the above generated output to /etc/portage/make.conf?

----------

## krinn

You don't need any tricky cflags, just use -march=native and your gcc will do that for you, all safely and nicely.

If really you have a reason to do so, like distcc, you should just copy the output of gcc using native.

```
gcc -march=native -### -E - $2 2>&1 | sed -r '/cc1/!d;s/(\")|(^.* - )//g'
```

----------

## josephg

 *krinn wrote:*   

> You don't need any tricky cflags, just use -march=native and your gcc will do that for you, all safely and nicely.

 

isn't native the default? why then so many experts have so much in their C*FLAGS as can be seen on so many threads?  :Surprised: 

i want to make my system as efficient as can be, using less cpu/memory. i have these lines below. these no good?

```
CFLAGS="-march=native -O2 --pipe -fomit-frame-pointer -ftree-vectorize"

CXXFLAGS="${CFLAGS}"

CPU_FLAGS_X86="mmx mmxext sse sse2 sse3 ssse3"
```

i think -O2 and --pipe are also default. i don't know about other default flags. if they are all defaults, then i'd rather take them out, as you say.Last edited by josephg on Thu Jan 11, 2018 10:19 pm; edited 1 time in total

----------

## NeddySeagoon

josephg,

-march=native cannot be the default.  You might not have the same CPU as was used to build the stage3.

The default is -march unset, and -mtune=generic so the output code will run on any 64 bit AMD/Intel CPU.

-march=native won't work with distcc either.  I really don't want amd64 code sent back to my Raspberry Pi  :)

```
$ gcc -march=native -### -E - $2 2>&1 | sed -r '/cc1/!d;s/(\")|(^.* - )//g'

-march=amdfam10 -mmmx -m3dnow -msse -msse2 -msse3 -mno-ssse3 -msse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul -mpopcnt -mabm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mno-avx -mno-avx2 -mno-sse4.2 -mno-sse4.1 -mlzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mprfchw -mno-adx -mfxsr -mno-xsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-mwaitx -mno-clzero -mno-pku --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=amdfam10
```

That's ugly. The no-* don't need to be listed in CFLAGS.

Its left as an exercise for the reader to improve that one liner to suppress the no-*

----------

## Dr.Willy

 *NeddySeagoon wrote:*   

> That's ugly. The no-* don't need to be listed in CFLAGS.
> 
> Its left as an exercise for the reader to improve that one liner to suppress the no-*

 

Now that you mention, I wrote that back when I tried distcc

```
gcc -v -E -x c -march=native -mtune=native - < /dev/null 2>&1 | sed -e '/cc1/!d; s:^.* - ::; s: -mno-\S\+::g'
```

----------

## Ant P.

It's no one-liner, but this is what I've gathered over the years (adding -E only just occurred to me though, thanks Dr. Willy).

```
#!/bin/sh

# prints expanded value of -march=native (distcc-compatible) on stdout

# shellcheck disable=SC2046,SC2005

# (we want echo to wordsplit and collapse this to one line)

echo $(

    gcc -v -E -march=native -x c /dev/null 2>&1 \

        | grep -F  -- '-march' \

        | grep -Eo -- ' (-m|--param )\S+' \

        | grep -Fv -- '-mno-'

)
```

----------

