# cryptsetup-1.0.5-r1 segfaults on amd64

## 102039

Hello,

i have a problem creating a encrypted partition using cryptsetup-1.0.5-r1 on an amd64/hardened system.

 *Quote:*   

> 
> 
> servername ~ # cryptsetup -c aes-lrw-benbi -y -s 384 luksFormat /dev/vg/data 
> 
> WARNING!
> ...

 

As you can see, i am trying to encrypt a lvm2 partition, but that shouldn't be a problem, because it works fine on a ~x86 system. All kernel options are correct, i am using hardened extensions, but i also tried by removing all pax/grsec features, which did not help.

Here is my emerge --info

 *Quote:*   

> Portage 2.1.3.19 (hardened/amd64, gcc-3.4.6, glibc-2.6.1-r0, 2.6.23-hardened-r4 x86_64)
> 
> =================================================================
> 
> System uname: 2.6.23-hardened-r4 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 6000+
> ...

 

Any ideas ?

----------

## schachti

Does it work if you remove -fforce-addr -msse3 from your CFLAGS and re-emerge cryptsetup?

----------

## pkerwien

Since your are using the LUKS extension, I assume you should install cryptsetup-luks. The latest stable amd64 in portage is cryptsetup-luks-1.0.4-r3.

----------

## 102039

 *schachti wrote:*   

> Does it work if you remove -fforce-addr -msse3 from your CFLAGS and re-emerge cryptsetup?

 

No, removed those flags and emerge'd cryptsetup again, still the same error.

 *pkerwien wrote:*   

> Since your are using the LUKS extension, I assume you should install cryptsetup-luks. The latest stable amd64 in portage is cryptsetup-luks-1.0.4-r3.

 

I also thought that in the first place, but LUKS is included in the cryptsetup package now, since version 1.0.5. I tried anyways with cryptsetup-luks, because you never know... But it didn't help :/

----------

## pkerwien

 *Quote:*   

> I also thought that in the first place, but LUKS is included in the cryptsetup package now, since version 1.0.5. I tried anyways with cryptsetup-luks, because you never know... But it didn't help :/

 

OK. Thanks for the info. I will now upgrade to cryptsetup-1.0.5...

----------

## pkerwien

FYI: Your command works on my Gentoo amd64 with kernel 2.6.23.12 on a plain IDE drive. But I'm not using hardened. My CFLAGS="-march=nocona -O2 -pipe".

Will now try to create a logical drive and see if I can create some errors...

----------

## Hu

Try building sys-fs/cryptsetup using the -hardenednopiessp variant of gcc.  If that produces a working binary, try again with -hardenednossp and -hardenednopie, then report back with which ones produced a working cryptsetup and which ones failed.

----------

## 102039

 *Hu wrote:*   

> Try building sys-fs/cryptsetup using the -hardenednopiessp variant of gcc.  If that produces a working binary, try again with -hardenednossp and -hardenednopie, then report back with which ones produced a working cryptsetup and which ones failed.

 

Hi,

have tried that, no change, still segfaulting :/

 *Quote:*   

> Will now try to create a logical drive and see if I can create some errors...

 

It works on a x86, i have one machine running, same useflags, same setup...and it worked perfectly. Only difference to this machine is amd64 and the cflags.

----------

## pkerwien

Forgot to post my results:   :Embarassed: 

Your cryptsetup command worked for me on my logical volume of 100GB (the volume group consists of 2 x 60GB harddrives). (I'm not sure if the LVM terms are correct. I'm a LVM n00b).

----------

## 102039

Hi pkerwien,

thanks for your tests, i guess it must be some kernel setting or cflag then, maybe a library used by cryptsetup must not be emerged with "-msse3" or "-march=athlon64. "-fforce-addr" seems to be ok, because i also use it on the x86 system, which works. I am totally clueless at the moment :/ 

Anyone else who has an idea ?

----------

## pkerwien

I can try to re-compile my whole testsystem first with -msse3 and then -fforce-addr and see what's happen.

----------

## 102039

Would be awesome if you could do this, i could also drop you my kernel config, so you can check that if the cflags make no problem. It would be great if i could get this finally working  :Smile: 

Big thanks for your help, very much appreciate it !

----------

## pkerwien

Could not reproduce any problem with the -msse3 flag and then later with the -msse3 + -fforce-addr flag.

----------

## 102039

Thanks for testing again! I sent you the my kernel config file, maybe you can try with that again. If that doesn't help i need to give up on the issue :/

----------

## pkerwien

Tested with your kernel config + hardened-sources-2.6.23-r4, just changed CPU from Athlon64 to Core 2. Still no problem with the cryptsetup luksFormat command.

----------

## 102039

Some great news...i tested it with the latest SVN version 1.0.6-pre1 of cryptsetup and it works! Thanks to Clemens Fruewirth

Thanks pkerwien again, for running all those tests anyway! I am glad it is running now   :Smile: 

----------

