# recommendations for server hd encryption

## turtles

I am looking for recommendations for performance encryption for a /var partition on a server. any thoughts?

----------

## Hu

That depends in part on how much traffic will be on /var.  If it is mostly logs, then performance is unlikely to matter much.  If it will hold a database, then performance could matter.  However, disks are generally considered slow, so most database engines will try to avoid hitting the disk whenever possible, which would again decrease the need for a high performance cipher.

DM-Crypt is the preferred in-kernel solution for encryption.  Within that, you can use a variety of ciphers and use (or not use) LUKS for key management.  If you want an out-of-kernel option, TrueCrypt is available.

----------

## turtles

I need it for a Postgresql / Apache server.

I plan on using dm-crypt with LUKS.

The /var partition is 1.4 TB in RAID 5.

The server has a U320 SCSI which I take to mean 320/mb sec throughput on the RAID controller. 

from lspci:

```
05:04.0 SCSI storage controller: Adaptec AHA-3960D / AIC-7899A U160/m (rev 01)

05:04.1 SCSI storage controller: Adaptec AHA-3960D / AIC-7899A U160/m (rev 01)

```

Google turned up:

http://blog.wpkg.org/2009/04/23/cipher-benchmark-for-dm-crypt-luks/

and found a csv file in this comment  which 

I sifted a comparison of setups I found here down to 256 bits. 

I am assuming anything under 256 bit is not very secure.

Then I eliminated insecure cypher modes.

Then sorted it by the last column mb/sec. 

Here is what I think is the top 20 in 256 bit, it looks like a huge performance hit down to 123 mb/sec from 320.

cipher,       chipher Mode,   iv alg, bits, create time, write time, read time, write throughput mb/sec

aes	xts	plain	256	5.52	9.42	8.31	108.7	123.23

twofish	cbc	plain	256	5.44	10.71	8.5	95.61	120.47

twofish	cbc	benbi	256	5.52	10.75	8.71	95.26	117.57

twofish	cbc	essiv:rmd128	256	5.43	10.91	8.74	93.86	117.16

twofish	cbc	essiv:wp256	256	5.45	11.09	8.77	92.34	116.76

twofish	cbc	essiv:tgr128	256	5.47	11.01	8.86	93.01	115.58

twofish	cbc	essiv:rmd256	256	5.46	10.89	8.91	94.03	114.93

twofish	xts	benbi	384	6.11	10.31	8.93	99.32	114.67

twofish	xts	plain	256	5.47	10.38	8.96	98.65	114.29

twofish	cbc	essiv:md5	256	5.44	10.95	8.97	93.52	114.16

twofish	cbc	essiv:sha256	256	5.46	10.94	9	93.6	113.78

twofish	cbc	essiv:md4	256	5.45	11.06	9.02	92.59	113.53

twofish	xts	plain	384	6.05	10.2	9.02	100.39	113.53

twofish	xts	benbi	512	6.74	10.25	9.04	99.9	113.27

twofish	xts	benbi	256	5.51	10.31	9.08	99.32	112.78

twofish	xts	plain	512	6.78	10.14	9.09	100.99	112.65

aes	xts	benbi	384	6.18	10.33	9.17	99.13	111.67

aes	xts	benbi	256	5.49	10.27	9.24	99.71	110.82

aes	cbc	benbi	256	5.55	11.78	9.38	86.93	109.17

Is that how I should be looking at this data?

Thanks

----------

## Hu

I suggest measuring the actual throughput on the unencrypted block device, rather than trusting the number in the hardware markings.  Similarly, you should test a few of those ciphers on your hardware.  Encryption performance is affected by both CPU speed and the presence or absence of hardware assistance for the cipher.  For example, if you have a CPU which can use AES-NI, you may see a significant performance boost for AES relative to other ciphers.

----------

## broken_chaos

 *turtles wrote:*   

> I am assuming anything under 256 bit is not very secure.

 

For practical purposes, 128-bit is just fine. Even 112-bit (such as 3DES) is currently unbreakable. Unless you have a very specific reason to need 256-bit keys (such as employer policy, government requirements, etc.), don't throw out sub-256-bit keys as a possibility if they meet your performance requirements.

Unless you have a very specific reason you need significantly longer keys, my off-the-cuff suggestion would be to start at aes-cbc-essiv:sha256 with a 128-bit key and see if that fits your needs.

----------

## turtles

 *Hu wrote:*   

> I suggest measuring the actual throughput on the unencrypted block device, rather than trusting the number in the hardware markings.  Similarly, you should test a few of those ciphers on your hardware.  Encryption performance is affected by both CPU speed and the presence or absence of hardware assistance for the cipher.  For example, if you have a CPU which can use AES-NI, you may see a significant performance boost for AES relative to other ciphers.

 

Thanks all I took both of your advice since I am not in a hurry I ran some tests. I figured i would post back in case anyone else is interested.  Here is what I have found for a test for measuring actual through put on the unencrypted device.

Read test:

```
dd if=/dev/zero of=/dev/sda1 bs=64k count=100k

102400+0 records in

102400+0 records out

6710886400 bytes (6.7[code] [/code]GB) copied, 73.863 s, 90.9 MB/s

```

Write test:

```

dd of=/dev/null if=/dev/sda1 bs=64k count=100k

102400+0 records in

102400+0 records out

6710886400 bytes (6.7 GB) copied, 35.5298 s, 189 MB/s

```

Here are some results I managed to get by running a script on the unformatted /var RAID array after booting the live CD.

Not sure if the live CD had all the kernel modules I wanted to try.

```
            

                        key         Read         Write

none   NA   NA         NA          90.9          189

aes   xts   plain         256         72.52         37.48

twofish   cbc   benbi      256         65.85         N/A

aes   xts   plain         256         60.16         35.78

twofish   cbc   benbi      128         58.85         46.33

twofish   cbc   benbi      256         56.92         35.18

aes   xts   plain         384         55.26         46.04

aes   xts   plain         256         51.12         40.88

aes   cbc   essiv:sha256   128         49.37         42.86

aes   xts   plain         256         48.01         52.41

aes   xts   plain         512         46.74         44.87

aes   cbc   essiv:sha256   192         45.69         53.31

aes   xts   plain         512         41.66         N/A

aes   cbc   essiv:sha256   128         40.72         61.84

aes   cbc   essiv:sha256   128         39.83         53.98

aes   cbc   essiv:sha256   128         39.81         50.57

aes   cbc   essiv:sha256   128         39.81         50.57

aes   cbc   essiv:sha256   192         38.11         50.37

aes   cbc   essiv:sha256   256         35.97         45.01

aes   xts   plain         384         34.75         36.73

aes   cbc   essiv:sha256   256         31.5         34.26

```

Some of the tests ran more than once like the default mode aes-cbc-essiv:sha256 for LUKS with a 128-bit key.

I from what I can tell from this the Xeon 3GHz, 2M cpu may support AES if the bios was upgraded. My guess it it was. It is still a big hit for encryption and a bigger hit for essiv not as much for benbi..

I may play around with it some more.

----------

## depontius

There are also "Full Disk Encryption" hard drives.  The AES engine is in the drive electronics, and key management is typically through the BIOS.

No performance or CPU hit.

----------

## gentoo_ram

What's the real goal here for encrypting a "server" drive, especially a /var partition?  The information will be unencrypted to all of the apps on the server.  So if someone gets into your server over the network, they are still going to be able to steal your un-encrypted data even though it is encrypted on the drive.  I can only think of two scenarios where encrypting data on a server drive might be useful:

1.  It's data that's only temporarily un-encrypted for a short time, then unmounted.  This would help protect against the "valuable" data being taken over the network.  Clearly this isn't your goal here.

2.  Data stored encrypted on the drive in case someone physically takes the server and its hard drives.  Maybe this is your goal?  Not sure.

But in scenario two, I'd think data on a laptop or something easy to steal makes more sense to encrypt than a server.

----------

## cach0rr0

 *gentoo_ram wrote:*   

> What's the real goal here for encrypting a "server" drive, especially a /var partition?  The information will be unencrypted to all of the apps on the server.  So if someone gets into your server over the network, they are still going to be able to steal your un-encrypted data even though it is encrypted on the drive.  I can only think of two scenarios where encrypting data on a server drive might be useful:
> 
> 1.  It's data that's only temporarily un-encrypted for a short time, then unmounted.  This would help protect against the "valuable" data being taken over the network.  Clearly this isn't your goal here.
> 
> 2.  Data stored encrypted on the drive in case someone physically takes the server and its hard drives.  Maybe this is your goal?  Not sure.
> ...

 

no. just, no. 

instead of going into a long diatribe as to the various use cases for disk encryption on a server, or hotlinking a dozen stories of lost confidential data (e.g. medical or financial information) here's a simplistic question that, unless you answer honestly, is of little utility: with the purchases you've made online, would you feel more comfortable if your payment information were stored on an a)an encrypted volume, or b)an unencrypted volume

Now ponder the various ways disks make their ways into places they dont belong; sometimes an external threat, many times an internal threat. Compound that with a home server where you don't have the multiple layers of physical security keeping people from getting at your data. 

there are also those cases where someone with the requisite legal authority can seize your property. At least here, there is legal precedent preventing you from being compelled to give up your cryptographic keys. 

that a server is less mobile than a laptop is of minuscule significance when objectively evaluating the reasons for/against deciding to encrypt. The justifications for wanting to encrypt are still *exactly* the same, the only thing that's variable is the ease/probability of the data being lifted. 

And why /var ? Among other things, because /var/log and /var/lib go there. 

Personally I'm a fan of nothing on my entire disk being discernable other than a handful of LUKS headers, but that's me.

----------

## titanofold

A more realistic test is to use pgbench to test the database.

I ran several tests using that on both encrypted and unencrypted. My best results were with Btrfs file system for read-only tests. (I only did read only because 99% of my queries are SELECTs. INSERT once, read many.)

Unencrypted I got about 120 TPS (transactions per second), encrypted I got about 100 TPS. I ended up having to use Ext4, though, becuase the Btrfs tool couldn't recognize the LVM as being something it could format. Ext4 with encryption is about 90 TPS. I used Twofish. I don't remember precisely the other options I gave to cryptsetup.

I wrote these down on a paper somewhere.

Anyway, what ended up happening is that most of the database on a smaller initialization set would get buffered in RAM. So, the size of the database and what actually gets queried will affect the TPS results as well.

----------

## cach0rr0

 *titanofold wrote:*   

> My best results were with Btrfs file system for read-only tests. 

 

dang...strange. How recent a kernel/btrfsprogs? I actually ended up dropping btrfs because of its performance with my crypto (serpent-xts-benbi), ended up with ext4. But, that was around ~2.6.34 or 2.6.36. For me this was just one large volume where my KVM guest images reside.

It was bad enough to the point of unusable. Measuring disk throughput from within the guest, it was frequently as slow as a few hundred kB/s. 

now that it's ext4 - same disk - there's degradation, as you'd expect, but it's the "normal" level of degradation.

----------

## titanofold

 *cach0rr0 wrote:*   

>  *titanofold wrote:*   My best results were with Btrfs file system for read-only tests.  
> 
> dang...strange. How recent a kernel/btrfsprogs?

 

This was maybe two months ago. So, definitely 3.0+ for the kernel.

----------

## turtles

Update results from a newer i7:

```
/root/crypt_test.sh 

Options                                Write     Read

-c aes-xts-plain -s 256               333.55   216.49

-c aes-cbc-essiv:sha256 -s 128        232.20   208.55

-c aes-xts-plain -s 384               288.45   187.20

-c aes-cbc-essiv:sha256 -s 192        259.24   186.86

-c aes-xts-plain -s 512               227.56   167.32

-c aes-cbc-essiv:sha256 -s 256        198.83   161.77
```

----------

