# Best supported RAID controller for SCSI drives?

## peaceful

I'm speccing out a 2U database server for my workplace and would like to select a good RAID controller to connect all my SCSI disks to.

Priorities for the RAID controller:

- stable under gentoo linux

- supports at least 6 SCSI drives (I'll be using RAID 1+0)

- high performance

- under US$1,000

- the following features fully supported in linux without requiring some ancient custom version of the kernel

```
I want...

  to know when something is wrong

  automatic Hot Swap allocation on volume degrade

  to blink and unblink drives (to find them),

  replace bad drives while running

  to be able to upgrade newly inserted drives to Hot Swap status

  to be able to shut off the beeper
```

The only experience I've had with hardware raid under linux is the Adaptec 2130SLP.  Not a good experience.  I had to use a custom 2.6.4 Suse kernel with adaptec's patches to get it working at all, and then the only CLI management feature that I could get working after setting the array up was getting individual drive lights to blink.  Adaptec support was unresponsive, and their driver support sucked.

From the comments on the latest BSD 3.8 song (http://www.openbsd.org/lyrics.html), I'm leaning towards towards LSI/AMI MegaRAID controllers (specifically the MegaRAID® SCSI 320-2E) -- though I'm disappointed that they don't have any Gentoo or generic linux kernel drivers--just drivers for specific Suse and RH versions.  I'd really like to hear some feedback from people who know better than me.  I'd rather avoid the 'find a specific kernel version from another distro and patch it' process.

What's out there that's good?  Is there anything that actually has drivers that I can install with a Gentoo or vanilla kernel?  Will the MegaRAID controllers do all that I need? Are there some other vendors I should be considering?  Am I asking the wrong questions?  Talk to me, people!

----------

## peaceful

Soooo, no one has any positive experience with RAID cards?

I'm now leaning more towards the Adaptec 2230 SLP.  It seems that Adaptec has started trying to support more generic linux setups recently.

http://adaptec.com/worldwide/product/proddetail.html?sess=no&language=English+US&prodkey=ASR-2230S&cat=%2fTechnology%2fSCSI%2fUltra320+SCSI+PCI-X+HBAs+%26+RAID

Anyone have one of those running?

----------

## statare

Without using hardware raid my self, I have seen 3ware mentioned in this forum as a good vendor with linux support. Search the forum for more 3ware info.

----------

## peaceful

 *statare wrote:*   

> Without using hardware raid my self, I have seen 3ware mentioned in this forum as a good vendor with linux support. Search the forum for more 3ware info.

 

I checked out 3ware.  They appear to have only PATA and SATA raid products (no scsi).

----------

## sentenza

Hey, you got the 2130SLP working on SuSE??? I'am actually totally in panic as i just received 2 big servers with 2 of these controllers in each and can't get anything working at all... The servers are 4xOpteron with 8G ram and 24 HD to run Oracle, so you'll certainly understand i'm quite under pressure from management  :Wink:  I've actually managed to boot, format, and start install, but it will always hang with an "offline device" error. I got the same results with gentoo 2005.1.

I'm actually thinking on going to LSI too, as they have acquired the technology of the famous Mylex controllers. I think that you don't need special drivers for them, they are directly in the kernel (LSI Logic New Generation RAID Device Drivers / LSI Logic Legacy MegaRAID Driver). I don't have them for the moment, as we're still investigating if can get those f***ing Adaptec working... If i get one, i'll post my comments.

However, regarding you're Idea on the 2230, if i were you, i won't do this : the driver for the linux kernel is the same and i don't think you'll get more from the utilities than with 2130. If you have any trick for installing SuSE (SLES9) on the 2130, i'll apreciate.

Cheers

----------

## peaceful

 *sentenza wrote:*   

> Hey, you got the 2130SLP working on SuSE??? I'am actually totally in panic as i just received 2 big servers with 2 of these controllers in each and can't get anything working at all... The servers are 4xOpteron with 8G ram and 24 HD to run Oracle, so you'll certainly understand i'm quite under pressure from management  I've actually managed to boot, format, and start install, but it will always hang with an "offline device" error. I got the same results with gentoo 2005.1.
> 
> I'm actually thinking on going to LSI too, as they have acquired the technology of the famous Mylex controllers. I think that you don't need special drivers for them, they are directly in the kernel (LSI Logic New Generation RAID Device Drivers / LSI Logic Legacy MegaRAID Driver). I don't have them for the moment, as we're still investigating if can get those f***ing Adaptec working... If i get one, i'll post my comments.
> 
> However, regarding you're Idea on the 2230, if i were you, i won't do this : the driver for the linux kernel is the same and i don't think you'll get more from the utilities than with 2130. If you have any trick for installing SuSE (SLES9) on the 2130, i'll apreciate.
> ...

 

Do you know if the LSI drivers that come in the kernel are any good?  That would be interesting to know...

I actually got the 2130SLP working with a specific Suse 2.6.4.something kernel, but the rest of the system is pure Gentoo.  It took me more than two weeks to get the machine booting off the array itself -- don't ask me how.  I'll not be doing that again--it's worth the extra few bucks to stick in a couple SATA drives to boot off of and backup the boot system, and mount the RAID array afterwards.

As far as how I got the kernel/drivers working, I downloaded adaptec's most current drivers at the time (which consisted of specific sets of patches against specific Suse/RHE kernels), and then downloaded Suse & RH kernels until I got one where the patches succeeded (that was painful).  Then I manually configured, compiled, and installed the patched kernel.

Anyway, my general advice without knowing more about the problem you're running into would be:

1) Update the RAID card's firmware to the latest version if you haven't already (never assume the firmware is up-to-date on a purchased card)

2) Install a working linux system on some regular HD first (I assume you're doing Suse since Oracle will 'Just Work' then)

3) Once you have a real bootable system, do the special kernel/driver thing and mount the RAID array to your favorite spot in the filesystem (/raid for example)

If you are doing that and have some more specific info about the errors you encounter, post them...

----------

## sentenza

 *peaceful wrote:*   

> 
> 
> Do you know if the LSI drivers that come in the kernel are any good?  That would be interesting to know...
> 
> I actually got the 2130SLP working with a specific Suse 2.6.4.something kernel, but the rest of the system is pure Gentoo.  It took me more than two weeks to get the machine booting off the array itself -- don't ask me how.  I'll not be doing that again--it's worth the extra few bucks to stick in a couple SATA drives to boot off of and backup the boot system, and mount the RAID array afterwards.
> ...

 

Just got that damned SuSE installed... Don't boot yet because the drivers are not in the initrd, but i'll fix this tommorow... I just can go to the sata disk because there's no space left in the chassis (It's quite hard to get a chassis with 24 HDs  :Wink:  ) and i'm stucked with SuSE or RH because otherwise you can't get any real support from Oracle...

Regarding driver quality, from megaraid_mbox.c (kernel 2.6.12)  :

```

/*

 *

 *                      Linux MegaRAID device driver

 *

 * Copyright (c) 2003-2004  LSI Logic Corporation.

 *

 *         This program is free software; you can redistribute it and/or

 *         modify it under the terms of the GNU General Public License

 *         as published by the Free Software Foundation; either version

 *         2 of the License, or (at your option) any later version.

 *

 * FILE         : megaraid_mbox.c

 * Version      : v2.20.4.6 (Mar 07 2005)

 *

 * Authors:

 *      Atul Mukker             <Atul.Mukker@lsil.com>

 *      Sreenivas Bagalkote     <Sreenivas.Bagalkote@lsil.com>

 *      Manoj Jose              <Manoj.Jose@lsil.com>

```

I think you might expect good drivers  :Wink: 

LSI has always been quite friendly to OSS and Linux, and i know that the Mylex are supported in Linux since at least 2.0 kernels. This is an example of what one can call a linux friendly company. Regarding the CLI, i don't know what you can expect, but have a look at lsi site http://www.lsilogic.com/products/megaraid/ and i think you'll learn more.

Cheers

----------

## peaceful

> I just can go to the sata disk because there's no space left in the chassis (It's quite hard to get a chassis with 24 HDs  )

Hmm, that is going to be a pain.  Have fun trying to boot off of that thing.  Been there.

> i'm stucked with SuSE or RH because otherwise you can't get any real support from Oracle... 

That's what I assumed.  We're using Postgresql, so I was able to stick with Gentoo.

> I think you might expect good drivers  

> LSI has always been quite friendly to OSS and Linux, and i know that the Mylex are supported in Linux since at least 2.0 kernels. This is an example of

> what one can call a linux friendly company. Regarding the CLI, i don't know what you can expect, but have a look at lsi site http://www.lsilogic.com/

> products/megaraid/ and i think you'll learn more. 

Interesting.  I've been all over their pages before, but I had some pretty discouraging email correspondence with their "tech support" people about whether or not I could get their drivers (from their site) working on a Gentoo or vanilla kernel.  I pretty much got "WE SUPPORT REDHAT AND SUSE" as a response from them, and we're a Gentoo shop.  Just for that comment, I think I'll see if they'll send me an evaluation unit...

----------

## sentenza

 *Quote:*   

> Hmm, that is going to be a pain. Have fun trying to boot off of that thing. Been there.

 

I'm fighting with the SuSEs sinc the 5.x series, so i hope to solve this just another problem in so many   :Wink: 

 *Quote:*   

> That's what I assumed.  We're using Postgresql, so I was able to stick with Gentoo.

 

That's what i'm doing for my mysql servers...

 *Quote:*   

> Interesting.  I've been all over their pages before, but I had some pretty discouraging email correspondence with their "tech support" people about whether or not I could get their drivers (from their site) working on a Gentoo or vanilla kernel.  I pretty much got "WE SUPPORT REDHAT AND SUSE" as a response from them, and we're a Gentoo shop. 

 

Do you know many hardware manufacturer that SUPPORT gentoo??? Nothing bad about gentoo itself, but frankly i don't know how they could do it. Gentoo is daily changing, and it's for sure impossible to really support gentoo. Don't mean things won't work, but there is real difference between having a working system and supporting it. Like it or not (personaly i don't) but the only mainstream distributions for enterprise solutions are RH and suse, and i think we're talking about "enterprise class" hardware here. 

That the hard way of gentoo or other "alternative" distributions, you can't get the best of both coporate and comunity sides. If you want to see if some hardware will work under gentoo, i think it's better to first check if it's supported directly in the kernel, then check the forums to see if someone has problems with it, and perhaps search google or LKML for a wider audience. Proprietary, binary drivers are crap, but that's not the case of LSI's or adaptec's. They open the sources, and try to get them as quick as possible in the mainline kernel (not so easy). They deliver their compiled drivers mainly for people comming from M$, or $$IX that don't wan't to ever launch a f***ing make command. If you choosed gentoo, that's certainly not your case  :Smile: 

 *Quote:*   

> Just for that comment, I think I'll see if they'll send me an evaluation unit...

 

Good luck on that one   :Laughing: 

On another topic, i was just wondering why you want to go to raid10? I know that looking at raw numbers, raid10 will beat quite any other solution except raid0, but you don't want to bet your job on a hard drive failure, do you? Raid 10 is perhaps what i'll have done 2 years ago, but i know more today regarding databases, and they are really a different thing. Generally, the most you have disks (logical not physical) the most performances you'll get. 

For example, with 8 physical disks, you'd better setup 4 raid1 than 2 raid10, and spread your tables on those subsystems, but this depends mostly on your application(s). Databases are about access time, and disk availibility, ie: your db should wait a minimum for your disks, and this is generally better achieved by spreading data on multiple disk subsystems than having a big powerfull 300MB/s array: i assume your users will never put a 300MB/s pressure on your DB (or you really nead a SAN, not a raid controller), but they could more easily put a 10000 read/s pressure. This will always be better handeld with more pointers (ie:logical heads) than with less but more powerfull pointers.

Just my 2 eurocents.

----------

## peaceful

 *Quote:*   

> Do you know many hardware manufacturer that SUPPORT gentoo??? Nothing bad about gentoo itself, but frankly i don't know how they could do it. Gentoo is daily changing, and it's for sure impossible to really support gentoo. Don't mean things won't work, but there is real difference between having a working system and supporting it. Like it or not (personaly i don't) but the only mainstream distributions for enterprise solutions are RH and suse, and i think we're talking about "enterprise class" hardware here. 

 

Heh, here's what I actually asked and got as a response:

```

Sir,

NO.

Thank you,

LSI Support-01

LSI Logic - Field Support Technician 

support@lsil.com 

http://www.lsilogic.com/downloads/selectDownload.do

Technical Support 678-728-1250 

-----Original Message-----

Do you support (or assist, or facilitate) download and configuration  

of the RH/Suse kernels for those who don't have paid-for RH/Suse  

subscriptions?
```

Not the most helpful response.  I assumed from that response that unless I paid RH/Suse money, they wouldn't support their own hardware.  Notice that I never even mentioned Gentoo.  They ignored my followup email after that one where I asked why they don't have drivers available for download for the vanilla linux kernel.  It wasn't until you mentioned it that I found out there were any built-in drivers.

As for expecting "supporting Gentoo" in general, when talking about device drivers for hardware, you're really talking about either 1) supporting the vanilla linux kernel (which is an available choice on Gentoo), and/or 2) supporting gentoo-sources.  No, I don't know a hardware manufacturer that supports gentoo-sources, although I don't think that is any harder than supporting other distro's kernels.  I DO know manufacturers who support the vanilla kernel, however (lots, obviosly).  Not supporting the vanilla kernel seems like a bad-move for a hardware company.

Of course, now that you pointed out that there are built-in drivers in the vanilla kernel, maybe they DO support the vanilla kernel, and the support guy who corresponded with me was clueless.

----------

## pactoo

I do not have any exact model anymore, but the ServeRAID Controller from IBM worked like a charm, even on brand new machines (those who where brand new a year ago, that is). Had various of those from the old UW up to UW160 in use and not a single, controller related problem. 

The stand alone pre-install configuration disk, which is also used for BIOS updates, is based on linux, the monitoring/maintenance tool is java and also runs (X needed). Both are available for free (beer? speech?) from IBM, though not easy to find, as the homepage is quite messy. 

Had quite a few of those in use with Redhat AS2.1 and ES3.0, both Kernel 2.4. based. I doubt support has gotten worse with Kernel 2.6, but no practical experience here

----------

## sentenza

 *pactoo wrote:*   

> I do not have any exact model anymore, but the ServeRAID Controller from IBM worked like a charm, even on brand new machines (those who where brand new a year ago, that is). Had various of those from the old UW up to UW160 in use and not a single, controller related problem. 
> 
> The stand alone pre-install configuration disk, which is also used for BIOS updates, is based on linux, the monitoring/maintenance tool is java and also runs (X needed). Both are available for free (beer? speech?) from IBM, though not easy to find, as the homepage is quite messy. 
> 
> Had quite a few of those in use with Redhat AS2.1 and ES3.0, both Kernel 2.4. based. I doubt support has gotten worse with Kernel 2.6, but no practical experience here

 

We also have multiple IBM ServeRAID here, and i can confirm that they are REALLY GOOD controllers. Get one with BBU so that you can enable Write Cache and still sleep at night. I would much have liked to use these again, but can buy them anymore with our standard reseller    :Mad: 

----------

## peaceful

Aha!  I knew there had to be some other options out their.

I'll go look at IBM's ServeRAID stuff.  Has anyone used these on a Gentoo system??

----------

## pactoo

And the vortex controllers, at least the older ones, did have a very good reputation and were among the first to be supported by linux, AFAIK. The first ServeRAID Generation also used the vortex driver. I think. However, I never had practiacal experience with those. 

Haven't heard anything about their recent hardware and Linux,  just ask google. Vortex as well as IBM do way better RAID hardware than adaptec, whos usable SCSI products can IMHO be completely be reduced to their 29xx line of controllers. 

Again, all my Info is about a year or more older, haven't got any contact to scsi since then.

----------

## linuxtuxhellsinki

& old Compaq SmartArray Controllers has been working as well   :Smile: 

----------

## peaceful

 *linuxtuxhellsinki wrote:*   

> & old Compaq SmartArray Controllers has been working as well  

 

Do you have them running on a Gentoo box?

----------

## linuxtuxhellsinki

 *peaceful wrote:*   

>  *linuxtuxhellsinki wrote:*   & old Compaq SmartArray Controllers has been working as well   
> 
> Do you have them running on a Gentoo box?

 

Of course  :Wink:    These are with some older Proliants, SmartArray 2DH for example works with 'cpqarray' driver like I think most of those Compaq's array controllers.

I've also some other Mylex & HP NetRaid controllers but haven't got time to test 'em, but I googled also some positive info from 'em and checked from kernel's doc dir that they're supported.

----------

## peaceful

Ok, so here's the list of vendors I've distilled:

LSI MegaRAID

Adaptec

IBM RaidServe (Vortex?)

Compaq SmartArray

Mylex

HP NetRaid

Next question: Are their any REVIEWS of these controllers?  I googled a bit, and for the life of me could not find any product reviews/comparisons by anybody.

----------

## sentenza

 *peaceful wrote:*   

> Ok, so here's the list of vendors I've distilled:
> 
> LSI MegaRAID
> 
> Adaptec
> ...

 

I'm not sure about this, but i think that:

- today Mylex==LSI

- ServeRaid!=Vortex , or at least "ICP Vortex" (german company, owned by adaptec, see: http://www.icp-vortex.com)

- The only Compaq SmartArray i used, was a f***ing s**t for performance... But that was under Windows 2k in a Raid5 config, so not revelant for your scenario.

What's clear from your scenario, is that you'll be doing raid1 or raid10, so you don't need a controller with much processing power. It is mainly the speed of your disks that will determine the overall speed of your raid system. What you need is a well supported controller, WITH A BATTERY UNIT, so you'll be able to use write cache and gain much write performance. 

Perhaps the most important factor for you seems to be the capabilities of the userland management tools. Have a look at them and choose accordingly.

----------

## peaceful

 *Quote:*   

> What's clear from your scenario, is that you'll be doing raid1 or raid10, so you don't need a controller with much processing power. It is mainly the speed of your disks that will determine the overall speed of your raid system. What you need is a well supported controller, WITH A BATTERY UNIT, so you'll be able to use write cache and gain much write performance. 
> 
> Perhaps the most important factor for you seems to be the capabilities of the userland management tools. Have a look at them and choose accordingly.

 

Good advice.  Unfortunately, it's hard to tell what the userland management tools are like without first purchasing and installing the hardware.

Does anyone have any advice about userland raid utils?

----------

## pactoo

ServeRaid is a family (or line) of Raidcontroller from IBM and early models used the vortex driver, afaik. Probably, because they all use the 960 (ppc? intel?) chip and the vortex where first to be supported. However, I still may be wrong about the driver. I possibly mix up mylex and vortex. 

Second, a battery cache has no performance impacts, it just makes sure, that data stored in the cache of the controller during a powerloss is also present after powering on again. The controller then will finish the write request and minimise data corruption. Hopfully.  

The writechache of the disks is usually disabled and replaced by that on the controller. That may still be one of the advantages of SCSI over IDE Raid, where you usually do not turn off the disk write cache and the controller has no knowledge, of what has for sure been written to disk and what is still volatile.

Btw.: Does anybody know, how to turn off the write cache on SATA disks at all? Independend of RAID or possible performance loss?

----------

## Galahad

I have a smartarray running on gentoo, even booting off of it, but that I don't think is a good idea, it's a PITA to get to work and I have to reboot to 2.4 and reinstall lilo every time I want to upgrade the kernel.

GDT Vortex works find, Mylex DAC960 should work, too, but I didn't get that to work, always hardlocked the system.

----------

## sentenza

 *pactoo wrote:*   

> 
> 
> Second, a battery cache has no performance impacts, it just makes sure, that data stored in the cache of the controller during a powerloss is also present after powering on again. The controller then will finish the write request and minimise data corruption. Hopfully.  
> 
> 

 

Well, it depends on the importance of your data...   :Wink: 

Write cache is a real advantage as it gives time to your controller to cache your writes in his internal memory, and if possible "re-arrange" them before sending to disk in one shot. This provides a huge writespeed boost due to less head move and biger block used. At each requested write by the kernel (or your application), the controller will acknowledge the write as soon as the write is in his cache, however it's not on disk. In a database, the software EXPECT that the data is on disk, and to ensure this it will generally open your datafile in O_SYNC mode, so that the end of a write operation occurs when the data really is on disk (ie: bypassing the kernel cache). 

Yes, you can always turn on write-cache on your controller, but doing this with no BBU is just insane if you care about your data, ie: a single powerloss could totally scratch your files or your filesystem in this scenario.

----------

