# RAID Setup [SOLVED]

## scottneh

I'm tryingto set up a RAID set with two hard drive that will be used for data from simulations.  I don not want to install linux on the RAID set is is strictly for data.  I have emerged dmraid and emerged device-mapper.  I created a RAID set in the BIOS and 

```
dmraid -s
```

 discovers the raid set, notes that it is type mirrored and is ok.  

```
dmraid -r
```

 shows both disks and notes that they are ok.  however when I type

[code]dmraid -ay

result

ERROR: device-mapper target type "mirror" not in kernel[code]

I have added RAID support in my kernel and turned on mirroring.  Anyone know where else this error might come from?[/code]Last edited by scottneh on Wed Sep 09, 2009 6:06 am; edited 1 time in total

----------

## energyman76b

yeah, using crap.

I am sorry, but why don't you use md raid?

----------

## NeddySeagoon

scottneh,

There are several sorts of raid supported by the kernel and you seem to have two sorts mixed up.

dmraid is also known as fakeraid. As you mention setting up the raid in the BIOS, you appear to be using this.

The in kernel support is not complete, you also need to emerge dmraid to get the third party kernel modules you need.

The kernel also provides kernel raid, which is a more robust, more mature and often faster raid solution.

Using this, you partition the drives and form the partitions into raid sets with the mdadm tool.

Kernel support for this sort of raid is in the kernel. No third party packages are required.

For the sake of completeness, the kernel supports some hardware raid cards and Windows Dynamic Disks, which is Microsofts version of software (kernel) raid.

----------

## scottneh

ok, so I deleted the BIOS/hardware raid set and have started to work with mdadm.  I partitioned the two drives and changed their ID to "fd" then issued the commands 

```
cd /dev && MAKEDEV md

mdadm --create --verbose --/dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
```

 but the command returns the error 

```
mdadm: error opening /dev/md0: No such device or address
```

Is ther something I need to do to create md0 before using mdadm?

----------

## energyman76b

a) makedev was wrong

b) you need to have all the md stuff you plan to use in the 

edit: humpf, I was wrong, makedev was right. Hmpf. Sorry.

----------

## Mike Hunt

For consistency, better use 1 (one) right through.

```
mknod /dev/md1 b 9 1

mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
```

In the kernel

```
Multi-device support (RAID and LVM)  --->

[*] Multiple devices driver support (RAID and LVM)

  <*>   RAID support

  < >     Linear (append) mode (NEW)

  < >     RAID-0 (striping) mode

  <*>     RAID-1 (mirroring) mode
```

----------

## scottneh

still get the same error mknod works butthe mdadm --create still fails.

I have emerged mdadm and I have the kernel configured as noted.  I do not have a /proc/mdstat file 

I saw some posts talking about loading the raid1 module but it looks like most of the descriptions for creating raid sets talks about doing it before building the OS verses after.

----------

## energyman76b

if you do ls -lh /dev/md* what do you get?

----------

## scottneh

```
ls -lh /dev/md*

returns

ls: access denied /dev/md*: No such file or directory

```

----------

## scottneh

sorry, that was premature.

```
mknod /dev/md1 b 9 1

ls -lh /dev/md* 

returns 

brw-r--r-- 1 root toot 9, 1 Sep 6 07:34 /dev/md1
```

----------

## energyman76b

ok, and when that device exists

mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1

without the -- in front of /dev/md0

oh, and change the permissions, try rwrw

----------

## scottneh

same thing

when I enter the command 

```
mdadm mdadm --create --verbose /dev/md1 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1 

returns

mdadm: error opening /dev/md1: No such device or address

```

Is there something i need to do to in order to make the device exist?  I have entered the previous commands...cc 

```
mknod /dev/md1 b 9 1 

cd /dev && MAKEDEV md 

mdadm --create --verbose --/dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
```

----------

## scottneh

Is raidtools still used or is mdadm the only thing needed for a 2.6 kernel?  The gentoo guide for setting up raid calls out raidtools or raidtools2 for a 2.4 kernel.

----------

## energyman76b

you only need mdadm.

so you made makedev - and have you checked that the md devices are really there afterwards? that they are at least rw/rw//?

----------

## scottneh

How do I check that they are really there? and set the permissions?

----------

## energyman76b

ls -lh /dev/md*

chmod 660 /dev/mdX

with X=0,1,2,3,....

----------

## scottneh

```

ls -lh /dev/md*

brw------- 1 root root 9, 0 Sep  6 12:46 /dev/md0

brw------- 1 root root 9, 1 Sep  6 12:46 /dev/md1

chmod 660 /dev/md1 

ls -lh /dev/md*

brw------- 1 root root 9, 0 Sep  6 12:46 /dev/md0

brw-rw---- 1 root root 9, 1 Sep  6 12:46 /dev/md1

```

----------

## energyman76b

could you change the ownership to root disk and retry?

----------

## scottneh

I'm working as root and it looks like the ownership of the disk is root

I entered

```
chown root /dev/md1
```

----------

## energyman76b

 *scottneh wrote:*   

> I'm working as root and it looks like the ownership of the disk is root
> 
> I entered
> 
> ```
> ...

 

chown root:disk /dev/mdX

try that

----------

## scottneh

```
chown root:disk /dev/md1

ls -h /dev/md1

returns

brw-rw---- 1 root disk 9, 1 Sep 6 12:46 /dev/md1

```

```
mdadm --create --verbose /dev/md1 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1 

returns 

mdadm: error opening /dev/md1: No such device or address 

```

----------

## energyman76b

so strange - but you do have support for md raid in the kernel? is there something in dmesg?

----------

## scottneh

I don't see anything in dmesg.

One thing i have as an issue and I'm not sure if this is the problem.

In my BIOS I have SATA configured as RAID instead of IDE and a hardware RAID set is online.  If I leave the SATA setting as RAID in the bios and delete the RAID set configuration then I can't boot from either my SATA CD-ROMS, they don't show up in the boot priority list.  If I configure SATA as IDE then the machine won't boot.  However, the IDE bus has two hard drives on it and one is the bootdrive.  The SATA drives are only used for data.

It seems like the mdadm --create command wouldn't care about this as far as creating /dev/md1 is concerned.

----------

## energyman76b

oh great.

you really have to turn off that bios-raid stuff. It is fake. And it really fucks up stuff.

----------

## scottneh

oh hell...that means I probalby have to change a buinch of settings.  

I deleted the RAID set and the mdadm tool still doesn;t work so I'll probably have to set the SATA devices to IDE and see what else I have t ochange.

----------

## scottneh

Yep, If I set SATA to IDE, I lose my IDE drive in the boot priority menu and can only boot from one of the first SATA drive.  My OS is built onto the IDE drive.  I could probably get away with just installing the MBR and grub on the SATA drive

----------

## scottneh

ok got rid of all the SATA RAID stuff in the BIOS.  New issue, I can't see any of my SATA drives now. in order to boor properly i had to change the order of my drives.

Originally, (with SATA set as RAID) my three SATA drives were sda, sdb and sdc.

After I set SATA to IDE, I had to rearrange my drives so I can see the drive with the MBR in the boot priority.  so now it is 

IDE primary = sda

IDE secondary = sdb

the SATA drives should be 

SATA 1 = sdc

SATA 2 = sdd

SATA 3 = sde

If I try to mount the SATA drives I get an error that they don't exist and fdisk can't open them either.

Is there a command to see what hard drives linux sees and what they are called?

----------

## scottneh

ok, set SATA devices to AHCI and everything is back to normal.  No RAID selection in BIOS at allAll devices are visible.  

However, mdadm still does not work.  I still get the same error about /dev/md1 does not exist.

----------

## scottneh

One thing I can;t do is load modules md and raid1.  I'm not sure if I should have to or not though.  I have those set to be built into the kernel.  There is no /proc/mdstat file as well

----------

## Mike Hunt

Did you try modprobe dm-mod

Do you have device-mapper support in the kernel?

```
Multi-device support (RAID and LVM)  --->

[*] Multiple devices driver support (RAID and LVM)

  <*>   RAID support

  < >     Linear (append) mode (NEW)

  < >     RAID-0 (striping) mode

  <*>     RAID-1 (mirroring) mode

  < >     RAID-10 (mirrored striping) mode (EXPERIMENTAL) (NEW)

  < >     RAID-4/RAID-5 mode (NEW)

  < >     RAID-6 mode (NEW)

  < >     Multipath I/O support (NEW)

  < >     Faulty test module for MD (NEW)

  <*>   Device mapper support

```

Look at the nice  raid wiki here.

----------

## scottneh

I tried modprobe dm-mod but it comes back with a fatal error noting that the module does not exist.  Here is the portion of my kernel for RAID

```
Multi-device support (RAID and LVM)  ---> 

[*] Multiple devices driver support (RAID and LVM) 

  <*>   RAID support 

  [*]      Autodetect RAID arrays during kernel boot

  < >     Linear (append) mode (NEW) 

  < >     RAID-0 (striping) mode 

  <*>     RAID-1 (mirroring) mode 

  < >     RAID-10 (mirrored striping) mode (EXPERIMENTAL) (NEW) 

  < >     RAID-4/RAID-5/RAID-6 mode

  < >     Multipath I/O support (NEW) 

  < >     Faulty test module for MD (NEW) 

  <*>     Device mapper support 

  [*]     Device mapper debugging support 

  <*>     Crypt target support

  <*>     Snapshot target 

  <*>     Mirror target 

  <*>     Zero target

  <*>     Multipath target

  <*>     I/O delaying target (EXPRIMENTAL)

  [*]     DM uevents [EXPERIMENTAL]

  <*>     Bad Block Relocation Device Target (EXPERIMENTAL)
```

I read smewhere that it was best to build in all the options under the Device mapper support.

----------

## NeddySeagoon

scottneh,

Device Mapper is for fake (BIOS) raid. Its not needed for kernel raid.

Its harmless - just kernel bloat.

mdadm is used for kernel raid. 

For kernel raid  partition your drives identically and set the partition types to the partitions that will be donated to raid sets to fd.

fdisk -1 will show you something like

```
/sbin/fdisk -l /dev/sda

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes

255 heads, 63 sectors/track, 121601 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Disk identifier: 0x0553caf4

   Device Boot      Start         End      Blocks   Id  System

/dev/sda1   *           1           5       40131   fd  Linux raid autodetect

/dev/sda2               6          70      522112+  82  Linux swap / Solaris

/dev/sda4              71      121601   976197757+   5  Extended

/dev/sda5              71         724     5253223+  fd  Linux raid autodetect

/dev/sda6             725      121601   970944471   fd  Linux raid autodetect
```

Thats one drive from my system

boot is a four way raid1 (or it won't boot) swap is what it says - I have four identical swaps.

sda5 is part of a raid5 set for root

sda6 is part of the raid5 under lvm2 for everything else.

udev will not make the /dev/mdX nodes unless the raid is assembled and you can't assemble it until you have a raid set.

The way round this is to make the mdX nodes you need by hand the first time. 

```
mknod /dev/md0 b 9 0
```

gets you started with /dev/md0

```
mknod /dev/md0 b 9 1
```

is  /dev/md1  and so on.

The permissions and ownerships will be wrong. They need to be chmod root:disk and chmod 660

----------

## scottneh

yeah, I have done all that and still no go.  I don't have an mdstat file, and can't modprobe raid1, dm-mod, or any of the other packages.  I have the support built into my kernel but its like there is still no support there.  Are there any known conflicts with other device drivers in the kernel regarding this?  I suppose if there is nothing else that I need to install then there must be some kind of kenel conflict that is prevent the support from being built in.

----------

## Mike Hunt

When you rebuilt your kernel with raid support, did you then copy it to /boot?

Was /boot mounted when you copied it over?

----------

## scottneh

Yeah, boot was mounted in fstab and i did copy it over.  I just rebuilt it yesterday just to make sure.

----------

## Mike Hunt

Please post your /etc/fstab and the output of

```
df -h
```

----------

## scottneh

I haven't mounted the two drives or built filesystems on them yet.  From the way the HOWTO's were reading it looked like you did that after creating hte raid set.

df -h

```
Filesystem            Size  Used Avail Use% Mounted on

/dev/sdd3             147G  6.2G  133G   5% /

udev                   10M  108K  9.9M   2% /dev

/dev/sda1             138G  1.1G  130G   1% /programs

shm                   3.9G     0  3.9G   0% /dev/shm

```

fstab

```
 /etc/fstab: static file system information.

#

# noatime turns off atimes for increased performance (atimes normally aren't

# needed; notail increases performance of ReiserFS (at the expense of storage

# efficiency).  It's safe to drop the noatime options if you want and to

# switch between notail / tail freely.

#

# The root filesystem should have a pass number of either 0 or 1.

# All other filesystems should have a pass number of 0 or greater than 1.

#

# See the manpage fstab(5) for more information.

#

# <fs>                  <mountpoint>    <type>          <opts>          <dump/p$

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.

/dev/sdd1               /boot           ext2            noauto,noatime  1 2

/dev/sdd3               /               ext3            noatime         0 1

/dev/sdd2               none            swap            sw              0 0

/dev/sda1               /programs       ext3            auto,noatime    0 2

#/dev/sdb1              /data           ext3            auto,noatime    0 2

#/dev/sdc1              /data           ext3            auto,noatime    0 2

#/dev/md1               /data           ext3            auto,noatime    0 2

/dev/cdrom              /mnt/cdrom      auto            noauto,ro       0 0

#/dev/fd0               /mnt/floppy     auto            noauto          0 0

# glibc 2.2 and above expects tmpfs to be mounted at /dev/shm for

# POSIX shared memory (shm_open, shm_unlink).

# (tmpfs is a dynamically expandable/shrinkable ramdisk, and will

#  use almost no memory if not populated with files)

shm                     /dev/shm        tmpfs           nodev,nosuid,noexec    $

```

----------

## Mike Hunt

Well, as you can see,  your /boot partition is not mounted because of the  noauto mount option in fstab.

Which means that your new kernel was not copied to /boot.

Do

```
mount /dev/sdd1 /boot
```

then copy your kernel to /boot and edit your grub.conf to point at your new raid enabled kernel.

Cheers  :Smile: 

----------

## scottneh

huh, I guess I have been working with the original seed kernel all this time...aarrgggg!  But this is why I'm here, to learn...

----------

## scottneh

that did it.  I have created the raid set.

----------

## Mike Hunt

Cool  :Smile: 

----------

## NeddySeagoon

scottneh,

learn about the 

```
uname -a
```

 command.

The date and time is the build date and time of the running kernel. We all forget to mount boot from time to time and its a very quick check of your kernel.

----------

## Mike Hunt

That's good to know.  :Smile: 

----------

## energyman76b

adding your own 'version' suffix helps too...

----------

