# Reassembling RAID [Solved]

## daemonflower

Hi all,

I recently upgraded my pretty old kernel to 2.6.16. In this course, I switched the SATA drivers. Don't ask me from what to what, but the upshot of it is that my SATA disks are now known as /dev/sda and /dev/sdb instead of hde and hdg. This is all right with me, because this has been the standard for a long time.

I had a RAID-1 on hde2 and hdg2, though. This is not recognized correctly any more. It comes up, but it consists of only one disk. I edited /etc/raidtab to replace the devices, but apparently that was not enough. 

The relevant portion of dmesg output is *Quote:*   

> md: raidstart(pid 5847) used deprecated START_ARRAY ioctl. This will not be supported beyond July 2006
> 
> md: could not open unknown-block(33,2).
> 
> md: autorun ...
> ...

 Three questions.

How do I add /dev/sdb2 back to the RAID?

On second thought, I rather had a RAID-5 instead of a RAID-1, so I can expand it more easily if I buy more disks in the future. Can I do that in place, without moving the data off?

I'm a bit worried about the "deprecated START_ARRAY ioctl". Does anybody know the implications of this? In the worst case I fear that my RAID will not work after July 2006...Any pointers to good HOWTOs, which help me to figure out the issues, welcome. 

mdadm --examine gives me *Quote:*   

> /dev/sda2:
> 
>           Magic : a92b4efc
> 
>         Version : 00.90.00
> ...

 Thanks,

HelgeLast edited by daemonflower on Sat Apr 22, 2006 9:52 am; edited 1 time in total

----------

## NeddySeagoon

daemonflower,

 *Quote:*   

> md: raidstart(pid 5847) used deprecated START_ARRAY ioctl. This will not be supported beyond July 2006

 

means this ioctl is planned to be removed from the kernek at that date, it will not suddenly stop working for you.

 *Quote:*   

> md: could not open unknown-block(33,2).

  Is /dev/hde2, so your /dev/sda2, in your raid is your old /dev/hdg2

It looks like your persistant raid superblock 'knows' about the IDE major, minor device numbers that compose the raid.

I can't think of a safe way of fixing that, nor can I explain why one partition comes up ok.

----------

## daemonflower

 *NeddySeagoon wrote:*   

>  *Quote:*   md: raidstart(pid 5847) used deprecated START_ARRAY ioctl. This will not be supported beyond July 2006 
> 
> means this ioctl is planned to be removed from the kernek at that date, it will not suddenly stop working for you.

 Hm, if I use a kernel without START_ARRAY ioctl and a raidstart which uses this, it will not work, will it? I was a bit worried wether the devs will upgrade raidtools in time, I guess. There are so many professional interests behind working RAID support that I should not think anything will go wrong there though.

 *NeddySeagoon wrote:*   

>  *Quote:*   md: could not open unknown-block(33,2).  Is /dev/hde2, so your /dev/sda2, in your raid is your old /dev/hdg2
> 
> It looks like your persistant raid superblock 'knows' about the IDE major, minor device numbers that compose the raid.

 I can confirm that, and go with your conclusion.

----------

## paulisdead

wow, looks like we're about in the same boat, I've got a similiar problem here https://forums.gentoo.org/viewtopic-t-453154.html.  I found the raidreconf command, and I think that might be what we need to transfer the array to different device nodes in /dev, I just can't seem to find confirmation that that's the correct way of doing things.

----------

## daemonflower

Phew, raidreconf seems dangerous. It hasn't even a --pretend option to see what's gonna happen   :Smile: 

I might use it though. The manpage looks promising, but is quite terse. Is there a more comprehensive tutorial about it somewhere, a forum, or anything where I can find others' experience about it?

----------

## overkll

 *Quote:*   

> How do I add /dev/sdb2 back to the RAID? 

 

```
mdadm /dev/md0 -a /dev/sdb2
```

or stop the array and reassemble it:

```
mdadm -S /dev/md0

mdadm -A /dev/md0 /dev/sda2 /dev/sdb2
```

If that fails to cooperate, then you will probably have to recreate the array.  I've had this problem on occassion with my raid 5 array.  I was able to add the device (or a new device) back to the array, but it would be missing on reboot.  Recreating the array adds the device back to the array and resyncs the array, with the exisiting data entact.  I'd make a backup before hand just to make sure.  See this thread for a recent real life example.

I'd also check the logs for any disk errors/failures.  There could be a good reason why the array member was dropped, hopefully not badblocks.

```
mdadm -C /dev/md0 --level=1 --raid-devices=2 /dev/sda2 /dev/sdb2
```

should recreate your array.

 *Quote:*   

> On second thought, I rather had a RAID-5 instead of a RAID-1, so I can expand it more easily if I buy more disks in the future. Can I do that in place, without moving the data off? 

 

No, raid 1 and raid 5 are different beasts, and you need a minium of 3 disks for raid 5.

If you haven't already, you may want to consider the following:

Use mdadm instead of raidtools

use /etc/mdadm.conf instead of /etc/raidtab

build your raid levels into your kernel - arrays will be autodetected upon startup/boot

----------

## daemonflower

 *overkll wrote:*   

> 
> 
> ```
> mdadm /dev/md0 -a /dev/sdb2
> ```
> ...

 Cool, that was what I'm looking for. I am not very familiar with RAIDs and I was not certain whether assembling a RAID can be done only once and whether it will result in data loss. I will do that as soon as I can go to single user mode. My RAID is mounted on /var, so I can't do that while running. 

On the backups... backing up 250 GB to DVDs is a real pain. OTOH, I'm running on a single disk now, so I'm inviting disaster (a mirror was backup enough for me before). Thinking about it, best thing to do would be to buy another drive and copy the data there...

 *overkll wrote:*   

> 
> 
> ```
> mdadm -C /dev/md0 --level=1 --raid-devices=2 /dev/sda2 /dev/sdb2
> ```
> ...

 Now I'm a little confused. From the manpage I would understand that -C creates a new array. mdadm -A is not enough?

Man, those man pages really aren't very comprehensible. Can't anyone recommend me a good tutorial on this, or at least some place that explains the terms used?

 *overkll wrote:*   

> No, raid 1 and raid 5 are different beasts, and you need a minium of 3 disks for raid 5.

 Ah, now I remember why I didn't create a RAID-5 in the first place...

Thanks!

----------

## overkll

 *Quote:*   

> Cool, that was what I'm looking for. I am not very familiar with RAIDs and I was not certain whether assembling a RAID can be done only once and whether it will result in data loss. I will do that as soon as I can go to single user mode. My RAID is mounted on /var, so I can't do that while running. 

 

and

 *Quote:*   

> Now I'm a little confused. From the manpage I would understand that -C creates a new array. mdadm -A is not enough? 

 

The first command example I gave you does what is known as a "hotadd" - adding a device to an existing array, while it is running.  The second example was reassembling the array.  AFAIK, this doesn't actually change the vital raid data that is located in the raid superblock which is used to auto assemble the array on boot.

At the end of your "mdadm --examine" output, you have

```
Number Major Minor RaidDevice State

this 1 34 2 1 active sync /dev/hdg2

0 0 33 2 0 active sync /dev/hde2

1 1 34 2 1 active sync /dev/hdg2
```

when it should be /dev/sda2 and /dev/sdb2.  Recreating the array would change it to the correct drives, assembling it won't.

But like I said before, I'd make a backup of the raid partition data first, just in case.  If all you have is /var on the raid device, just back that up.  You got 250GB there?  If so, find a local retail outlet that will accept returns (like Fry's), buy a large disk, back it up, do your repair and return the new disk.  :Wink: 

 *Quote:*   

> On the backups... backing up 250 GB to DVDs is a real pain. OTOH, I'm running on a single disk now, so I'm inviting disaster (a mirror was backup enough for me before). Thinking about it, best thing to do would be to buy another drive and copy the data there... 

 

Raid should never be a replacement for backups.  If one gets file corruption, one is screwed.    Raid will do its job and propagate the corruption.  IMHO, raid is mainly for redundancy in case of disk failure and/or for speed gains, not for backup.  You are right, one is better off using one disk for the live filesystem and a cron job utilizing rsync to duplicate the live file system to a second disk.  I do this on all my linux systems, even the raided ones, and it has saved my ass more than once!

 *Quote:*   

> Man, those man pages really aren't very comprehensible. Can't anyone recommend me a good tutorial on this, or at least some place that explains the terms used? 

 

LOL!  I agree completely!  I'vd found undocumented alternative switches:  -D for --detail, -E for --examine and -Q for --query.  Googling "mdadm raid howto" should bring up something useful.  I found a very comprehensive howto when I switched from raidtools/raidtab to mdadm/mdadm.conf but don't remember the site.

Good luck!

----------

## daemonflower

 *overkll wrote:*   

> But like I said before, I'd make a backup of the raid partition data first, just in case.  If all you have is /var on the raid device, just back that up.  You got 250GB there?  If so, find a local retail outlet that will accept returns (like Fry's), buy a large disk, back it up, do your repair and return the new disk. 

 Heh, now that's an idea that hasn't occured to me before! I'll look into that.  :Smile: 

 *overkll wrote:*   

> Raid should never be a replacement for backups.  If one gets file corruption, one is screwed.    Raid will do its job and propagate the corruption.  IMHO, raid is mainly for redundancy in case of disk failure and/or for speed gains, not for backup.  You are right, one is better off using one disk for the live filesystem and a cron job utilizing rsync to duplicate the live file system to a second disk.  I do this on all my linux systems, even the raided ones, and it has saved my ass more than once!

 I totally agree with you. I keep my vital files in a subversion repository which is rsynced nightly to two different remote servers. I have data though, which are less than vital, but still important to me. I just can't currently afford to buy another 250 gig drive, and certainly can't afford a remote backup. There's always the tradeoff between Bang and Buck...

I've had drives failing on me on several occasions, and it has always been a pain to restore the backups. Never really went as expected. I think a RAID can save you trouble in case of hardware failure, not in case of "user failure". I may have to think again. Maybe two separate disks really give you more flexibility. Hmm. I'll have a while to think about it. I won't touch that RAID before I've got a spare drive.

Thinking about it, can I fuck up anything if I reformat my second drive, which currently isn't used for the RAID, and manually mirror the data there? Just for the time till I have a "real" solution? I can't see a problem there off the top of my head, but maybe you can?

----------

## overkll

I use a daily cron job script (rsync) to replicate the live file system except for /etc/fstab, /etc/mtab, /dev, /proc, and /sys.  I also exclude /usr/portage/distfiles/*, /var/tmp/portage/* and the runtime pid files.  I then have an grub entry that will boot from the backup disk if necessary.  Power off, select the backup boot grub entry and voila, back up and running in no time in case of any hd/file corruption issues.  I can then use the live backup system to resolve any issues and restore the primary system while essential services are basically unaffected  :Wink: 

The partitions (/boot, swap, /, /home) are identical to the main drive.  I only need to make sure that /etc/fstab on the backup disk is modified to successfully boot and mount the backup partitions instead of the main disk's partitions.

Is that what you mean by manually mirroring?

----------

## daemonflower

Not exactly, nothing so involved. I just think about duplicating the /var tree to the second disk.

What I worry about is: can I destroy data on the first RAID disk by formatting the second RAID disk? I can't imagine how, but hey, I've been outimagined by Real Life(TM) at times. 

So, do you have any experience with that procedure?

----------

## overkll

 *Quote:*   

> What I worry about is: can I destroy data on the first RAID disk by formatting the second RAID disk?

 

As long as the disk/partition is not part of a running array, no.  Softraid (md) is smart enough to know that the active partition (sda2) is newer and wouldn't use the data from the older, out-of-sync partition (sdb2).  You check the status of the array with "cat /proc/mdstat".

I reread this thread again and I came across something I feel needs to be addressed.

 *Quote:*   

> Cool, that was what I'm looking for. I am not very familiar with RAIDs and I was not certain whether assembling a RAID can be done only once and whether it will result in data loss.  

 

There is no limit on the number of times an array is "assembled".   MD devices are assembled every time one boots.  If it needs to be done manually, that's definitely possible.

 *Quote:*   

> I will do that as soon as I can go to single user mode. My RAID is mounted on /var, so I can't do that while running.

 

Au Contraire!  One can hotadd, hotremove and mark faulty with a live array/filesystem.  MD devices and the data they contain are even available while they are in the process of assembly/syncing!  One can even format a MD device while it is being created/reconstructed!

As far as the good howto, see this site.  It's a little outdated but contains plenty of good information.  It should be compulsory reading for anyone administering softraid.

----------

## daemonflower

 *overkll wrote:*   

> As long as the disk/partition is not part of a running array, no.  Softraid (md) is smart enough to know that the active partition (sda2) is newer and wouldn't use the data from the older, out-of-sync partition (sdb2).

 Good. It wouldn't make much sense otherwise either. Just wanted to make sure.

 *overkll wrote:*   

> There is no limit on the number of times an array is "assembled".   MD devices are assembled every time one boots.  If it needs to be done manually, that's definitely possible.

 I confused the terminology there. I wondered about reconstruction, not about assembling. I see clearer now.

From the RAID-HOWTO I learned that reconstruction should be painless and safe with the command 

```
raidhotadd /dev/md0 /dev/sdb2
```

I guess the mdadm commands you mentioned earlier, 

```
mdadm /dev/md0 -a /dev/sdb2; mdadm -C /dev/md0 --level=1 --raid-devices=2 /dev/sda2 /dev/sdb2
```

does the same for mdadm? I'm still not sure whether "create" is the same as "reconstruct". I guess I'll just use raidhotadd.

----------

## overkll

Basically, there are two packages for managing raid - raidtools and mdadm.  AFAIK, raidtools came first, but mdadm is preferred by many.  Even can send you an email when there is trouble.  There was a section call "Why mdadm?" in that howto mentioned in my previous post.

raidhotadd is from raidtools and is the equivelant of mdadm <mddevice> -a /dev/<partition>.  Both commands basically restore the raid device, syncing the new partition with the raid device.  Didn't mean to confuse the issue with the reconstrution term.

In my experience, sometimes the hotadd will only work while the machine is up and upon subsequent booting the device that was hotadded will need to be hotadded at every boot.  That's where the create option comes in.  I've used it to fix the issue I've just described.  As long is the md device is running and in degraded mode (partion missing but still functioning as raid device) the create function will add the partition back into the array, rewrite the correct superblock raid info and sync the disc with the existing data from the degraded array.  The array will need no more intervention.

When I refer to the "create" function, I'm referring to the "mdadm -C" command not the equivelant raidtools command.  Don't know about that one since I haven't tried/used it in this manner.

----------

## daemonflower

Hah. So I finally went ahead and did it:

```
mdadm /dev/md0 -a /dev/sdb2
```

and it worked perfectly, started to recreate the RAID right away and finished a couple of hours later.

So, I guess, much ado about very little.  :Shocked: 

Still, better safe than sorry, and in the process I learned a thing or two about RAIDs and mdadm. Thanks for guiding me!

----------

## overkll

You're very welcome.  Just giving back to the community.

The forums are one of the best things about gentoo...  that and portage!   :Smile: 

----------

## daemonflower

You're right. Every time I post a question on a non-Gentoo forum, I am reminded of the high quality of the Gentoo forums. And portage is the next major step in the evolution of mankind since the invention of writing   :Very Happy: 

----------

