# lvm2 and / on raid5 [Solved]

## grooveman

Okay... after a very exhaustive (and exhausting) search on the forums and on the net and countless hours of trial and error, I still have not found any help on this (bear in mind that the wiki is still down, though much of the work I have tried is based upon google cached pages).

My system will not boot.  Grub loads, but it always tells me "File not found" when it goes to look for my kernel.  It just isn't seeing the raid device.

/boot is setup on raid1

swap is setup on raid5 via a vg

/ is setup on raid5 (10GB) via a vg

/var is setup on raid5 (6GB) via a vg

I have compiled in my kernel:

```
General Setup

      [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support
```

```
  Loadable Module Support

      [*] Enable loadable module support

                [*]   Module unloading          

                [*]     Forced module unloading

                [*]   Module versioning support 

                [*]   Source checksum for all modules

                [*]   Automatic kernel module loading
```

```
  Device Drivers

    Block Devices

      <*> RAM disk support       

                  (16)  Default number of RAM disks          

                 (16384) Default RAM disk size (kbytes)   

                  (1024) Default RAM disk block size (bytes)
```

```
  Multi Device Support (RAID and LVM)

     [*] Multiple devices driver support (RAID and LVM) 

                  <*>   RAID support                                

                  < >     Linear (append) mode                  

                  <*>     RAID-0 (striping) mode                 

                  <*>     RAID-1 (mirroring) mode             

                  < >     RAID-10 (mirrored striping) mode (EXPERIMENTAL)      

                  <*>     RAID-4/RAID-5/RAID-6 mode                                   

                  [*]       Support adding drives to a raid-5 array                    

                  < >     Multipath I/O support                                           

                  < >     Faulty test module for MD                         

                  <*>   Device mapper support                              

                  [ ]     Device mapper debugging support              

                  < >     Crypt target support                                 

                  <*>     Snapshot target (EXPERIMENTAL)              

                  < >     Mirror target (EXPERIMENTAL)                  

                  < >     Zero target (EXPERIMENTAL)                    

                  < >     Multipath target (EXPERIMENTAL)              

                  < >     I/O delaying target (EXPERIMENTAL)   

                  < >     Bad Block Relocation Device Target (EXPERIMENTAL)   
```

I used genkernel to make my initrd like so:

```
genkernel --dmraid --lvm2
```

Booted to the live CD and chrooted:

```
# cat /proc/mdstat

Personalities : [raid0] [raid1] [raid6] [raid5] [raid4]

md3 : active raid5 sda3[0] sdc3[2] sdb3[1]

      471716352 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md2 : active raid5 sda2[0] sdc2[2] sdb2[1]

      15647104 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

md1 : active raid1 sda1[0] sdb1[1]

      72192 blocks [2/2] [UU]

unused devices: <none>
```

```
# vgs

  VG   #PV #LV #SN Attr   VSize   VFree

  vg1    1   1   0 wz--n-  14.92G      0

  vg2    1   3   0 wz--n- 449.86G 393.86G

```

```
# lvs

  LV    VG   Attr   LSize  Origin Snap%  Move Log Copy%

  swap  vg1  -wi-ao 14.92G

  slash vg2  -wi-ao 10.00G

  var   vg2  -wi-ao  6.00G
```

My fstab:

```
/dev/md1                /boot           ext3            noauto,noatime  1 2

/dev/vg2/slash          /               ext3            noatime         0 1

/dev/vg2/var            /var            ext3            noatime         0 1

/dev/vg1/swap           none            swap            sw              0 0

/dev/cdrom              /mnt/cdrom      audo            noauto,ro       0 0

#/dev/fd0               /mnt/floppy     auto            noauto          0 0

```

Now... my grub.conf... I have tried a MILLION different kernel parameters... but none work.  I'll show a couple:

grub.conf:

```
default 0

timeout 3

splashimage=(hd0,0)/grub/splash.xpm.gz

title=Gentoo Linux  (2.6.x) (64 bit)

root (hd0,0)

kernel /bzImage udev dolvm2 lvmraid=/dev/md3 root=/dev/ram0 real_root=/dev/vg2/slash

initrd /initramfs-genkernel-x86_64-2.6.22-gentoo-r9

```

```

...

title=Gentoo Linux  (2.6.x) (64 bit)

root (hd0,0)

kernel /bzImage udev dolvm2 lvmraid=/dev/md3 root=/dev/ram0 lvm2root=/dev/vg2/slash

initrd /initramfs-genkernel-x86_64-2.6.22-gentoo-r9
```

I likewise I have subsituted numbers [0-5] for md# and from ram# and I have tried a few other desperate options I picked up from around the web.. No dice.

I have nothing in my /etc/mdadm.conf, as it states that I need nothing, and none of the docs state to put anything in it.

Please, I could use some help here!

Thanks!

G

(BTW, I know that "/" on a raid5 is not recommended, but I know it is possible, and it is what I am after.  Thanks!)

----------

## wardred

See: http://www.sabayonlinux.org/forum/viewtopic.php?t=6050

apparently genkernel 3.4.7 may have a bug.  I don't know about newer versions.  Sigh.  I may have to downgrade my genkernel.

(Are you getting the error about lvm.conf when running genkernel?)

----------

## likewhoa

did you setup grub on both drives that are on the mirror array?

i.e

```

>grub root (hd0,0)

>grub root (hd1,0)

>quit

```

I don't have much experience with genkernel as i only tested it once and never used it again because at the time it didn't support crypted /.

hope this helps.

----------

## grooveman

 *wardred wrote:*   

> See: http://www.sabayonlinux.org/forum/viewtopic.php?t=6050
> 
> apparently genkernel 3.4.7 may have a bug.  I don't know about newer versions.  Sigh.  I may have to downgrade my genkernel.
> 
> (Are you getting the error about lvm.conf when running genkernel?)

 

No, no errors.  Went off without a hitch, in fact. My version of Genkernel is sys-kernel/genkernel-3.4.9_pre6.

 *likewhoa wrote:*   

> did you setup grub on both drives that are on the mirror array?
> 
> 

 

Yes, I most definitely did my grub on both.  

I really think the problem here has to do with the ramdisk... I'm pretty confident I have everything in the kernel that needs to be there.  I don't think the ramdrive is getting made, but I have no way of finding out.  If it is getting made, then the raid modules aren't loaded to it.

Although, it could be possible I am missing something else...

I am not married to genkernel.  If there is a better way, I would love to hear it.  Surely someone has done this...

----------

## grooveman

Eureka!

We have boot!

I switched to lvm2create_initrd as was menioned in this wiki:

http://gentoo-wiki.com/HOWTO_Install_Gentoo_on_an_LVM2_root_partition#bash_fails_on_libncurses.so.5

and did the "ncurses" fix, as he stated.

My problem, I'm ashamed to admit, was a bad symlink to my kernel image in /boot

BUT -- it isn't all my fault. I checked for this specifically, but was misled to believe it was fine.

It used to be that when doing an "ls" (with --color implied through an alias as has always been the case in the gentoo baselayout), bad symlinks were always blocked with a big, flashy, ochre-colored background to warn you that a simlink is bad.  It isn't doing that... they must have changed the dircolors for this baselayout...  it showed it as a happy cyan.. which always used to mean that the link was good.  An "ls -l", however, did show correctly that the link was to "bzIMage" and not "bzImage", as it should have been...

We are now happy  :Smile: 

----------

## likewhoa

funny that was my second suggestion but was gonna save it for next reply  :Very Happy: 

FYI that usually happens when you emerge grub without /boot being mounted.

----------

## mcc666

 *grooveman wrote:*   

> 
> 
> I switched to lvm2create_initrd as was menioned in this wiki:
> 
> http://gentoo-wiki.com/HOWTO_Install_Gentoo_on_an_LVM2_root_partition#bash_fails_on_libncurses.so.5
> ...

 

I've tried 1k different solutions - without success yet!

Is there any chance to collect dmesg logs for further analysis?

best regards,

MCC

----------

