# udev-182 HOWTO

## NeddySeagoon

NeddySeagoon's Rough Guide to >=udev-182

WARNING: You might be first to follow this guidecorrections, comments and improvements welcome. 

DISCLAIMER: You follow this guide at your own risk. If your system breaks as a result, you get to keep both pieces.

Allow plenty of time - this is a major change.  For help, post to this thread. >=udev-182 is only in ~ARCH at the time of writing.

If you run ~ARCH you are supposed to know what you are doing.

Credits The Gentoo Wiki which provided much of the insperation and gentoo-wiki.com which I have used for static root on lvm over raid for many years.

Aim of this guide

To keep you in control of the boot process on your system.

You need this guide if you have a separate /usr or /var partition and want to understand and have control of the boot process on your system after the upgrade to >=udev-182

The use of raid and lvm is covered as I had to make that work but encrypted filesystems are not as I have no way to test.

An initrd is required but you build it with the aid of a kernel provided script. Neither dracut nor genkernel are required, well if they were needed you would not

be in control would you?

The idea is to have a system that when it fails to boot, you can fix it yourself.  Its likely that during the executopn of this procedure you will get to practice

that, so have a bootable USB stick, or CDROM handy.  You will not be able to boot into an old kernel.

Prerequisties

A Gentoo system with a separate /usr and or /var partition, optionally, with everything except /boot on lvm on raid.  >=udev-182 should be in your package.mask, or

you have seen emerge wanting to upgrade to it.  If you already have udev-182, you box won't boot again until you make arrangeents to mount /usr and /var before udev

is started.

Overview

Preparation - you can do this now and test it.  These steps are required before you reboot useing you new initrd but they can be tested and debugged before you

make the leap to >=udev-182

Initrd preparation

This can be done but not tested until your have >=udev-182 installed.

The leap to >=udev-182

Going back after this stage is pointless, long term the only way is on.

The reboot

If it doesn't work first time, fetch the boot media you hoped you would not need, get into your chroot and fix it.

Preparation

As the initrd will contain /sbin/mount change /etc/fstab to mount everything by UUID.  This is a robustness improvement you get for free. You cannot yet use a UUID

in grub.conf to mount root.  That's covered later, once the system boots using the initrd.

To disciver your UUIDs run the command blkid

```
$ sudo blkid

Password:

/dev/sda1: UUID="9392926d-6408-6e7a-8663-82834138a597" TYPE="linux_raid_member"

/dev/sda2: UUID="b6633d8e-41ef-4485-9bbe-c4c2d69f4e8c" TYPE="swap"

/dev/sda5: UUID="5e3cadd4-cfd2-665d-9690-1ac76d8f5a5d" TYPE="linux_raid_member"

/dev/sda6: UUID="9657e667-5b60-f6a3-0391-65e6dcf662fa" TYPE="linux_raid_member"

[other drives omitted]

/dev/md125: UUID="741183c2-1392-4022-a1d3-d0af8ba4a2a8" TYPE="ext2"

/dev/md126: UUID="ff5730d5-c28d-4276-b300-5b0b0fc60300" TYPE="ext4"

/dev/md127: UUID="7b2KgY-NHef-kuNk-WBAp-VnLa-h03A-b4ehGy" TYPE="LVM2_member"

/dev/mapper/vg-usr: UUID="fc23601b-e2af-443f-959e-50148154a91b" TYPE="ext4"

/dev/mapper/vg-local: UUID="7e0d0ae4-2c57-44e9-995f-5723ab0670b3" TYPE="ext4"

/dev/mapper/vg-tmp: UUID="1ff5e3b5-c2d8-41a6-b172-e1ede8fe22ff" TYPE="ext2"

/dev/mapper/vg-var: UUID="57b34894-f80d-47ab-a522-46fb6e1a19b8" TYPE="ext4"

/dev/mapper/vg-vmware: UUID="e41a94cc-0817-449f-8ac8-f3055d33879c" TYPE="ext4"

/dev/mapper/vg-opt: UUID="7f0fad34-6130-42d8-8246-033de9717005" TYPE="ext4"

/dev/mapper/vg-distfiles: UUID="54b46458-d411-4ea8-8920-4c5fcdb56a81" TYPE="ext4"

/dev/mapper/vg-packages: UUID="59c06fd1-8471-4efb-b055-8f1cfb39645f" TYPE="ext4"

/dev/mapper/vg-portage: UUID="8675cb9c-4251-489f-8e9b-0244ca80176c" TYPE="ext2"

/dev/mapper/vg-home: UUID="8d5d5691-ceb7-4e58-bed3-28803cb88bfe" TYPE="ext4"

/dev/sde1: SEC_TYPE="msdos" UUID="FC30-3DA9" TYPE="vfat"
```

Notice that every block device has a UUID. Carefully choose the right ones for use in your

```
/etc/fstab
```

In my case

```
# <fs>                                    <mountpoint>    <type>          <opts>          <dump/pass>

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.

#/dev/md125             /boot           ext2            noauto,noatime  1 2

UUID=741183c2-1392-4022-a1d3-d0af8ba4a2a8 /boot           ext2            noauto,noatime  1 2

UUID=8d5d5691-ceb7-4e58-bed3-28803cb88bfe /home           ext4            noatime         1 2

UUID=7f0fad34-6130-42d8-8246-033de9717005 /opt            ext4            noatime         1 2
```

and so on.

Save the changes and test with 

```
mount -a
```

 fix any syntax errors.

The initrd requires some packages to be built with the static USE flag. Edit 

```
/etc/portage/package.use
```

 to include

```
# static bits and pieces for an initrd

sys-fs/lvm2 static

sys-fs/mdadm static

sys-apps/busybox static
```

mdadm and lvm2 are not requried if you do not use kernel raid or Logical Volume Manager. Do not put the static USE flag in /etc/make.conf

busybox will provide the shell in the initrd.

```
emerge -uDNav world
```

 to rebuild packages with the changed USE flags.

>=udev-182 depends on a kernel option

```
    |    [*] Maintain a devtmpfs filesystem to mount at /dev                    | |

  | |    [*]   Automount devtmpfs at /dev, after the kernel mounted the rootfs
```

The Automount is not required as our initrd init script will mount it anyway. If you have a kernel with the automount option missing, this guide will still work.

If you are using kernel raid auto assembly, turn it off

```
[ ]     Autodetect RAID arrays during kernel boot 
```

 This will break rebooting meanwhile.

Rebuild and reinstall your kernel. If you need to reboot, raid autoassembly users cannot reboot into this kernel without following the rest of this guide.

At the outset, I mentioned a kernel provided script. The kernel sources provide the gen_init_cpio and gen_initramfs_list.sh utilities. The gen_init_cpio utility

does not come prepackaged and needs to be built. 

```
cd /usr/src/linux/usr

make gen_init_cpio
```

Check these files are execuatable

```
chmod +x gen_init_cpio

chmod +x ../scripts/gen_initramfs_list.sh
```

Initrd Preparation

With the aid of the kernel scripts, there are two files to prepare.  The init script itself and a list of files to be included in the initrd.

Old habits die hard an I like to prepare the initrd in /root/initrd

The  /root/initrd/initramfs_list contains

```
# directory structure

dir /proc       755 0 0

dir /usr        755 0 0

dir /bin        755 0 0

dir /sys        755 0 0

dir /var        755 0 0

#dir /lib        755 0 0

dir /lib64      755 0 0

dir /sbin       755 0 0

dir /mnt        755 0 0

dir /mnt/root   755 0 0

dir /etc        755 0 0

dir /root       700 0 0

dir /dev        755 0 0

# busybox

file /bin/busybox /bin/busybox  755 0 0

# for raid on lvm

file /sbin/mdadm                /sbin/mdadm              755 0 0

file /sbin/lvm.static           /sbin/lvm.static         755 0 0

# libraries required by /sbin/fsck.ext4 and /sbin/fsck

slink   /lib                            /lib64                          777 0 0

file    /lib64/ld-linux-x86-64.so.2     /lib64/ld-linux-x86-64.so.2     755 0 0

file    /lib64/libext2fs.so.2           /lib64/libext2fs.so.2           755 0 0

file    /lib64/libcom_err.so.2          /lib64/libcom_err.so.2          755 0 0

file    /lib64/libpthread.so.0          /lib64/libpthread.so.0          755 0 0

file    /lib64/libblkid.so.1            /lib64/libblkid.so.1            755 0 0

file    /lib64/libuuid.so.1             /lib64/libuuid.so.1             755 0 0

file    /lib64/libe2p.so.2              /lib64/libe2p.so.2              755 0 0

file    /lib64/libc.so.6                /lib64/libc.so.6                755 0 0

file    /sbin/fsck              /sbin/fsck                      755 0 0

file    /sbin/fsck.ext4         /sbin/fsck.ext4                 755 0 0

# our init script

file    /init                   /root/initrd/init               755 0 0
```

Omit 

```
# for raid on lvm

file /sbin/mdadm                /sbin/mdadm              755 0 0

file /sbin/lvm.static           /sbin/lvm.static         755 0 0
```

if you do not require raid or lvm support.

Use

```
ldd <your_fsck_helper>
```

 to discover the libraries you need if you do not use ext4.

32 bit installs do not have /lib64. See the gentoo wiki for a 32 bit initramfs_list.

If you made your initrd elsewhere, update

```
file    /init                   /root/initrd/init               755 0 0
```

The init script itself

```

#!/bin/busybox sh

rescue_shell() {

    echo "$@"

    echo "Something went wrong. Dropping you to a shell."

    /bin/busybox --install -s

    exec /bin/sh

}

# allow the use of UUIDs or filesystem lables

uuidlabel_root() {

    for cmd in $(cat /proc/cmdline) ; do

        case $cmd in

        root=*)

            type=$(echo $cmd | cut -d= -f2)

            echo "Mounting rootfs"

            if [ $type == "LABEL" ] || [ $type == "UUID" ] ; then

                uuid=$(echo $cmd | cut -d= -f3)

                mount -o ro $(findfs "$type"="$uuid") /mnt/root

            else

                mount -o ro $(echo $cmd | cut -d= -f2) /mnt/root

            fi

            ;;

        esac

    done

}

check_filesystem() {

    # most of code coming from /etc/init.d/fsck

    local fsck_opts= check_extra= RC_UNAME=$(uname -s)

    # FIXME : get_bootparam forcefsck

    if [ -e /forcefsck ]; then

        fsck_opts="$fsck_opts -f"

        check_extra="(check forced)"

    fi

    echo "Checking local filesystem $check_extra : $1"

    if [ "$RC_UNAME" = Linux ]; then

        fsck_opts="$fsck_opts -C0 -T"

    fi

    trap : INT QUIT

    # using our own fsck, not the builtin one from busybox

    /sbin/fsck -p $fsck_opts $1

    ret_val=$?

    case $ret_val in

        0)      return 0;;

        1)      echo "Filesystem repaired"; return 0;;

        2|3)    if [ "$RC_UNAME" = Linux ]; then

                        echo "Filesystem repaired, but reboot needed"

                        reboot -f

                else

                        rescue_shell "Filesystem still have errors; manual fsck required"

                fi;;

        4)      if [ "$RC_UNAME" = Linux ]; then

                        rescue_shell "Fileystem errors left uncorrected, aborting"

                else

                        echo "Filesystem repaired, but reboot needed"

                        reboot

                fi;;

        8)      echo "Operational error"; return 0;;

        16)     echo "Use or Syntax Error"; return 16;;

        32)     echo "fsck interrupted";;

        127)    echo "Shared Library Error"; sleep 20; return 0;;

        *)      echo $ret_val; echo "Some random fsck error - continuing anyway"; sleep 20; return 0;;

    esac

# rescue_shell can't find tty so its broken

    rescue_shell

}

# start for real here

# temporarily mount proc and sys

mount -t proc none /proc

mount -t sysfs none /sys

mount -t devtmpfs none /dev

# disable kernel messages from popping onto the screen

###echo 0 > /proc/sys/kernel/printk

# clear the screen

###clear

# assemble the raid set(s) - they got renumbered from md1, md5 and md6

# /boot

/sbin/mdadm --assemble /dev/md125 --uuid=d678d02e-28ab-84e0-c44c-77eb7ee19756

# don't care if /boot fails to assemble

# /  (root)  I wimped out of root on lvm for this box

/sbin/mdadm --assemble /dev/md126 --uuid=ad5fe0cb-775d-38b4-7169-e221fc96089f || rescue_shell

# if root won't assemble, we are stuck

# LVM for everything else

/sbin/mdadm --assemble /dev/md127 --uuid=52be4797:edab2349:eb21497e:52035eaa || rescue_shell

# and if the LVM space won't assemble there is no /usr or /var so we are really in a mess

# TODO could auto cope with degraded raid operation

# lvm runs as whatever its called as and we need vgchange

ln -s /sbin/lvm.static /sbin/vgchange

# start the vg volume group - we only have one volume group

/sbin/vgchange -ay vg || rescue_shell

# if this failed we have no /usr or /var

# get here with raid sets assembled and logical volumes available

# mounting rootfs on /mnt/root

uuidlabel_root || rescue_shell "Error with uuidlabel_root"

# space separated list of mountpoints that ...

mountpoints="/usr /var"

# ... we want to find in /etc/fstab ...

ln -s /mnt/root/etc/fstab /etc/fstab

# ... to check filesystems and mount our devices.

for m in $mountpoints ; do

#echo $m

    check_filesystem $m

    echo "Mounting $m"

    # mount the device and ...

    mount $m || rescue_shell "Error while mounting $m"

    # ... move the tree to its final location

    mount --move $m "/mnt/root"$m || rescue_shell "Error while moving $m"

done

echo "All done. Switching to real root."

# clean up. The init process will remount proc sys and dev later

umount /proc

umount /sys

umount /dev

# switch to the real root and execute init

exec switch_root /mnt/root /sbin/init

```

Fill in your own mdadm parameters and volume group names.

Remove the mdadm and lvm items from the init script if you don't use mdadm or lvm

Read the comments in the init file - you may need to make other adjustments.

Notice that if you make a mess of your included libraries, there is a 20 second pause then fsck returns as if it had worked but without doing anything.

This avoids the need to get into your chroot to fix your initrd, provided everything else works. Fix it.  Mounting unchecked filesystems is ok for debugging but not

reccomenred.

Mount /boot, so the script gen_initramfs_list.sh can install the initrd there.

Run /usr/src/linux/scripts/gen_initramfs_list.sh -o /boot/initrd.cpio.gz /root/initrd/initramfs_list

Fix grub.conf.  Mount root using the UUID of the root filesystem and add in the initrd line.

```
#3.3.1-gentoo

title=Kernel 3.3.1-gentoo (hd0)

root (hd0,0)

kernel (hd0,0)/3.3.1-gentoo root=UUID=ff5730d5-c28d-4276-b300-5b0b0fc60300 vga=0x317 video=vesafb:mtrr:3,ywrap

initrd /boot/initrd.cpio.gz

title=Kernel 3.3.1-gentoo (hd1)

root (hd1,0)

kernel (hd1,0)/3.3.1-gentoo root=/dev/md126

initrd /boot/initrd.cpio.gz
```

The leap to >=udev-182 

Update udev and anything else that is manually masked due to udev.  Check your  /etc/portage/package.mask

be sure you have >=openrc-0.9.9.3

run

```
 etc-update
```

Reboot to test.

TODO:  Put all the files needed in the initrd in a filesystem in /root/initrd so they don't get messed up by random system updates.

TODO:  Done - Use UUIDs in the mdadm --assemble commands post updated above but initscript examples copy/pasted from a second box.

UUIDs are unique, so readers need to use their own anyay. Not the the UUIDS here are the UUIDS foing using

```
mdadm -E <componet device>
```

not the UUIDs of the filesystem on the /dev/md device.

TODO:  Test with a static /dev in the initrd and DEVTMPFS off in the kernel.  If that works, this becomes a complex static initrd that can be hand rolled with cpio.

--- edit 1 ---

Well that should have booted but not cleanly. The filesystems that the initrd now checks and mounts the normal init system tryes to check and mount.  Edit /etc/fstab to stop that.

```
UUID=57b34894-f80d-47ab-a522-46fb6e1a19b8 /var                    ext4  noatime,noauto    1 0

UUID=fc23601b-e2af-443f-959e-50148154a91b /usr                    ext4  noatime,noauto    1 0
```

The filesystems that are mounted by the initrd need to be set to noauto, so that the normal init system does not attempt to mount them.  Do not remove the entries as the init script in the initrd uses them.

The pass number, thats th final field on the line needs to be set to zero, which means don't check this filesystem.

--- edit 2 ---

Init script updated to use UUIDs in mdadm assemble commands

----------

## ExecutorElassus

upon emerging mdadm, I get the following advice:

```
* If you're not relying on kernel auto-detect of your RAID devices,

 * you need to add 'mdraid' to your 'boot' runlevel:

 * rc-update add mdraid boot

```

Should I heed it?

Which of those directories in the initramfs directory list (the first list of directories) are necessary, and are there others?

emerging busybox results is:

```
* You cannot have USE='static pam'.  Assuming static is more important.
```

Do I need to rebuild busybox, or does this mean that it has automatically dropped the pam USE-flag, and built static?

What is the difference between an initramfs and an initrd? Also, this relates to the graphics options you have on your first kernel option in your grub.conf. I remember explicitly disabling framebuffer support and directfb a couple years ago (due to some problem with the X server which may not even exist anymore; support for Radeon? Xcomposite? I can't even remember =/ ). Is there instruction somewhere where I can set that up, besides the framebuffer wiki that's here?

Why does the "--uuid=..." line for your large lvm use colons innstead of dashes? And what do you mean you wimped out of root on lvm?

What is /mnt/root? I don't have one on my system. Is that / ?

Thanks for the guide. Adventure!

Cheers,

EELast edited by ExecutorElassus on Mon Apr 30, 2012 10:21 pm; edited 1 time in total

----------

## NeddySeagoon

ExecutorElassus,

You need mdadm in the boot runlevel if your root is not on raid and the initrd does not assemble yur raid sets. (It does.)

If you have mdadm set up to monitor your raid sets and send you emails you should add it to the defualt runlevel unless you need it started earlier.

The initrd only assembles your raid sets - it does not start the mdadm service.

Search in menuconfig is your friend.  press / and enter devtmpfs.

Busybox must be built with stactic, as its libraries are not in the initrd. This meanes busybox does not have pam support.

The difference between and initrd and initramfs is one of internal format.  An initrd is an old term the really refers to an ext2 filesystem in a gzipped file.

An initramfs is a compressed cpio archive ove other cpio achives. The compression is user defined at kernel build time ... at least, the decompression support is.

I still use a framebuffer console as I use the nvidia binary blob for Xorg.  You continue to use hwatever you have been using. This initrd/initramfs does not affect any video options.

----------

## ExecutorElassus

Hi Neddy,

okay, so I can leave mdadm to start at default, as previously. 

I found the kernel options. Hurrah!

I haven't been using any console framebuffer at all, so my bootup is an ugly 600x800 startup console. I'd really like it to be … not ugly.

When I run 'make gen_init_cpio' I get this:

```
# make gen_init_cpio

make: `gen_init_cpio' is up to date.

```

Is that normal behavior?

Sorry for all the questions. I'm new at this.

Cheers,

EE

----------

## NeddySeagoon

ExecutorElassus,

The 

```
make: `gen_init_cpio' is up to date
```

means the binary already existed and is newer than the .c and .h files use to build it, so there was nothing to do.

If you are nervous, you can run 

```
make clean

make gen_init_cpio
```

That will remove any binaries before the build starts.

Its normal after the first 

```
make gen_init_cpio
```

in any kernel tree.

----------

## ExecutorElassus

okay, I'm almost at the end of updating your script to fit my system. What is the /mnt/root option you have? Is that what I would call / on my system?

----------

## NeddySeagoon

ExecutorElassus,

The initrd is a little like a boot CD , when you installed originally.

You attached your root partition at /mnt/gentoo to being the install, untarred the stage3 then chrooted into it.

The initrd is the boot CD, your install is the already untarred stage3 and the init script in the initrd lists the steps requited to mount things, check them and get into the chroot. 

The initrd attaches it at /mnt/root because it needs to read /etc/fstab to get the UUIDs of /usr and /var

It also needs to switch_root right at the very end, so your root filesystem has to be mounted somewhere before that.

----------

## ExecutorElassus

Hi Neddy

okay, so in that case, I need to make /mnt/root to serve as the temporary root fs, but that won't affect anything post-mount, correct?

Also noteworthy: the /usr/src/linux/scripts/gen_initramfs_list.sh has to be run from /usr/src/linux/, as its line 298 refers to "usr/gen_init_cpio" as a relative - that is, not absolute - path. 

Cheers,

EE

----------

## NeddySeagoon

ExecutorElassus,

/mnt/root only exists in the initrd.

----------

## ExecutorElassus

Hi Neddy,

ah, okay. then I'll delete that directory once I can mount / again. For the moment, the initrd is not mounting any of the / partitions, and is consequently unable to find the kernel's root (that is, the block device specified by "root=…" in grub.conf). Neither "/dev/md126" (which is the raid device built out of the three mirrored partitions from each of the three HDDd), nor the equivalent UUID of that array device, seem to work. Is there some other UUID I should be giving? Just the UUID from one of the partitions?

Cheers,

EE

----------

## udeved

Hello neddy,

I hope you can help me with not created symlinks in /dev/disk/by-label(uuid) for other linux LVs.

My setup:

everything on lvm except /boot and swap

custom kernel 3.3.4 and 3.2.12, udev 182-r3, openrc 0.9.9.3, lvm 2.0.2.95-r1, genkernel created ramdisk with lvm support

same result with lower version of udev and lvm

Fix for lvm 2.0.2.95-r1

https://bugs.gentoo.org/show_bug.cgi?id=409921#c52

If I boot the system, only symlinks for the gentoo PV's containing LVs  are created in  /dev/disk/by-label(uuid), I assume by udev.

Despite eg /dev/arch/ and /dev/ubuntu/ are present, the /dev/mapper/  only contains the gentoo LVs.

If I start after boot the  lvm service, still no symlinks.

However, if I restart the lvm service, the symlinks are created.

I have basically the same setup for arch and ubuntu, and it works there.

Also, I don't get the system started without the "dolvm" kernel parameter, but I guess it is genkernel init script related

Any help would be appreciated.

PS: I probably give your cpio "arch" approach to initramfs a try and see if that works better.Last edited by udeved on Sun May 06, 2012 5:50 pm; edited 1 time in total

----------

## NeddySeagoon

udeved,

I don't get any symlinks at all for my logical volumes.

In fact, I get error messages from lvm saying that udev was supposed to make the links but didn't ... falling back in direct linke creation.

genkernel will do something similar to my initrd but it will do a lot more too as genkernel builds you a fully modular kernel, therfore, it needs to load the kernel modules needed to boot your box too.  My initrd is not tied to any partitular kernel as it does not contain bits of the kernel. With genkernel, you need a new initrd with every kernel build.

Are your arch and ubuntu logical vlomes started by genkernels initrd?

I would be surprised if they were.

----------

## udeved

 *NeddySeagoon wrote:*   

> udeved,
> 
> I don't get any symlinks at all for my logical volumes.
> 
> In fact, I get error messages from lvm saying that udev was supposed to make the links but didn't ... falling back in direct linke creation.
> ...

 

Yes, perhaps I didn't get the problem along.

I run a custom kernel, without many modules, almost all is built in, but the initrd is generated with 

```
genkernel --lvm --install initramfs --kernname=kernel

```

Only eg ext3 fs module in initrd, its about 5 modules, and very small in size

So, all VGs are activated, so in my case arch, ubuntu, chakra and gentoo(the booted system)

Thus, udev creates and populates eg /dev/arch/lv-name, besides the /dev/gentoo/LV-name

however, the /dev/disk/by-label or /dev/disk/by-uuid don't have any symlinks to the activated LVs, only the gentoo LVs are symlinked.

No lvm service in runlevels.

But, I get the missing LV in the mentioned directories after twice starting lvm service, strange...

Output without lvm service, as after boot:

```
# ls -la /dev/disk/by-label

insgesamt 0

drwxr-xr-x 2 root root 200  6. Mai 19:59 .

drwxr-xr-x 5 root root 100  6. Mai 2012  ..

lrwxrwxrwx 1 root root  10  6. Mai 2012  Daten -> ../../sda2

lrwxrwxrwx 1 root root  10  6. Mai 2012  System -> ../../sda1

lrwxrwxrwx 1 root root  11  6. Mai 2012  arch-boot -> ../../sda11

lrwxrwxrwx 1 root root  10  6. Mai 2012  boot -> ../../sda3

lrwxrwxrwx 1 root root  11  6. Mai 2012  chakra-boot -> ../../sda13

lrwxrwxrwx 1 root root  11  6. Mai 18:35 gentoo-boot -> ../../sda10

lrwxrwxrwx 1 root root  11  6. Mai 18:38 ubuntu -> ../../dm-20

lrwxrwxrwx 1 root root  11  6. Mai 18:38 ubuntu-usr -> ../../dm-17

ls -la /dev/disk/by-uuid

insgesamt 0

drwxr-xr-x 2 root root 220  6. Mai 19:59 .

drwxr-xr-x 5 root root 100  6. Mai 2012  ..

lrwxrwxrwx 1 root root  10  6. Mai 2012  1E1816A318167A4B -> ../../sda1

lrwxrwxrwx 1 root root  10  6. Mai 2012  2b2a518d-60be-4c99-bb7b-7b2041251a2b -> ../../sda5

lrwxrwxrwx 1 root root  11  6. Mai 18:38 2c9bf884-120c-4fcb-8901-5d6e711f742b -> ../../dm-20

lrwxrwxrwx 1 root root  11  6. Mai 18:35 42b3d32d-b892-4252-861e-5fd727792ef9 -> ../../sda10

lrwxrwxrwx 1 root root  11  6. Mai 2012  7987c32a-6d8a-4074-97cd-94862b921df3 -> ../../sda11

lrwxrwxrwx 1 root root  10  6. Mai 2012  C4EE10EDEE10D98E -> ../../sda2

lrwxrwxrwx 1 root root  10  6. Mai 2012  ce99c81a-032a-4254-9acb-8c4bddef5982 -> ../../sda3

lrwxrwxrwx 1 root root  11  6. Mai 18:38 e2bfcc75-4650-4c27-9575-78d893b3bf90 -> ../../dm-17

lrwxrwxrwx 1 root root  11  6. Mai 2012  fe3e1a6a-e034-4356-a82d-e5d49f614880 -> ../../sda13

```

Output with twice started lvm service:

```
ls -la /dev/disk/by-label

insgesamt 0

drwxr-xr-x 2 root root 400  6. Mai 20:01 .

drwxr-xr-x 5 root root 100  6. Mai 2012  ..

lrwxrwxrwx 1 root root  10  6. Mai 2012  Daten -> ../../sda2

lrwxrwxrwx 1 root root  10  6. Mai 2012  System -> ../../sda1

lrwxrwxrwx 1 root root  11  6. Mai 20:01 arch -> ../../dm-16

lrwxrwxrwx 1 root root  11  6. Mai 2012  arch-boot -> ../../sda11

lrwxrwxrwx 1 root root  11  6. Mai 20:01 arch-tmp -> ../../dm-15

lrwxrwxrwx 1 root root  11  6. Mai 20:01 arch-usr -> ../../dm-13

lrwxrwxrwx 1 root root  11  6. Mai 20:01 arch-var -> ../../dm-14

lrwxrwxrwx 1 root root  10  6. Mai 2012  boot -> ../../sda3

lrwxrwxrwx 1 root root  10  6. Mai 20:01 chakra -> ../../dm-3

lrwxrwxrwx 1 root root  11  6. Mai 2012  chakra-boot -> ../../sda13

lrwxrwxrwx 1 root root  10  6. Mai 20:01 chakra-tmp -> ../../dm-2

lrwxrwxrwx 1 root root  10  6. Mai 20:01 chakra-usr -> ../../dm-0

lrwxrwxrwx 1 root root  10  6. Mai 20:01 chakra-var -> ../../dm-1

lrwxrwxrwx 1 root root  11  6. Mai 18:35 gentoo-boot -> ../../sda10

lrwxrwxrwx 1 root root  11  6. Mai 18:38 ubuntu -> ../../dm-20

lrwxrwxrwx 1 root root  11  6. Mai 20:01 ubuntu-tmp -> ../../dm-19

lrwxrwxrwx 1 root root  11  6. Mai 18:38 ubuntu-usr -> ../../dm-17

lrwxrwxrwx 1 root root  11  6. Mai 20:01 ubuntu-var -> ../../dm-18

# ls -la /dev/disk/by-uuid

insgesamt 0

drwxr-xr-x 2 root root 420  6. Mai 20:01 .

drwxr-xr-x 5 root root 100  6. Mai 2012  ..

lrwxrwxrwx 1 root root  11  6. Mai 20:01 05a1302b-a68d-4d6d-bd42-5b44d4579f02 -> ../../dm-18

lrwxrwxrwx 1 root root  10  6. Mai 2012  1E1816A318167A4B -> ../../sda1

lrwxrwxrwx 1 root root  10  6. Mai 2012  2b2a518d-60be-4c99-bb7b-7b2041251a2b -> ../../sda5

lrwxrwxrwx 1 root root  11  6. Mai 18:38 2c9bf884-120c-4fcb-8901-5d6e711f742b -> ../../dm-20

lrwxrwxrwx 1 root root  10  6. Mai 20:01 355db144-8d49-4833-a771-f3fdd9ebac1d -> ../../dm-3

lrwxrwxrwx 1 root root  11  6. Mai 20:01 37bb65e7-7c71-46ff-aeef-3d0a7481f23a -> ../../dm-15

lrwxrwxrwx 1 root root  11  6. Mai 18:35 42b3d32d-b892-4252-861e-5fd727792ef9 -> ../../sda10

lrwxrwxrwx 1 root root  11  6. Mai 20:01 584b24ae-67c9-4ccd-8e3f-02d9d2fc1c1a -> ../../dm-14

lrwxrwxrwx 1 root root  11  6. Mai 2012  7987c32a-6d8a-4074-97cd-94862b921df3 -> ../../sda11

lrwxrwxrwx 1 root root  11  6. Mai 20:01 915ff97c-d501-415e-b2e5-47973790ccb1 -> ../../dm-13

lrwxrwxrwx 1 root root  10  6. Mai 2012  C4EE10EDEE10D98E -> ../../sda2

lrwxrwxrwx 1 root root  11  6. Mai 20:01 a1635db3-44eb-491f-91a7-35d31f498d35 -> ../../dm-16

lrwxrwxrwx 1 root root  10  6. Mai 20:01 b6cf29f5-f46c-4014-853a-ee4f6f329c67 -> ../../dm-2

lrwxrwxrwx 1 root root  10  6. Mai 2012  ce99c81a-032a-4254-9acb-8c4bddef5982 -> ../../sda3

lrwxrwxrwx 1 root root  11  6. Mai 20:01 d49e80af-2499-448d-9240-52b3a7fb54e4 -> ../../dm-19

lrwxrwxrwx 1 root root  10  6. Mai 20:01 d7026d92-2b96-4f2c-b2c9-3fbf84a26128 -> ../../dm-1

lrwxrwxrwx 1 root root  11  6. Mai 18:38 e2bfcc75-4650-4c27-9575-78d893b3bf90 -> ../../dm-17

lrwxrwxrwx 1 root root  10  6. Mai 20:01 f188ac91-b7d9-4221-ab5b-d2a46432ade9 -> ../../dm-0

lrwxrwxrwx 1 root root  11  6. Mai 2012  fe3e1a6a-e034-4356-a82d-e5d49f614880 -> ../../sda13

```

I forgot, the result of getting the LVs symlinked, is that they show up in dolphin sidebar.

----------

## NeddySeagoon

udeved,

I suspect that your genkernel initrd is only doing

```
vgchange -ay gentoo
```

so only your gentoo logical volume group is started. 

/usr/src/linux/Documentation/initrd.txt tells how to examine the content of the initrd.

What does the init script in your initrd show?

----------

## udeved

 *NeddySeagoon wrote:*   

> udeved,
> 
> I suspect that your genkernel initrd is only doing
> 
> ```
> ...

 

This is the method from initrd's ./etc/initrd.scripts...

 *Quote:*   

> startVolumes() {
> 
> 	#good_msg 'Checking if volumes need to be started...'
> 
> 	# Here, we check for /dev/device-mapper, and if it exists, we setup a
> ...

 

----------

## udeved

I quickly booted ubuntu, to post some output:

```
~$ ls -la /dev/disk/by-label

insgesamt 0

drwxr-xr-x 2 root root 580 Mai  6  2012 .

drwxr-xr-x 6 root root 120 Mai  6  2012 ..

lrwxrwxrwx 1 root root  11 Mai  6 21:05 arch -> ../../dm-16

lrwxrwxrwx 1 root root  11 Mai  6 21:05 arch-boot -> ../../sda11

lrwxrwxrwx 1 root root  11 Mai  6 21:05 arch-tmp -> ../../dm-15

lrwxrwxrwx 1 root root  11 Mai  6 21:05 arch-usr -> ../../dm-13

lrwxrwxrwx 1 root root  11 Mai  6 21:05 arch-var -> ../../dm-14

lrwxrwxrwx 1 root root  10 Mai  6 21:05 boot -> ../../sda3

lrwxrwxrwx 1 root root  10 Mai  6 21:05 chakra -> ../../dm-3

lrwxrwxrwx 1 root root  11 Mai  6 21:05 chakra-boot -> ../../sda13

lrwxrwxrwx 1 root root  10 Mai  6 21:05 chakra-tmp -> ../../dm-2

lrwxrwxrwx 1 root root  10 Mai  6 21:05 chakra-usr -> ../../dm-0

lrwxrwxrwx 1 root root  10 Mai  6 21:05 chakra-var -> ../../dm-1

lrwxrwxrwx 1 root root  10 Mai  6 21:05 Daten -> ../../sda2

lrwxrwxrwx 1 root root  10 Mai  6 21:05 gentoo -> ../../dm-5

lrwxrwxrwx 1 root root  11 Mai  6 21:05 gentoo-boot -> ../../sda10

lrwxrwxrwx 1 root root  11 Mai  6 21:05 gentoo-distfiles -> ../../dm-11

lrwxrwxrwx 1 root root  11 Mai  6 21:05 gentoo-opt -> ../../dm-12

lrwxrwxrwx 1 root root  10 Mai  6 21:05 gentoo-portage -> ../../dm-8

lrwxrwxrwx 1 root root  10 Mai  6 21:05 gentoo-src -> ../../dm-7

lrwxrwxrwx 1 root root  11 Mai  6 21:05 gentoo-tmp -> ../../dm-10

lrwxrwxrwx 1 root root  10 Mai  6 21:05 gentoo-usr -> ../../dm-6

lrwxrwxrwx 1 root root  10 Mai  6 21:05 gentoo-var -> ../../dm-9

lrwxrwxrwx 1 root root  10 Mai  6 21:05 home -> ../../dm-4

lrwxrwxrwx 1 root root  10 Mai  6 21:05 System -> ../../sda1

lrwxrwxrwx 1 root root  11 Mai  6 21:05 ubuntu -> ../../dm-20

lrwxrwxrwx 1 root root  11 Mai  6 21:05 ubuntu-tmp -> ../../dm-19

lrwxrwxrwx 1 root root  11 Mai  6 21:05 ubuntu-usr -> ../../dm-17

lrwxrwxrwx 1 root root  11 Mai  6 21:05 ubuntu-var -> ../../dm-18

```

Note, contrary to gentoo, ubuntu also symlinks the ubuntu LVs.

Gentoo doesn't do that.

Arch also creates all symlinks.

```
~$ ls -la /dev/mapper

insgesamt 0

drwxr-xr-x  2 root root     480 Mai  6  2012 .

drwxr-xr-x 20 root root    4820 Mai  6 21:05 ..

lrwxrwxrwx  1 root root       8 Mai  6 21:05 arch-root -> ../dm-16

lrwxrwxrwx  1 root root       8 Mai  6 21:05 arch-tmp -> ../dm-15

lrwxrwxrwx  1 root root       8 Mai  6 21:05 arch-usr -> ../dm-13

lrwxrwxrwx  1 root root       8 Mai  6 21:05 arch-var -> ../dm-14

lrwxrwxrwx  1 root root       7 Mai  6 21:05 chakra-root -> ../dm-3

lrwxrwxrwx  1 root root       7 Mai  6 21:05 chakra-tmp -> ../dm-2

lrwxrwxrwx  1 root root       7 Mai  6 21:05 chakra-usr -> ../dm-0

lrwxrwxrwx  1 root root       7 Mai  6 21:05 chakra-var -> ../dm-1

crw-------  1 root root 10, 236 Mai  6 21:05 control

lrwxrwxrwx  1 root root       8 Mai  6 21:05 gentoo-distfiles -> ../dm-11

lrwxrwxrwx  1 root root       8 Mai  6 21:05 gentoo-opt -> ../dm-12

lrwxrwxrwx  1 root root       7 Mai  6 21:05 gentoo-portage -> ../dm-8

lrwxrwxrwx  1 root root       7 Mai  6 21:05 gentoo-root -> ../dm-5

lrwxrwxrwx  1 root root       7 Mai  6 21:05 gentoo-src -> ../dm-7

lrwxrwxrwx  1 root root       8 Mai  6 21:05 gentoo-tmp -> ../dm-10

lrwxrwxrwx  1 root root       7 Mai  6 21:05 gentoo-usr -> ../dm-6

lrwxrwxrwx  1 root root       7 Mai  6 21:05 gentoo-var -> ../dm-9

lrwxrwxrwx  1 root root       7 Mai  6 21:05 home-home -> ../dm-4

lrwxrwxrwx  1 root root       8 Mai  6 21:05 ubuntu-root -> ../dm-20

lrwxrwxrwx  1 root root       8 Mai  6 21:05 ubuntu-tmp -> ../dm-19

lrwxrwxrwx  1 root root       8 Mai  6 21:05 ubuntu-usr -> ../dm-17

lrwxrwxrwx  1 root root       8 Mai  6 21:05 ubuntu-var -> ../dm-18

```

Very strange.

----------

## NeddySeagoon

udeved,

That init script fragment looks OK to me.

----------

## udeved

 *NeddySeagoon wrote:*   

> udeved,
> 
> That init script fragment looks OK to me.

 

I just had a look into ubuntu initrd.

Ubuntu seems to do the lvm activation purely with udevadm besides a vgscan

```
# It's all over netlink now

echo "" > /proc/sys/kernel/hotplug

   

# Start the udev daemon to process events

/sbin/udevd --daemon --resolve-names=never

# Iterate sysfs and fire off everything; if we include a rule for it then

# it'll get handled; otherwise it'll get handled later when we do this again

# in the main boot sequence.

( /sbin/udevadm trigger --action=add --subsystem-match=block; \

  /sbin/udevadm trigger --action=add --subsystem-nomatch=block; ) &
```

I can't find any reference to vgchange -a y in lvm init related scripts.

If you follow the bug report I posted, iirc post 54 gives the solution.

It says to comment out some lines in /lib/rcscripts/addons/lvm-start-sh, which I bvelieve is used in initrd.

I will generate a new initrd, after I commented out the vgchange line as well, and test it.

----------

## Goverp

I've been following the udev-182 saga, but not dared touch anything so far.  My box has a 4-disk RAID 5 array partitioned into /, /var, /home and /var/tmp, so /usr is part of rootfs, but /var isn't.  The system is an amd64 Gentoo stable setup, kernel 3.2.12 (no initramfs) sys-fs/udev-171-r5.  The mdadm service is in the default runlevel, and doesn't start until quite late (after cups).  The kernel assembles the RAID array from command-line parameters (not auto-assemble, but the explicit version of the same).  Here are the md-related lines from syslog:

```
 May  8 12:01:59 kernel: Command line: root=/dev/md127p3 md=127,/dev/sda3,/dev/sdb3,/dev/sdc3,/dev/sdd3 vga=0x317

 May  8 12:01:59 kernel: Kernel command line: root=/dev/md127p3 md=127,/dev/sda3,/dev/sdb3,/dev/sdc3,/dev/sdd3 vga=0x317

 May  8 12:01:59 kernel: md: Will configure md127 (super-block) from /dev/sda3,/dev/sdb3,/dev/sdc3,/dev/sdd3, below.

 May  8 12:01:59 kernel: ata5: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xff00 irq 14

 May  8 12:01:59 kernel: ata6: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xff08 irq 15

 May  8 12:01:59 kernel: md: raid1 personality registered for level 1

 May  8 12:01:59 kernel: md: raid6 personality registered for level 6

 May  8 12:01:59 kernel: md: raid5 personality registered for level 5

 May  8 12:01:59 kernel: md: raid4 personality registered for level 4

 May  8 12:01:59 kernel: md: Skipping autodetection of RAID arrays. (raid=autodetect will force)

 May  8 12:01:59 kernel: md: Loading md127: /dev/sda3

 May  8 12:01:59 kernel: md: bind<sda3>

 May  8 12:01:59 kernel: md: bind<sdb3>

 May  8 12:01:59 kernel: md: bind<sdc3>

 May  8 12:01:59 kernel: md: bind<sdd3>

 May  8 12:01:59 kernel: md/raid:md127: device sdd3 operational as raid disk 3

 May  8 12:01:59 kernel: md/raid:md127: device sdc3 operational as raid disk 2

 May  8 12:01:59 kernel: md/raid:md127: device sdb3 operational as raid disk 1

 May  8 12:01:59 kernel: md/raid:md127: device sda3 operational as raid disk 0

 May  8 12:01:59 kernel: md/raid:md127: allocated 4270kB

 May  8 12:01:59 kernel: md/raid:md127: raid level 5 active with 4 out of 4 devices, algorithm 2

 May  8 12:01:59 kernel: md127: detected capacity change from 0 to 805295751168

 May  8 12:01:59 kernel: md127: p1 p2 p3 p4

 May  8 12:01:59 kernel: md127: p1 p2 p3 p4

 May  8 12:01:59 kernel: EXT4-fs (md127p3): mounted filesystem with ordered data mode. Opts: (null)

 May  8 12:01:59 kernel: mdadm: sending ioctl 1261 to a partition!

 May  8 12:01:59 kernel: EXT4-fs (md127p3): re-mounted. Opts: (null)

 May  8 12:01:59 kernel: EXT4-fs (md127p4): mounted filesystem with ordered data mode. Opts: (null)

 May  8 12:01:59 kernel: EXT4-fs (md127p1): mounted filesystem with ordered data mode. Opts: (null)

 May  8 12:01:59 kernel: EXT4-fs (md127p2): mounted filesystem with ordered data mode. Opts: (null)
```

.  md127p1 is /var, md127p2 is /var/tmp (and /tmp bind-mounted from within that), md127p3 is rootfs, and md127p4 is /home.  /etc/fstab locates the partitions by LABEL=, which may not be as precise as UUID, but it's pretty similar in function.

My question is, will this need an initramfs after udev-182?  I'd have thought not, as the kernel is already assembling the RAID array without udev's or mdadm's assistance.  It will need something to mount /var and so forth before udev starts, such as greyspoke's script, or maybe just a one-liner to mount /var.

If it's got a reasonable chance, I'll try upgrading udev to test it, but not if I'm almost certain to need to build an initramfs.

----------

## NeddySeagoon

Goverp,

You will need something to mount /var before udev starts, thats normally in the sysinit runlevel and well before local-mount.

An initrd or one possible solution.  A mount /var early in the in init script is not a good idea. The filesystem should be checked before its mounted, so now you need the right fsck too.

There are two or three other approaches, greyspokes script.  SteveLs changes to the existing init scrips and a latecommer, vapiers busybox with the USE=sep-usr flag set.

I'm not sure how that works but I guess its intended for use in an initrd.

----------

## udeved

 *NeddySeagoon wrote:*   

> Goverp,
> 
> You will need something to mount /var before udev starts, thats normally in the sysinit runlevel and well before local-mount.
> 
> An initrd or one possible solution.  A mount /var early in the in init script is not a good idea. The filesystem should be checked before its mounted, so now you need the right fsck too.
> ...

 

Thanks a lot for mentioning the buybox approach.

Google found me the thread, and it states it works with busybox 1.20.0 without initrd.

Exactly what I was looking for, besides the earlymount script.

http://www.gossamer-threads.com/lists/gentoo/dev/252734

I will report back how that one worked out, since its in the portage tree it says. 

Do you see any other LVs, or any LVs at all in your filemanager?

----------

## steveL

I don't think busybox with sep-usr works with /usr on LVM since it runs before init (thus before lvm service):

 *vapier wrote:*   

> [the] new applet has a hand written set of commands to automatically mount /dev /proc /sys /usr and seed /dev, and then execute 
> 
> the real init (defaulting to /sbin/init)

 

I'd like openrc using busybox in any case, so I'll play with that sometime in next couple of weeks.

You could also try earlymounts or the patched scripts, which I prefer as they seem to have less issues with other stuff, although they are a bit trickier to setup due to the patching needed, and switching udev runlevel. (They just make udev run after localmount, so don't require any configuration beyond a single variable in udev.conf, eg for fsck, lvm or mdadm, last of which I don't use, so haven't tested.)

----------

## udeved

I can confirm, that busybox sep-usr does not work with root on lvm.

I tried to configure busybox with mdev support, no difference, LVM volumes don't get activated.

I also messed with different initrd structures.

I took arch's initrd created with mkinitcpio, customized it a bit, and the thingy boots. Probably I post the init script here, after I got everything like demons to work properly.

After closer inspection of arch's initrd, I found that arch includes the udevd in lib.

This seems to solve the problem of LVs not getting properly symlinked and thus refuse to show in filemanager.

I also simply copied the whole arch kernel with ramdisk to gentoo boot partition.

If I boot arch's kernel with gentoo, everything LV related works as intended in desktop environment.

Atm, arch's mkinitcpio seems to have the better approach with their "hooks" for ramdisk than genkernel.

----------

## NeddySeagoon

udeved,

Root on LVM has always required an inird.  LVM is a userspace tool and you need it working to mount root.

Root on a real partition or autodetected raid with everything else in LVM did not used to need an initrd but will with >=udev-182 as /usr and /var are in LVM, so LVM must be started to mount /usr and /var before udev starts.

I'm not sure if steveLs patched scripts will start LVM early enough.  I think they will as local-mount used to mount /usr and /var anyway, so LVM had to be running before local-mount.

----------

## udeved

 *NeddySeagoon wrote:*   

> udeved,
> 
> Root on LVM has always required an inird.  LVM is a userspace tool and you need it working to mount root.
> 
> Root on a real partition or autodetected raid with everything else in LVM did not used to need an initrd but will with >=udev-182 as /usr and /var are in LVM, so LVM must be started to mount /usr and /var before udev starts.
> ...

 

Yes, you are totally right. I completely forgot about userspace when made my enthusiast post.

I assumed wrongly that ginit also allows the kernel parameter dolvm, which produces a kernel panic.

I forgot to mention, I moved my root to a non LVM partition as well and tested busybox sep-usr

The result was still unmounted partitions with that ginit kernel paramenter for non root LVM partitions.

I tested a non LV /usr and /var as well.

I got now a very well working initrd which symlinks LVs properly.

If want me to post the scripts, say so.  :Smile: 

----------

## NeddySeagoon

udeved,

Please do post your scripts and your process.  Start a new thread in Documentation Tips and Tricks.

Sharing what you have learned is the Open Source Community way.

This /usr and /var early mounting is a very contraversial change, the more options the better.

Please stay on the forums to give support to your followers (you will get some).

----------

## udeved

 *NeddySeagoon wrote:*   

> udeved,
> 
> Please do post your scripts and your process.  Start a new thread in Documentation Tips and Tricks.
> 
> Sharing what you have learned is the Open Source Community way.
> ...

 

I can do that, however, don't you think the proper place would be your thread?

I use udev 182, and basically followed the udev guide on gentoo-wiki, with the exception that my ramdisk contains the udevd and arch structure.

You decide, I'll gladly share what I found out.

The genkernel basically misses a udev "hook"/include, and I found no way of telling genkernel to include udev.

So a port of mkinitcpio seems the way to go for me, if its all not too complicated..

----------

## NeddySeagoon

udeved,

 *udeved wrote:*   

> The genkernel basically misses a udev "hook"/include.

 

To bring that to the attention of the genkernel developers, please post a but at bugs.gentoo.org

You should start a new thread as you are offering a different solution to me.  If you post your scripts in this thread, the two solutions will get mixed up and hard to follow.

If you are offering a minor change to my first post here, by all means post in this thread.

----------

## udeved

 *NeddySeagoon wrote:*   

> udeved,
> 
>  *udeved wrote:*   The genkernel basically misses a udev "hook"/include. 
> 
> To bring that to the attention of the genkernel developers, please post a but at bugs.gentoo.org
> ...

 

I hesitated to file a bug report, since I found out that genkernel used to have a  --udev parameter.

My conclusion was, its intended, until udev >182 hits stable release?

The idea was to offer just a different solution to initrd, but yes, probably a small thread on udev182 initrd will do, with a link to your guide.

I would want to focus on initrd, but this requires a bit more testing before I start a thread.

----------

## steveL

 *NeddySeagoon wrote:*   

> Root on LVM has always required an inird.  LVM is a userspace tool and you need it working to mount root.
> 
> Root on a real partition or autodetected raid with everything else in LVM did not used to need an initrd but will with >=udev-182 as /usr and /var are in LVM, so LVM must be started to mount /usr and /var before udev starts.
> 
> I'm not sure if steveLs patched scripts will start LVM early enough.  I think they will as local-mount used to mount /usr and /var anyway, so LVM had to be running before local-mount.

 

Yeah exactly, they will. That's my setup: root, /tmp and /home on standard partitions, /usr, /var, /usr/portage, distfiles etc on lvm, so I can confirm it works fine on udev-171-r5 at least. The same approach should be fine on 182+ since the base requirements are met.

That's why I feel the approach is cleaner, even if it does mean a couple of (small) patches to initscripts: we're using the standard initialisation for practically everything, just we start udev late, once we know all our partitions are available. I just did the simplest thing given the spec: that udev needs to be started after partitions are mounted.

That means anything localmount depends on will be started, including lvm and mdadm (though I don't use the latter: feedback would be welcome. :)

Of course that only works if you don't need any modules or udev-created nodes to localmount (iow if you never needed an initrd/initramfs before) which is the case if you manually configure the kernel as part of your install, and root is not on lvm, raid (non-detected?), or encrypted (all which have always required an initramfs or initrd.)

afair encryption of any partition requires an initramfs but I don't know.

----------

## udeved

*deleted post*

Moved it here:

https://forums.gentoo.org/viewtopic-t-923168.html

----------

## hydrapolic

Thank you for the guide! 

Now I'm running grub-2.00/udev-1.82-r3/openrc-0.10.4

/dev/sda1 - ext2 boot

/dev/sda2 - ext4 root

/dev/sda3 - lvm (separate usr,var on jfs)

----------

## Knute

Your guide says to make changes to some kernel options, but fails to provide any information as to where in the kernel those options are!

That's like saying meet me on main street, but fail to say what city, country etc....  Very annoying!   :Rolling Eyes: 

----------

## SamuliSuominen

 *Knute wrote:*   

> Your guide says to make changes to some kernel options, but fails to provide any information as to where in the kernel those options are!
> 
> That's like saying meet me on main street, but fail to say what city, country etc....  Very annoying!  

 

All you have to do is press the / key in the menuconfig to get 'Search' window. 

No guides should ever pinpoint the exact location since the location keeps changing from version to version, thus the guides that does this get outdated very fast.

----------

## Knute

 *ssuominen wrote:*   

> All you have to do is press the / key in the menuconfig to get 'Search' window. 
> 
> No guides should ever pinpoint the exact location since the location keeps changing from version to version, thus the guides that does this get outdated very fast.

 

When I did the search, it showed no matches, so I had no clue as to what option you were trying to set.  (And you didn't even give what it's listed as in the .config file either!)

As far as pinpointing the exact location, your comment points to an attitude that I have had the misfortune of running across before.  You try to make it sound harder than it is so that you can retain some mystical guru status.  (It was in a windows hacking group, where they gave a big long complicated explaination about how to get around the login screen in windows 95, where all you have to do is press Esc.)

By the way, those options are in Device Drivers.  (Really hard to just pony up the information, right?)  :Shocked: 

----------

## miroR

 *hydrapolic wrote:*   

> Thank you for the guide! 
> 
> Now I'm running grub-2.00/udev-1.82-r3/openrc-0.10.4
> 
> /dev/sda1 - ext2 boot
> ...

 

Well, not using Grub Legacy anymore me neither.

So what are the equivalent lines to these in the guide:

 *NeddySeagoon wrote:*   

> 
> 
> ```
> #3.3.1-gentoo
> 
> ...

 

If I do manage to translate into working grub2 lines before someone else, I'll post them here myself.

EDIT: Oh, I remembered (ran grub2-mkconfig several times already...). Grub2 is generally not configured manually, but with that command.

It must be something like

```
grub2-mkconfig some-option-pointing-at-initrd.cpio.gz-here -o /boot/grub2/grub.cfg
```

So the question is only which option? end of EDIT

EDIT2 No such option. I'll try with

```
# ln initrd.cpio.gz initrd.img-$version
```

as /etc/grub.d/10_linux shows it'll be accepted for the kernel named

vmlinuz-$version end of EDIT2

Since we're talking...

NeddySeaGoon, I hope I'll thank you too, but I first haveto make it, and I'm still only getting my hands dirty only. Pls, if you're around, does the line in your guide:

```
# TODO could auto cope with degraded raid operation
```

mean degraded raid - won't boot?

I am moving to LVM2/plain raid (some partitions lvm2 on raid, some plain raid, root is plain like in your configuration).

They are all on degraded raid5 of 3 devices (3x1TB drives, degraded, so I can add the third once I `rsync -aHAXz /plain-system /lvm2-raid-system` over from plain to raid partitions), being the third obviously in use, i.e. holding the current Gentoo filesystem on plain partition.

So that mean degraded raid - won't boot? I hope not, but just in case...Last edited by miroR on Fri Sep 21, 2012 1:26 am; edited 2 times in total

----------

## miroR

Just in case someone else run into the following problem.

My initramfs is /usr/src/initramfs/, and there in is the initramfs_list.

From /usr/src/linux/scripts I issue:

```

# ./gen_initramfs_list.sh -o /boot/initrd.cpio.gz /usr/src/initramfs/initramfs_list 

./gen_initramfs_list.sh: line 298: usr/gen_init_cpio: No such file or directory

# 
```

The command, indeed, failed.

Still from /usr/src/linux/scripts, I added the symlink:

```
# ln -s ../usr/ usr
```

and the line now worked. I got a nearly 5M /boot/initrd.cpio.gz  made for me...

----------

## hydrapolic

@miroR

I followed the instructions at: http://wiki.gentoo.org/wiki/GRUB2_Quick_Start

You may also check out: https://forums.gentoo.org/viewtopic-t-928972.html

Happy grubbing  :Smile: 

----------

## miroR

 *hydrapolic wrote:*   

> @miroR
> 
> I followed the instructions at: http://wiki.gentoo.org/wiki/GRUB2_Quick_Start
> 
> You may also check out: https://forums.gentoo.org/viewtopic-t-928972.html
> ...

 

Thanx!

I was a frequent, still am occasional, visitor at wiki.gentoo.org and en.gentoo-wiki.com Gurb2 guides, and I did remember and am regularly past grub loading stage fine now.

I load initrd, it starts, mounts proc, dev and sys and throws me out into rescue shell.

But I got stuck with just in the grub cfg code

```
insmod mdraid09
```

necessary for my boot raid1 /dev/md0 device (identified by its uuid, as blkid tells me) and not this onw at all::

```
insmod mdraid1x
```

so all the other raid5 devices, and on one of them is a vg and lv volumes, they none of them mount at all.

But how and why exactly the other mdraid1x does not load at all is a mistery to me.

At first I thougt it had to do something with, while I followed, too blindly, the requirement for the installation of mkinitcpio, which required these two options to be activated in the kernel:

```
CONFIG_MD_LINEAR

CONFIG_DM_UEVENT
```

to be set so  mkinitcpio could install. That requirement, while no need for them in my system, of a program, I'm not happy with.

But I set them, and still had to overcome another hurdle to install  mkinitcpio, as can be seen in my bug report:

https://bugs.gentoo.org/show_bug.cgi?id=435642

That was in my first hours of studying this guide (it's not just reading, when it's over and over, isn't it studying then?)...

Later, I realized that I would very much like this clean and hard approach that Neddy, who is still not anywhere around for us, yet, gave us, to work for me, even though I might have to revert to the well documented Gentoo ported mkinitcpio package, if I don't make it with this NeddySeaGoon'guide.

But what should I do to get the necessary module 

```
insmod mdraid1x
```

 inserted.

OTOH, it's not a module in the kernel. Because:

```
# zgrep MDRAID /proc/config.gz

#
```

no output at all, and:

```
# zgrep RAID /proc/config.gz

# CONFIG_RAID_ATTRS is not set

CONFIG_MD_RAID0=y

CONFIG_MD_RAID1=y

CONFIG_MD_RAID10=y

CONFIG_MD_RAID456=y

# CONFIG_MULTICORE_RAID456 is not set

# CONFIG_DM_RAID is not set

# CONFIG_ASYNC_RAID6_TEST is not set

CONFIG_ASYNC_RAID6_RECOV=y

CONFIG_RAID6_PQ=y

#

```

as can be seen, I followed I think Neddy's idea, and installed some of those which were modules before, into the kernel...

So, anyone can tell me how to move on from this hurdle, thanx in advance!

I used to have old "-e 0.90" created raid styles along with raid_autodetect  (CONFIG_MD_AUTODETECT, not *not* set of course) up until the current upgrade, which is in uncussesful many tries so far, in which I only create the booting /boot containing raid with that option, and all others with the default 1.2 version in mdadm.

So I'll try and perform the system shutdown to make it forget if it has those in the cache somewhere that prevent new mraid1x from kicking in.

Otherwise, no more ideas at this time.

----------

## hydrapolic

Why did you set the raid drivers as modules?

----------

## miroR

 *hydrapolic wrote:*   

> Why did you set the raid drivers as modules?

 

No, I didn't. I set them in the kernl ("y" there, not "m").

----------

## miroR

Here I am inching ever closer to solving this.

Some of my previous assumptions are wrong/probably wrong. Sorry. Hours upon hours of slow progress here.

Here go my newest tries.

I'm in my friend's François Dupoux's sysresccd, because for some reason, even though the new lvm2 on standard raid5 setup (except for /boot on version 0.90 mdadm made raid1), all rsync'd over from single /dev/sda installation...

Because even though the new plain /boot-on-raid1 and lvm2-on-raid5 setup is completely independently made (raid devices are degraded, the one missing partition in the raid1/raid5 is the old /dev/sdaX which will be added upon successful reinstallation on the new setup...

Because independent even though new setup is, I can't use mdadm nor mount devices when I chroot into it from the old setup and don't know how I could do so.

From sysresccd I can do all of these.

/dev/md0 is the raid1 made of /dev/sdb2 and /dev/sdc2 (on both the GPT-EFI partition is sd[bc]1, has to be)

/dev/sdb and /dev/sdc are sure partitioned in the exact same way, but with Roderick Smith's great gdisk tool, http://www.rodsbooks.com/gdisk/ (that is a man whom we all owe for this tool! I would donate, but am poor. The aforementioned François Dupoux is also great, except he banned me from sysresccd.org for stupid reasons, he is quirky, seems. Never mind my old man's ramblings. Just easy-going. Don't ban me from here, pls!)...

I can just fine mount /dev/md0 through it's uuid which I stuck in /etc/fstab, meaning it is correct. So are the other partitions's uuid's.

Also, grub2 recognizes and starts booting from my plain raid1 /dev/md0. But. But!!

Here is what gets in real, unedited consecutive tries I performed:

```
sysresccd / # grub2-install --boot-directory=/boot/  --modules="part_gpt mdraid mdraid09 mdraid1x ext2 fs_uuid lvm" /dev/md0 

/usr/bin/grub2-mkimage: error: cannot stat `/usr/lib64/grub/i386-pc/mdraid.mod': No such file or directory.

sysresccd / # grub2-install --boot-directory=/boot/  --modules="part_gpt mdraid mdraid09 mdraid1x ext2 fs_uuid lvm" /dev/sda 

/usr/bin/grub2-mkimage: error: cannot stat `/usr/lib64/grub/i386-pc/mdraid.mod': No such file or directory.

sysresccd / # grub2-install --boot-directory=/boot/  --modules="part_gpt mdraid09 mdraid1x ext2 fs_uuid lvm" /dev/md0 

/usr/bin/grub2-mkimage: error: cannot stat `/usr/lib64/grub/i386-pc/fs_uuid.mod': No such file or directory.

sysresccd / # grub2-install --boot-directory=/boot/  --modules="part_gpt mdraid09 mdraid1x ext2 lvm" /dev/md0 

/usr/sbin/grub2-bios-setup: warning: File system `ext2' doesn't support embedding.

/usr/sbin/grub2-bios-setup: warning: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and their use is discouraged..

/usr/sbin/grub2-bios-setup: error: will not proceed with blocklists.

sysresccd / # grub2-install --boot-directory=/boot/  --modules="part_gpt mdraid09 mdraid1x lvm" /dev/md0 

/usr/sbin/grub2-bios-setup: warning: File system `ext2' doesn't support embedding.

/usr/sbin/grub2-bios-setup: warning: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and their use is discouraged..

/usr/sbin/grub2-bios-setup: error: will not proceed with blocklists.

sysresccd / # grub2-install --boot-directory=/boot/  --modules="part_gpt ext2 mdraid09 mdraid1x lvm" /dev/md0 

/usr/sbin/grub2-bios-setup: warning: File system `ext2' doesn't support embedding.

/usr/sbin/grub2-bios-setup: warning: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and their use is discouraged..

/usr/sbin/grub2-bios-setup: error: will not proceed with blocklists.

sysresccd / # grub2-install --boot-directory=/boot/  --modules="part_gpt ext2 mdraid09 mdraid1x lvm" /dev/md0 

/usr/sbin/grub2-bios-setup: warning: File system `ext2' doesn't support embedding.

/usr/sbin/grub2-bios-setup: warning: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and their use is discouraged..

/usr/sbin/grub2-bios-setup: error: will not proceed with blocklists.

sysresccd / # grub2-install --boot-directory=/boot/  --modules="part_gpt ext2 mdraid09 mdraid1x lvm" /dev/sda 

Installation finished. No error reported.

sysresccd / # grub2-install --boot-directory=/boot/  --modules="part_gpt ext2 mdraid09 mdraid1x lvm" /dev/sdb

Installation finished. No error reported.

sysresccd / # 
```

Not exactly what I wanted, but it wouldn't install in /dev/md0... Or should I try force, by adding the "-f" option iirc?

Because,this below:

```
sysresccd / # grub2-mkconfig -o /boot/grub2/grub.cfg 

Generating grub.cfg ...

/usr/sbin/grub2-probe: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image..

Found linux image: /boot/vmlinuz-3.5.4-hardened-120921_0700

Found initrd image: /boot/initrd.img-3.5.4-hardened-120921_0700

Found linux image: /boot/vmlinuz-3.5.4-hardened-120920_0200

Found linux image: /boot/vmlinuz-3.5.4-hardened-120919_1700

Found linux image: /boot/vmlinuz-3.5.3-hardened-r3-120917_2100

Found linux image: /boot/vmlinuz-3.5.3-hardened-r3-120914_1500

/usr/sbin/grub2-probe: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image..

done

sysresccd / # 
```

doesn't let me inch not even half an inch on from the current impass.

----------

## miroR

I stopped the raid1 /dev/md0, zero-superblock the /dev/sdb2 and /dev/sdc2.

The same raid1 uuid still remains reported by blkid, even after removing blkid cache:

```
# blkid -c /dev/null | egrep 'db2|sdc2'

/dev/sdb2: UUID="e7f482cb-ca79-497d-85db-43041277ca2e" TYPE="ext2" PARTLABEL="Linux filesystem" PARTUUID="cf771640-1e0f-4ddf-a0cd-10a191d86f75" 

/dev/sdc2: UUID="b42ecfc7-4b56-4e41-9d85-6a7215ebb8c1" TYPE="ext2" PARTLABEL="Linux filesystem" PARTUUID="cf771640-1e0f-4ddf-a0cd-10a191d86f75" 

#
```

So now I still  can't install drivers because even though:

```
# grub2-install  --boot-directory=/boot --modules="part_gpt mdraid09 mdraid1x ext2 lvm"  /dev/sdb

Installation finished. No error reported.

 # grub2-install  --boot-directory=/boot --modules="part_gpt mdraid09 mdraid1x ext2 lvm"  /dev/sdc

Installation finished. No error reported.

 # 
```

still:

```

 # grub2-mkconfig -o /boot/grub2/grub.cfg

Generating grub.cfg ...

/usr/sbin/grub2-probe: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image..

Found linux image: /boot/vmlinuz-3.5.4-hardened-120921_0700

Found initrd image: /boot/initrd.img-3.5.4-hardened-120921_0700

Found linux image: /boot/vmlinuz-3.5.4-hardened-120920_0200

Found linux image: /boot/vmlinuz-3.5.4-hardened-120919_1700

Found linux image: /boot/vmlinuz-3.5.3-hardened-r3-120917_2100

Found linux image: /boot/vmlinuz-3.5.3-hardened-r3-120914_1500

/usr/sbin/grub2-probe: warning: Couldn't find physical volume `(null)'. Some modules may be missing from core image..

done

# 

```

I am also trying to post this issue (broached long ago there)

EDIT exactly here:   http://marc.info/?l=linux-raid&m=129790712716243&w=2   end of EDIT

on linux-raid mailing list.

----------

## miroR

There, asking for help now on Linux-raid as well:

http://marc.info/?l=linux-raid&m=134825983112176&w=2

EIDIT: That particular hurdle solved:

http://marc.info/?l=linux-raid&m=134831845825240&w=2

----------

## miroR

After about a week of downtime, troubling experience having lasted for longer period yet, but the downtime for about a week, which first time so long downtime since I've been using Gentoo, I made what I best described in this thread.

See above:

 *miroR wrote:*   

> I am moving to LVM2/plain raid (some partitions lvm2 on raid, some plain raid, root is plain like in your configuration). 

 

and around there.

1670.00 GiB raid5 of three 835.00 GiB devices, containing lv volumes (no /root, no /usr, no /var among those, just data volumes, and swap and /build) is syncing last, and it'll take its time.

But I didn't make it with this approach, and it can be seen where and what wouldn't pass (in the previous posts in this thread).

And neither, yet, with mkinitcpio:

https://forums.gentoo.org/viewtopic-t-923168.html#7145030

(but there at least I got developer udeved to find the bug that someone reckless or lazy or malicious (how can I tell?) seem to have introduced in the ebuild, and am really looking forward to using that program, but it is not yet well adapted for Gentoo, iiuc)

During that time I was miserably hit by unstable udev (which is such for, my guess can still be read why in the linked thread just below, but I got called to order for my big mouth and digressions out of place in many people's opinion, but you judge for yourself, gentle reader:

https://forums.gentoo.org/viewtopic-t-934678-postdays-0-postorder-asc-start-100.html#7142000

I finally got my system to boot (it's a real nightmare, not being able to boot for about a week!) with neither this guide where you're reading my conclusion (which seems to me similar or same as https://wiki.gentoo.org/wiki/Early_Userspace_Mounting, nor with the lean and mean, but not fully Gentoo-ready mkinitcpio, for the reasons mentioned above (link to thread given too)...

I got my first in my lifetime initramfs of my own making, simply, without some of the udev newfangled stuff and things, as prescribed here:

http://en.gentoo-wiki.com/wiki/Root_filesystem_over_LVM2,_DM-Crypt_and_RAID#Initram

except for the cryptsetup part (encryption need much more profound understanding than I currently have; I don't want to be one of those poor souls who lost their data; I could anyways become one, even with what I'm playing with, but much more likely so if I go and play the tricky dm-crypt luks and associates...).

I will only need, but it's really not urgent, to get the lvm to also start at boot, because it current doesn't. I start it manually.

Everything else is basically just normally functioning again.

Only I am not, yet, normally functioning, because of too much stress, but I'm slowly recovering...

My thanks do go to people who develop Gentoo.

Along with my thanks goes the recommendation to get the evil one out: the secret services' SELinux, which I wrote about quite a lot on this Forum ( https://forums.gentoo.org/viewtopic-t-905472.html#7125428 ), making fair use, fair use, dear friends, of what democracy is left in the Declining SuperPower (where the headquarters of Gentoo are, and a fine backdoor there is for the NSA's SELinux, as most anywhere... maybe only not on gnu.org, actually, as it happens), that plays tricks, the US's NSA, on us in this particular issue, which tricks the better among you, dear Americans, whose history and achievements I admire, should not let happen. GNU Linux is sacred, don't let it be broken in by backdoors and surveillance! Gentoo won't remain our pride any longer! It'll become like RedHat or SuSE! which go the Widoze ways of Billy the Gangster and his Gang... which is thriving their greed and deceit on the world stupidity, ignorance and laziness.

That's it.

God bless you!

Or, iow, may the Good of the Life be with you!

Miroslav Rovis

Zagreb, Croatia

EDIT: I lost, as usual, a better worded variant, and had to redo what I remembered, of the lost corrections. Sorry!

----------

## nerdboy

Not being entirely happy with the approaches I've tried so far:

1) genkernel initramfs with the appropriate options for lvm and mdadm

2) mkinitcpio (which didn't really work at all, even after patching for the latest udev)

3) This very nice guide from Neddy, which left me with odd problems like no /dev/pts mount, etc

so I stole Neddy's fsck idea and applied it to genkernel as a klugey hack (or would that be hacky kluge?) enabled by USE=premount.  It currently supports ext2/3/4 and jfs checks of /usr before it gets mounted (and should be easy to extend by modifying a couple of file listings in $FILESDIR).

In case anyone wants to try it out, I'm about to push it to my (portage) overlay on github; it works for me on several machines with lvm and lvm on kernel raid.

https://github.com/sarnold/portage-overlay

Edit: I still do my kernels manually, but now i use genkernel to generate the initramfs.  The only thing left seems to be deactivating /usr when on lvm.  Not sure how to hack that one yet...

----------

## steveL

 *nerdboy wrote:*   

> The only thing left seems to be deactivating /usr when on lvm.  Not sure how to hack that one yet...

 

Have a look here for some ideas.

Also, anyone who needs to explore openrc deptree can use this.

----------

## ExecutorElassus

hi Neddy!

I'm back, after several month of your script working just fine. It still does, but now I have a question about a different box (my laptop, finally back from the shop).

That box has the following partition scheme on each of two drives:

/dev/sdX1: boot partition, starting on sector 2048 (because grub2 says it needs it?)

/dev/sdX2: swap space

/dev/sdX3: extended partition

/dev/sdX5: logical partition

The root partition is a logical volume on /dev/md5, which is the raid-0 array on /dev/md5.

Your guide says you wimped out putting root on lvm for the example, but I am apparently HARDCORE, and am doing that. So, how do I modify your init script to deal with root being on lvm?

Cheers, thanks again for all your help.

EE

----------

## ExecutorElassus

Hi Neddy!

So, I went and wimped out myself, and re-did the partitions to match the gentoo guide. Now, though, I have a problem with grub2.

Specifically, it isn't properly finding the kernel files on /boot. It says something like "error: file /boot/kernel-3.8.1-gentoo not found. You need to load the kernel first."

What am I doing wrong with grub that this is the case? The file is indeed on the partition named /boot: grub's own shell can find it. Any guesses?

Cheers,

EE

----------

