# Separate /usr on Linux requires initramfs

## jenkler

Hi gentooers!

I have some questions here.

 *Quote:*   

> Linux systems which have / and /usr on separate file systems but do not
> 
> use an initramfs will not be supported starting on 01-Nov-2013.
> 
> If you have / and /usr on separate file systems and you are not
> ...

 

I am running gentoo on a server with minimal server stuff, No X, no nothing. I use GPT, ext4 and softwareraid (MD) autodetect from kernel WITHOUT any sort of initrd

```

/dev/md1        /boot                   ext4            noatime,noauto                          1       2

/dev/md2        none                    swap            loop=/dev/loop6,encryption=AES256,sw    0       0

/dev/md3        /                       ext4            noatime                                 0       1

/dev/md5        /home                   ext4            noatime,nodev,nosuid                    0       2

/dev/md6        /tmp                    ext4            noatime,nodev,noexec,nosuid             0       2

/dev/md7        /usr                    ext4            noatime,nodev                           0       2

/dev/md8        /var                    ext4            noatime,nodev,nosuid                    0       2

/dev/md9        /server                 ext4            noatime,nodev,noexec,nosuid             0       2

```

As it seams, a Separate /usr partition will not work in the future.

1. Does this mean that it may work or does it mean 100% unbootabe system?

2. I have a static kernel with no modules support with Separate /boot,  / and /usr

This have nothing todo with a  Separate /boot and / right? 

3. It seams to be 2 options here. Move /usr content to / or use an initrd, i would move it but then i must resize and stuff.

4. If i want a initrd as mimial as possible (Just be able to boot, NO modules support in kernel) , is it possible to use an initrd without module support in the kernel and what exactly kernel config options is needed?

5. Should i use dracut, or make my own generic initrd that should work with all kernels?

Any tips, whats your solution?

----------

## ulenrich

```
etc # zcat /proc/config.gz |grep -i initrd ; emerge -p -1 dracut

CONFIG_BLK_DEV_INITRD=y

# CONFIG_ACPI_INITRD_TABLE_OVERRIDE is not set

These are the packages that would be merged, in order:

Calculating dependencies  .... ....... done!

[ebuild   R    ] sys-kernel/dracut-034  USE="-debug device-mapper -net (-selinux)" DRACUT_MODULES="-biosdevname -bootchart -btrfs caps -cifs crypt -crypt-gpg -crypt-loop dash -dmraid -dmsquash-live -gensplash -iscsi -livenet -lvm -mdraid -multipath -nbd -nfs -plymouth -ssh-client -syslog systemd" 0 kB
```

I guess initrd is working with forbidden modules load configured.

Many Redhat-conspiracy devotees think dracut is bloated, but for me

- it runs in 9 seconds: "dracut -H initrd-3.11.4-1.bfs 3.11.4-1.bfs"

- it results a 6 MByte initrd file

I don't trust Gentoo maintainers of genkernel hacking around to get a slim initrd. One Gentoo developer in another thread just admitted Udev is needed in an initrd when using LVM2. The Redhat-haters in this forum made a self-full-filling prophecy out of it ...

Instead I split out device-mapper from Lvm2 to get a dmsetup only out of it (As it is supposed to do by upstream and done by Debian), because I need it for LUKS cryptsetup my /home partition:

https://bugs.gentoo.org/show_bug.cgi?id=479950Last edited by ulenrich on Thu Oct 10, 2013 10:47 am; edited 1 time in total

----------

## jenkler

OK, so. For a working initrd you only need:

CONFIG_BLK_DEV_INITRD=y

In the kernel. Why the hell is dracut masked and if its bloated what tool to use instead? Or is manual the only way?

If i choose the initrd solution for this issue what do i need to do more than 1 Kernel option, create an initrd file and put it in lilo.conf

```

 image=/boot/vmlinuz

  label="Linux"

  root="/dev/md3"

  initrd=/boot/initrd.img

```

I still wondering if this initrd file is needed on a server, what will break?. Is there anyone here that have som issues without initrd and a separate /usr part on a server system?

MD autodetect is in the kernel and all drivers is static. If you look att application such as PHP, Mysql, Nginx, syslog-ng, they should work fine. Or are they going to move commands like mount, bash to /usr too? I am confused  :Wink: 

----------

## frostschutz

A lot of things may be moving to /usr in the future. For that reason I do not use a separate /usr anymore. If you already have a partition for everything, your / will be a mostly empty filesystem anyway, so no reason not to put those couple of files onto the /usr partition as well, making /usr your new /. This transition can be handled easily from a rescue system, and you could keep using your Initramfs-free setup as long as the deprecated RAID autodetect keeps working.

Not that there's anything wrong with a nice and custom made Initramfs. ( http://gentoo-en.vfose.ru/wiki/Initramfs ) Plus it enables you to use newer md raid metadata, gpt partitions, etc. (things that aren't supported by the md autodetection).

Mine currently looks like this:

```

#!/bin/busybox sh

rescue_shell() {

    echo "Something went wrong. Dropping you to a shell."

    busybox --install -s

    exec /bin/sh

}

# Prepare

mount -t devtmpfs none /dev

mount -t proc none /proc

mount -t sysfs none /sys

echo 0 > /proc/sys/kernel/printk

# Assemble RAID:

( sleep 2 # disk not ready?

  mdadm --assemble --scan

  sleep 2

) &

# Unlock Key

cryptsetup luksOpen --header /root/key.luks /root/key KEY

wait # for mdadm

# Unlock SSD

( cryptsetup luksOpen --allow-discards --key-file=/dev/mapper/KEY --keyfile-offset=0 --keyfile-size=512 `findfs UUID="ae797aa3-83af-4f5d-9f87-9461044d7fd9"` luksSSD1

  # lvm vgscan

  lvm lvchange -a y SSD/root

  sleep 2

) &

# Unlock HDD

for i in 1 2 3 4 5 6 7 8

do

    cryptsetup luksOpen --key-file=/dev/mapper/KEY --keyfile-offset=$(($i*512)) --keyfile-size=512 /dev/md"$i" luksHDD"$i" &

done

wait # for cryptsetup / LVM

# Mount Root

mount -o ro `findfs UUID="fa15678f-7e7e-4a47-8ed2-7cea7a5d037d"` /mnt/root || rescue_shell 

# Clean up

cryptsetup luksClose KEY

echo 1 > /proc/sys/kernel/printk

umount /dev /proc /sys

# Switcheroo

exec switch_root /mnt/root /sbin/init

```

----------

## jenkler

Nice one frostschutz!

 *Quote:*   

> 
> 
> Embedding into the Kernel
> 
> If you want the initramfs to be embedded into the kernel image, edit your kernel config and set Initramfs source file(s) to the root of your initramfs, (e.g /usr/src/initramfs):

 

This seams to be my best option, because its completely transparent

What binarys do you have in your Initramfs?

----------

## frostschutz

busybox, cryptsetup, mdadm, lvm, all of them built statically by Gentoo. If Gentoo removes those static use flags, I'll have to start including libraries as well...

----------

## hydrapolic

My computer is partitioned like the following:

sigle disk /dev/sda

/dev/sda1: boot

/dev/sda2: root

/dev/sda3: lvm

The partitions usr,var,tmp,opt,home are on lvm.

This is an amd64~ machine, and messages like the following started to come up:

 * setting up tmpfiles.d entries for /dev ...

/lib64/rc/sh/tmpfiles.sh: line 237: uniq: command not found

With dracut-034 USE="device-mapper" and DRACUT_MODULES="lvm" and with "rd.auto rd.lvm=1" kernel options, it seems like /usr is still not mounted and the error appears. Anyone got it working with dracut?

With genkernel-3.4.47 (genkernel --lvm initramfs) and with dolvm kernel option it works just fine and the error disappeared.

----------

## jenkler

Sorry, I dont use genkernel or modules at all. I want my system as static as possible. I have decided to wait until some problem occur on my client first  :Wink: 

If anyone having issues with a plain separate / and /usr (Not related to LVM, btrfs or other layers) only ext4 like me, post your issues here  :Wink: 

----------

## lepgalle

 *jenkler wrote:*   

> If anyone having issues with a plain separate / and /usr (Not related to LVM, btrfs or other layers) only ext4 like me, post your issues here 

 

Yes, since yesterday. Nothing serious I guess however after update tmpfiles.d appeared in sysinit boot level which requires uniq to run. Sure enough it is in /usr/bin This is the first time an error is generated during boot because of the separate /usr partition.

----------

## Hu

The failure due to uniq being on /usr is funny, since the line that failed for you is probably tmpfiles_basenames="`printf "${tmpfiles_basenames}

" | sort | uniq`", printf is a shell command, sort is in /bin, and sort understands -u to render the sorted output unique.  Therefore, the first failure is entirely because someone wrote a suboptimal shell pipeline.

----------

## lepgalle

 *Hu wrote:*   

> The failure due to uniq being on /usr is funny, since the line that failed for you is probably tmpfiles_basenames="`printf "${tmpfiles_basenames}
> 
> " | sort | uniq`", printf is a shell command, sort is in /bin, and sort understands -u to render the sorted output unique.  Therefore, the first failure is entirely because someone wrote a suboptimal shell pipeline.

 

Indeed, you are right. It complains about line 237 which is exactly the one you picked. So, just for fun I edited the corresponding line in /lib64/rc/sh/tmpfiles.sh to tmpfiles_basenames="`printf "${tmpfiles_basenames}

" | sort -u`" and watch what is happening during next boot (I understand that this edit will not be permanent).

Thanks

----------

## Hu

If it works correctly, it would be worth filing a bug to have that change incorporated upstream.  Fewer processes in a pipeline is almost always a good thing.

----------

## hydrapolic

Thank you for the tip Hu, a bug report is opened: https://bugs.gentoo.org/show_bug.cgi?id=493736

----------

## jenkler

I Solved it by one big partition (ext4) 3 TB and a swap file. No more issues yey  :Wink: 

----------

## hydrapolic

@Hu,

your fix will appear in the next release of OpenRC, thanks for it

@jenkler,

I use a single root partition for xen dom0, but on other servers it makes sense to split up to increase security (mount options) or to prevent the logs to fill up your whole disk or maybe to use different filesystems for different tasks (ext4 is fine, but I had problems with it on vmware, others like jfs/xfs are also worth considering - check out the kernel changelog, you'll find commits fixing data corruption/memory leaks for ext4)

----------

## jenkler

I use monit to monitor disk and no users have shell access on my systems  :Wink:  I use 5% of my disk now so the log fillup is no problem for me. And you can allways use logrotate.

Its nice to have only one part (easy to backup and so on)

----------

