# [SOLVED] 2.6.32 doesnt find LvmVg after lksOpen, 2.6.31 does

## pansolero

Hello,

I need some help from LVM/LUKS/initramfs/boot-experts because my server does not boot after upgrading from 2.6.31 to 2.6.32 - using the same kernel-.config!

My athlon2-gentoo-amd64-server has one 500GB SATA drive

```

   Device Boot      Start         End      Blocks   Id  System

/dev/sda1               1          13      104391   83  Linux

/dev/sda2              14       60801   488279610   8e  Linux LVM

```

attached to an onboard 00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller. Sda1 ist just the ext2-bootpartition, while sda2 is LUKS-encrypted and contains

```
# cryptsetup luksDump /dev/sda2

LUKS header information for /dev/sda2

Version:        1

Cipher name:    aes

Cipher mode:    cbc-essiv:sha256

Hash spec:      sha1

Payload offset: 2056

MK bits:        256

MK digest:      30 e7 06 d2 72 3a.....1d a6 78

MK salt:        1d 6d bc .....b9 0a 82 6e 48

                d3 dc 3f 1f 27.... 6d e3 e7 a7

MK iterations:  10

UUID:           a38c7c....e11

Key Slot 0: ....
```

```
# ls -l /dev/vg/

total 0

lrwxrwxrwx 1 root root 19 2010-01-09 19:57 data -> /dev/mapper/vg-data

lrwxrwxrwx 1 root root 19 2010-01-09 19:57 home -> /dev/mapper/vg-home

lrwxrwxrwx 1 root root 20 2010-01-09 19:57 slash -> /dev/mapper/vg-slash

lrwxrwxrwx 1 root root 19 2010-01-09 19:57 swap -> /dev/mapper/vg-swap
```

```
# pvdisplay

  --- Physical volume ---

  PV Name               /dev/block/253:0

  VG Name               vg

  PV Size               465.66 GB / not usable 2.55 MB

  Allocatable           yes (but full)

  PE Size (KByte)       4096

  Total PE              119208

  Free PE               0

  Allocated PE          119208

  PV UUID               sFw3KV-uueQ-NCL2-28bI-JE86-oFuw-1t6mCd
```

```
# vgdisplay                                                                                                                              

  --- Volume group ---                                                                                                                                

  VG Name               vg                                                                                                                            

  System ID                                                                                                                                           

  Format                lvm2                                                                                                                          

  Metadata Areas        1                                                                                                                             

  Metadata Sequence No  5                                                                                                                             

  VG Access             read/write                                                                                                                    

  VG Status             resizable                                                                                                                     

  MAX LV                0                                                                                                                             

  Cur LV                4                                                                                                                             

  Open LV               4                                                                                                                             

  Max PV                0                                                                                                                             

  Cur PV                1                                                                                                                             

  Act PV                1                                                                                                                             

  VG Size               465.66 GB                                                                                                                     

  PE Size               4.00 MB                                                                                                                       

  Total PE              119208                                                                                                                        

  Alloc PE / Size       119208 / 465.66 GB                                                                                                            

  Free  PE / Size       0 / 0                                                                                                                         

  VG UUID               1GfpGK-G7p9-2LVh-s0Ms-6LPB-gL21-MMes0Y
```

```
# lvdisplay                                                                                                                              

  --- Logical volume ---                                                                                                                              

  LV Name                /dev/vg/swap                                                                                                                 

  VG Name                vg                                                                                                                           

  LV UUID                FZ6mal-HlzY-Yakx-b6d4-5vGT-nyY0-PLcQpT                                                                                       

  LV Write Access        read/write

  LV Status              available

  # open                 1

  LV Size                2.00 GB

  Current LE             512

  Segments               1

  Allocation             inherit

  Read ahead sectors     auto

  - currently set to     256

  Block device           253:1

  --- Logical volume ---

  LV Name                /dev/vg/slash

  VG Name                vg

  LV UUID                dfrUI8-TO7F-ZBBd-Xw2w-KswJ-r7fR-5n5qex

  LV Write Access        read/write

  LV Status              available

  # open                 1

  LV Size                30.00 GB

  Current LE             7680

  Segments               1

  Allocation             inherit

  Read ahead sectors     auto

  - currently set to     256

  Block device           253:2

  --- Logical volume ---

  LV Name                /dev/vg/home

  VG Name                vg

  LV UUID                dbuD4Y-C60k-TWz8-N9qp-zaoa-iPzq-3whMFC

  LV Write Access        read/write

  LV Status              available

  # open                 1

  LV Size                60.00 GB

  Current LE             15360

  Segments               1

  Allocation             inherit

  Read ahead sectors     auto

  - currently set to     256

  Block device           253:3

  --- Logical volume ---

  LV Name                /dev/vg/data

  VG Name                vg

  LV UUID                eMwBoS-640u-Xjpo-J2qC-EmRk-cAz1-l9xZ9e

  LV Write Access        read/write

  LV Status              available

  # open                 1

  LV Size                373.66 GB

  Current LE             95656

  Segments               1

  Allocation             inherit

  Read ahead sectors     auto

  - currently set to     256

  Block device           253:4
```

Long ago, I created the 2.6.31 kernel using genkernel. Then I emerged 2.6.32, copied the .config from 2.6.31, ran "make oldconfig"  and used 

```
genkernel --no-menuconfig --oldconfig --install --makeopts=-j5 --lvm --luks --disklabel all
```

 (also tried many other settings) to create the new 2.6.32 kernel. It installed fine.

This is my kernel configuration: https://politbuero.dyndns.org/2.6.32-kernelconfig.txt

This is my grub.conf:

```
title=Gentoo Linux (2.6.32-gentoo)

root (hd0,0)

kernel /boot/kernel-genkernel-x86_64-2.6.32-gentoo-r1 root=/dev/ram0 crypt_root=/dev/sda2 real_root=/dev/vg/slash dolvm init=/linuxrc

initrd /boot/initramfs-genkernel-x86_64-2.6.32-gentoo-r1

title=Gentoo Linux (2.6.31-gentoo-r1)

root (hd0,0)

kernel /boot/kernel-genkernel-x86_64-2.6.31-gentoo-r1 root=/dev/ram0 crypt_root=/dev/sda2 real_root=/dev/vg/slash dolvm init=/linuxrc

initrd /boot/initramfs-genkernel-x86_64-2.6.31-gentoo-r1
```

Its boots fine using 2.6.31 and fails using 2.6.32, as you can see on the following pictures:

https://politbuero.dyndns.org/2.6.31-works.jpg

https://politbuero.dyndns.org/2.6.32-fails.jpg

I have spent *many* hours recompiling the kernel and cursing at make menuconfig, trying to find the missing option. When I type "shell" after 2.6.32 has failed to find the vg, there indeed is no "/dev/vg/" as there is when the volume group "vg" is found. PLEASE, do you have an idea why its not finding the volume group after decrypting the LUKS device?

Sorry for the crappy pictures. And thanks for your help!

P.S:

```
# emerge --info

Portage 2.1.6.13 (default/linux/amd64/10.0, gcc-4.3.2, glibc-2.9_p20081201-r2, 2.6.31-gentoo-r1 x86_64)

=================================================================

System uname: Linux-2.6.31-gentoo-r1-x86_64-AMD_Athlon-tm-_II_X4_600e_Processor-with-gentoo-2.0.1

Timestamp of tree: Sun, 10 Jan 2010 04:00:01 +0000

ccache version 2.4 [disabled]

app-shells/bash:     4.0_p28

dev-lang/python:     2.6.2-r1

dev-util/ccache:     2.4-r7

dev-util/cmake:      2.6.4

sys-apps/baselayout: 2.0.1

sys-apps/openrc:     0.5.1-r1

sys-apps/sandbox:    1.6-r2

sys-devel/autoconf:  2.63-r1

sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2

sys-devel/binutils:  2.18-r3

sys-devel/gcc-config: 1.4.1

sys-devel/libtool:   2.2.6a

virtual/os-headers:  2.6.27-r2

ACCEPT_KEYWORDS="amd64"

CBUILD="x86_64-pc-linux-gnu"

CFLAGS="-march=amdfam10 -O2 -pipe"

CHOST="x86_64-pc-linux-gnu"

CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/lib/fax /usr/share/config /var/spool/fax/etc"

CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/udev/rules.d"

CXXFLAGS="-march=amdfam10 -O2 -pipe"

DISTDIR="/usr/portage/distfiles"

FEATURES="distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch"

GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"

LANG="en_US.utf8"

LC_ALL="en_US.utf8"

LDFLAGS="-Wl,-O1"

LINGUAS="en"

MAKEOPTS="-j5"

PKGDIR="/usr/portage/packages"

PORTAGE_CONFIGROOT="/"

PORTAGE_RSYNC_EXTRA_OPTS="-q"

PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"

PORTAGE_TMPDIR="/var/tmp"

PORTDIR="/usr/portage"

SYNC="rsync://rsync.gentoo.org/gentoo-portage"

USE="acl amd64 berkdb bzip2 cli consolekit cracklib crypt cups cxx dri fortran gdbm gnutls gpm iconv ipv6 loop-aes mmx modules mudflap multilib ncurses nls nptl nptlonly openmp pam pcre pppd python readline reflection session spl sse sse2 ssh ssl sysfs tcpd unicode xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="dav_fs dav_lock disk_cache access auth auth_basic authn_alias authn_anon authn_default authn_file dbd ext_filter file_cache mem_cache mime_magic proxy_connect proxy_http ident vhost_alias authn_dbd authz_default authn_dbm authz_dbm authz_groupfile authz_owner authz_user auth_dbm auth_anon auth_digest authz_host alias file-cache filter echo charset-lite cache disk-cache mem-cache ext-filter case_filter case-filter-in deflate mime-magic cern-meta expires headers usertrack unique-id proxy proxy-connect proxy-ftp proxy-http info include cgi cgid dav dav-fs vhost-alias speling rewrite log_config logio env setenvif mime status autoindex asis negotiation dir imap actions userdir so" ELIBC="glibc" FRITZCAPI_CARDS="fcusb" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa via vmware voodoo"

Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTDIR_OVERLAY
```

Last edited by pansolero on Wed Jan 13, 2010 2:22 pm; edited 1 time in total

----------

## Hu

What is the output of pvdisplay, vgdisplay, and lvdisplay when booted into 2.6.32?  I assume the output you gave was for 2.6.31, since everything was found and shown.

[Edit: fixed markup mistake.]Last edited by Hu on Sun Jan 10, 2010 10:21 pm; edited 1 time in total

----------

## pansolero

Hu, thank you for your answer!

I am currently not physically at the machine in question, but I can log into it. Yes, those outputs are from 2.6.31 running (not the rescue shell, but the real system). I tried using these commands in the 2.6.32 rescue-shell and I got "command not found", so I thought this was normal and the init scripts used other means of detecting LVs.

Now I just unpacked the initramfs's of both .31 and .32, and their bin/lvm files have the same md5sum. Found out that the init scripts just create e.g. vgscan symlinks to bin/lvm and then call e.g. "vgscan 2>/dev/null"

I'll see what the 2.6.32 vgscan says when its stderr is not redirected to /dev/null and then report back.

Thanks again, Hu, you made me discover the init scripts  :Smile: 

----------

## pansolero

I just got back from the machine and took more crappy photos...

https://politbuero.dyndns.org/problem/

There's really not much more to say than whats visible on the pictures... vgscan doesn't give any errors while finding nothing, dmesg has no errors either. There simply are no volume groups found. Even specifying --debug and --verbose didn't really help.

I was surpised to find no mention of the words LVM/lvm in dmesg, but its the same with the working 2.6.31 kernel, so I guess thats fine.

Does maybe https://politbuero.dyndns.org/problem/2.6.32-busybox-no-dev-vg-directory2.jpg give a clue to you?

I'm lost. Please advise!

----------

## pansolero

https://bugs.gentoo.org/show_bug.cgi?id=292833

----------

