# HELP - what driver for gentoo guest under kvm? SOLVED

## Moriah

KVM has a virtual IDE disk interface, and my system recognizes the virtual disk as /dev/sda1 and /dev/sda2 during the initramfs (I need initramfs because I use LVM2 for teh root filesystem on all my linux boxes so I can use snapshots during backups.), but when I try to run vgscan during the initramfs, it does not find the volume group, so I think (after much trial and error) that I must have the wrong disk driver installed.

The latest gentoo minimal live cd boots fine under kvm, so there has to be a driver that works; I just don't seem to be able to find it.

What disk driver should be configured for the qemu-kvm virtual disk?

----------

## Ant P.

You should be using the virtio driver for everything, including disks. They'll show up as /dev/vda when configured correctly.

----------

## NeddySeagoon

Moriah,

If you can see /dev/sda in the guest, you can run lspci to see what the emulated hardware is.

----------

## Moriah

I already did that with the live cd; it was the first thing that I did.  Here is the debian decode of the lspci -n, per standard pappy seeds method:

```

PCI ID   Works?   Vendor   Device   Driver   Kernel

80861237      Intel Corporation   440FX - 82441FX PMC [Natoma]      

80867000      Intel Corporation   82371SB PIIX3 ISA [Natoma/Triton II]      

80867010   Yes   Intel Corporation   82371SB PIIX3 IDE [Natoma/Triton II]   ata_piix   v2.6.25-

80867113   Yes   Intel Corporation   82371AB/EB/MB PIIX4 ACPI   i2c-piix4   v2.6.25-

15ad0405   Yes   VMware   SVGA II Adapter   vmwgfx   v3.2.35-

10ec8139   Yes   Realtek Semiconductor Co., Ltd.   RTL-8139/8139C/8139C+   8139cp,8139too   v2.6.25-

```

Looks like it wants piix to me:

```

# CONFIG_ATA_PIIX is not set

CONFIG_PATA_OLDPIIX=y

# CONFIG_PATA_MPIIX is not set

# CONFIG_I2C_PIIX4 is not set

```

Oops!    :Embarassed: 

Looks like I did not get everything set the way I thought.  I guess I need CONFIG_ATA_PIIX=y also?  Actually, on another real system that still uses IDE, CONFIG_ATA_PIIX=y is all that is set.

I'll try changing to that and see what I get.  Been a long time since I config-ed a kernel for IDE hardware...    :Confused: 

----------

## Moriah

Still no joy.    :Evil or Very Mad: 

Here is the image of my boot screen under lvm right after it tries the vgscan.  Note that it shows sda as scsi:

http://www.elilabs.com/~rj/kvm_boot.gif

Here is the init script from the initramfs:

```

(chroot) livecd initramfs # cat init

#!/bin/busybox sh

echo Mount the /proc and /sys filesystems.

mount -t proc none /proc

mount -t sysfs none /sys

sleep 10

#echo doing luksOpen

#cryptsetup luksOpen /dev/sda cryptoroot

#sleep 10

echo doing vgscan

lvm vgscan

sleep 10

echo doing vgchange

lvm vgchange -a y

sleep 10

echo mounting root

mount /dev/gentoo/rootfs /mnt/root

sleep 10

echo unmounting /proc and /sys

umount /proc

umount /sys

sleep 10

echo doing the pivot root

exec switch_root /mnt/root /sbin/init

sleep 10

echo 'OOPS!'

sleep 10

(chroot) livecd initramfs # 

```

Here is my manual lvm setup after ssh-ing into the live cd vm:

```

Please report any bugs you find to http://bugs.gentoo.org. Be sure to include

detailed information about how to reproduce the bug you are reporting.

Thank you for using Gentoo Linux!

livecd ~ # vgscan

  Reading all physical volumes.  This may take a while...

  Found volume group "gentoo" using metadata type lvm2

livecd ~ # vgchange -a y

  2 logical volume(s) in volume group "gentoo" now active

livecd ~ # swapon /dev/gentoo/swap

livecd ~ # mount /dev/gentoo/rootfs /mnt/gentoo

livecd ~ # mount /dev/sda1 /mnt/gentoo/boot

livecd ~ # mount -t proc none /mnt/gentoo/proc

livecd ~ # mount --rbind /sys /mnt/gentoo/sys

livecd ~ # mount --rbind /dev /mnt/gentoo/dev

livecd ~ # chroot /mnt/gentoo /bin/bash

livecd / # source /etc/profile

livecd / # export PS1="(chroot) $PS1"

(chroot) livecd / # 

```

Here is the lscpi from my vm when booting the live cd:

```

(chroot) livecd / # lspci

00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)

00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]

00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]

00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)

00:02.0 VGA compatible controller: VMware SVGA II Adapter

00:03.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 20)

(chroot) livecd / # 

```

Also, here are excerps from the dmesg of the live cd:

```

[    1.852718] ACPI: bus type scsi registered

[    1.854434] libata version 3.00 loaded.

[    6.040398] ata_piix 0000:00:01.1: version 2.13

[    6.040585] ata_piix 0000:00:01.1: setting latency timer to 64

[    6.048347] scsi0 : ata_piix

[    6.050047] scsi1 : ata_piix

[    6.050945] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc100 irq 14

[    6.050946] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc108 irq 15

[    6.203608] ata2.01: NODEV after polling detection

[    6.203958] ata1.01: NODEV after polling detection

[    6.204371] ata2.00: ATAPI: QEMU DVD-ROM, 1.2.2, max UDMA/100

[    6.204692] ata1.00: ATA-7: QEMU HARDDISK, 1.2.2, max UDMA/100

[    6.204694] ata1.00: 167772160 sectors, multi 16: LBA48 

[    6.205287] ata2.00: configured for MWDMA2

[    6.205924] ata1.00: configured for MWDMA2

[    6.206060] scsi 0:0:0:0: Direct-Access     ATA      QEMU HARDDISK    1.2. PQ: 0 ANSI: 5

[    6.208025] sd 0:0:0:0: [sda] 167772160 512-byte logical blocks: (85.8 GB/80.0 GiB)

[    6.208059] sd 0:0:0:0: [sda] Write Protect is off

[    6.208061] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00

[    6.208076] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[    6.210264]  sda: sda1 sda2

[    6.211362] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     1.2. PQ: 0 ANSI: 5

[    6.215467] sr0: scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray

[    6.215474] cdrom: Uniform CD-ROM driver Revision: 3.20

[    6.216383] sd 0:0:0:0: [sda] Attached SCSI disk

[    6.218067] sr 1:0:0:0: Attached scsi CD-ROM sr0

[    6.693937] sd 0:0:0:0: Attached scsi generic sg0 type 0

[    6.694851] sr 1:0:0:0: Attached scsi generic sg1 type 5

```

```

[  214.237625] Adding 3145724k swap on /dev/mapper/gentoo-swap.  Priority:-1 extents:1 across:3145724k 

[  219.606819] XFS (dm-0): Mounting Filesystem

[  219.622159] XFS (dm-0): Ending clean mount

```

Here are the likely relevant non-commented .config lines to consider:

```

CONFIG_ATA=y

CONFIG_ATA_SFF=y

CONFIG_ATA_BMDMA=y

CONFIG_ATA_PIIX=y

CONFIG_HAVE_IDE=y

CONFIG_SCSI_MOD=y

CONFIG_SCSI=y

CONFIG_SCSI_DMA=y

CONFIG_SCSI_TGT=y

CONFIG_SCSI_PROC_FS=y

CONFIG_SCSI_MULTI_LUN=y

CONFIG_SCSI_SCAN_ASYNC=y

```

So what drivers should I have chosen in the make menuconfig?

----------

## NeddySeagoon

Moriah,

You need 

```
 <*>       Intel ESB, ICH, PIIX3, PIIX4 PATA/SATA support 
```

under the SATA menu.

-- edit --

Thats there 

```
CONFIG_ATA_PIIX=y
```

do you have DEVTMPFS and mount DEVTEMPFS?

What about partition table support.  You probably only need MSDOS Partition Table support but what sort of partition table did you create?

----------

## Moriah

I used Pappy's seed 3.7.10-gentoo-r1-x86_64-08.config, which has:

```

(chroot) livecd linux # grep DEVTMPFS .config

CONFIG_DEVTMPFS=y

# CONFIG_DEVTMPFS_MOUNT is not set

(chroot) livecd linux # 

```

This is exactly how it is also set on my real physical laptop, which is the host.

The partitioning was done with fdisk, and I took the default in the seed, which has always worked before on real machines for me:

```

(chroot) livecd linux # grep -i part .config

# Partition Types

# CONFIG_PARTITION_ADVANCED is not set

CONFIG_MSDOS_PARTITION=y

(chroot) livecd linux # 

```

The live cd inside the chroot shows:

```

(chroot) livecd linux # fdisk -l

Disk /dev/sda: 85.9 GB, 85899345920 bytes, 167772160 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x94b38560

   Device Boot      Start         End      Blocks   Id  System

/dev/sda1            2048      104447       51200   83  Linux

/dev/sda2          104448   167772159    83833856   83  Linux

Disk /dev/mapper/gentoo-rootfs: 75.2 GB, 75161927680 bytes, 146800640 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/gentoo-swap: 3221 MB, 3221225472 bytes, 6291456 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

(chroot) livecd linux # 

```

And:

```

(chroot) livecd linux # lvdisplay 

  --- Logical volume ---

  LV Path                /dev/gentoo/rootfs

  LV Name                rootfs

  VG Name                gentoo

  LV UUID                5IEXIW-5rb2-hS8w-rsIn-hTcZ-YHEA-9uX0Bq

  LV Write Access        read/write

  LV Creation host, time livecd, 2013-04-24 00:57:04 -0400

  LV Status              available

  # open                 1

  LV Size                70.00 GiB

  Current LE             17920

  Segments               1

  Allocation             inherit

  Read ahead sectors     auto

  - currently set to     256

  Block device           253:0

   

  --- Logical volume ---

  LV Path                /dev/gentoo/swap

  LV Name                swap

  VG Name                gentoo

  LV UUID                oH73Sn-s1A5-9g4S-D9NO-Jp0H-gqfC-HefYSv

  LV Write Access        read/write

  LV Creation host, time livecd, 2013-04-24 00:57:25 -0400

  LV Status              available

  # open                 2

  LV Size                3.00 GiB

  Current LE             768

  Segments               1

  Allocation             contiguous

  Read ahead sectors     auto

  - currently set to     256

  Block device           253:1

   

(chroot) livecd linux # 

```

----------

## NeddySeagoon

Moriah,

About all thats left is to suggest that you are not booting the kernel you think you are.

Chase the date stamps from the .config, to the bzImage in the build location to the kernel in /boot, they should get progressively later. The last two may be identical.

Check for typos in file names in grub.conf and in /boot, maybe you didn't mount /boot ?

----------

## Moriah

All dates, names, and lengths check out.  I am booting the kernel I mean to be booting.

'Tis a puzzlement...  :Neutral: 

However, since my kernel seems to be able to find /dev/sda, /dev/sda1, and /dev/sda2, does that mean that the correct driver is present and working?  Maybe its a device manager LVM problem?  The logical volumes show up when I run the live cd, so its OK on the disk.  I've checksd and re-checked that LVM2 has been emerged, that the init script is correct in the initramfs, and that the device manager is installed in the kernel .config file. 

Whatever could the matter be?    :Crying or Very sad: 

This is driving me *NUTS*   :Exclamation:   :Exclamation:   :Exclamation: 

----------

## Moriah

To simplify the problem, I am building a simple gentoo system to run under kvm with a 10G virtual disk and 3 partitions: boot, swap, and root.

It will not use LVM.  We shall see if I can get it to boot.    :Twisted Evil: 

----------

## Moriah

Well, I got the simplified vm to boot, but there's no network device.    :Surprised: 

If the new udev is supposed to assign uniquely determined device names to my network devices, how come the name the live cd used (enp0s3) is not the name the freshly made kernel also uses?    :Question: 

The dmesg shows that the 8139cp driver was used for the hardware, but I can't tell what it got called.  If I ls /dev it shows no network devices that I can recognize.    :Crying or Very sad: 

What *IS* then name of my NIC in the new kernel?    :Question: 

----------

## NeddySeagoon

Moriah,

You don't get /dev entries for NICs.  Look in 

```
ifconfig -a
```

your lspci says

```
10ec8139   Yes   Realtek Semiconductor Co., Ltd.   RTL-8139/8139C/8139C+   8139cp,8139too 
```

Real 8139cp cards are very rare, its unlikely the KVM emulates one, I suspect you need the 8139too driver.

----------

## Moriah

In reality, I had a completely different problem: a typo in the /etc/conf.d/net file that set up the tap device the vm was connected to.    :Embarassed: 

Also in reality, it really *IS* a 8139cp, and my dmesg told me so, but before I looked at that, I changed to the 8139too driver, and only after that did I discover the above typo.    :Embarassed: 

So now with the /etc/conf.d/net configuring the tap device correctly, and the 8139cp driver, my simple stripped down gentoo vm is working.    :Very Happy: 

So I now have to go take a look at the kernel on my *REAL* gentoo vm and make sure the driver is set to 8139cp, and also figure out why the vgscan in the initramfs is not picking up my volume groups.    :Confused: 

I suspect I botched something in the lvm binary in the initramfs /bin directory, but who knows?  I will report back when I have investigated deeper.    :Mr. Green: 

----------

## NeddySeagoon

Moriah,

vgscan ?

My init from the initrd contains ...

```
# lvm runs as whatever its called as

ln -s /sbin/lvm.static /sbin/vgchange

# everything on the SDD

/sbin/vgchange -ay ssd || rescue_shell

# start the vg volume group - /home and everything for portage - need not die here

/sbin/vgchange -ay vg || rescue_shell

# if this failed we have no /usr or /var

# get here with raid sets assembled and logical volumes available

```

its the vgchange that runs your logical volumes. vgscan is not required.

----------

## Moriah

Neddy:

I boot to a root filesystem that is running LVM so that I can use the LVM snapshot target to freeze the state of the filesystem while I take a  backup while the system continues to run.

You example looks like it sets up LVM on a second drive/partition after you have already gotten the root filesystem mounted.

I don't know what the problem is.  From within the init script,  can see /dev/sda, /dev/sda1 (the boot partition, not under LVM), and /dev/sda2 (the partition that acts as the physical volume).  When the init script does ls /dev/sd*, it shows these drives and partitions.  But when the init script issues lvmdiskscan, it shows NO DRIVES!    :Shocked: 

I tried rebuilding lvm.static and copying it into the initramfs, but it made no difference.

I use this same technique to boot real physical machines all the time, so I am really boggled why it doesn't also work here.    :Crying or Very sad: 

----------

## NeddySeagoon

Moriah,

Well, yes and no.  I have root on LVM, usually on raid5, but that snippet is from a system with SSD, so no raid yet.

Its from the initrd, which is indeed the root elsewhere that you refer too It starts the ssd volume group (on the ssd) and the vg volume group (on raid5), as /home and /usr/portage is still there.

My main point was that vgscan is not required, only vgchange.

----------

## Moriah

Yes, I agree, the vgscan is not required, provided everything else is working properly, but if there *ARE* problems, the output of vgscan can provide some diagnostic help.

And right now, I *NEED* help.     :Sad: 

I am just learning about kvm and the associated qemu options and types of filesystems and networking to support it.  I have networking working with a natted bridge and taps to each vm, but it is not elegant yet, just kind of duct taped together.  I still need to learn more about bridge devices and the commands associated with them, and about taps.  The closest to these I have come so far is tun devices when setting up openvpn tunnels.

I can boot the gentoo minimal livecd in a vm and lvm works fine there.  I use lvm for my root filesystem on all my gentoo boxes already, with an initramfs to set things up, but the same technique is not working for me when I try to do it under a vm.    :Confused: 

I have win-xp-pro and w7-32bit working fine, but the images are on an xfs filesystem.  At least the images are on their own logical volume.  I have since learned that there are definate advantages to using a filesystem that supports sparse files, so the images for the vm's can be over-provisioned on the available filesystem space.  

I have also just learned that there is apparently a qemu option to say that a backing store for the vm's memory can be provided, so suspend/resume and migrate operations can work, and that there is an option to specify that the vm is to use the original image as read-only, and provide an additional file to use for copy-on-write.  I was thinking how to neatly to that with lvm and snapshots, but that got messy when I thought about sparse filesystems.  It wasn't too bad with fully pre-allocated images, each on a separate logical volume.  Apparently all that thinking was not necessary, as qemu looks like it takes care of that for me, so running a vm with a throw-away disk, starting with a known good image, should be easy.  Can I also spawn multiple levels of copy-on-write disk images, to implement chains of snapshots?  That was feature of vmware workstation that I liked a lot.

Where is the man page for qemu-kvm?  I do not seem to have one.  Is there a good book on qemu and kvm anywhere?  I have search my Safari-books-online subscription, but all I see is a bunch of system administration books that just touch on qemu and kvm.  The best thing I have found so far is a pdf from IBM on best practices, but it is not a complete reference, and assumes one already knows qemu and kvm pretty well.

----------

## NeddySeagoon

Moriah,

I used http://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/index.html as a beginners guide.

Its written around libvirtd but anything it can do, you can do at the command lime too.

There may well be an update to that document now.  I used it about two years ago.

----------

## Moriah

I think I might have found my problem with lvm in my initramfs.  I had just lvm vgscan, but I *SHOULD* have had lvm vgscan --mknodes instead.  I also should have had:

```

echo /sbin/mdev > /proc/sys/kernel/hotplug

mdev -s

```

before the vgscan.

Nevertheless, I am now trying to build a guest kernel (3.7.10-gentoo-r1) to use virtio for the disk and network, so I need to set VIRTIO_BLK for the disk, and VIRTIO_NET for the network device.  I am using make menuconfig to configure the kernel, along with Pappy's seeds as my starting point, but I cannot seem to find VIRTIO_BLK.  When I do "/virtio_blk" in menuconfig, it shows:

```

  | Symbol: VIRTIO_BLK [=n]      

  | Type  : tristate            

  | Prompt: Virtio block driver            

  |   Defined at drivers/block/Kconfig:511      

  |   Depends on: BLK_DEV [=y] && VIRTIO [=n]   

  |   Location:    

  |     "-> Device Drivers

  | (1)   -> Block devices (BLK_DEV [=y])    

```

Yet when I go to -> Device Drivers -> Block devices, I cannot find Virtio block driver anywhere.  What gives?    :Question: 

In a similar manner, "/virtio_net" shows that the Prompt: Virtio network driver (EXPERIMENTAL) should be in -> Device Drivers -> Network device support (NETDEVICES [=y]) -> Network core driver support (NET_CORE [=y])", but I cannot find that prompt either.

How do I configure for virtio disk and network devices if the prompts aren't there?    :Question: 

I was able to start kvm from the live cd with these virtio devices and it "just worked", so the underlying stuff is there; I just can't see how to configure my kernel to use it.     :Sad: 

I cannot even find these symbols in the .config file:

```

livecd gentoo # grep VIRT usr/src/3.7.10-gentoo-r1-x86_64-08.config 

# CONFIG_PARAVIRT_GUEST is not set

CONFIG_VIRT_TO_BUS=y

# CONFIG_FB_VIRTUAL is not set

# CONFIG_VIRTIO_PCI is not set

# CONFIG_VIRTIO_MMIO is not set

# CONFIG_VIRT_DRIVERS is not set

# CONFIG_DEBUG_VIRTUAL is not set

# CONFIG_VIRTUALIZATION is not set

livecd gentoo # 

```

Is the seed faulty, or is menuconfig faulty, or what?    :Question: 

----------

## Hu

Your understanding of menuconfig is faulty.  :Wink:   The output you posted shows that, as you said, Virtio block driver is under Block devices.  However, what you did not remark on, and may not have understood, is that you can only choose a value for that symbol after the expression in the Depends on line is true.  The output you posted shows the expression is currently false by way of VIRTIO=n.  Find the symbol VIRTIO (not VIRTIO_BLK or VIRTIO_NET), set it to true, and then you should have access to the Virtio block driver entry.

You can use Z in menuconfig to show unsatisfied entries, but you still will not be permitted to change them until you satisfy their dependencies.

----------

## Moriah

Thanks!    :Very Happy: 

menuconfig is a big improvement over the old line by line questionaire we had to use in the early days of slackware circa 1994, but all the current tools are pretty crude compared to what ought to be possible.  I have long wished for an expert system type of kernel configurator, and with tools like jess, it ought to be possible.  After all, Forgy's R1 that later became VAXCON was production rule based, and was probably the most successful expert system of the 1980's.  Maybe this could be a summer intern project for somebody.    :Idea: 

----------

## NeddySeagoon

/me looks at Pappy

----------

## Moriah

OK, I have a system all built and ready to boot except grub is giving me a hard time.  The system is built using a single /dev/vda virtual disk under kvm, and has 2 partitions: /dev/vda1 and /dev/vda2.  When I run grub, it refuses to accept the root command:

```

    GNU GRUB  version 0.97  (640K lower / 9216K upper memory)

 [ Minimal BASH-like line editing is supported.  For the first word, TAB

   lists possible command completions.  Anywhere else TAB lists the possible

   completions of a device/filename. ]

grub> root (hd0,0)

Error 21: Selected disk does not exist

grub> 

```

Tab completion does not seem to be working either, or at least is not helpful.

How do I set up grub to boot from a /dev/vda virtual disk?

----------

## NeddySeagoon

Moriah,

You will need to feed grub a device.map file.  You will be used to seeing these as grub will normally write one to tell what it found.

```
info grub
```

covers it, iirc.

----------

## Moriah

Thanks Neddy!  I never had to use the --device-map option before; I did not even know it existed.  It seems to have done the trick.    :Very Happy: 

----------

## s4e8

You need a /etc/lvm/lvm.conf in initramfs, with udev and sysfs off. eg:

```

devices {

    dir = "/dev"

    scan = [ "/dev" ]

    obtain_device_list_from_udev = 0

    filter = [ "r|/dev/nbd.*|", "a/.*/" ]

    sysfs_scan = 0

}

global {

    umask = 077

    units = "h"

    activation = 1

    proc = "/proc"

}

```

----------

## Moriah

i should have closed this thread some time ago, as after fixing grub, gentoo runs fine as a guest under kvm using virtio for both disk and network.

----------

