# Qemu KVM VGA Passthrough

## bmadonna1401

Hello Everyone,

I have 2 Nvidida cards in my system a GTX 770 and a GTX TITAN Z].  I'm trying to do VGA pass through on my system for qemu KVM visualization.  It don't seem to matter if I put it before my systemd statement in my grub.cfg or after either way it doesn't work I do a demsg and it shows it then 2 seconds latter I check again and its disabled dmesg just comes up blank.  I am very confused needless to say.  Per the below comments I have updated my kernel to the gentoo 4.2 sources and I have attempted to update the Intel Microcode per the kernel.  When I do dmesg afterwards I don't see any output about my microcode being updated.  Also I could be mistaking believe vfio-pci and pci_stub were conflicting with each other so I attempted to black list pci_stub and it still continued to load.  So I rebuilt my kernel without PCI stub.  Which didn't help anything so I've gone back to trying to use PCI_stub again verses vfio-pci to reserve my devices.   Please see below for all my config files.  Any help would be greatly appreciated.  I'm not really understanding what is different on Gentoo verses if I run a non source distro because I've done this on Debian and Ubuntu and it worked fine with no patches and both of those were much older kernels.  Obviously I would prefer to run Gentoo because I'm on this forum.  Again I would appreciate any help or suggestions there seems to be very little on doing vga pass though on Gentoo on the internet at all.  

# PUT YOUR CONFIG HERE OR IN separate files named *.conf

# in /etc/dracut.conf.d

# SEE man dracut.conf(5)

# Sample dracut config file

#logfile=/var/log/dracut.log

#fileloglvl=6

# Exact list of dracut modules to use.  Modules not listed here are not going

# to be included.  If you only want to add some optional modules use

# add_dracutmodules option instead.

#dracutmodules+=""

# dracut modules to omit

#omit_dracutmodules+=""

# dracut modules to add to the default

add_dracutmodules+="crypt lvm mdraid resume"

# additional kernel modules to the default

add_drivers+="vfio vfio_iommu_type1 vfio_pci vfio_virqfd"

# list of kernel filesystem modules to be included in the generic initramfs

#filesystems+=""

# build initrd only to boot current hardware

#hostonly="yes"

#

# install local /etc/mdadm.conf

mdadmconf="yes"

# install local /etc/lvm/lvm.conf

#lvmconf="no"

# A list of fsck tools to install. If it's not specified, module's hardcoded

# default is used, currently: "umount mount /sbin/fsck* xfs_db xfs_check

# xfs_repair e2fsck jfs_fsck reiserfsck btrfsck". The installation is

# opportunistic, so non-existing tools are just ignored.

#fscks=""

# inhibit installation of any fsck tools

#nofscks="yes"

# mount / and /usr read-only by default

#ro_mnt="no"

# set the directory for temporary files

# default: /var/tmp

#tmpdir=/tmp

kernel_cmdline+=" rd.md.uuid=b4e75530:a2e57081:49ee5be2:083e7cb0 "  

kernel_cmdline+=" rd.md.uuid=3c8265c1:3681bfc6:1505f61b:9f5bbac1 "  

kernel_cmdline+=" rd.lvm.lv=ubuntu/usr "

kernel_cmdline+=" rd.luks.uuid=60203fec-148d-4efb-af42-af0c9c551cde "

kernel_cmdline+=" resume=UUID=caabc852-d622-4835-b72f-f9a4e183e26d "

/etc/default/grug

# Copyright 1999-2015 Gentoo Foundation

# Distributed under the terms of the GNU General Public License v2

# $Id$

#

# To populate all changes in this file you need to regenerate your

# grub configuration file afterwards:

#     'grub2-mkconfig -o /boot/grub/grub.cfg'

#

# See the grub info page for documentation on possible variables and

# their associated values.

GRUB_DISTRIBUTOR="Gentoo"

# Default menu entry

#GRUB_DEFAULT=0

# Boot the default entry this many seconds after the menu is displayed

GRUB_TIMEOUT=15

#GRUB_TIMEOUT_STYLE=menu

# Append parameters to the linux kernel command line

GRUB_CMDLINE_LINUX="init=/usr/lib/systemd/systemd"

#

# Examples:

#

# Boot with network interface renaming disabled

# GRUB_CMDLINE_LINUX="net.ifnames=0"

#

# Boot with systemd instead of sysvinit (openrc)

# GRUB_CMDLINE_LINUX="init=/usr/lib/systemd/systemd"

# Append parameters to the linux kernel command line for non-recovery entries

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on pci-stub.ids=10de:1184,10de:0e0a,8086:8d20,046d:0a44"

# Uncomment to disable graphical terminal (grub-pc only)

#GRUB_TERMINAL=console

# The resolution used on graphical terminal.

# Note that you can use only modes which your graphic card supports via VBE.

# You can see them in real GRUB with the command `vbeinfo'.

#GRUB_GFXMODE=640x480

# Set to 'text' to force the Linux kernel to boot in normal text

# mode, 'keep' to preserve the graphics mode set using

# 'GRUB_GFXMODE', 'WIDTHxHEIGHT'['xDEPTH'] to set a particular

# graphics mode, or a sequence of these separated by commas or

# semicolons to try several modes in sequence.

#GRUB_GFXPAYLOAD_LINUX=

# Path to theme spec txt file.

# The starfield is by default provided with use truetype.

# NOTE: when enabling custom theme, ensure you have required font/etc.

#GRUB_THEME="/boot/grub/themes/starfield/theme.txt"

# Background image used on graphical terminal.

# Can be in various bitmap formats.

#GRUB_BACKGROUND="/boot/grub/mybackground.png"

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to kernel

#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries

#GRUB_DISABLE_RECOVERY=true

# Uncomment to disable generation of the submenu and put all choices on

# the top-level menu.

# Besides the visual affect of no sub menu, this makes navigation of the

# menu easier for a user who can't see the screen.

#GRUB_DISABLE_SUBMENU=y

# Uncomment to play a tone when the main menu is displayed.

# This is useful, for example, to allow users who can't see the screen

# to know when they can make a choice on the menu.

#GRUB_INIT_TUNE="60 800 1"

/boot/grub/grub.cfg 

#

# DO NOT EDIT THIS FILE

#

# It is automatically generated by grub2-mkconfig using templates

# from /etc/grub.d and settings from /etc/default/grub

#

### BEGIN /etc/grub.d/00_header ###

if [ -s $prefix/grubenv ]; then

  load_env

fi

if [ "${next_entry}" ] ; then

   set default="${next_entry}"

   set next_entry=

   save_env next_entry

   set boot_once=true

else

   set default="0"

fi

if [ x"${feature_menuentry_id}" = xy ]; then

  menuentry_id_option="--id"

else

  menuentry_id_option=""

fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then

  set saved_entry="${prev_saved_entry}"

  save_env saved_entry

  set prev_saved_entry=

  save_env prev_saved_entry

  set boot_once=true

fi

function savedefault {

  if [ -z "${boot_once}" ]; then

    saved_entry="${chosen}"

    save_env saved_entry

  fi

}

function load_video {

  if [ x$feature_all_video_module = xy ]; then

    insmod all_video

  else

    insmod efi_gop

    insmod efi_uga

    insmod ieee1275_fb

    insmod vbe

    insmod vga

    insmod video_bochs

    insmod video_cirrus

  fi

}

if [ x$feature_default_font_path = xy ] ; then

   font=unicode

else

insmod part_gpt

insmod part_gpt

insmod diskfilter

insmod mdraid1x

insmod lvm

insmod ext2

set root='lvmid/bMDGKI-zyTL-6C0p-58ca-4wUs-n7D2-EnbUPw/7AiFek-h3T1-8CW3-T4tZ-TsVj-PX9C-w3sfo8'

if [ x$feature_platform_search_hint = xy ]; then

  search --no-floppy --fs-uuid --set=root --hint='lvmid/bMDGKI-zyTL-6C0p-58ca-4wUs-n7D2-EnbUPw/7AiFek-h3T1-8CW3-T4tZ-TsVj-PX9C-w3sfo8'  5e69f903-1d04-4264-a805-b6c7d1a8dba1

else

  search --no-floppy --fs-uuid --set=root 5e69f903-1d04-4264-a805-b6c7d1a8dba1

fi

    font="/share/grub/unicode.pf2"

fi

if loadfont $font ; then

  set gfxmode=auto

  load_video

  insmod gfxterm

  set locale_dir=$prefix/locale

  set lang=en_US

  insmod gettext

fi

terminal_output gfxterm

if [ x$feature_timeout_style = xy ] ; then

  set timeout_style=menu

  set timeout=15

# Fallback normal timeout code in case the timeout_style feature is

# unavailable.

else

  set timeout=15

fi

### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/10_linux ###

menuentry 'Gentoo GNU/Linux' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-422c5dc6-8c37-4c68-9a85-913b2728a1f7' {

	load_video

	if [ "x$grub_platform" = xefi ]; then

		set gfxpayload=keep

	fi

	insmod gzio

	insmod part_gpt

	insmod ext2

	set root='hd0,gpt2'

	if [ x$feature_platform_search_hint = xy ]; then

	  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  fee8ca78-6dbe-4db7-9edc-b2453925b51d

	else

	  search --no-floppy --fs-uuid --set=root fee8ca78-6dbe-4db7-9edc-b2453925b51d

	fi

	echo	'Loading Linux 4.2.4-gentoo ...'

	linux	/vmlinuz-4.2.4-gentoo root=/dev/mapper/ubuntu-root ro init=/usr/lib/systemd/systemd intel_iommu=on pci-stub.ids=10de:1184,10de:0e0a,8086:8d20,046d:0a44

	echo	'Loading initial ramdisk ...'

	initrd	/initramfs-4.2.4-gentoo.img

}

submenu 'Advanced options for Gentoo GNU/Linux' $menuentry_id_option 'gnulinux-advanced-422c5dc6-8c37-4c68-9a85-913b2728a1f7' {

	menuentry 'Gentoo GNU/Linux, with Linux 4.2.4-gentoo' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.2.4-gentoo-advanced-422c5dc6-8c37-4c68-9a85-913b2728a1f7' {

		load_video

		if [ "x$grub_platform" = xefi ]; then

			set gfxpayload=keep

		fi

		insmod gzio

		insmod part_gpt

		insmod ext2

		set root='hd0,gpt2'

		if [ x$feature_platform_search_hint = xy ]; then

		  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  fee8ca78-6dbe-4db7-9edc-b2453925b51d

		else

		  search --no-floppy --fs-uuid --set=root fee8ca78-6dbe-4db7-9edc-b2453925b51d

		fi

		echo	'Loading Linux 4.2.4-gentoo ...'

		linux	/vmlinuz-4.2.4-gentoo root=/dev/mapper/ubuntu-root ro init=/usr/lib/systemd/systemd intel_iommu=on pci-stub.ids=10de:1184,10de:0e0a,8086:8d20,046d:0a44

		echo	'Loading initial ramdisk ...'

		initrd	/initramfs-4.2.4-gentoo.img

	}

	menuentry 'Gentoo GNU/Linux, with Linux 4.2.4-gentoo (recovery mode)' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.2.4-gentoo-recovery-422c5dc6-8c37-4c68-9a85-913b2728a1f7' {

		load_video

		if [ "x$grub_platform" = xefi ]; then

			set gfxpayload=keep

		fi

		insmod gzio

		insmod part_gpt

		insmod ext2

		set root='hd0,gpt2'

		if [ x$feature_platform_search_hint = xy ]; then

		  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  fee8ca78-6dbe-4db7-9edc-b2453925b51d

		else

		  search --no-floppy --fs-uuid --set=root fee8ca78-6dbe-4db7-9edc-b2453925b51d

		fi

		echo	'Loading Linux 4.2.4-gentoo ...'

		linux	/vmlinuz-4.2.4-gentoo root=/dev/mapper/ubuntu-root ro single init=/usr/lib/systemd/systemd

		echo	'Loading initial ramdisk ...'

		initrd	/initramfs-4.2.4-gentoo.img

	}

	menuentry 'Gentoo GNU/Linux, with Linux 4.2.4-gentoo.old' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.2.4-gentoo.old-advanced-422c5dc6-8c37-4c68-9a85-913b2728a1f7' {

		load_video

		if [ "x$grub_platform" = xefi ]; then

			set gfxpayload=keep

		fi

		insmod gzio

		insmod part_gpt

		insmod ext2

		set root='hd0,gpt2'

		if [ x$feature_platform_search_hint = xy ]; then

		  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  fee8ca78-6dbe-4db7-9edc-b2453925b51d

		else

		  search --no-floppy --fs-uuid --set=root fee8ca78-6dbe-4db7-9edc-b2453925b51d

		fi

		echo	'Loading Linux 4.2.4-gentoo.old ...'

		linux	/vmlinuz-4.2.4-gentoo.old root=/dev/mapper/ubuntu-root ro init=/usr/lib/systemd/systemd intel_iommu=on pci-stub.ids=10de:1184,10de:0e0a,8086:8d20,046d:0a44

		echo	'Loading initial ramdisk ...'

		initrd	/initramfs-4.2.4-gentoo.img

	}

	menuentry 'Gentoo GNU/Linux, with Linux 4.2.4-gentoo.old (recovery mode)' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.2.4-gentoo.old-recovery-422c5dc6-8c37-4c68-9a85-913b2728a1f7' {

		load_video

		if [ "x$grub_platform" = xefi ]; then

			set gfxpayload=keep

		fi

		insmod gzio

		insmod part_gpt

		insmod ext2

		set root='hd0,gpt2'

		if [ x$feature_platform_search_hint = xy ]; then

		  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  fee8ca78-6dbe-4db7-9edc-b2453925b51d

		else

		  search --no-floppy --fs-uuid --set=root fee8ca78-6dbe-4db7-9edc-b2453925b51d

		fi

		echo	'Loading Linux 4.2.4-gentoo.old ...'

		linux	/vmlinuz-4.2.4-gentoo.old root=/dev/mapper/ubuntu-root ro single init=/usr/lib/systemd/systemd

		echo	'Loading initial ramdisk ...'

		initrd	/initramfs-4.2.4-gentoo.img

	}

	menuentry 'Gentoo GNU/Linux, with Linux 4.1.10' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.1.10-advanced-422c5dc6-8c37-4c68-9a85-913b2728a1f7' {

		load_video

		if [ "x$grub_platform" = xefi ]; then

			set gfxpayload=keep

		fi

		insmod gzio

		insmod part_gpt

		insmod ext2

		set root='hd0,gpt2'

		if [ x$feature_platform_search_hint = xy ]; then

		  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  fee8ca78-6dbe-4db7-9edc-b2453925b51d

		else

		  search --no-floppy --fs-uuid --set=root fee8ca78-6dbe-4db7-9edc-b2453925b51d

		fi

		echo	'Loading Linux 4.1.10 ...'

		linux	/vmlinuz-4.1.10 root=/dev/mapper/ubuntu-root ro init=/usr/lib/systemd/systemd intel_iommu=on pci-stub.ids=10de:1184,10de:0e0a,8086:8d20,046d:0a44

		echo	'Loading initial ramdisk ...'

		initrd	/initramfs-4.1.10.img

	}

	menuentry 'Gentoo GNU/Linux, with Linux 4.1.10 (recovery mode)' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.1.10-recovery-422c5dc6-8c37-4c68-9a85-913b2728a1f7' {

		load_video

		if [ "x$grub_platform" = xefi ]; then

			set gfxpayload=keep

		fi

		insmod gzio

		insmod part_gpt

		insmod ext2

		set root='hd0,gpt2'

		if [ x$feature_platform_search_hint = xy ]; then

		  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  fee8ca78-6dbe-4db7-9edc-b2453925b51d

		else

		  search --no-floppy --fs-uuid --set=root fee8ca78-6dbe-4db7-9edc-b2453925b51d

		fi

		echo	'Loading Linux 4.1.10 ...'

		linux	/vmlinuz-4.1.10 root=/dev/mapper/ubuntu-root ro single init=/usr/lib/systemd/systemd

		echo	'Loading initial ramdisk ...'

		initrd	/initramfs-4.1.10.img

	}

	menuentry 'Gentoo GNU/Linux, with Linux 4.0.5-gentoo' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.0.5-gentoo-advanced-422c5dc6-8c37-4c68-9a85-913b2728a1f7' {

		load_video

		if [ "x$grub_platform" = xefi ]; then

			set gfxpayload=keep

		fi

		insmod gzio

		insmod part_gpt

		insmod ext2

		set root='hd0,gpt2'

		if [ x$feature_platform_search_hint = xy ]; then

		  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  fee8ca78-6dbe-4db7-9edc-b2453925b51d

		else

		  search --no-floppy --fs-uuid --set=root fee8ca78-6dbe-4db7-9edc-b2453925b51d

		fi

		echo	'Loading Linux 4.0.5-gentoo ...'

		linux	/vmlinuz-4.0.5-gentoo root=/dev/mapper/ubuntu-root ro init=/usr/lib/systemd/systemd intel_iommu=on pci-stub.ids=10de:1184,10de:0e0a,8086:8d20,046d:0a44

		echo	'Loading initial ramdisk ...'

		initrd	/initramfs-4.0.5-gentoo.img

	}

	menuentry 'Gentoo GNU/Linux, with Linux 4.0.5-gentoo (recovery mode)' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-4.0.5-gentoo-recovery-422c5dc6-8c37-4c68-9a85-913b2728a1f7' {

		load_video

		if [ "x$grub_platform" = xefi ]; then

			set gfxpayload=keep

		fi

		insmod gzio

		insmod part_gpt

		insmod ext2

		set root='hd0,gpt2'

		if [ x$feature_platform_search_hint = xy ]; then

		  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2  fee8ca78-6dbe-4db7-9edc-b2453925b51d

		else

		  search --no-floppy --fs-uuid --set=root fee8ca78-6dbe-4db7-9edc-b2453925b51d

		fi

		echo	'Loading Linux 4.0.5-gentoo ...'

		linux	/vmlinuz-4.0.5-gentoo root=/dev/mapper/ubuntu-root ro single init=/usr/lib/systemd/systemd

		echo	'Loading initial ramdisk ...'

		initrd	/initramfs-4.0.5-gentoo.img

	}

}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###

### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/30_os-prober ###

### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/40_custom ###

# This file provides an easy way to add custom menu entries.  Simply type the

# menu entries you want to add after this comment.  Be careful not to change

# the 'exec tail' line above.

### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###

if [ -f  ${config_directory}/custom.cfg ]; then

  source ${config_directory}/custom.cfg

elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then

  source $prefix/custom.cfg;

fi

### END /etc/grub.d/41_custom ###

current kernel config

https://bpaste.net/show/d599bfbcf33c

/etc/modules-load.d/vfio.conf

vfio

vfio_iommu_type1

vfio_pci

vfio_virqfdLast edited by bmadonna1401 on Sun Oct 25, 2015 8:59 pm; edited 2 times in total

----------

## Roman_Gruber

First.

I believe your kernel config is truncated anyway.

Better way:

```
wgetpaste /usr/src/linux/.config
```

2nd:

 I'm trying to dod VGA pass through on my system for qemu KVM visualization.

Your basic question is probably about qemu KVM visualization. Therefore you should probably change your topic to that qemu .... thing.

your iommnu thing may be or may be not related...

3rd gentoo vs vanilla sources may be not that different anyway regarding this iommnu thing.

so just use gentoo sources and try using something more recent.

afaik 4.2 slot will be anyway next stable kernel.org long term supported kernel.

I assume / expect you already updated every firmware / microcode and use those microcode utils also.

I also assume that you will try a more recent kernel and report back.

---

Usually on every forums there are some guidelines on how to ask questions.

your motivation is to get the virtualization running, i suppose, not the other stuff, therefore you should ask for that.

--

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on pci-stub.ids=10de:1184,10de:0e0a,8086:8d20,046d:0a44" 

some stuff can be tweaked with the proc directory usually when you change some values there. did you already tried that?

----------

## TigerJr

Much peoples have faced with intel_iommu interrupt remapping hardware bug, iommu need interrupt remapping for USB\PCI Passthrough for most hypervisors

On this forum i aready have answered about errata hardware bug, just take a closer look on this forum

----------

