# unable to load radeon in vfio-pci  host

## guido.hatzsis

After much playing around with vfio-pci I was able to get the PCIe controller, and the ATI card and its audio controller on the same VFIO grouping. I added the vfio driver to the kernel and added the devices. The command line is

 intel_iommu=on iommu=pt rd.driver.pre=vfio-pci vfio-pci.ids=8086:0c01,1002:679a,1002:aaa0

When I run qemu with the two devices:

    -device vfio-pci,host=01:00.0,multifunction=on,x-vga=on,rombar=0,romfile=ati.rom \

     -device vfio-pci,host=01:00.1

the guest (ubuntu) can see the radeon card, but fails to load the module 

Oct 08 19:30:16 ubuntu kernel: [drm:radeon_get_bios [radeon]] *ERROR* Unable to locate a BIOS ROM

Oct 08 19:30:16 ubuntu kernel: radeon 0000:00:05.0: Fatal error during GPU init

Oct 08 19:30:16 ubuntu kernel: [drm] radeon: finishing device.

Oct 08 19:30:16 ubuntu kernel: [TTM] Memory type 2 has not been initialized

Oct 08 19:30:16 ubuntu kernel: radeon: probe of 0000:00:05.0 failed with error -22

lspci shows the device and being present but the module wont load. The same radeon driver works fine when running on the same card.

Also when I start qemu, the colors and mode on the intel graphics go all funny (like VGA 16 color mode)...

Whats going on here?

----------

## Anon-E-moose

Have you made sure the (kernel) drivers for radeon are not loaded before you do vfio?

I have a radeon as a second card and don't load any radeon drivers at startup, then I can either use the radeon for X or vfio passthrough.

On my system lspci -nnk shows

```
04:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Turks XT [Radeon HD 6670/7670] [1002:6758]

   Subsystem: PC Partner Limited / Sapphire Technology Turks XT [Radeon HD 6670/7670] [174b:e194]

   Kernel modules: radeon
```

Notice no "Kernel driver in use: " line.

edit to add:

I do have to bind the radeon after the vfio module is loaded

```
# radeon 6670 vga

  echo "1002 6758" >/sys/bus/pci/drivers/vfio-pci/new_id 

  echo 0000:04:00.0 > /sys/bus/pci/drivers/vfio-pci/bind

# radeon 6670 audio hdmi

  echo "1002 aa90" >/sys/bus/pci/drivers/vfio-pci/new_id 

  echo 0000:04:00.1 > /sys/bus/pci/drivers/vfio-pci/bind
```

and this is on the qemu line

```
-device vfio-pci,host=04:00.0 -device vfio-pci,host=04:00.1
```

----------

## guido.hatzsis

Yeah, thanks for that. No, I have removed the radeon kernel module completely. I wasn't able to get the card into the same group as the PCIe controller because the PCIe driver was being loaded first.

There is no module loaded on the host (radeon doesn't exist) and fglrx is blacklisted (because it crashes on loading).

The strange thing is that device appears on the guest running ubuntu, but the module doesn't load.

----------

## guido.hatzsis

[quote="Anon-E-moose"]Have you made sure the (kernel) drivers for radeon are [b]not[/b] loaded before you do vfio?

and this is on the qemu line

[code]-device vfio-pci,host=04:00.0 -device vfio-pci,host=04:00.1[/code][/quote]

Without the romfile (which should work) qemu says

[code]Device option ROM contents are probably invalid (check dmesg).

Skip option ROM probe with rombar=0, or load from file with romfile=

[/code]

and

[code]Oct 08 20:30:48 ubuntu kernel: radeon 0000:00:04.0: Invalid ROM contents

Oct 08 20:30:48 ubuntu kernel: radeon 0000:00:04.0: Invalid ROM contents

Oct 08 20:30:48 ubuntu kernel: [drm:radeon_get_bios [radeon]] *ERROR* Unable to locate a BIOS ROM

Oct 08 20:30:48 ubuntu kernel: radeon 0000:00:04.0: Fatal error during GPU init[/code]

----------

## Anon-E-moose

A google with "linux vfio R9 280" gives me this link which might give you some clues (I haven't read the whole link)

https://bbs.archlinux.org/viewtopic.php?id=162768&p=60

----------

## guido.hatzsis

[quote="Anon-E-moose"]A google with "linux vfio R9 280" gives me this link which might give you some clues (I haven't read the whole link)

https://bbs.archlinux.org/viewtopic.php?id=162768&p=60[/quote]

Thanks, I had read that link a few times. Actually I tried to use the VGA arb patch for the i915 but I could not get the patch to apply easily (it was a bit old).

I do suspect that its something to do with VGA arbitration, and I might look into disabling that. I also noticed that the audio driver on the i915 (ie. the one that plays audio out the HDMI) fails to work with IOMMU enabled, and for some reason (with both ubuntu and the stock udev) the snd-intal-hda controller is used to run the ATI radeon audio IP (which might be the reason why the audio is crap).

I will try to remove VGA arb from the kernel and see how I go.

----------

## hceline

I remember seeing something about Radeon cards and PCIe in KVM.

Found this doing a quick search http://vfio.blogspot.no/2014/08/dual-vga-assignment-geforce-radeon.html especially this part:

 *Quote:*   

> The importance of Q35 has been largely exaggerated.  If you are mostly concerned with assigning GPUs to Windows guests, Windows seems to be perfectly happy using the default 440FX chipset model.  Linux guests won't like this, particularly with Radeon cards because the driver blindly attempts to poke at the upstream PCIe root port.

 

Saw an more throughout explanation with fix somewhere but can't find it right now, hope the above helps.

----------

