# qemu-kvm not allowing 64-bit kvm [resolved]

## dvh

Finally, I am having some success with kvm, running on linux on an mac mini (core2duo, 64-bit CPU).  I have successfully installed and run (with lots of help from this forum) a 32-bit arch linux VM using a kvm domain, so it is fast.  I now want to install a 64-bit guest, and this command:

```
virt-install -n a64-kvm -r 1024 --connect=qemu:///system -v --import --disk path=/opt/a64-kvm.vdi,format=vdi -w bridge=br0 --vnc --prompt --arch x86_64
```

 installs, starts, and runs the guest...but it runs slowly, so I know I am using qemu SW emulation.  examining the .xml config file for this guest, I see:

```
<domain type='qemu'>

  <name>a64-kvm</name>

  <uuid>deda9a94-817d-bec6-3088-4624098c4e85</uuid>

  <memory>1048576</memory>

  <currentMemory>1048576</currentMemory>

  <vcpu>1</vcpu>

  <os>

    <type arch='x86_64' machine='pc-0.12'>hvm</type>

    <boot dev='hd'/>

  </os>

  <features>

    <acpi/>

    <apic/>

    <pae/>

  </features>

  <clock offset='utc'/>

  <on_poweroff>destroy</on_poweroff>

  <on_reboot>restart</on_reboot>

  <on_crash>restart</on_crash>

  <devices>

    <emulator>/usr/bin/qemu-kvm</emulator>

    <disk type='file' device='disk'>

      <driver name='qemu' type='vdi'/>

      <source file='/opt/a64-kvm.vdi'/>

      <target dev='hda' bus='ide'/>

      <address type='drive' controller='0' bus='0' unit='0'/>

    </disk>

    <controller type='ide' index='0'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>

    </controller>

    <interface type='bridge'>

      <mac address='52:54:00:57:a3:9c'/>

      <source bridge='br0'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

    </interface>

    <serial type='pty'>

      <target port='0'/>

    </serial>

    <console type='pty'>

      <target type='serial' port='0'/>

    </console>

    <input type='mouse' bus='ps2'/>

    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>

    <video>

      <model type='cirrus' vram='9216' heads='1'/>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>

    </video>

    <memballoon model='virtio'>

      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>

    </memballoon>

  </devices>

</domain>

```

...verifying the qemu mode.

so I thought to force the domain by adding another parameter to the install command:

```
#virt-install -n a64-kvm -r 1024 --connect=qemu:///system -v --import --disk path=/opt/a64-kvm.vdi,format=vdi -w bridge=br0 --vnc --prompt --arch x86_64 --virt-type kvm
```

  the "virt type" should force use of kvm.  the response to this command is:

```
ERROR    Host does not support domain type 'kvm' for virtualization type 'hvm' arch 'x86_64'
```

...seems like maybe qeum-kvm doesn't recognize that I have a 64-CPU (and that would explain why it defaulted to SW emulation by default.

virsh capabilities yields this:

```
<capabilities>

  <host>

    <uuid>9cfe245e-d0c8-bd45-a79f-54ea5fbd3d97</uuid>

    <cpu>

      <arch>i686</arch>

      <model>n270</model>

      <vendor>Intel</vendor>

      <topology sockets='1' cores='2' threads='1'/>

      <feature name='lahf_lm'/>

      <feature name='lm'/>

      <feature name='xtpr'/>

      <feature name='cx16'/>

      <feature name='tm2'/>

      <feature name='est'/>

      <feature name='vmx'/>

      <feature name='ds_cpl'/>

      <feature name='pbe'/>

      <feature name='tm'/>

      <feature name='ht'/>

      <feature name='ss'/>

      <feature name='acpi'/>

      <feature name='ds'/>

      <feature name='pse36'/>

    </cpu>

    <migration_features>

      <live/>

      <uri_transports>

        <uri_transport>tcp</uri_transport>

      </uri_transports>

    </migration_features>

  </host>

...

```

so for some reason, it would appear that my CPU arch is set to i686, instead of x86_64.  This looks like something I may have done incorrectly when I first installed something.  how are these capabilites set, and how would I change them?  Am I maybe missing some kernel setting?  why would I be limited to effectively 32-bit HW virtualization with this CPU?  my 64-bit gentoo system boots and runs fine directly on the HW.  any ideas?

-dvhLast edited by dvh on Wed Sep 29, 2010 1:01 pm; edited 4 times in total

----------

## V-Li

Which QEmu package are you using and which QEmu targets are set?

----------

## dvh

Used the latest qemu-kvm package (with ~arch keyword) if I am not mistaken (will check for sure when I get home).  I followed this guide: http://en.gentoo-wiki.com/wiki/KVM when I installed...and I don't see anything about "targets", so my answer is that I really don't know.  how would I check this?

-dvh

----------

## V-Li

equery u qemu-kvm

----------

## idella4

dvh

you are lucky to have a developer of gentoo reply.  I have kvm working well.  Do you have this in your kernel .config

```

genny linux # grep KVM .config

CONFIG_HAVE_KVM=y

CONFIG_HAVE_KVM_IRQCHIP=y

CONFIG_HAVE_KVM_EVENTFD=y

CONFIG_KVM_APIC_ARCHITECTURE=y

CONFIG_KVM_MMIO=y

CONFIG_KVM=m

CONFIG_KVM_INTEL=m

# CONFIG_KVM_AMD is not set

```

emerge -pv reveals the required use flags which include the QEmu targets. A demo

```

genny idella # equery u qemu-kvm

[ Legend : U - flag is set in make.conf       ]

[        : I - package is installed with flag ]

[ Colors : set, unset                         ]

 * Found these USE flags for app-emulation/qemu-kvm-0.12.5:

 U I

 + + aio                             : Enables support for Linux's Async IO

 + + alsa                            : Enable alsa output for sound emulation

 + + bluetooth                       : Enables Bluetooth Support

 - - curl                            : Adds support for client-side URL transfer library

 + + esd                             : Enable esound output for sound emulation

 - - fdt                             : Enables firmware device tree support

 + + gnutls                          : Enable TLS support for the VNC console server

 - - hardened                        : activate default security enhancements for

                                       toolchain (gcc, glibc, binutils)

 - - kvm-trace                       : Allows you to use KVM tracing

 + + ncurses                         : Enable the ncurses-based console

 - - pulseaudio                      : Enable pulseaudio output for sound emulation

 - - qemu-ifup                       : Provides the qemu-ifup script for use with QEMU's

                                       built in bridging

 + + qemu_softmmu_targets_arm        : system emulation target

 + + qemu_softmmu_targets_cris       : system emulation target

 + + qemu_softmmu_targets_i386       : system emulation target

 + + qemu_softmmu_targets_m68k       : system emulation target

 + + qemu_softmmu_targets_microblaze : system emulation target

 + + qemu_softmmu_targets_mips       : system emulation target

 + + qemu_softmmu_targets_mips64     : system emulation target

 + + qemu_softmmu_targets_mips64el   : system emulation target

 + + qemu_softmmu_targets_mipsel     : system emulation target

 + + qemu_softmmu_targets_ppc        : system emulation target

 + + qemu_softmmu_targets_ppc64      : system emulation target

 + + qemu_softmmu_targets_ppcemb     : system emulation target

 + + qemu_softmmu_targets_sh4        : system emulation target

 + + qemu_softmmu_targets_sh4eb      : system emulation target

 + + qemu_softmmu_targets_sparc      : system emulation target

 + + qemu_softmmu_targets_sparc64    : system emulation target

 + + qemu_softmmu_targets_x86_64     : system emulation target

 + + qemu_user_targets_alpha         : userspace emulation target

 + + qemu_user_targets_arm           : userspace emulation target

 + + qemu_user_targets_armeb         : userspace emulation target

 + + qemu_user_targets_cris          : userspace emulation target

 + + qemu_user_targets_i386          : userspace emulation target

 + + qemu_user_targets_m68k          : userspace emulation target

 + + qemu_user_targets_microblaze    : userspace emulation target

 + + qemu_user_targets_mips          : userspace emulation target

 + + qemu_user_targets_mipsel        : userspace emulation target

 + + qemu_user_targets_ppc           : userspace emulation target

 + + qemu_user_targets_ppc64         : userspace emulation target

 + + qemu_user_targets_ppc64abi32    : userspace emulation target

 + + qemu_user_targets_sh4           : userspace emulation target

 + + qemu_user_targets_sh4eb         : userspace emulation target

 + + qemu_user_targets_sparc         : userspace emulation target

 + + qemu_user_targets_sparc32plus   : userspace emulation target

 + + qemu_user_targets_sparc64       : userspace emulation target

 + + qemu_user_targets_x86_64        : userspace emulation target

 + + sasl                            : Adds support for the Simple Authentication and

                                       Security Layer

 + + sdl                             : Enable the SDL-based console

 - - static                          : !!do not set this during bootstrap!! Causes

                                       binaries to be statically linked instead of

                                       dynamically

 - - vde  

```

----------

## dvh

Thanks to both Idella4 and V-Li.  Idella4: I believe that my kernel is correctly built...

```

dvh@gentoo-64 /usr/src/linux $ grep KVM .config

CONFIG_HAVE_KVM=y

CONFIG_HAVE_KVM_IRQCHIP=y

CONFIG_HAVE_KVM_EVENTFD=y

CONFIG_KVM_APIC_ARCHITECTURE=y

CONFIG_KVM_MMIO=y

CONFIG_KVM=y

CONFIG_KVM_INTEL=y

# CONFIG_KVM_AMD is not set
```

the only difference is that I built in the intel support as opposed to making a module

V-Li: I built qemu-kvm-0.12.5 with the default use flags, and ~keyword.  here is the result of the equery:

```

gentoo-64 portage # equery u qemu-kvm

[ Searching for packages matching qemu-kvm... ]

[ Colour Code : set unset ]

[ Legend : Left column  (U) - USE flags from make.conf              ]

[        : Right column (I) - USE flags packages was installed with ]

[ Found these USE variables for app-emulation/qemu-kvm-0.12.5 ]

 U I

 - + aio                             : Enables support for Linux's Async IO

 - - alsa                            : Adds support for media-libs/alsa-lib (Advanced Linux Sound Architecture)

 - - bluetooth                       : Enables Bluetooth Support

 - - curl                            : Adds support for client-side URL transfer library

 - - esd                             : Adds support for media-sound/esound (Enlightened Sound Daemon)

 - - fdt                             : Enables firmware device tree support

 - - gnutls                          : Adds support for net-libs/gnutls (TLS 1.0 and SSL 3.0 support)

 - - hardened                        : activate default security enhancements for toolchain (gcc, glibc, binutils)

 - - kvm-trace                       : Allows you to use KVM tracing

 + + ncurses                         : Adds ncurses support (console display library)

 - - pulseaudio                      : Adds support for PulseAudio sound server

 - - qemu-ifup                       : Provides the qemu-ifup script for use with QEMU's built in bridging

 - + qemu_softmmu_targets_arm        : system emulation target

 - + qemu_softmmu_targets_cris       : system emulation target

 - + qemu_softmmu_targets_i386       : system emulation target

 - + qemu_softmmu_targets_m68k       : system emulation target

 - + qemu_softmmu_targets_microblaze : system emulation target

 - + qemu_softmmu_targets_mips       : system emulation target

 - + qemu_softmmu_targets_mips64     : system emulation target

 - + qemu_softmmu_targets_mips64el   : system emulation target

 - + qemu_softmmu_targets_mipsel     : system emulation target

 - + qemu_softmmu_targets_ppc        : system emulation target

 - + qemu_softmmu_targets_ppc64      : system emulation target

 - + qemu_softmmu_targets_ppcemb     : system emulation target

 - + qemu_softmmu_targets_sh4        : system emulation target

 - + qemu_softmmu_targets_sh4eb      : system emulation target

 - + qemu_softmmu_targets_sparc      : system emulation target

 - + qemu_softmmu_targets_sparc64    : system emulation target

 - + qemu_softmmu_targets_x86_64     : system emulation target

 - + qemu_user_targets_alpha         : userspace emulation target

 - + qemu_user_targets_arm           : userspace emulation target

 - + qemu_user_targets_armeb         : userspace emulation target

 - + qemu_user_targets_cris          : userspace emulation target

 - + qemu_user_targets_i386          : userspace emulation target

 - + qemu_user_targets_m68k          : userspace emulation target

 - + qemu_user_targets_microblaze    : userspace emulation target

 - + qemu_user_targets_mips          : userspace emulation target

 - + qemu_user_targets_mipsel        : userspace emulation target

 - + qemu_user_targets_ppc           : userspace emulation target

 - + qemu_user_targets_ppc64         : userspace emulation target

 - + qemu_user_targets_ppc64abi32    : userspace emulation target

 - + qemu_user_targets_sh4           : userspace emulation target

 - + qemu_user_targets_sh4eb         : userspace emulation target

 - + qemu_user_targets_sparc         : userspace emulation target

 - + qemu_user_targets_sparc32plus   : userspace emulation target

 - + qemu_user_targets_sparc64       : userspace emulation target

 - + qemu_user_targets_x86_64        : userspace emulation target

 - - sasl                            : Adds support for the Simple Authentication and Security Layer

 - - sdl                             : Adds support for Simple Direct Layer (media library)

 - - static                          : !!do not set this during bootstrap!! Causes binaries to be statically linked instead of dynamically

 - - vde                             : Enable VDE-based networking

```

from this, it would look like my target should be supported.  I will keep looking...

-dvh

----------

## idella4

dvh

recompile & try as modules.  Go the whole hog and install virt-manager.  You may not want to use it in the long term but it can be a feedback tool.  It tells you whether or not kvm is active.

On creating a new vm, it offers to choose between qemu and kvm.

Add the modules kvm and kvm_intel or kvm_amd to autoloading.

On mine it works just right.

 *Quote:*   

> 
> 
> seems like maybe qemu-kvm doesn't recognize that I have a 64-CPU 
> 
> 

 

Your battle is not with qemu-kvm, it's with libvirt.  libvirt is used as a back-end by virtualisation platforms.

It is temperamental, but it generally is good with kvm.

The fault is not in libvirt.

I think modules in the kernel could fix it.

I shall copy your setup and see.  You have your guest already made so I have to catch you up.

----------

## dvh

idella4:  no need for you to try to duplicate my issue.  i recompiled the kernel with kvm and kvm_intel as modules, and I think I am making progress.  at least "virsh capabilities" now shows "X86_64" for my host cpu arch.  I'm not really sure what happened here...but it was probably some form of operator error.  You have been very helpful...helped me through Xen and now KVM.  I really appreciate it.

-dvh

----------

## Hu

=app-emulation/qemu-kvm-0.12.5 works fine with KVM_INTEL=y in the kernel.

----------

## dvh

for whatever reason, after inserting kvm and kvm_intel as modules (as opposed to built into the kernel), all seems to be working.  I now have a KVM 64-bit domain which is operating at what appears to be native speed.  This is WAY different than before, so I know that I now have kvm, and not just qemu.

in fact, my 64-bit domain is running arch linux using a .vdi filesystem image, directly from a virtualbox install.  I can now run the same VM from virtualbox (running on OSX) and from kvm (running on gentoo linux).  cool.

...and thanks for all the help.

-dvh

----------

## dvh

Hu:  I would have thought so too, and that's why I must have committed some other error somewhere in my process.  I don't know what it might be, but I do not have anything too exotic in my configuration.  I do know that earlier, when I first tried kvm_intel built into the kernel, there was some issue with creating the /dev/kvm device, which I had to work manually.  is that normal?

-dvh

----------

## idella4

 *dvh wrote:*   

> idella4:  no need for you to try to duplicate my issue.  i recompiled the kernel with kvm and kvm_intel as modules, and I think I am making progress.  at least "virsh capabilities" now shows "X86_64" for my host cpu arch.  I'm not really sure what happened here...but it was probably some form of operator error.  You have been very helpful...helped me through Xen and now KVM.  I really appreciate it.
> 
> -dvh

 

 :Very Happy:   :Cool:   :Laughing: 

Too late, I have constructed a copy.  Oh well, another vm to the list.

```

gentoo64 ~ # virsh -c qemu:///system

Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands

       'quit' to quit

virsh # list

 Id Name                 State

----------------------------------

  3 genny64              running

```

If you are all fixed up, please insert fixed up or solved in your thread title.

----------

## dvh

again, I appreciate the time you have given to help me as I stumble through this.   My stated problem is indeed resolved.  I have generally been using these VMs via ssh, with X forwarding their windows on my local desktop.  I have yet to start an entire X desktop on a VM in a window on the host (like virtualbox is setup to do).  Maybe that is next, or maybe my X forwarding is enough.

-dvh

----------

## Hu

 *dvh wrote:*   

> there was some issue with creating the /dev/kvm device, which I had to work manually.  is that normal?

 No.  After building KVM into the kernel, the system worked without further work by me.  Anything not handled by the kernel was done by the Gentoo system scripts/processes.

----------

## dvh

so I have had KVM working great for a few days now.  I rebooted my machine into OSX to take care of some other documentation task, and when I reboot now into my Gentoo-supported KVM system, I can no longer start a VM that was working prior...and I don't believe that I changed ANYTHING.  when I try to start a VM, I get this message:

```

gentoo-64-kvm dvh # virsh start a64-vdi

error: Failed to start domain a64-vdi

error: internal error Process exited while reading console log output: libvir: error : cannot open /sys/devices/system/cpu/cpu1/online: No such file or directory
```

looking at the subject directory, I see this:

```

gentoo-64-kvm cpu # pwd

/sys/devices/system/cpu

gentoo-64-kvm cpu # ls

cpu0  cpu1  cpufreq  cpuidle  kernel_max  offline  online  perf_events  possible  present
```

then...

```

gentoo-64-kvm cpu1 # pwd

/sys/devices/system/cpu/cpu1

gentoo-64-kvm cpu1 # ls

cache  cpufreq  cpuidle  microcode  thermal_throttle  topology
```

so you can see that the target file "online" does not exist in this directory.  I am reasonably sure that it DID exist in the past, as my VMs started fine.  I believe that this entire /sys directory is created on startup using the products of I think udev.  what I don't understand is why I see "online" at the cpu parent directory, but not where qemu-kvm expects it, at the cpu1 child directory.  just what is it that triggers the generation of these files...and what would change that generation?  if I grep the dmesg for any CPU indications, I see:

```

dvh@gentoo-64-kvm ~ $ dmesg | grep CPU

[    0.000000] SMP: Allowing 2 CPUs, 0 hotplug CPUs

[    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:2 nr_node_ids:1

[    0.000000] PERCPU: Embedded 28 pages/cpu @ffff880001a00000 s82536 r8192 d23960 u1048576

[    0.000000] SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1

[    0.000000] RCU-based detection of stalled CPUs is enabled.

[    0.001196] CPU: Physical Processor ID: 0

[    0.001200] CPU: Processor Core ID: 0

[    0.001205] mce: CPU supports 6 MCE banks

[    0.001215] CPU0: Thermal monitoring enabled (TM2)

[    0.018829] CPU0: Intel(R) Core(TM)2 CPU         T5600  @ 1.83GHz stepping 02

[    0.094014] Brought up 2 CPUs

[    0.198242] microcode: CPU0 sig=0x6f2, pf=0x20, revision=0x57

[    0.198253] microcode: CPU1 sig=0x6f2, pf=0x20, revision=0x57
```

so I believe that the startup process is indeed seeing two CPUs.  Any ideas what might be causing this?

-dvh

----------

## jeffk

dvh - 

If your qemu-kvm just stopped working in the last day or two, you might be seeing the same problem I am:

Forum post: qemu-kvm segfault ~amd64 0.12.5-r1 and 9999

----------

## dvh

Jeffk:

thanks for helping.  I had read your posting before, and maybe our issues are related, but I am not getting any segfaulting, and I can load the kvm and kvm-intel modules just fine.  my issue seems to be with the state of sysfs after the boot sequence is finished.  for some reason, the directory structure under /sys/devices/system/cpu does not contain the "online" and "offline" files within the cpu0 and cpu1 directories.  not sure why not, or what entity should create these.  I tried to create them manually, by copying the same name file from the parent directory, but I got an error and could not do so.

any help relative to the startup process and what might inhibit the creation of these files would be greatly appreciated.

-dvh

----------

