# [solved] KVM + Network

## pactoo

Hello,

I am just in the process of getting into KVM and of course I am stuck with networking. I find a lot of documentation that uses nat and/or bridges, but none that uses the dummy interfaces. 

So I would like to know, where is the problem/disadvantage in creating and assigning a seperate dummy interface for each virtual machine? On first sight, way less hassle than bridging or configure tap interfaces?

Or why is it, that nobody seems to use them? Instead of dummy it of course could also be an eth0:1, eth0:2 a.s.o. Any implications I do not see?Last edited by pactoo on Sat Nov 07, 2009 5:02 pm; edited 1 time in total

----------

## jormartr

I am using vde right now. It gives flexibility.

----------

## pactoo

Thanks. I did not know about vde and I don't think I am able to grasp it's concept advantages yet, except maybe when you want to run virtual machines on multiple hosts which shall all appear to reside in the same subnet.  Which of course has its uses. But still, the documentation I found so far uses tap interfaces with vde. Why?

However, if you have one host with a couple of virtual machines, let's say way less then 10, what is wrong using dummy interfaces? Why mess with bridge, tun/tap or vde? Still I think I am missing something very basic.

----------

## Mad Merlin

What do you mean "dummy interfaces"? You can't have a VM use an aliased IP on the host unless you also put the guest into a (separate) bridge and do some fancy routing/NAT.

----------

## pactoo

The linux kernel can provide a dummy interface, that is a nic, that does not have a hardware associated with it, but otherwise treated like an ethernet card: 

ifconfig dummy0 192.168.10.1 netmask 255.25.255.0 up

You may need to "modprobe dummy", as long as this option has been compiled into your kernel. Of course.  And somewhere you have to provide a module option of many dummy interfaces are allowed at all. 

So running 10 dummy interfaces is like having 10 ethernet cards without a cable connected. Just, you do not need the hardware. 

So my question could also be: if I had 10 real Networkinterfaces, only one or two connected to the outer world, which then of course won't be used for the guests, why can't I simply assign each of my, lets say four, guests an individual interface? So guest one uses eth4 (or dummy0), guest two eth5 (or dummy1) and so on. 

I do not see any need for NAT or iptables here at all. And nowadays the kernel sets up obvious routes by itself, but even if not, loading a set of routes upon booting is still way less effort and easier to set up than hassle around with bridges, tap or vde and other userspace stuff. 

Which raises the question: Is there (and if, why) potential trouble if our four guest-cards are on the same subnet? If not, we do not even need routes - or only one to the hosting system. 

Again, wether we talk about dummy or ethernet does not make any difference. AFAIK

But, as nobody seems to do it that way, there must be reason which I currently fail to see.

----------

## Mad Merlin

Oh, I didn't know such a facility existed. However, I think you're thinking you can assign an existing interface to be used by the guest, which isn't actually possible.[1] Tap devices are special virtual devices that are meant to be plugged into bridges and used in unconventional ways (like bridging a guest, or creating a VPN connection). You can also plug real ethernet devices devices into bridges (obviously), but I don't think KVM can use a dummy interface in the same way it can use a tap device for the guest (though I could be mistaken, but if the dummy interface also worked, there would be seemingly no difference between it and a tap device).

The bridging setup is actually very simple, I'm not sure what your aversion to it is. In the simplest case (switching eth0 for br0 and plugging eth0 into br0), your new tap interfaces get automagically created and plugged into br0 and eth0 in the guest works just like it was a real NIC plugged into the same switch your host's eth0 is plugged into. No VDE, no routing, and no iptables is necessary. You can, of course, still do lots of crazy stuff with bridging if you want/need to.

[1] Actually that's partially a lie, with VT-d, you can pass through some types of hardware to your guests (network cards are the primary example, video cards will not work), but it requires hardware support that's nearly impossible to find, so if you don't know you have it, you likely don't have it.

----------

## Hu

As best I can tell, the dummy interface is the network interface equivalent of /dev/null, which makes it useless for virtual machines.  Bridging, NAT, and VDE are the typical choices for using networking with the guest, depending on your isolation, control, and topology requirements.  What are you trying to do?

----------

## pactoo

@Hu:

The /dev/null analogy is wrong. dummy interfaces behave like ehternet. Firewall, routing rules apply and work as if were real ethernet. I do have services running on multiple dummy interfaces and so far there is no noticable difference then when they were bound to real ethernet interfaces. So they are more like loopback devices.

But dummy is not the topic, I just used them because if I would have asked for real ethernet interfaces, people would have answered, that most people do not have so much of those to use one dedicated for each guest instead of going into my question. 

Mad Merlin comes pretty close to my problem: For one I still fail to understand why I can bridge from a virtual nic to a real nic, but not connect directly to a real one. In fact, as the virtual nic someway or the other needs to communicate with a real one, there must some form of commincation channel and why not make it communcate directly with the real stuff through this means?

Maybe this is the point where tap differs from dummy. Which leaves the question, why I cannot use tap like an ethernet interface, but have to bridge it? But at least there is some light at the end of the tunnel. 

For two, my hope went even further. Besides being able to establish commucation between virtual guests using classical routing, I was expecting not only to assign, but rather dedicate interfaces to the guest. 

So that traffic entering an dedicated interface will be passed on straight to a guest and cannot even be used to access the host, further, not even configured by it as it is under full control of the guest. The host just sees the routing target. Like I would love to be able to dedicate partitions to a certain guest.  

This way I could easily establish different and isolated subnets on my host. 

What problem do I have with bridging? Well, besides that this another set of tools required, which add a little complexity, my main problem ist, that bridges connect segments of the same network. That means, the guest have to be in the same subnet as at least one real interface on the host. This is not desired. Management of the host of course has to happen in a different subnet than the services are running. 

So even if the host has only one nic, that should be in a different subnet that the guests its hosting. As the host knows the IP adresses of its guests, it can easily route incoming request to the inquired guests/services

So what do I want to archive? Understanding of the concept, basically, and therefore I originally planned to implement following scenario, which pretty much should take it all:

Guest 1: Only local access, shall not be able to comunicate with anybody else.

Guest 2+3 (172.16.16.0/24): Shall talk to each other and some external client in the same subnet, but not to Guest 4+5 

Guest 4+5 (172.16.32.0/24): Shall talk to each other and some external client in a different subnet (10.1.1.0/24), but not to guest 2+3 or their client subnet

Guest 6 (172.16.48.10/24) Shall talk to Guest 7 only

Guest 7 (172.16.64.10/24) Shall talk to Guest 6 only

No guest should be able to establish a networkconnection to the host itself. 

1. Version

The host hast 2 Ethernetinferfaces:

eth0: Management and access to guests (192.168.0.1)

eth1: goes to internetrouter

2. Version:

The host itself has 4 Ethernetinterfaces:

eth0 Managementinterface. Access to the host should only be possible via this interface (of course, I can configure the ssh daemon that way...)

eth1 Same subnet as guest 2+3

eth2 Same subnet as the clients of guest 4+5

eth3 goes out to the internetrouter, which does some firewalling to prevent guest 4+5 to talk to the rest of the world.

And I would have liked to archive this by clean design, not by ugly Firewallhacks. NAT is a no go anyway. However, this seems currently to be impossible

----------

## pactoo

Post Scriptum:

 *Mad Merlin wrote:*   

>  ... but it requires hardware support that's nearly impossible to find, so if you don't know you have it, you likely don't have it.

 

I know I do not have it, currently I do not have any form of VT. However I wonder, wether AMDs iommu is not the same as VT-d? Not sure though, what processors do support it. Off course, this is off topic, but new hardware is needed soon anyway. Albeit most likely without VT-d or iommu.

----------

## Mad Merlin

You can actually use tap devices like real ethernet devices as well, just give them an IP address. Normally with KVM the tap devices are not given IP addresses as they just function as conduits to connect a guest to the bridge. It might be possible to give the tap device an IP address and then using routing to shuffle the packets off to somewhere else without connecting it to a bridge, but I've never heard of someone doing that before, so you'd probably be on your own with that one. (Actually, that sounds a lot like tun mode in OpenVPN... maybe it does work.)

The one key piece you're missing is that bridges do not actually need to be connected to any real interfaces, they're perfectly happy connecting any number of tap devices, or nothing at all.

What you want to achieve is actually pretty easy to do with bridges. For guests 6 and 7 for example, just create another bridge (say br1) without an IP address, which is not attached to any real ethernet devices, it only has the two tap devices for guest 6 and guest 7 plugged into it. This setup is functionally identical to 2 physical computers and an unmanaged switch, where the only thing plugged into the switch is the network cables for the two computers. They can talk to eachother, but not the outside or to the switch, and you can use whatever network addresses you want.

For guest 4 and 5, do as for 6 and 7 but with a new bridge (say br2), but also bridge in eth1 (which is physically connected to 172.16.32.0/24, and which somehow routes to 10.1.1.0/24). eth1 and br2 do not have IP addresses and the host is not accessable from the guest or outside clients through this interface.

For guest 2 and 3, create a third bridge (say br3) and do as for guest 4 and 5, but bridge in eth2 (which is physically connected to 172.16.16.0/24).

Finally, for guest 1, create a fourth bridge (say br4) and give br4 an IP address in the same subnet as the guest, plug the guest's tap into br4 and you're off to the races. Actually I'm not sure if you want the host to be reachable here or not, if not, I don't think you need the bridge at all here (or even an eth0 in the guest).

None of the above involves any special routing rules (at least on the host), nor any iptables/NAT, because you are effectively building whole new distinct networks. If you're not already, just think of bridges like a physical network switch, albeit with an effectively limitless number of ports.

Also, the host doesn't actually really know what the guest's IPs are, because the host doesn't configure that, the guest does. Of course, the host could find it out by sniffing network traffic as it all still passes through the host, but it's not as convenient as you might think.

 *pactoo wrote:*   

> Post Scriptum:
> 
>  *Mad Merlin wrote:*    ... but it requires hardware support that's nearly impossible to find, so if you don't know you have it, you likely don't have it. 
> 
> I know I do not have it, currently I do not have any form of VT. However I wonder, wether AMDs iommu is not the same as VT-d? Not sure though, what processors do support it. Off course, this is off topic, but new hardware is needed soon anyway. Albeit most likely without VT-d or iommu.

 

As I understand it, most CPUs support it, however most motherboards do not. Also, yes, AMD's IOMMU is basically the same thing as VT-d. To use KVM you will need VT though, otherwise it'll fall back of plain old QEMU (which is really really slow).

----------

## Hu

 *Mad Merlin wrote:*   

> It might be possible to give the tap device an IP address and then using routing to shuffle the packets off to somewhere else without connecting it to a bridge, but I've never heard of someone doing that before, so you'd probably be on your own with that one.

 

I do this sometimes.  It works fine.  The Gentoo Home Router guide is applicable here.

----------

## pactoo

Thanks very much four time an patience. I got a rather big step closer to understand the whole thing as opposed to simply follow howtos. Now I got something to play with, when my new hardware arrives.

----------

