# kvm and networking, not sure I understand the tutorials.

## DaggyStyle

hello,

I want to set kvm image with bridge networking where the guest will get the ip via dhcp, I've read the tutorials (hope I didn't missed anything) and I'm failing to understand two things:

how exactly instruct the adapter to get the ip?

what exactly is a tap interface and why would I need it.

anyone?

Thanks.

----------

## NeddySeagoon

DaggyStyle,

Network in the kvm guest works the same as in any hardware install.

You either sert up static data in /etc/resolv.conf and /etc/config.d/net or you install dhcpcd and have  a dynamic IP in the guest.

----------

## Mad Merlin

Just picture a bridge like a physical switch and a tap like a physical network cable. Your VM's tap (network cord) plugs into the host's bridge (switch) and it's just like it's on the same physical network (provided you also plug eth0 into the bridge).

----------

## DaggyStyle

 *NeddySeagoon wrote:*   

> DaggyStyle,
> 
> Network in the kvm guest works the same as in any hardware install.
> 
> You either sert up static data in /etc/resolv.conf and /etc/config.d/net or you install dhcpcd and have  a dynamic IP in the guest.

 

so I only make sure there is a different mac addr in general do the router will know it is hw card on his part?

 *Mad Merlin wrote:*   

> Just picture a bridge like a physical switch and a tap like a physical network cable. Your VM's tap (network cord) plugs into the host's bridge (switch) and it's just like it's on the same physical network (provided you also plug eth0 into the bridge).

 

mmm I see, good explanation, so I must have both br and tap.

ok, thanks.

----------

## DaggyStyle

regrading these configs (from the tutorial):

```
config_tap0="null"

tuntap_tap0="tap"

tunctl_tap0="-u joeuser"

mac_tap0="00:00:00:00:00:00"
```

for dhcp I should set config_tap0 to dhcp right?

why tunctl_tap0 needs user? to which user should I assign it? as I understand it, I just turn my computer to a virtual switch, so what should be the config for what is now eth0?

if my router is dhcp based on mac address, mac_tap0 should define what card I'm representing?

thanks.

----------

## NeddySeagoon

DaggyStyle,

Thats wildly different to my setup.  My KVM host has 5 real nics, so I have trimmed my net file for clarity.

```
config_eth1="null"

bridge_br0="eth1"

config_br0="192.168.10.254/24"
```

You could use dhcpcd there if you wanted to but this kvm host also runs my firewall in a VM, so its important its IP addresses don't move around.

One of he the VMs has 

```
config_eth0="192.168.10.119/24 brd 192.168.10.255"

routes_eth0="default via 192.168.10.253"
```

 Again, you could use dhcpcd here and just leave the file blank

Notice how this VM connects to the same subnet as br0 in the host, which uses the hosts eth1.

Its all kernel tun/tap in the background.

----------

## cach0rr0

 *DaggyStyle wrote:*   

> regrading these configs (from the tutorial):
> 
> ```
> config_tap0="null"
> 
> ...

 

-dhcp should be handled within the guest itself, not the host. As far as the host is concerned, tap0 for example, will have no IP. As far as the guest is concerned, however, tap0 is how the guest's eth0 communicates with the host. So within the guest, its eth0 will have an IP assigned (via DHCP, or static). In other words, on the host, you set tap0, tap1, etc, to have mostly 'null' configuration settings, and within the guest you tell eth0 to use DHCP (eth0 being how the guest sees its NIC)

-tunctl_tap0 only needs a user if you fire off 'qemu-kvm' using your regular user, or some other user besides root. 

-your router will see the MAC address defined in /etc/conf.d/net , but the networking bits on your machine will do the job of translating between the MAC the router/switch sees, and the MAC the guest itself things is its interface's MAC. 

To give you a working example, from my own machine with a single NIC: I have 2 KVM guests running currently, and this is my /etc/conf.d/net

```

bridge_br0="eth0 tap0 tap1 tap2"

brctl_br0="setfd 0" "sethello 0" "stp off"

rc_need_br0="net.tap0 net.tap1 net.tap2"

config_br0="192.168.1.85/24"

routes_br0="default via 192.168.1.1"

dns_domain_br0="whitehathouston.com"

dns_servers_br0="192.168.1.1"

dns_search_br0="whitehathouston.com"

config_tap0="null"

tuntap_tap0="tap"

tunctl_tap0="-u meat"

mac_tap0="52:54:00:12:34:56"

config_tap1="null"

tuntap_tap1="tap"

tunctl_tap1="-u meat"

mac_tap1="52:54:00:12:34:57"

config_tap2="null"

tuntap_tap2="tap"

tunctl_tap2="-u meat"

mac_tap2="52:54:00:12:34:58"

config_eth0="null"

```

What's important to pay attention to here is twofold:

-each TAP interface should have a different MAC address. These are somewhat arbitrary in the sense that they do not have any relation to your NIC's actual MAC address

-when I show you the command-line for how I start my KVM guests, you will notice that the MAC specified on the kvm command-line, is not the same as the MAC specified in /etc/conf.d/net ; again, the MAC I decided to use is somewhat arbitrary, and all that's really important is that, in both /etc/conf.d/net, and on the KVM command-line, the MAC you decide to use isn't one that's in use elsewhere on the network

So without further ado, this is the kvm command-line I use for these two guests

```

sudo /usr/bin/qemu-kvm -drive file=/share/kvm-img-backup/web/apache.img,if=virtio,boot=on -net nic,model=virtio,macaddr=00:1d:92:ab:3f:77 -net tap,ifname=tap0,script=no,downscript=no -m 768 -vnc 192.168.1.85:5 -balloon virtio &

/usr/bin/sudo /usr/bin/qemu-kvm -drive file=/share/kvm-img-backup/mail/postfix.img,if=virtio -net nic,model=virtio,macaddr=00:1d:92:ab:3f:78 -net tap,ifname=tap1,script=no,downscript=no -m 512 -vnc 192.168.1.85:6 -balloon virtio &

```

Notice the 'macaddr' setting. Also notice how there are two "-net' parameters. The first -net is what you specify as the guest sees things. The second -net is what you specify for host the host sees things. I am instructing traffic destined for tap0 to go to 00:1d:92:ab:3f:77, and traffic destined for tap1 to go to 00:1d:92:ab:3f:78, for lack of a long explanation. 

The bridge is what handles all incoming traffic, whether it's traffic destined for your NIC, or one of the virtual NIC's. 

So, if you make sure /etc/conf.d/net 's specified MAC address for say, tap0, is different from both the MAC address you specify on the kvm command line, and, unique on the network, you should be fine. And within the guest, you just run 'dhcpcd eth0' as you would if the guest were in fact a real machine. 

I don't use DHCP in my guests, but it should work just fine. My guests just have simple /etc/conf.d/net like so

```

config_eth0="192.168.1.80 netmask 255.255.255.0 brd 192.168.1.255"

routes_eth0="default via 192.168.1.1"

```

Nothing special as far as the guest is concerned. It has a real (private, but real) IP, and uses my little Linksys router as the gateway. 

Now, running services on these machines (e.g. http), there is no special treatment. If I configure my Linksys router (running ddwrt) to do 1:1 NAT, I can set my guest to use a private IP, as per above. But before I put this system behind my wireless router, I had my box connected directly to my ISP's router instead, and the same "it doesnt matter" approach applied; I made sure I enabled IP forwarding on the host machine, gave the host machine .90, and gave my guests .91 .92 .93 (the last octet of my available public IP's), told them to use .94 as a gateway. 

HTH

----------

## DaggyStyle

Neddy,

thanks for the example.

cach0rr0,

so all I care is about the guest mac and not the tup mac, I think I understood, but what about the eth0 connection? e.g. what should replace eth0 when it comes to the non kvm users?

----------

## Mad Merlin

There's really no reason to explicitly set the tap's MAC address, the kernel will general a unique one for you. Furthermore, if you (like many others in the past) try and set your tap's MAC address to the same MAC address as you set in the guest, the VM's network will cease to work.

----------

## cach0rr0

 *DaggyStyle wrote:*   

> 
> 
> cach0rr0,
> 
> so all I care is about the guest mac and not the tup mac, I think I understood, but what about the eth0 connection? e.g. what should replace eth0 when it comes to the non kvm users?

 

br0 becomes the "real" interface, through which eth0, tap0, tap1, etc, all communicate. 

br0 is the interface for which you will set up actual network settings for the Host OS, such as IP, netmask, gateway (gateway will be your router/firewall), etc

everything in and out of the host OS, whether it's related to KVM or not, even if the host happens to be your desktop (as mine is) and you're doing web browsing, even the web browsing is going out via br0. 

the stipulations on the MAC addresses are as follows:

-the MAC addresses assigned to any tap interfaces on the host OS should be unique. So, tap0 and tap1 must have different MAC addresses, and all of them should have a different MAC address from what eth0 has. Note that you do not specify a MAC for eth0 inside /etc/conf.d/net, eth0's config should just be "null". 

-the MAC address you tell the Host OS to give to tap0, should be different from the MAC address that the guest OS sees (which, is determined by the macaddr parameter on the kvm command-line)

as far as setting up DHCP, you do not care about telling tap0, tap1, etc to send DHCP requests. The host OS will not send DHCP requests for tap0, tap1, etc. The guest OS will send DHCP requests. 

a further example:

```

meat@ricker ~ $ sudo ifconfig br0

br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet 192.168.1.85  netmask 255.255.255.0  broadcast 192.168.1.255

        ether 00:24:8c:23:82:b9  txqueuelen 0  (Ethernet)

        RX packets 53590488  bytes 29458005066 (27.4 GiB)

        RX errors 0  dropped 270209  overruns 0  frame 0

        TX packets 51515671  bytes 17248106076 (16.0 GiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

meat@ricker ~ $ sudo ifconfig eth0

eth0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500

        ether 00:24:8c:23:82:b9  txqueuelen 1000  (Ethernet)

        RX packets 56221912  bytes 31198513438 (29.0 GiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 53885105  bytes 17777054305 (16.5 GiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

        device interrupt 18

$ sudo ifconfig tap0

tap0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500

        ether 52:54:00:12:34:56  txqueuelen 500  (Ethernet)

        RX packets 1929415  bytes 567106950 (540.8 MiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 2708876  bytes 1098365635 (1.0 GiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

$ ps -ef |grep qemu-system.*tap0

root     22639 22638  0 Jun18 ?        08:31:14 /usr/bin/qemu-system-x86_64 --enable-kvm -drive file=/share/kvm-img-backup/web/apache.img,if=virtio,boot=on -net nic,model=virtio,macaddr=00:1d:92:ab:3f:77 -net tap,ifname=tap0,script=no,downscript=no -m 768 -vnc 192.168.1.85:5 -balloon virtio

$ ssh root@bauer

Enter passphrase for key '/home/meat/.ssh/id_dsa': 

Last login: Tue Aug 28 00:21:17 CDT 2012 from 192.168.1.85 on pts/0

bauer ~ # ifconfig eth0

eth0      Link encap:Ethernet  HWaddr 00:1d:92:ab:3f:77  

          inet addr:192.168.1.80  Bcast:192.168.1.255  Mask:255.255.255.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:898896 errors:0 dropped:136 overruns:0 frame:0

          TX packets:793688 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000 

          RX bytes:318539549 (303.7 MiB)  TX bytes:259128917 (247.1 MiB)

```

Note the mac addresses in each^^^

NB: you can indeed let the kernel craft MAC addresses for the tapN interfaces. Personal preference, I've assigned them.

----------

## DaggyStyle

let me see if I got it right, I can do the following:

create br0 with tup0-1, assign mac address only to tup0 - ip will be assigned per mac address id.

tunctl_tap0 will be set to null (I want all users to be able to use it when running linux).

tunctl_tap1 will be set to "-g virt_users" for all virt users to be able to use it while running a vm.

bottom line, I get tup0 will be used for linux, tup1 will be assigned to vms where my router sets the ip per mac address given by tup config (linux session) or guest mac (vm sessions)

am I right?

----------

## cach0rr0

 *DaggyStyle wrote:*   

> let me see if I got it right, I can do the following:
> 
> create br0 with tup0-1, assign mac address only to tup0 - ip will be assigned per mac address id.
> 
> tunctl_tap0 will be set to null (I want all users to be able to use it when running linux).
> ...

 

br0 should have tap0-1 *and* eth0

so, eth0, tap0, tap0, should all be bridged *together* as members of br0. 

think of br0 the same way you would a physical switch; eth0, tap0, tap1, are all plugged into it, and they can communicate directly with each other just via the switch without a router in the picture. 

as for tap0 - "all users to be able to use it" - what are they using it for? Something completely unrelated to KVM? 

as for tap1 - this will be the tap interface for *one* VM. If you want a second VM, make a tap2, if you want a third, make tap3

your router will set the IP based on the mac address that it sees making the DHCP request. 

the mac address that it sees will be the mac address of the tapN interface; however, the guest (VM) is what's making the DHCP request.

----------

## DaggyStyle

 *cach0rr0 wrote:*   

>  *DaggyStyle wrote:*   let me see if I got it right, I can do the following:
> 
> create br0 with tup0-1, assign mac address only to tup0 - ip will be assigned per mac address id.
> 
> tunctl_tap0 will be set to null (I want all users to be able to use it when running linux).
> ...

 

that is what I've thought, thanks for the clarification.

 *cach0rr0 wrote:*   

> 
> 
> as for tap0 - "all users to be able to use it" - what are they using it for? Something completely unrelated to KVM? 
> 
> 

 

I'm running a multiseat system, one of the users is running a W7 vm (currently using vb but I want to switch to kvm or at least vmplayer, I just want to run some benchies)

so if I have 3 users on the system and only one is using tap1, I want the other two to be able to use tap0.

 *cach0rr0 wrote:*   

> 
> 
> as for tap1 - this will be the tap interface for *one* VM. If you want a second VM, make a tap2, if you want a third, make tap3
> 
> 

 

understood, just to make sure, 

```
tunctl_tap1="-g virt_users" 
```

 will work, right?

 *cach0rr0 wrote:*   

> 
> 
> your router will set the IP based on the mac address that it sees making the DHCP request. 
> 
> the mac address that it sees will be the mac address of the tapN interface; however, the guest (VM) is what's making the DHCP request.

 

Mad Merlin mentioned that there is no need to set mac addrs for the tap, in that case the router should see the mac of the guest, right?

----------

## cach0rr0

[quote="DaggyStyle"]

understood, just to make sure, 

```
tunctl_tap1="-g virt_users" 
```

 will work, right?

that should work, yes

 *DaggyStyle wrote:*   

> 
> 
> Mad Merlin mentioned that there is no need to set mac addrs for the tap, in that case the router should see the mac of the guest, right?

 

it will still see the tapN interface. 

the difference - if you dont specify a mac address for the tap interface in /etc/conf.d/net, from what he's saying the kernel will create one for you. 

in other words, it doesnt change which interface the router sees. It still sees the tap interface. But instead of seeing a tap interface whose MAC you have hard-coded yourself, it sees a tap interface for which the kernel has created a MAC address. As far as the router is concerned, whether or not you specify a mac address in /etc/conf.d/net makes no difference.

----------

## DaggyStyle

so either way, the kernel will assign a mac to the tup which means that it will get a ip, this means that for each vm, I'll have two ips assigned?

----------

## Mad Merlin

There's a MAC address for tap0, this is for the local kernel's uses, and will never be seen in the guest or on external hosts (hence the desire for autogeneration, it really doesn't matter what it is, as long as it is unique).

There's also a MAC address that you assign to the virtual NIC in the guest (this is passed on the qemu-kvm command line). This is what the guest will see its MAC address as, and this is the MAC address that external hosts will see as being associated with the guest. This you must assign manually (unless you only have 1 VM, then the hardcoded default will do).

----------

## DaggyStyle

 *Mad Merlin wrote:*   

> There's a MAC address for tap0, this is for the local kernel's uses, and will never be seen in the guest or on external hosts (hence the desire for autogeneration, it really doesn't matter what it is, as long as it is unique).
> 
> There's also a MAC address that you assign to the virtual NIC in the guest (this is passed on the qemu-kvm command line). This is what the guest will see its MAC address as, and this is the MAC address that external hosts will see as being associated with the guest. This you must assign manually (unless you only have 1 VM, then the hardcoded default will do).

 

I want each vm to have one ip address, from what I understand, I need to set it up in tup config and pass it to the vm, right?

also, is there a way to prevent the br from getting ip? meaning only the tups should get ip.

----------

## NeddySeagoon

DaggyStyle,

I admit to being lazy and using virt-manager for these things.

----------

## DaggyStyle

 *NeddySeagoon wrote:*   

> DaggyStyle,
> 
> I admit to being lazy and using virt-manager for these things.

 virt-manager means that I need to install parts of gnome that I won't accept.

but still there is no guarantee that it does what I need.

thanks for the help though.

btw, is there a hint on where to ask?

----------

## Mad Merlin

 *DaggyStyle wrote:*   

>  *Mad Merlin wrote:*   There's a MAC address for tap0, this is for the local kernel's uses, and will never be seen in the guest or on external hosts (hence the desire for autogeneration, it really doesn't matter what it is, as long as it is unique).
> 
> There's also a MAC address that you assign to the virtual NIC in the guest (this is passed on the qemu-kvm command line). This is what the guest will see its MAC address as, and this is the MAC address that external hosts will see as being associated with the guest. This you must assign manually (unless you only have 1 VM, then the hardcoded default will do). 
> 
> I want each vm to have one ip address, from what I understand, I need to set it up in tup config and pass it to the vm, right?
> ...

 

The tap config says nothing about the guest's network config aside from what network it will have access to (by virtue of what bridge it's plugged into). Just remember the network configuration in the guest is exactly like it would be on real hardware. The host config controls:

1) How many physical network interfaces the guest has.

2) What network said interfaces are plugged into.

3) The MAC address of the network interfaces.

That is to say, everything hardware related, stuff that you'd physically change on a real machine. If you want one IP address in the guest, then set one IP address in the guest, just like you would on real hardware.

Also, if you don't want the bridge to have an IP address, then don't assign it one. Please note that if you only have one network interface on the host, and that network interface is plugged into the bridge, the host will not be able to communicate with the rest of the network (but the guests will be able to).

----------

## DaggyStyle

so if br0 doesn't have an ip, it is impossible to use both tun0 and tun1 at the same time?

----------

## Mad Merlin

 *DaggyStyle wrote:*   

> so if br0 doesn't have an ip, it is impossible to use both tun0 and tun1 at the same time?

 

No, whether or not the bridge has an IP has exactly zero impact on the behaviour of the taps (and guests).

----------

## DaggyStyle

 *Mad Merlin wrote:*   

>  *DaggyStyle wrote:*   so if br0 doesn't have an ip, it is impossible to use both tun0 and tun1 at the same time? 
> 
> No, whether or not the bridge has an IP has exactly zero impact on the behaviour of the taps (and guests).

 

so I didn't understood you, let's try the following:

I have:

two seats on my comp, meaning two can use the machine at the same time.

two groups of users, ones that use linux and others that uses windows via visualization sw.

I need:

when a user uses linux, he should have is own network adapter with a specific mac address (linux configured) for dhcp usage.

when a user uses windows via visualization sw, he should have is own network adapter with a specific mac address (visualization sw based) for dhcp usage.

I do:

create BR0 without mac address.

create tun0 with mac address for the linux group.

create tun1 without mac for the vm group.

define a mac address within the vm.

I get: two users can use the system, have network working for both, each using it's own virtual adapter where each is getting a reserved ip from the router based on the mac address.

am I right?

----------

## Mad Merlin

 *DaggyStyle wrote:*   

> 
> 
> I do:
> 
> create BR0 without mac address.
> ...

 

 Create br0 without MAC address

 Plug eth0 into br0

 qemu-kvm -net nic,macaddr=52:54:00:00:00:00 -net tap -hda linux.img ...

 qemu-kvm -net nic,macaddr=52:54:00:00:00:01 -net tap -hda windows.img ...

Replace 52:54:00:00:00:00 and 52:54:00:00:00:01 with whatever MAC addresses your router expects.

----------

## DaggyStyle

 *Mad Merlin wrote:*   

>  *DaggyStyle wrote:*   
> 
> I do:
> 
> create BR0 without mac address.
> ...

 

ok, shouldn't that be this?

qemu-kvm -net nic,macaddr=52:54:00:00:00:00 -net tap0 -hda linux.img ...

qemu-kvm -net nic,macaddr=52:54:00:00:00:01 -net tap1 -hda windows.img ...

----------

## michaelbenson

Is there any other thread which has more information on this topic or where this topic has been discussed in detail.

----------

## Mad Merlin

 *DaggyStyle wrote:*   

>  *Mad Merlin wrote:*    *DaggyStyle wrote:*   
> 
> I do:
> 
> create BR0 without mac address.
> ...

 

No, -net tap will have qemu-kvm dynamically create a new tap device and use it. -net tap0 will not work.

There is the option of using already created tap devices, which is mainly useful for invoking qemu-kvm without root privileges (tap creation requires root), this happens with -net tap,ifname=tap0, for example.

----------

## DaggyStyle

 *Mad Merlin wrote:*   

>  *DaggyStyle wrote:*    *Mad Merlin wrote:*    *DaggyStyle wrote:*   
> 
> I do:
> 
> create BR0 without mac address.
> ...

 

ok, I understand, I'll give it a try, thanks.

----------

