# Bridged eth0, KVM, access to guest from network [solved]

## dkasak

Hi all. I'm having an interesting problem. I have to run a bunch of VMs, that should sit on the same network as other PCs on my local network. I've set up my bridged networking ( bridging an ethernet device, not wireless ) as per the instructions for systemd users, at: http://wiki.gentoo.org/wiki/Network_bridge#Host_Configuration_.28using_systemd.29. ie I have:

```
# bridge ports defined empty to avoid DHCP being run for their configuration

config_eth0="null"

# Bridge-config

config_br0="192.168.1.128"

routes_br0="192.168.1.254"

brctl_br0="setfd 0 sethello 10 stp on"

bridge_br0="eth0"
```

 ... and the /etc/systemd/system/bridge@.service file as per the instructions.

I've set up a couple of VMs now. I can ssh into them from the host, and also ssh to the host from the guest. But I can't ssh to guests from other PCs on the network, or visa-versa. I can, however, ping things. eg:

Pinging a VM from another PC on network:

```
ultrabook ~ # ping 192.168.1.130

PING 192.168.1.130 (192.168.1.130) 56(84) bytes of data.

64 bytes from 192.168.1.130: icmp_seq=1 ttl=64 time=2.21 ms

64 bytes from 192.168.1.130: icmp_seq=2 ttl=64 time=1.94 ms

^C

--- 192.168.1.130 ping statistics ---

2 packets transmitted, 2 received, 0% packet loss, time 1002ms

rtt min/avg/max/mdev = 1.941/2.078/2.215/0.137 ms

ultrabook ~ #

```

Attempting to ssh into it:

```
ultrabook ~ # ssh 192.168.1.130

ssh: connect to host 192.168.1.130 port 22: No route to host

ultrabook ~ #

```

Why would that be? I'm admittedly not so hot on networking   :Confused: 

Oh - on the VM host:

```
tesla ~ # brctl show

bridge name     bridge id               STP enabled     interfaces

br0             8000.d050991071d2       no              eth0

                                                        vnet0

virbr0          8000.000000000000       yes

virbr1          8000.525400af33dd       yes             virbr1-nic

                                                        vnet1

                                                        vnet2

tesla ~ #

```

Any help greatly appreciated  :Smile: Last edited by dkasak on Fri Jun 13, 2014 12:09 am; edited 1 time in total

----------

## schorsch_76

You need to add tap devices to the bridge. These tap devices are used by the qemu/kvm VMs to access the network.

I do it the same. Here is my config:

```

#################################################

# the bridge br0 contains eth0 and tap0

# tap0: virtual machine

# tap1: VPN 

# eth0: Network

#################################################

config_eth0="null"

tuntap_tap0="tap"

config_tap0="null"

tunctl_tap0="-t tap0 -u me"

config_br0="192.168.178.19/24"

routes_br0="default via 192.168.178.1"

brctl_br0="setfd 0 sethello 10 stp off"

bridge_br0="eth0 tap0"

depend_br0() {

    need net.eth0

    need net.tap0

}

```

----------

## dkasak

Thanks for the response. Alas, this config doesn't work for me, and in fact it breaks connectivity completely. Maybe it's because I'm using systemd? I end up with BOTH eth0 and br0 'active' and with IP 192.168.1.128. I've never seen eth0 get an IP address when using bridging. I guess for now, I can try to revert to 'vanilla' networking, and use KVM's "macvtap" thing. It's a bit annoying that you can access the host from the guest, but I'm looking at workarounds, and they're not TOO bad.

----------

## schorsch_76

systemd should not have effect on the network. I am not sure how systemd manages "depends_br0()". I use plain openrc without networkmanager/wicd/whatever.

In bridge setup only the bridge interface has an ip address. Maybe you need to shut the network complete down and start it again. The important thing is, that the tap device and eth0 are in the same bridge and only the br0 has the ip adress. The tap device get assigned to the QEMU VM's. If you use systemd or openrc to setup this config doesn't matter.

----------

## dkasak

Yeah I rebooted multiple times while attempting to figure things out.

I'm pretty sure systemd is involved in all this, as it's the init scripts that parse the config files, bring up the interfaces, etc. The WIKI ( http://wiki.gentoo.org/wiki/Network_bridge#Host_Configuration_.28using_systemd.29 ) includes a new service file /etc/systemd/system/bridge@.service that has 'simple' support for bridging, but doesn't appear to bring up the tap0 interface. So I guess this isn't "strictly" a systemd issue, but an issue with the much simpler systemd service file that I'm using, compared to the more complex networking scripts that Gentoo has had up until systemd arrived.

----------

## schorsch_76

Ok, then it is most probably a issue with the systemd service file.

----------

## steveL

Lovely tip schorsch. I've saved it off in my tips thread. 

Would be cool to see the setup for the other VPN interface as well.

Thanks,

steveL.

----------

## Aiken

 *schorsch_76 wrote:*   

> You need to add tap devices to the bridge. These tap devices are used by the qemu/kvm VMs to access the network.
> 
> 

 

I do it differently. Instead of setting up tapX in /etc/conf.d/net I let quemu configure the tap device at runtime. At boot time my bridge consists of eth0, eth1 and wlan0. The tap devices are created by qemu at runtime and it has options to name scripts to be run when it brings up and takes down the tap device. Qemu passes the name of the tap device to the script. I use simple scripts for when I wanted to start qemu and for qemu-ifup. This adds the tapX devices to br0 on demand instead of preallocating them at boot time.

```

qemu-system-x86_64 -enable-kvm -cpu host \

   -drive file=${1},if=virtio \

   -net nic -net tap,script=qemu-ifup,downscript=qemu-ifdown \

   -m 2000M \

   -daemonize \

   -vnc :$2 -smp 2 \

   -pidfile pid-$2 \

   -usbdevice tablet

```

```

#!/bin/bash

#echo Interface $1

ifconfig $1 0.0.0.0 promisc up

brctl addif br0 $1

```

----------

## toralf

I'm just curious about qemu-ifdown script ...

----------

## Aiken

My qemu-ifdown script only had an echo in it to see what arguments were passed to it. Thinking I forgot to do anything with it as when that qemu instance stops the tap device is destroyed and removed from the bridge.

```

#!/bin/bash

#echo Interface ${1}

brctl delif br0 ${1}

```

While writing this post I changed my qemu-ifdown to that. That removes the tapX device from the bridge instead of hoping it is automatically removed when qemu stops.

----------

## dkasak

OK I've changed my approach. virt-manager, and quite possibly libvirt and other related things, are flakey shit. I had a massive number of iptables rules, most of them totally invalid, which were responsible for not only 75% of my VM issues, but also prevented the system from booting properly, and eventually Gnome from starting at all. This seems to be related to me passing br0 to virt-manager and telling it to set up VMs using this interface.

I ended up deleting /etc/libvirt and starting again, passing in my disk images, and avoiding using gentoo's "kinda half-way-there" bridged support with systemd. I now have my normal eth0 active, and I've set up macvtap bridging. I then created a host-only VPN and attached a 2nd network interface to it. So now:

- PCs on the LAN can access the guest via the macvtap interface

- the host can access the guest via the host-only VPN

<phew>  :Razz: 

Incidentally, it would be great if this wasn't necessary - I don't remember this being necessary with other VMs, but then I guess I can't complain - it's open-source.

If I abandoned virt-manager and used the command-line, I might have a better experience, but I unfortunately don't have time for that now - this is for work, required yesterday. If I have to abandon virt-manager, I might as well go back to virtualbox / vmware, and run with an 'evaluation' license or whatever  :Wink: 

----------

## steveL

 *dkasak wrote:*   

> I now have my normal eth0 active, and I've set up macvtap bridging. I then created a host-only VPN and attached a 2nd network interface to it. So now:
> 
> - PCs on the LAN can access the guest via the macvtap interface
> 
> - the host can access the guest via the host-only VPN
> ...

 

Great :) Can you post (redacted-for-work) configs to show the setup, and how the various things tie together?

Even if it's only two files, or you just refer to above, it helps to have the specific solution to the problem asked, as a guide to others in the future.

----------

## dkasak

OK sure. It's pretty simple. I'd taken some screenshots and gone to upload them, but ... the forums don't let you post images. WTF? Whatever. I use virt-manager to configure my VMs. Those who want to use the command-line can post the command-line alternatives ...

First I go into virt-manager, and double-click on 'localhost (QEMU). This opens a 'Connection Details' window. Click the 'Virtual Networks' page / tab / whatever. Click on the 'add' button in the bottom-left of the window. Fill in the wizard. It doesn't really matter what network you define, I guess as long as it's not the same network as your LAN.

Typically people would use the built-in DHCP server, but for my purpose I disabled this, as one of the VMs that I'm using actually needed a VPN for guest-to-guest communication, and it provides it's own DHCP server. If you have something similar going on in your guests, note that your host will create a network interface that sits inside this VPN, and you should be careful that you don't have multiple DHCP servers, or IP address clashes between host and guests.

Next I go to the guest configuration, and define ...

1st network interface:

```
Source device: Host device eth0: macvtap

Device model: virtio

Source mode: VEPA
```

2nd network interface:

```
Virtual network 'nznet' : Isolated network, internal and host routing only

Device model: virtio
```

The 'Device model' you choose must be supported in the guest(s) you're running. 'virtio' is recommended if your guests have kernel support for this device. Otherwise use something else. The 'Source mode' in the device running in 'macvtap' mode doesn't appear to make much difference for what I'm using it for.

After you've configured this way, everything on the LAN except for the host itself ( and possibly other guests ) can communicate with the guest via the 1st network interface, which will have an IP address in your LAN. Your host can communicate with the guest via the 2nd network interface, which sits in the VPN.

---

It's not quite perfect, but it's an acceptable hack, and much less hassle than doing bridging 'properly'. It also doesn't freak out network-manager, which can have flow-on effects, with other network-manager-dependent services not starting up. I've actually had bridging working properly before with both VirtualBox ( before it supported doing it via it's own kernel module ... and also before everything depended on network-manager ) and KVM / QEMU ... but as noted above, it doesn't seem to work so well these days :/ Such is life.

----------

## szatox

My way to make it work with command line. Just in case someone might find it useful.

```
cat ./qemu_launcher.sh

#! /bin/bash

# HELP: This script is intended to be ran by user with limited priviliges.

# HELP: User must be allowed to run 'sudo tunctl', 'sudo brctl' and 'sudo ifconfig'  without password in order to work.

# HELP: Intended invocation of this script is:

# HELP: "cmd=<params you want to append to qemu command line> <THIS FILE'S NAME> <virtual machines name> [name of the bridge to bind]"

# HELP: Multiple bridges are allowed. Bridges must already exist and be configured before this script is started (e.g. configured by init scripts).

# HELP: Feel free to edit contents of "static_params" near the end of this file to define set of options you want always active.

add_ifaces ()

{

i=1

USER=$(whoami)

while [ ${#1} -gt 0 ]

        do sudo /usr/bin/tunctl -u $USER -t "${name}-${i}" > /dev/null

        sudo /sbin/brctl addif $1 "${name}-${i}" > /dev/null

        sudo /bin/ifconfig "${name}-${i}" up promisc > /dev/null

        # HINT: VIRTIO NETWORK IS DISABLED BY DEFAULT, ADJUST THE FOLLOWING LINE IF YOU NEED IT

        params="${params} -net nic,macaddr=$( printf 'DE:AD:BE:EF:%02X:%02X\n' $((RANDOM%256)) $((RANDOM%256)) ),vlan=$i -net tap,ifname=$name-$i,script=no,downscript=no,vlan=$i"

        shift

        i=$(($i+1))

done

echo $params

}

del_ifaces ()

{

i=1

while [ ${#1} -gt 0 ]

        do sudo /bin/ifconfig "${name}-${i}" down > /dev/null

        sudo /sbin/brctl delif $1 "${name}-${i}" > /dev/null

        sudo /usr/bin/tunctl -d "${name}-${i}" > /dev/null

        shift

        i=$(($i+1))

done

}

main ()

{

name=$1

shift

bridges="$@"

networking="$(add_ifaces ${bridges})"

RUN="qemu-system-x86_64 -name ${name} ${static_params} ${networking} ${cmd}"

echo starting machine $name with command

echo $RUN

$RUN

del_ifaces ${bridges}

}

        # HINT: PUT YOUR FAVOURITE OPTIONS HERE

static_params="-enable-kvm -m 1024"

main $@

```

----------

