# KVM networking

## Wizumwalt

I'm trying to get networking going between the host OS and the guest OS, and eventually between two guest OS's later. I'd also like for the guest OS's to be able to access the internet. I've been trying to follow this link ... http://en.gentoo-wiki.com/wiki/KVM#Networking_2 ... and have my setup using Direct bridging which AFAIK, is like the virtual hub being described and I think fits my needs.

Here is the network config on my host OS ...

```

$ ifconfig

br0       Link encap:Ethernet  HWaddr 9e:09:8f:ac:36:19  

          inet addr:192.168.0.254  Bcast:192.168.0.255  Mask:255.255.255.0

          inet6 addr: fe80::c494:acff:febb:a528/64 Scope:Link

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:8 errors:0 dropped:0 overruns:0 frame:0

          TX packets:3495 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0 

          RX bytes:440 (440.0 B)  TX bytes:168363 (164.4 KiB)

eth0      Link encap:Ethernet  HWaddr 00:25:64:b8:c5:93  

          inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0

          inet6 addr: fe80::225:64ff:feb8:c593/64 Scope:Link

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:4247704 errors:0 dropped:0 overruns:0 frame:0

          TX packets:2284561 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000 

          RX bytes:5949327203 (5.5 GiB)  TX bytes:177768857 (169.5 MiB)

          Interrupt:17 

lo        Link encap:Local Loopback  

          inet addr:127.0.0.1  Mask:255.0.0.0

          inet6 addr: ::1/128 Scope:Host

          UP LOOPBACK RUNNING  MTU:16436  Metric:1

          RX packets:7079 errors:0 dropped:0 overruns:0 frame:0

          TX packets:7079 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0 

          RX bytes:448135 (437.6 KiB)  TX bytes:448135 (437.6 KiB)

qtap0     Link encap:Ethernet  HWaddr d6:0a:81:f5:71:ab  

          inet6 addr: fe80::d40a:81ff:fef5:71ab/64 Scope:Link

          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1

          RX packets:58 errors:0 dropped:0 overruns:0 frame:0

          TX packets:3560 errors:0 dropped:27 overruns:0 carrier:0

          collisions:0 txqueuelen:500 

          RX bytes:6159 (6.0 KiB)  TX bytes:182289 (178.0 KiB)

qtap1     Link encap:Ethernet  HWaddr 9e:09:8f:ac:36:19  

          inet6 addr: fe80::9c09:8fff:feac:3619/64 Scope:Link

          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1

          RX packets:0 errors:0 dropped:0 overruns:0 frame:0

          TX packets:0 errors:0 dropped:3483 overruns:0 carrier:0

          collisions:0 txqueuelen:500 

          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

```

The guest OS is configured with the following ...

```

config_eth0=( "192.168.0.15 netmask 255.255.255.0 brd 192.168.0.255" )

routes_eth0=( "default gw 192.168.0.1" )

```

Of course, I am currently unable to ping either one from the other end. 

I start my VM using these network params ...

```

kvm -net nic,macaddr=52:54:00:12:34:56,model=virtio -net tap,ifname=qtap0,script=no,downscript=no ...

```

Anyone help on what might be wrong?

----------

## Anarcho

Could you please post the config of your bridge (brctl show)?

Also I wonder (as I guess that your bridge is using eth0) why both br0 and eth0 have IP addresses assigend. If the bridge is setup correctly, only br0 should have an IP address.

Also, could it be that you need to enable ip forwarding:

cat /proc/sys/net/ipv4/ip_forward

----------

## Wizumwalt

Here's the script that I used to start things up.

```

    ebegin "Loading the kvm module"

    /sbin/modprobe kvm

    eend $? "Loading the kvm-intel module"

    /sbin/modprobe kvm-intel

    eend $? "Failed to load the kvm-intel module"

    ebegin "Loading the tun module"

        /sbin/modprobe tun

    eend $? "Failed to load the tun module"

    ebegin "Setting up the bridge device (br0)"

    /sbin/brctl addbr br0

    /sbin/ifconfig br0 192.168.0.254 netmask 255.255.255.0 up

    eend $? "Failed to create the bridge interface"

    for ((i=0; i < NUM_OF_DEVICES; i++)); do

        ebegin "Setting up tap interface: qtap$i"

        /usr/bin/tunctl -b -u $USERID -t qtap$i >/dev/null

        eend $? "Failed to create the tap interface: qtap$i"

        ebegin "Linking the bridge interface with qtap$i"

        /sbin/brctl addif br0 qtap$i

        eend $? "Failed to link the bridge interface to qtap$i"

        ebegin "Bring qtap$i interface up"

        /sbin/ifconfig qtap$i up 0.0.0.0 promisc

        eend $? "Failed to bring qtap$i up"

    done

    

    ebegin "Allowing Internet access"

    echo "1" > /proc/sys/net/ipv4/ip_forward

    eend $? "Failed to allow forwarding"

    ebegin "Allowing masquerade"

    iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

    eend $? "Failed to allow masquerade (eth0)"

    eend 0

```

And here's what was added to /etc/sysctl.conf

```

net.bridge.bridge-nf-call-arptables = 0

net.bridge.bridge-nf-call-iptables = 0

net.bridge.bridge-nf-call-ip6tables = 0                                

```

/etc/conf.d/net of the host OS ...

```

config_eth0=( "192.168.0.10 netmask 255.255.255.0 brd 192.168.0.255" )

routes_eth0=( "default gw 192.168.0.1" )

#config_eth1=( "192.168.2.200 netmask 255.255.255.0 brd 192.168.2.255" )

#routes_eth1=( "default gw 192.168.2.1" )

bridge_br0="eth0 tap0 tap1"

brctl_br0=( "setfd 0" "sethello 0" "stp off" )

rc_need_br0="net.tap0 net.tap1"

config_br0=( "192.168.0.10/24" )

routes_br0=( "default via 192.168.0.1" )

dns_domain_br0="mydomain.com"

dns_servers_br0="xxx.xxx.xxx.xxx"

dns_search_br0="mydomain.com"

config_tap0=( "null" )

tuntap_tap0="tap"

tunctl_tap0="-u winky"

mac_tap0="52:54:00:12:34:56"

config_tap1=( "null" )

tuntap_tap1="tap"

tunctl_tap1="-u winky"

mac_tap1="52:54:00:12:34:57"

```

----------

## AngelKnight

Your conf.d/net indicates that eth0 gets an IP address, your br0 gets an IP address, then you're adding eth0 to br0.  This is confusing; erase routes_eth0 and set config_eth0="null" instead.

Your script seems to explicitly configure br0, but if br0 is up before hand it doesn't seem likely that the /sbin/brctl addbr br0" will succeed.

Any chance that net.eth0 and net.br0 don't exist and/or aren't in the runlevel you're using?

I think folks will have an easier time helping you if you do as Anarcho requested and post the output of "/sbin/brctl show" once you've got the scripts run, the guest started, and reached the point where things don't work as you expect.

I'd ask that you also include output from "/sbin/ip addr show" and "/sbin/ip neigh show" on the host after you've got all this running and then attempted to ping the guest from the host.

----------

## Wizumwalt

Until I get things working, I'm starting net.eth0 and net.br0 manually. And I'd really like to keep eth0 w/ the 0.10 ip address. Here's more of the requested commands.

My /etc/conf.d/net only now includes the following, and my startup script posted above shows no errors when starting.

```

config_eth0=( "192.168.0.10 netmask 255.255.255.0 brd 192.168.0.255" )

routes_eth0=( "default gw 192.168.0.1" )

```

```

$ brctl show

bridge name     bridge id               STP enabled     interfaces

br0             8000.5e36bdad7787       no              qtap0

                                                        qtap1

```

```

$ ip addr show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo

    inet6 ::1/128 scope host 

       valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000

    link/ether 00:25:64:b8:c5:93 brd ff:ff:ff:ff:ff:ff

    inet 192.168.0.10/24 brd 192.168.0.255 scope global eth0

    inet6 fe80::225:64ff:feb8:c593/64 scope link 

       valid_lft forever preferred_lft forever

3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc mq state DOWN qlen 1000

    link/ether 00:10:18:59:d7:65 brd ff:ff:ff:ff:ff:ff

4: sit0: <NOARP> mtu 1480 qdisc noop state DOWN 

    link/sit 0.0.0.0 brd 0.0.0.0

23: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 

    link/ether 5e:36:bd:ad:77:87 brd ff:ff:ff:ff:ff:ff

    inet 192.168.0.254/24 brd 192.168.0.255 scope global br0

    inet6 fe80::d479:99ff:fed2:3be8/64 scope link 

       valid_lft forever preferred_lft forever

24: qtap0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500

    link/ether 6e:5e:97:2c:a3:7b brd ff:ff:ff:ff:ff:ff

    inet6 fe80::6c5e:97ff:fe2c:a37b/64 scope link 

       valid_lft forever preferred_lft forever

25: qtap1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500

    link/ether 5e:36:bd:ad:77:87 brd ff:ff:ff:ff:ff:ff

    inet6 fe80::5c36:bdff:fead:7787/64 scope link 

       valid_lft forever preferred_lft forever

```

```

$ ip neigh show

192.168.0.1 dev eth0 lladdr 00:1f:33:d3:6a:c8 STALE

```

----------

## AngelKnight

eth0 has 192.168.0.10/24 and br0 has 192.168.0.254/24.  Interesting; you'll probably need proxy ARP on to get your gateway to convince it to switch ethernet frames towards the guests through your host.

An actual bridging design would be:

br0 carries 192.168.0.10/24 and 192.168.0.254/24

br0 ties together eth0, qtap0 and qtap1 into an L2 broadcast segment

Remember: in Linux bridge members never have L3 addresses; if you need to "keep" 192.168.0.10/24, make it an address on br0 as well.

----------

## Anarcho

If there are only the qtaps in the bridge, then of course you can't ping them from outside. You probably can't ping them because both eth0 and br0 are in the same subnet.

Why don't you bridge eth0 together with qtaps and then assign .10 to the bridge? You shouldn't need the .254 at all.

----------

