# Xen multi-homed networking ?

## clockwork06

So I have a test box that I am playing around with. Its hooked into two different networks a 10.10.x.x and a 172.16.1.x (its essentially a shared server.) and I have been trying to create a xen guest that has two network devices (eth0 & eth1) one on each network. The physical host has two bond's setup, one uses eth1 & 2 (bond0) the other eth3 & 4 (bond1). So I need xenbr0 to use bond0 and xenbr1 to use bond1, then pass off the devices as eth0 and eth1 to the "guest".

Does anyone know how to do this with xen ? I have been using a script like this:

```

#!/bin/sh

/etc/xen/scripts/network-bridge start vifnum=0 bridge=xenbr0 netdev=bond0 $*

/etc/xen/scripts/network-bridge start vifnum=1 bridge=xenbr1 netdev=bond1 $*

```

but that doesnt work at all. Matter of fact it craters networking on the box whenever I try it. What am I missing ?

----------

## atatut

Quite the same problem;

I have 3 physical NICs. Problem is that by default the xend-config launches the network-bridge script. I found dozen of scripts telling me to change the reference to this script in my xend-config but non of them seems to work. Here's what I'd like to do:

eth0 => xenbr0

eth1 => xenbr1

eth2 => remains eth2 and must remain out of xen for administration purposes of the DOM0 itself and for reaching the gentoo file system.

I have tried configurations like this:

#!/bin/sh

BASEPATH=/etc/xen/scripts/

"$BASEPATH/network-bridge" "$@" vifnum=0 netdev=eth0 bridge=xenbr0

"$BASEPATH/network-bridge" "$@" vifnum=1 netdev=eth1 bridge=xenbr1

after altering the /etc/xen/xend-config.sxp to point on my-network-script and not the network-bridge called in the:

(network-script network-bridge)

But is still does what ever it wants and I'd love to correct this issue.

All the NICs are in the same subnet and eth0 and eth1 will be used to load balance the network charge while the eth2 remains unchanged to directly attack the gentoo/xen system through ssh.

There's maybe the answer in the /etc/xen/xend-config.sxp here:

# The script used to control virtual interfaces.  This can be overridden on a

# per-vif basis when creating a domain or a configuring a new vif.  The

# vif-bridge script is designed for use with the network-bridge script, or

# similar configurations.

#

# If you have overridden the bridge name using

# (network-script 'network-bridge bridge=<name>') then you may wish to do the

# same here.  The bridge name can also be set when creating a domain or

# configuring a new vif, but a value specified here would act as a default.

#

# If you are using only one bridge, the vif-bridge script will discover that,

# so there is no need to specify it explicitly.

#

(vif-script vif-bridge)

but those who posted their custom bridge script on which I based "my-network-script" never refered to changes whithin the above section. Does anyone have an idea on how to achieve the setup of the network interfaces inside xen?

----------

## atatut

There's a post related with this problem on another forum:

http://www.nabble.com/how-to-setup-two-bridges-with-two-physical-interfaces--tf4057032.html#a11524821

which seems to be the "Xen forum" but there's no much in there as well. I'm still trying to paste together the puzzle pieces but there's not much info out there. Maybe some guru will want to share his knowledge. Durring that time, I'll continue add what I found in this post for future gentoo users to setup their Xen install in minutes.

----------

## atatut

Another post on another forum:

http://wiki.xensource.com/xenwiki/XenNetworking

I'll try to see what I can get out of it and post the results, whatever they are.

----------

## DNAspark99

I know this is an old post, but after struggling with xen & bridging bonded interfaces, I've finally figured it out; and since this is one of the only threads to come up on the topic, I'll share the solution I've found to work, based off info here: https://bugs.gentoo.org/111684

My setup: 

 I've got a xen host (dom0) with 4 nics. I want to create two bonded interfaces out of these that go to different networks. 

Obviously I want the xen domU's (xen client vm's) to access these networks. The default xen networking scripts seem to mess up the bonding setup, somewhere around the point where they pull the real bond0 interface and rename it to pbond0. So, to work around this, I've done the following:

edit /etc/xen/xend-config.sxp and comment out the network-bridge script to prevent it from messing things up.

```
#(network-script network-bridge)
```

Now, in /etc/conf.d/net, we can manually set up networking to first bond the desired interfaces, then bridge them;

```
# Bridged Bonded Interfaces for Xen

# Null config for physical interfaces

config_eth0=( null )

config_eth1=( null )

config_eth2=( null )

config_eth3=( null )

# Bond physical interfaces, nullify config

RC_NEED_bond0="net.eth0 net.eth2"

RC_NEED_bond1="net.eth1 net.eth3"

slaves_bond0="eth0 eth2"

slaves_bond1="eth1 eth3"

config_bond0=( "null" )

config_bond1=( "null" )

# Bridge bonded interfaces, configure with network settings

RC_NEED_xenbr0="net.bond0"

RC_NEED_xenbr1="net.bond1"

bridge_xenbr0="bond0"

bridge_xenbr1="bond1"

brctl_xenbr0=( "setfd 0" "sethello 0" "stp off" )

brctl_xenbr1=( "setfd 0" "sethello 0" "stp off" )

config_xenbr0=( "dhcp" )

config_xenbr1=( "dhcp" )

#Configure Default Gateway (optional)

#routes_xenbr0=( "default gw 10.100.101.1" )

# Bond Settings

preup() {

        # Adjusting the bonding mode / MII monitor

        # Possible modes are : 0, 1, 2, 3, 4, 5, 6,

        #     OR

        #   balance-rr, active-backup, balance-xor, broadcast,

        #   802.3ad, balance-tlb, balance-alb

        # MII monitor time interval typically: 100 milliseconds

        if [[ ${IFACE} == "bond0" ]] ; then

                BOND_MODE="active-backup"

                #BOND_MODE="broadcast"

                BOND_MIIMON="100"

                echo ${BOND_MODE} >/sys/class/net/bond0/bonding/mode

                echo ${BOND_MIIMON}  >/sys/class/net/bond0/bonding/miimon

                einfo "Bonding mode is set to ${BOND_MODE} on ${IFACE}"

                einfo "MII monitor interval is set to ${BOND_MIIMON} ms on ${IFACE}"

        elif [[ ${IFACE} == "bond1" ]] ; then

                BOND_MODE="active-backup"

                #BOND_MODE="broadcast"

                BOND_MIIMON="100"

                echo ${BOND_MODE} >/sys/class/net/bond1/bonding/mode

                echo ${BOND_MIIMON}  >/sys/class/net/bond1/bonding/miimon

                einfo "Bonding mode is set to ${BOND_MODE} on ${IFACE}"

                einfo "MII monitor interval is set to ${BOND_MIIMON} ms on ${IFACE}"

        else

                einfo "Doing nothing on ${IFACE}"

        fi

        return 0

}
```

Now, create init scripts for every interface, even though not all we be used directly. 

```

cd /etc/init.d

ln -s net.lo net.eth0

ln -s net.lo net.eth1

ln -s net.lo net.eth2

ln -s net.lo net.eth3

ln -s net.lo net.bond0

ln -s net.lo net.bond1

ln -s net.lo net.xenbr0

ln -s net.lo net.xenbr1

```

Then, I've removed any net.eth* and net.bond* from the default runlevel, and replaced them with net.xenbr* 

```

rc-update del net.bond0 default

rc-update del net.bond1 default

rc-update add net.xenbr0 default

rc-update add net.xenbr1 default

```

Finally, the domU (xen client vm) config will need to specify the bridge to connect the virtual interface(s) to:

```
vif = ["bridge=xenbr0", "bridge=xenbr1"]
```

So far, this works for me.

----------

## DNAspark99

WOW

ok, so for what it's worth, I've got 2 ProCurve 2510 switches, and 2 2510G gigabit switches.

For some 'odd reason', the bridged/bonded nic failover WAS NOT WORKING on the gigabit switches!

I went through configs. I fiddled with module options. I changed switch settings. I tested everything over and over again. 

It would just NOT work on the 2510G, but WOULD on the 2510's. This was driving me nuts!

After hours speaking with ProCurve support, it turns out that an "unreleased firmware upgrade" fixed the issue! (need: Y_11_08.swi to flash the 2510G). Turns out that the gratuitous arp was not traversing over the switch interconnect, so it was relying on the mac-age-time to expire the bonded interface's mac/port# relationship from it's cache - in my case, set to the lowest setting: 60 seconds. A minute of downtime does not equal high availability!! But the firmware flash fixed it. yay.

Luckily I only pulled out HALF my hair troubleshooting this. uggg.

Hopefully someone else is saved this headache now!

----------

