# HOWTO setup dual wan

## duxsco

Hi,

I have setup a dual wan network on my Gentoo Linux server and would like to discuss this solution, which I have come up with, with the help of various websites. Unfortunately, I am not able to recall the various website that have come in handy as my research lies some time back. But, anyway thanks a lot all of you. In the end, someone, who is experienced with the gentoo wiki, can create an entry. 

1. This is the exemplary network used (based on: http://lartc.org/howto/lartc.rpdb.multiple-links.html):

```

                                                                   ________

                                          +--------------+        /

                           192.168.0.0/24 |    router0   |       |

                            +-------------+IP:           +-------

                            |             | 192.168.0.254|     /

 eth2 IP:   _        +------+-------+     +--------------+    |

 192.168.2.1 \       |     eth0     |                         |

              \      |IP:           |                         |

               \     | 192.168.0.1  |                         |

        __      \    |Default GW IP:|                         |

    ___/  \_     \   | 192.168.0.254|                         |

  _/        \__   \  |              |                        /

 /             \ eth2|              |                        |

| Local network -----+    server    |                        |     Internet

 \_           __/    |              |                        |

   \__     __/       |              |                        |

      \___/          |     eth1     |                        \

 192.168.2.0/24      |IP:           |                         |

                     | 192.168.1.1  |                         |

                     |Default GW IP:|                         |

                     | 192.168.1.254|                         |

                     +------+-------+     +--------------+    |

                            |             |    router1   |     \

                            +-------------+IP:           +-------

                           192.168.0.1/24 | 192.168.1.254|       |

                                          +--------------+        \________

```

The server will provide dual wan access with round-robin load balancing and failover. router0 and router1 can be a DD-WRT, OpenWrt or FritzBox router. Other router platforms are currently not supported for this failover approach. But, you can just google around to get it running.

2. First, "emerge -av sys-apps/iproute2" for routing. Then, add the following code to "/etc/iproute2/rt_tables" in order to create two new routing tables. One for each wan network at eth0 and eth1:

```

100     first

101     second

```

3. These are basic iptables routes. They protect the local network from the internet, provide NAT and mark packets passing eth0 and eth1 in order to keep a network session running on the same wan port. In this example, I save it at "/root/fw.sh".

```

#!/bin/bash

#tools

IPTABLES="/sbin/iptables"

IP6TABLES="/sbin/ip6tables"

###interfaces

WAN0="eth0"; WAN0_IP="192.168.0.1"

WAN1="eth1"; WAN1_IP="192.168.1.1"

LAN="eth2"

###clearing iptables

$IPTABLES -F

$IPTABLES -t nat -F

$IPTABLES -t mangle -F

$IPTABLES -X

$IPTABLES -t nat -X

$IPTABLES -t mangle -X

$IPTABLES -P INPUT DROP

$IPTABLES -P OUTPUT ACCEPT

$IPTABLES -P FORWARD DROP

$IP6TABLES -F

$IP6TABLES -t mangle -F

$IP6TABLES -X

$IP6TABLES -t mangle -X

$IP6TABLES -P INPUT DROP

$IP6TABLES -P OUTPUT ACCEPT

$IP6TABLES -P FORWARD DROP

###nat

$IPTABLES -t nat -A POSTROUTING -o $WAN0 -j SNAT --to-source $WAN0_IP

$IPTABLES -t nat -A POSTROUTING -o $WAN1 -j SNAT --to-source $WAN1_IP

###mangle (used for load balancing)

$IPTABLES -t mangle -A POSTROUTING -o $WAN0 -m conntrack --ctstate NEW -j CONNMARK --set-mark 0x100

$IPTABLES -t mangle -A POSTROUTING -o $WAN1 -m conntrack --ctstate NEW -j CONNMARK --set-mark 0x101

$IPTABLES -t mangle -A PREROUTING -i $WAN0 -m conntrack --ctstate NEW -j CONNMARK --set-mark 0x100

$IPTABLES -t mangle -A PREROUTING -i $WAN1 -m conntrack --ctstate NEW -j CONNMARK --set-mark 0x101

$IPTABLES -t mangle -A PREROUTING -m conntrack --ctstate ESTABLISHED,RELATED -j CONNMARK --restore-mark

$IPTABLES -t mangle -A OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j CONNMARK --restore-mark

###must have rules

$IPTABLES  -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

$IPTABLES  -A INPUT -p icmp   -j ACCEPT

$IPTABLES  -A INPUT -i lo -j ACCEPT

$IPTABLES  -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

$IP6TABLES -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

$IP6TABLES -A INPUT -p icmpv6   -j ACCEPT

$IP6TABLES -A INPUT -i lo -j ACCEPT

$IP6TABLES -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

###internet access

$IPTABLES -A FORWARD -i $LAN -o $WAN0 ! -d 192.168.0.0/16 -j ACCEPT

$IPTABLES -A FORWARD -i $LAN -o $WAN1 ! -d 192.168.0.0/16 -j ACCEPT

###default reject

$IPTABLES  -A INPUT   -j REJECT

$IPTABLES  -A FORWARD -j REJECT

$IP6TABLES -A INPUT   -j REJECT

$IP6TABLES -A FORWARD -j REJECT

```

Now, do the following as root in order to apply the iptables rules:

```

# chmod 700 /root/fw.sh

# /root/fw.sh

# /etc/init.d/iptables save

# /etc/init.d/ip6tables save

# rc-update add iptables boot

# rc-update add ip6tables boot

```

4. With the mangle rules, iptables marks packets for iproute2. Now, we need to tell iproute what to do with packets that have been marked. We execute the following code as root:

```

# ip rule add fwmark 0x100 table 100

# ip rule add fwmark 0x101 table 101

```

And, place the code in "/etc/local.d/local.start" to get it executed at boot time, which should look like:

```

#!/bin/bash

ip rule add fwmark 0x100 table 100

ip rule add fwmark 0x101 table 101

```

Above code tells iproute to lookup table 100 when getting packets marked with 0x100 and table 101, if getting 0x101 marked packets.

5. We need to fill table 100 and 101 with routing rules. For this purpose, I created the file "/usr/local/bin/dualwan_routes.sh" with this code:

```

#!/bin/bash

ip r flush table 0x100

ip r flush table 0x101

ip r | grep -v default | grep -v nexthop | while read I; do ip r add table 0x100 $I; ip r add table 0x101 $I; done

ip r add default via 192.168.0.254 dev eth0 table 0x100

ip r add default via 192.168.1.254 dev eth1 table 0x101

```

I made it executable "chmod 700 /usr/local/bin/dualwan_routes.sh", ran "emerge -av sys-process/daemontools" to get setlock and added this line to cron:

```

*/1 * * * * /usr/bin/setlock -n /tmp/dualwan_routes.lock /usr/local/bin/dualwan_routes.sh > /dev/null 2>&1

```

6. The last thing to do is setting up failover. The approach depends on the router you have in use. I explain the steps I took in order to get it running on DD-WRT, FritzBox and OpenWrt.

For DD-WRT and OpenWrt, you have to create an ssh keypair without password as normal user on the Gentoo server by executing:

```

~ ssh-keygen

```

6.a) DD-WRT:

- Go to "Administration" -> "Management" -> "Remote Access" and enable "SSH Management"

- Go to "Services" -> "Services" -> "Secure Shell", enable "SSHd" and add the following code to "Authorized Keys" (ssh-... is the ssh public key, e.g. id_rsa.pub, created in step 6.):

```

command="nvram get wanup",no-port-forwarding,no-X11-forwarding,no-pty ssh-...

```

Save and apply.

6.b) OpenWrt:

Go to "System" -> "Administration" -> "SSH Access" -> "SSH-Keys" and save this in the field (ssh-... is the ssh public key, e.g. id_rsa.pub, created in step 6.):

```

command="ifstatus wan",no-port-forwarding,no-X11-forwarding,no-pty ssh-...

```

Save and apply.

6.c) FritzBox:

Unfortunately, I only have the German version. Perhaps, someone can get the English names.

Go to "Heimnetz" -> "Netzwerk" -> "Programme" and enable "Statusinformationen über UPnP übertragen (empfohlen)".

English names might be:

"Home network" -> "network" -> "programs" and enable "status information over UPnP"

Save and  apply.

7. You need to create the failover script "/usr/local/bin/dualwan_failover.sh" and adjust it to your needs (IP adresses, interface names and non-root username on the Gentoo server). I assume router0 being OpenWrt and router1 FritzBox:

```

#!/bin/bash

#DD-WRT

#wan0=1

#if [ $(sudo -u david ssh root@192.168.0.254 2>/dev/null) -eq 1 ]; then

#   wan0=0

#fi

#OpenWrt

wan0=1

if [ $(sudo -u david ssh root@192.168.0.254 2>/dev/null | grep -q '"up": true,'; echo $?) -eq 0 ]; then

   wan0=0

fi

#FritzBox

sudo -u david curl -s "http://192.168.1.254:49000/upnp/control/WANIPConn1" \

-H "Content-Type: text/xml; charset="utf-8"" \

-H "SoapAction:urn:schemas-upnp-org:service:WANIPConnection:1#GetStatusInfo" \

-d "<?xml version='1.0' encoding='utf-8'?> \

<s:Envelope s:encodingStyle='http://schemas.xmlsoap.org/soap/encoding/' xmlns:s='http://schemas.xmlsoap.org/soap/envelope/'> \

<s:Body> <u:GetStatusInfo xmlns:u='urn:schemas-upnp-org:service:WANIPConnection:1' /> \

</s:Body> </s:Envelope>" |\

sudo -u david grep -q "<NewConnectionStatus>Connected</NewConnectionStatus>"

wan1=$?

ip r | sudo -u david grep -q ^default[[:space:]]\*$

both=$?

ip r | sudo -u david grep -q "default via 192.168.0.254 dev eth0"

first=$?

ip r | sudo -u david grep -q "default via 192.168.1.254 dev eth1"

second=$?

if [[ $wan0 -eq 0 ]] && [[ $wan1 -eq 0 ]] && [[ $both -ne 0 ]]; then

   ip r del default

   ip r add default nexthop via 192.168.0.254 dev eth0 weight 1 nexthop via 192.168.1.254 dev eth1 weight 1

elif [[ $wan0 -eq 0 ]] && [[ $wan1 -ne 0 ]] && [[ $first -ne 0 ]]; then

   ip r del default

   ip r add default via 192.168.0.254 dev eth0

elif [[ $wan0 -ne 0 ]] && [[ $wan1 -eq 0 ]] && [[ $second -ne 0 ]]; then

   ip r del default

   ip r add default via 192.168.1.254 dev eth1

fi

```

Make it executable as root:

```

# chmod 700 /usr/local/bin/dualwan_failover.sh

```

And, add this to cron:

```

*/1 * * * * /usr/bin/setlock -n /tmp/dualwan_failover.lock /usr/local/bin/dualwan_failover.sh > /dev/null 2>&1

```

Adjust the weight to your needs. In above code, half of the traffic is distributed to each wan port (both "weight 1").Last edited by duxsco on Fri Oct 04, 2013 3:56 pm; edited 1 time in total

----------

## duxsco

I don't know why, but somehow the Gentoo forum  kicks me out regularly. I don't have this problem on other websites yet.

----------

## NeddySeagoon

Moved from Networking & Security to Documentation, Tips & Tricks.

by request of OP

----------

## SlashBeast

You don't need the whole iptables maddness and you one can ignore the `ip rule add fwmark`.

http://lartc.org/howto/lartc.rpdb.multiple-links.html

----------

## duxsco

From larc.org:

```

                                                                 ________

                                          +------------+        /

                                          |            |       |

                            +-------------+ Provider 1 +-------

        __                  |             |            |     /

    ___/  \_         +------+-------+     +------------+    |

  _/        \__      |     if1      |                      /

 /             \     |              |                      |

| Local network -----+ Linux router |                      |     Internet

 \_           __/    |              |                      |

   \__     __/       |     if2      |                      \

      \___/          +------+-------+     +------------+    |

                            |             |            |     \

                            +-------------+ Provider 2 +-------

                                          |            |       |

                                          +------------+        \________

```

 *Quote:*   

> 
> 
> Let us first set some symbolical names. Let $IF1 be the name of the first interface (if1 in the picture above) and $IF2 the name of the second interface. Then let $IP1 be the IP address associated with $IF1 and $IP2 the IP address associated with $IF2. Next, let $P1 be the IP address of the gateway at Provider 1, and $P2 the IP address of the gateway at provider 2. Finally, let $P1_NET be the IP network $P1 is in, and $P2_NET the IP network $P2 is in. 
> 
> 

 

The solution, provided at larc.org, won't work. The Linux router only knows of the networks $P1_NET and $P2_NET. It is not able to assign incoming packets with source IPs, that are not part of the subnet $P1_NET and $P2_NET, to if1 and if2. Let's say you are currently not connected to the local network, but via UMTS, and try to ssh into your linux router. The laptop you use got the ip 123.123.123.123 assigned. $P1_NET is 123.123.150.0/24 and $P2_NET is 123.123.151.0/24. Your linux router gets a packet with source IP 123.123.123.123 and tries to answer, but doesn't know which WAN NIC to use, because the routing tables contain only the $P1_NET and $P2_NET subnets.

You can use larg.org solution if you do the following:

You need to place a router between linux router and provider1. And, one between linux router and provider2.

Then, you have the following setttings:

```

                                                                   ________

                                          +--------------+        /

                           192.168.0.0/24 |    router0   |       |

                            +-------------+IP:           +-------

                            |             | 192.168.0.254|     /

 eth2 IP:   _        +------+-------+     +--------------+    |

 192.168.2.1 \       |     eth0     |                         |

              \      |IP:           |                         |

               \     | 192.168.0.1  |                         |

        __      \    |Default GW IP:|                         |

    ___/  \_     \   | 192.168.0.254|                         |

  _/        \__   \  |              |                        /

 /             \ eth2|              |                        |

| Local network -----+    server    |                        |     Internet

 \_           __/    |              |                        |

   \__     __/       |              |                        |

      \___/          |     eth1     |                        \

 192.168.2.0/24      |IP:           |                         |

                     | 192.168.1.1  |                         |

                     |Default GW IP:|                         |

                     | 192.168.1.254|                         |

                     +------+-------+     +--------------+    |

                            |             |    router1   |     \

                            +-------------+IP:           +-------

                           192.168.0.1/24 | 192.168.1.254|       |

                                          +--------------+        \________ 

```

On router0 you need to execute:

```

iptables -t nat -A POSTROUTING -d 192.168.0.1 -j SNAT --to-source 192.168.0.254

```

And on router1:

```

iptables -t nat -A POSTROUTING -d 192.168.1.1 -j SNAT --to-source 192.168.1.254

```

I don't like this solution, because all packets from the internet get the source address 192.168.0.254 or 192.168.1.254 assigned.

----------

## duxsco

Or, lets say you are in the local net and try to connect to google (173.194.112.136). Your packet gets forwarded correctly because of lartc.org code:

```

ip route add default scope global nexthop via $P1 dev $IF1 weight 1 \

       nexthop via $P2 dev $IF2 weight 1

```

We assume the packet got out on the first wan NIC

Then, google responds and sends a packet over the first wan NIC. This packet has the source IP 173.194.112.136. Your laptop now tries to respond to this packet, but doesn't know which NIC to use, because only 123.123.150.0/24 and 123.123.151.0/24 are in the routing tables.

----------

## duxsco

Another solution to this problem is splitting all non-private subnets up between the two wan NICs.

Very simple (and wrong) example:

-first wan gets IP ranges 0.0.0.0-127.255.255.255

-second wan gets IP ranges 128.0.0.0-255.255.255.255

But, I don't like this solution, because it's not round-robin.

----------

## SlashBeast

I use lartc.org soluton with boxes where I have multiple wans and everything works great, either I have by far other usecase or we have great misunderstanding.

At this moment whatever goes from eth0 reply goes out thru eth0, same goes for eth1, eth2 and eth3.

----------

## duxsco

What do you use for $P1_NET and $P2_NET?

Larc.org says:

```

Finally, let $P1_NET be the IP network $P1 is in, and $P2_NET the IP network $P2 is in. 

```

In both cases, the network is the internet. So $P1_NET == $P2_NET.

----------

## SlashBeast

The networks of ip addresses I use, so if I have 1.0.0.10 and 2.0.0.20 with 24bit subnet my networks are 1.0.0.0/24 and 2.0.0.0/24. And in my case both ip addresses are the public one, but still I do have 'upstream' which is ISP, I still have ip network and gateway I go thru on each link.

to make it crystal clear, your isp gave you ip address, which is in specified network, this is this network.

edit: something with real world examples https://kindlund.wordpress.com/2007/11/19/configuring-multiple-default-routes-in-linux/

----------

## duxsco

If I am not mistaken, your routing tables are like the following:

- table T1:

```

default via 1.0.0.254 dev eth0 proto static

1.0.0.0/24 dev eth0  proto kernel  scope link  src 1.0.0.10

```

- table T1:

```

default via 2.0.0.254 dev eth1 proto static

2.0.0.0/24 dev eth1  proto kernel  scope link  src 2.0.0.10

```

- table main:

```

1.0.0.0/24 dev eth0  proto kernel  scope link  src 1.0.0.10

2.0.0.0/24 dev eth1  proto kernel  scope link  src 2.0.0.10

default 

   nexthop via 10.0.0.254  dev eth0 weight 1

   nexthop via 20.0.0.254  dev eth1 weight 1

```

- ip rules:

```

0:   from all lookup local 

32762:   from 1.0.0.10 lookup T1 

32763:   from 2.0.0.20 lookup T2

32766:   from all lookup main 

32767:   from all lookup default

```

I think it's working for you because your Gentoo server is directly connected to the internet (running pppoe). Unfortunately, I cannot connect mine directly, because I use IP-TV and Dell's iDrac:

```

                                                                                        ________

                                              +--------+       +--------------+        /

                           192.168.0.0/24     |        |       |    router0   |       |

                            +-----------------+ switch +-------+IP:           +-------

                            |                 |        |       | 192.168.0.254|     /

 eth2 IP:   _        +------+-------+         +---+----+       +--------------+    |

 192.168.2.1 \       |     eth0     |             |                                |

              \      |IP:           |             |                                |

               \     | 192.168.0.1  |             |                                |

        __      \    |Default GW IP:|             |                                |

    ___/  \_     \   | 192.168.0.254|             |                                |

  _/        \__   \  |              |          +--+------+                        /

 /             \ eth2|              |  iDrac   |         |                        |

| Local network -----+    server    +----------+ router2 +                        |     Internet

 \_           __/    |              |          |         |                        |

   \__     __/       |              |          +--+------+                        |

      \___/          |     eth1     |             |                                \

 192.168.2.0/24      |IP:           |             |                                |

                     | 192.168.1.1  |             |                                |

                     |Default GW IP:|             |                                |

                     | 192.168.1.254|             |                                |

                     +------+-------+         +---+----+       +--------------+    |

                            |                 |        |       |    router1   |     \

                            +-----------------+ switch +-------+IP:           +-------

                           192.168.0.1/24     |        |       | 192.168.1.254|       |

                                              +---+----+       +--------------+        \________ 

                                                  |

                                                  |

                                              +---+----+

                                              |        |

                                              | IP-TV  |

                                              |        |

                                              +--------+

```

router2 has openvpn running. If my server crashes, I can still connect to it over iDrac over both wan connections. I could place the TVs in the local network, if I setup an IGMP proxy on my Gentoo server. But, I need router1 anyway in order to provide router2 with an internet connection. Therefore, I avoid the hassle and just connect the TVs to router1.

----------

## 666threesixes666

you guys should wiki this brew with no iptv, routers, or switches, server acting as the router.

 eth1 192.168.0.1/24  should be  192.168.1.0/24  representing the network.  if it was wiki, id fix that typo that has been plaguing this thread.

```

                                                     ________

                                                    /

                           192.168.0.0/24          |

                            +----------------------

                            |                     /

 eth2 IP:   _        +------+-------+             |

 192.168.2.1 \       |     eth0     |             |

              \      |IP:           |             |

               \     | 192.168.0.1  |             |

        __      \    |Default GW IP:|             |

    ___/  \_     \   | 192.168.0.254|             |

  _/        \__   \  |              |            /

 /             \ eth2|              |            |

| Local network -----+    server    |            |     Internet

 \_           __/    |              |            |

   \__     __/       |              |            |

      \___/          |     eth1     |             \

 192.168.2.0/24      |IP:           |             |

                     | 192.168.1.1  |             |

                     |Default GW IP:|             |

                     | 192.168.1.254|             |

                     +------+-------+             |

                            |                     \

                            +----------------------

                           192.168.1.0/24          |

                                                   \________

```

only switching 192.168.0.0, and 192.168.1.0 to random dhcp addresses from isp, and their default gw....

----------

## duxsco

There might be use cases where a direct connection of the server to the internet is not possible, e.g. IP-based telephony, IP-TV, restrictions of TV-cable based connections (KabelDeutschland), VDSL. The advantage of my solution is that it is applicable under more use cases. As long as you have OpenWRT, DD-WRT or FritzBox between the linux server and the internet, failover works. You don't need to think of dynamically changing wan IPs.

----------

## duxsco

I think you should use the larc.org tutorial and ping based fail-over in order to get a dual wan setup where the server is directly connected to the internet.

----------

## Nreal

How could you bond two connections - wifi and wired to one? Wired 10 mbs from router enp4s0 and wifi from adroid phone 300mbs/s.

----------

## duxsco

 *Nreal wrote:*   

> How could you bond two connections - wifi and wired to one? Wired 10 mbs from router enp4s0 and wifi from adroid phone 300mbs/s.

 

Afaik, that's not possible. See:

https://unix.stackexchange.com/questions/132048/bonding-two-broadband-connections-to-serve-lan-as-a-gateway/132105#132105

----------

## NeddySeagoon

Nreal'

Both ends of the bond have to go to the same place so that they can be unbonded.

If you got your phone and landline from the same ISP, in theory it might be possible but your ISP would need to support it. 

You can do failover but that's not a bond.

----------

## Nreal

Ok thank You for the answers. It sure would be nice, to be able to combine all 4gs in a house + landline..

----------

