# Understanding Routing issue

## adlerweb

Hello,

i've got some network fsckup on one of my machines. While this state only shows rarely and a reboot quickly "solves" the issue it somewhat puzzles me because i couldn't find a reason for this.

First of all my network setup: I'm connected to a local network with a static IP. Since i use VMs the corresponding eth0 is bound to a bridge br0:

```
brctl show

bridge name   bridge id      STP enabled   interfaces

br0      8000.0025xxx   no      eth0

```

 *Quote:*   

> ip addr
> 
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
> 
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> ...

 

As you can see there is another interface: tap0. This is a openvpn-tunnel. Since 192.168.1.0/24 is a more or less public and rather insecure network i tend to route everything except local networks through this vpn, so my routing looks like this:

```
ip route show

default via 192.168.2.1 dev tap0 

87.98.xxx.xxx via 192.168.1.1 dev br0  <- Thats my VPN-Server

127.0.0.0/8 dev lo  scope link 

192.168.2.0/24 dev tap0  proto kernel  scope link  src 192.168.2.3 <- Added by OpenVPN

192.168.1.0/24 dev br0  scope link 

192.168.10.0/24 via 192.168.1.1 dev br0 

192.168.11.0/24 via 192.168.1.1 dev br0 -|

192.168.12.0/24 via 192.168.1.1 dev br0  |

192.168.13.0/24 via 192.168.1.1 dev br0  |- Other Networks on eth0

192.168.14.0/24 via 192.168.1.1 dev br0  |

192.168.15.0/24 via 192.168.1.1 dev br0 -|

```

This whole setup works...most of the time at least. My problem starts when the openvpn-connection drops - tap0 goes down, the routes are removed and i cant connect to external servers because i've got no default route. No problem so far. Now i reestablish the VPN, routes go back to the state shown above and i am able to ping 192.168.2.1 (the router inside the VPN) and some external servers - but not all. Some external Servers that i tried to contact when the VPN was down do not respond. I first thought of an borked routing cache so i used "ip route flush cache" to get rid of old items. Still nothing. A tcpdump on the VPN shows me this:

 *Quote:*   

> tcpdump -i tap0
> 
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> 
> listening on tap0, link-type EN10MB (Ethernet), capture size 96 bytes
> ...

 

Routing cache says the same:

 *Quote:*   

> ...
> 
> 94.23.xxx tos lowdelay via 192.168.1.60 dev tap0  src 192.168.2.3 
> 
>     cache  ipid 0x1f3e
> ...

 

Wait, what? 192.168.1.60 is a gateway located on eth0/br0 - it is used for the locally routed subnets and could also route to the external server. (While 192.168.1.1 is given for this subnets, the .1-router usually sends an ICMP Redirect Message pointing to the "real" router 192.168.1.60). OK, but now my question:

Why is 192.168.1.60 used for the external server - there is no route related to the external server pointing to .60 or .1. I also flushed again and monitored br0 - no traffic related to the external server or the gateways. Are there other caches involved in routing that i could flush? Could routing protocols on eth0 interfere even if there is a matching static route?

Edit: Even if i set a static route to the external host like shown below (and clear the cache) the pc still tries to route to .1.60 on tap0

```
94.23.xxx1 via 192.168.104.1 dev tap0
```

----------

## Veldrin

could you also post the routing table when the vpn is down. I am curious, where the default route points to.

 *Quote:*   

> Are there other caches involved in routing that i could flush? Could routing protocols on eth0 interfere even if there is a matching static route? 

 

Depends on what you mean by routing protocol. If you are thinking of OSPF, RIP, BGP, IS-IS and a like, then yes, if a more specific route has been setup, that one will be taken. 

If you are thinking about the routes added by openvpn, (which are static ones), then probably no (that is unless there is a more specific one).

V.

----------

## adlerweb

 *Veldrin wrote:*   

> could you also post the routing table when the vpn is down. I am curious, where the default route points to.

 

OpenVPN's gateway gets deleted, so there is no default gateway:

```
ip route show

87.98.xxx.xxx via 192.168.1.1 dev br0 

127.0.0.0/8 dev lo  scope link 

192.168.10.0/24 via 192.168.1.1 dev br0

192.168.1.0/24 dev br0  scope link 

192.168.11.0/24 via 192.168.1.1 dev br0

192.168.12.0/24 via 192.168.1.1 dev br0

192.168.13.0/24 via 192.168.1.1 dev br0

192.168.14.0/24 via 192.168.1.1 dev br0

192.168.15.0/24 via 192.168.1.1 dev br0

```

```
ping 8.8.8.8

connect: Network is unreachable

```

 *Veldrin wrote:*   

> Depends on what you mean by routing protocol. If you are thinking of OSPF, RIP, BGP, IS-IS and a like, then yes, if a more specific route has been setup, that one will be taken.

 

On br0 a default gateway is published via EIGRP but i thought EIGRP (or IGRP) will not be honored by a linux client unless a corresponding daemon is running. Anyhow: Shouldn't my local static route have a higher priority?

Add:

I was surfing wikipedia while writing this post... Before i killed openvpn "ip route show cache" showed:

```
91.198.174.225 via 192.168.2.1 dev tap0  src 192.168.2.3 
```

when i kill openvpn it is

```
91.198.174.225 from 192.168.104.3 via 192.168.199.63 dev tap0
```

If i reestablish the vpn it stays at this state - even after "ip route flush cache"

----------

## AngelKnight

 *adlerweb wrote:*   

> I was surfing wikipedia while writing this post... Before i killed openvpn "ip route show cache" showed:
> 
> ```
> 91.198.174.225 via 192.168.2.1 dev tap0  src 192.168.2.3 
> ```
> ...

 

So who's 192.168.199.63, and who is 192.168.104.3?  192.168.104.X doesn't appear in your routing tables anywhere, nor does 192.168.199.X.  Looks like you've got some additional investigating to do.

----------

