# SOLVED-bridged openvpn problem - arp traffic but cannot ping

## bladebreak

Apologies to those on the openvpn list, I am crossposting here after getting no response on the list. I know it's a bit specific to one app but I've seen a number of you who have got this successfully working under gentoo and hope one of you can help me out.

I've been unsuccessfully attempting to establish a bridged VPN connection between two linux boxes for some time now. Openvpn can successfully connect, and when running tcpdump on tap0 I can see arp requests passing across the network and being answered, but neither side can ping the opposite endpoint or any machines on the opposite subnet.

Systems: Two gentoo linux boxes, both on 2.6.7 kernels with OpenVPN version 1.5.0

Network setup:

```

|--------------|

|LAN 1         |

|192.168.1.0/24|

|--------------|

     |||

     br0

 (phys. eth1)

 192.168.1.1

      |||

|--------------|

| ilium        |

|(linux box 1) |

|--------------|

     |||

    eth0

  24.57.x.xx1

     |||

  internet

     |||

    eth0

  24.57.x.xx2

     |||

|--------------|

| fishbox      |

|(linux box 2) |

|--------------|

     |||

     br0

 (phys. eth1)

 192.168.10.1

      |||

|---------------|

|LAN 2          |

|192.168.10.0/24|

|---------------| 

```

The VPN endpoint for ilium is 192.168.254.1, and for fishbox is 192.168.254.2

For testing's sake I have created a minimalistic firewall file, with default policies of allow on all interfaces:

```

#!/bin/bash

PRIVATE=192.168.10.0/24

LOOP=127.0.0.1

iptables -F

# Set default policies

iptables -P OUTPUT ACCEPT

iptables -P INPUT ACCEPT

iptables -P FORWARD ACCEPT

iptables -t filter -P INPUT ACCEPT

iptables -t filter -P OUTPUT ACCEPT

iptables -t filter -P FORWARD ACCEPT

iptables -t nat -P PREROUTING ACCEPT

iptables -t nat -P OUTPUT ACCEPT

iptables -t nat -P POSTROUTING ACCEPT

iptables -t mangle -P PREROUTING ACCEPT

iptables -t mangle -P OUTPUT ACCEPT

# Flush all old rule sets

iptables -t filter -F INPUT

iptables -t filter -F OUTPUT

iptables -t filter -F FORWARD

iptables -t nat -F PREROUTING

iptables -t nat -F OUTPUT

iptables -t nat -F POSTROUTING

iptables -t mangle -F PREROUTING

iptables -t mangle -F OUTPUT

iptables -A FORWARD -p tcp --sport 137:139 -o eth0 -j DROP

iptables -A FORWARD -p udp --sport 137:139 -o eth0 -j DROP

iptables -A OUTPUT -p tcp --sport 137:139 -o eth0 -j DROP

iptables -A OUTPUT -p udp --sport 137:139 -o eth0 -j DROP

# Allow local loopback

iptables -A INPUT -s $LOOP -j ACCEPT

iptables -A INPUT -d $LOOP -j ACCEPT

# Allow incoming pings (can be disabled)

iptables -A INPUT -p icmp --icmp-type echo-request -j ACCEPT

# Allow services such as www and ssh (can be disabled)

iptables -A INPUT -p tcp --dport http -j ACCEPT

iptables -A INPUT -p tcp --dport ssh -j ACCEPT

#Allow OpenVPN

iptables -A INPUT -p udp --dport 5000 -j ACCEPT

# Allow packets from TUN/TAP devices.

iptables -A INPUT -i tun+ -j ACCEPT

iptables -A FORWARD -i tun+ -j ACCEPT

iptables -A INPUT -i tap+ -j ACCEPT

iptables -A FORWARD -i tap+ -j ACCEPT

# Allow packets from private subnets

iptables -A INPUT -i br0 -j ACCEPT

iptables -A FORWARD -i br0 -j ACCEPT

# Keep state of connections from local machine and private subnets

iptables -A OUTPUT -m state --state NEW -o eth0 -j ACCEPT

iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

iptables -A FORWARD -m state --state NEW -o eth0 -j ACCEPT

iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT

# Masquerade local subnet

iptables -t nat -A POSTROUTING -s $PRIVATE -o eth0 -j MASQUERADE

```

(the setup is the same on both linux boxes with the appropriate range set for PRIVATE).

The config file for ilium:

```

remote 24.57.x.xx2

dev tap0

secret vpn.key

comp-lzo

ping 15

verb 5

ifconfig 192.168.254.1 255.255.255.0

```

and for fishbox:

```

remote 24.57.x.xx1

dev tap0

secret vpn.key

comp-lzo

ping 15

verb 5

ifconfig 192.168.254.2 255.255.255.0

```

Routing on ilium:

```

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 br0

192.168.10.0    192.168.254.2   255.255.255.0   UG    0      0        0 tap0

192.168.254.0   0.0.0.0         255.255.255.0   U     0      0        0 tap0

24.57.0.0       0.0.0.0         255.255.240.0   U     0      0        0 eth0

127.0.0.0       127.0.0.1       255.0.0.0       UG    0      0        0 lo

0.0.0.0         24.57.0.1       0.0.0.0         UG    0      0        0 eth0

```

and fishbox:

```

192.168.1.0     192.168.254.1   255.255.255.0   UG    0      0        0 tap0

192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 br0

192.168.254.0   0.0.0.0         255.255.255.0   U     0      0        0 tap0

24.57.80.0      0.0.0.0         255.255.240.0   U     0      0        0 eth0

127.0.0.0       127.0.0.1       255.0.0.0       UG    0      0        0 lo

0.0.0.0         24.57.80.1      0.0.0.0         UG    0      0        0 eth0

```

(the routes to the opposite networks were created manually by me)

ifconfig on ilium:

```

br0       Link encap:Ethernet  HWaddr 00:00:C0:B2:D5:74

         inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0

         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

         RX packets:18756 errors:0 dropped:0 overruns:0 frame:0

         TX packets:16883 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:0

         RX bytes:7923296 (7.5 Mb)  TX bytes:6381819 (6.0 Mb)

eth0      Link encap:Ethernet  HWaddr 00:80:C8:D4:20:A5

         inet addr:24.57.x.xx1  Bcast:255.255.255.255  Mask:255.255.240.0

         UP BROADCAST NOTRAILERS RUNNING MULTICAST  MTU:1500  Metric:1

         RX packets:10408 errors:0 dropped:0 overruns:0 frame:0

         TX packets:10951 errors:0 dropped:0 overruns:0 carrier:0

         collisions:1 txqueuelen:1000

         RX bytes:4087067 (3.8 Mb)  TX bytes:2374911 (2.2 Mb)

         Interrupt:5 Base address:0x340

eth1      Link encap:Ethernet  HWaddr 00:00:C0:B2:D5:74

         UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1

         RX packets:18745 errors:0 dropped:0 overruns:0 frame:0

         TX packets:16883 errors:0 dropped:0 overruns:0 carrier:0

         collisions:99 txqueuelen:1000

         RX bytes:8202837 (7.8 Mb)  TX bytes:6393849 (6.0 Mb)

         Interrupt:3 Base address:0x290 Memory:d0000-d4000

lo        Link encap:Local Loopback

         inet addr:127.0.0.1  Mask:255.0.0.0

         UP LOOPBACK RUNNING  MTU:16436  Metric:1

         RX packets:52 errors:0 dropped:0 overruns:0 frame:0

         TX packets:52 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:0

         RX bytes:5190 (5.0 Kb)  TX bytes:5190 (5.0 Kb)

tap0      Link encap:Ethernet  HWaddr 00:FF:0A:FE:9D:05

         inet addr:192.168.254.1  Bcast:192.168.254.255  Mask:255.255.255.0

         UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1

         RX packets:60 errors:0 dropped:0 overruns:0 frame:0

         TX packets:206 errors:0 dropped:165 overruns:0 carrier:0

         collisions:0 txqueuelen:1000

         RX bytes:2520 (2.4 Kb)  TX bytes:16742 (16.3 Kb)

```

and on fishbox:

```

br0       Link encap:Ethernet  HWaddr 00:0D:88:B5:26:09

         inet addr:192.168.10.1  Bcast:192.168.10.255  Mask:255.255.255.0

         UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

         RX packets:310531 errors:0 dropped:0 overruns:0 frame:0

         TX packets:490220 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:0

         RX bytes:15812216 (15.0 Mb)  TX bytes:699162697 (666.7 Mb)

eth0      Link encap:Ethernet  HWaddr 00:0D:88:B5:26:0E

         inet addr:24.57.x.xx2  Bcast:255.255.255.255  Mask:255.255.240.0

         UP BROADCAST NOTRAILERS RUNNING MULTICAST  MTU:1500  Metric:1

         RX packets:2277821 errors:0 dropped:0 overruns:0 frame:0

         TX packets:331616 errors:0 dropped:0 overruns:0 carrier:0

         collisions:750 txqueuelen:1000

         RX bytes:810673387 (773.1 Mb)  TX bytes:23717664 (22.6 Mb)

         Interrupt:11

eth1      Link encap:Ethernet  HWaddr 00:0D:88:B5:26:09

         UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1

         RX packets:303579 errors:0 dropped:0 overruns:0 frame:0

         TX packets:496908 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:1000

         RX bytes:21035578 (20.0 Mb)  TX bytes:699695084 (667.2 Mb)

         Interrupt:5 Base address:0x2000

lo        Link encap:Local Loopback

         inet addr:127.0.0.1  Mask:255.0.0.0

         UP LOOPBACK RUNNING  MTU:16436  Metric:1

         RX packets:330 errors:0 dropped:0 overruns:0 frame:0

         TX packets:330 errors:0 dropped:0 overruns:0 carrier:0

         collisions:0 txqueuelen:0

         RX bytes:35748 (34.9 Kb)  TX bytes:35748 (34.9 Kb)

tap0      Link encap:Ethernet  HWaddr 00:FF:A7:8B:63:02

         inet addr:192.168.254.2  Bcast:192.168.254.255  Mask:255.255.255.0

         UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1

         RX packets:7079 errors:0 dropped:0 overruns:0 frame:0

         TX packets:450 errors:0 dropped:586 overruns:0 carrier:0

         collisions:0 txqueuelen:1000

         RX bytes:610357 (596.0 Kb)  TX bytes:23664 (23.1 Kb)

```

OpenVPN can establish a connection, and each box can ping its own endpoint (ilium can ping 254.1 and fishbox can ping 254.2). When attempting to ping the opposite side's endpoint, OpenVPN registers traffic flowing back and forth across the link, and tcpdump's on each of the tap0 devices display arp data flowing across the bridge:

fishbox root # ping 192.168.254.1

PING 192.168.254.1 (192.168.254.1) 56(84) bytes of data.

>From 192.168.254.2 icmp_seq=1 Destination Host Unreachable

...

ilium's tap0:

20:34:14.171025 arp who-has 192.168.254.1 tell 192.168.254.2

20:34:14.171236 arp reply 192.168.254.1 is-at 00:00:c0:b2:d5:74

...

fishbox's tap0:

15:58:22.426266 arp who-has 192.168.254.1 tell 192.168.254.2

15:58:22.446864 arp reply 192.168.254.1 is-at 00:00:c0:b2:d5:74

...

I can also see various other arp requests from other machines on each of the LANs traversing the bridge, but nobody can actually ping across the bridge.

A couple of points I noticed that may help spark ideas to those who know more about this than I:

- Originally rp_filter was turned on by my old firewall setup, which was causing the arp's to be dropped on the other side of the bridge. I've disabled it and now they are responding. Are there other kernel settings (besides that and ip_forward) that are critical to OpenVPN working correctly which I should double-check?

- I notice many dropped packets in the tap0 adapters in the ifconfig listings. Is there some way I can figure out why they are being dropped?

Any feedback or pointers would be most appreciated - I'm completely stumped, and judging from the ease at which most people on the list seem to be able to set up this software, I must be missing a setting or two.

TIA,

ScottLast edited by bladebreak on Fri Aug 06, 2004 1:43 pm; edited 1 time in total

----------

## devon

I don't use OpenVPN, but I noticed that the ARP replies says 00:00:C0:B2:D5:74 owns 192.168.254.1, but the MAC address for tap0 is 00:FF:0A:FE:9D:05. The first MAC address is for eth1 on ilium. Something to check out.  :Smile: 

----------

## bladebreak

Thanks so much devon - your sharp eye indeed spotted the problem!

For some reason the bridged endpoint was responding with the MAC address of eth1 instead of what it needed to (TAP0), so the route could not be properly established.

I fixed this by manually associating the addresses of the opposite endpoints with the correct MACs of the TAP0s:

On ilium:

```

arp -s 192.168.254.2 00:FF:A7:8B:63:02

```

and on fishbox:

```

arp -s 192.168.254.1 00:FF:0A:FE:9D:05 

```

I'm not sure why I had to do this, or if there is an automagic setting in OpenVPN that will do it for me, but for now the problem is resolved.

Thanks so much!!!

Scott

----------

