# [SOLVED] fwmark based routing stopped working

## tnt

I have two ADSL links on eth2 and eth3.

ADSL1 with IP 10.5.18.18 is default gateway in main routing table.

ADSL2 with IP 10.5.18.22 is used just for marked packets:

```
#!/bin/bash

ip route add default via 10.5.18.22 dev eth3 table 20

ip rule add fwmark 0x351 table 20

ip rule add fwmark 0x352 table 20

ip rule add fwmark 0x353 table 20

ip route flush cache

```

everything worked fine for years using kernels 2.6.24 and 2.6.29.

recently I upgraded to 2.6.32-r2 and traffic through ADSL2 stopped.

the moment I delete table 20 and ip rules, everything works fine:

I can set both ADSL1 or ADSL2 as default gateway and they will work.

again, the moment I start making routing decision considering firewall marks, I get traffic only on ADSL1 (main table default gw) interface.

any info on what could be changed between kernels 2.6.29 and 2.6.32 regarding this issue?

----------

## tnt

I've found out that when I mark ICMP protocol with 0x351 fwmark and try too ping something ping packets are sent via eth3 indeed:

iptraf detailed eth3 statistics shows that there are constatnly outgoing ICMP packages.

even more interesting is fact that there is exactly the same number of incoming ICMP packages, but my ping output is empty:

there is no "Destination Host Unreachable" or similar - nothing.

this leeds me to believe that ICMP packages are routed right, I receive some answer, but those answer packages are discarded.

so, I've flushed all firewall rules except marking for ICMP, and added explicit 

```
iptables -t mangle -A OUTPUT -p ICMP -j MARK --set-mark 0x351
```

that didn't help.

I've added explicit rule

```
iptables -I INPUT -i eth3 -j ACCEPT
```

that didn't help.

I've checked, and my source route verification is turned off for these ifaces:

```
etc # sysctl net.ipv4.conf.default.rp_filter

net.ipv4.conf.default.rp_filter = 1

etc # sysctl net.ipv4.conf.eth2.rp_filter

net.ipv4.conf.eth2.rp_filter = 0

etc # sysctl net.ipv4.conf.eth3.rp_filter

net.ipv4.conf.eth3.rp_filter = 0
```

changing that to "=1" doesn't solve the problem.

any idea what could go wrong and why does my system discard packages from eth3 if they are not routed by main ruting table?

----------

## tnt

tcpdump on eth3 after 3 pings to 216.239.34.10

 *Quote:*   

> ping -I eth3 -c3 216.239.34.10
> 
> PING 216.239.34.10 (216.239.34.10) from 10.5.18.21 eth3: 56(84) bytes of data.
> 
> --- 216.239.34.10 ping statistics ---
> ...

 

```
13:24:23.556436 00:23:54:07:e9:6a > 00:90:d0:da:d2:06, ethertype IPv4 (0x0800), length 98: 10.5.18.21 > 216.239.34.10: ICMP echo request, id 51300, seq 1, length 64

13:24:23.605304 00:90:d0:da:d2:06 > 00:23:54:07:e9:6a, ethertype IPv4 (0x0800), length 98: 216.239.34.10 > 10.5.18.21: ICMP echo reply, id 51300, seq 1, length 64

13:24:24.555536 00:23:54:07:e9:6a > 00:90:d0:da:d2:06, ethertype IPv4 (0x0800), length 98: 10.5.18.21 > 216.239.34.10: ICMP echo request, id 51300, seq 2, length 64

13:24:24.603520 00:90:d0:da:d2:06 > 00:23:54:07:e9:6a, ethertype IPv4 (0x0800), length 98: 216.239.34.10 > 10.5.18.21: ICMP echo reply, id 51300, seq 2, length 64

13:24:25.563105 00:23:54:07:e9:6a > 00:90:d0:da:d2:06, ethertype IPv4 (0x0800), length 98: 10.5.18.21 > 216.239.34.10: ICMP echo request, id 51300, seq 3, length 64

13:24:25.610497 00:90:d0:da:d2:06 > 00:23:54:07:e9:6a, ethertype IPv4 (0x0800), length 98: 216.239.34.10 > 10.5.18.21: ICMP echo reply, id 51300, seq 3, length 64

```

so, I'm definitely getting those packets back, but system ignoress them.  :Sad: 

----------

## tnt

setting

```
# Enables source route verification

net.ipv4.conf.default.rp_filter = 2

# Enable reverse path

net.ipv4.conf.all.rp_filter = 2
```

in /etc/sysctl.conf solved the problem!

----------

