# Can I force a local user to a specific output interface?

## JSheridan

My gentoo box has one ethernet connection eth0 and two ppp connections.

The internet is available through both ppp connections and through another router in the local network.

local network: 192.168.2.0/24

gentoo box eth0: 192.168.2.1

secondary router: 192.168.2.2

ppp0 & ppp1: provider dependent

I managed to route clients using the box as router through different connections (ppp0,1 and the other router on 192.168.2.2) to the internet.

Now i want that local user a on the box uses i.e. ppp0 and local user b uses the router on 192.168.2.2 or ppp1.

Currently all local processes of all users only use the default route via ppp0.

Any help is appreciated.

----------

## truc

see iptables with the owner, mark and connmark modules and iproute (rules..)   :Question: 

----------

## JSheridan

 *truc wrote:*   

> see iptables with the owner, mark and connmark modules and iproute (rules..)  

 

As far as I know the routing decision for local generated packets is made before iptables. So the moment a packet enters the first iptables chain it already has a output interface. I think when a process opens a socket with not default interface/ip, the kernel chooses the best match.

If I'm not wrong it should look like this:

application -> routing decision -> mangle/output -> nat/output -> filter/output -> mangle/postrouting -> nat/postrouting -> interface

I first thought I could mark the packet in mangle/output and add an ip rule to match the mark like I did it with the client connections. Maybe I just messed up my routing table but the time I wrote my post I thought that I might not be able to correct the initial routing decision as easy as this.

But now, after I looked at the netfilter packet flow graph which shows a reroute check after mangle/output, I think I just might have missed a detail.

I tried something like this:

iptables -t mangle -A OUTPUT -m owner --uid-owner user_a -j MARK --set-mark 1

iptables -t mangle -A OUTPUT -m owner --uid-owner user_b -j MARK --set-mark 2

ip rule add fwmark 1 table 1

ip rule add fwmark 2 table 2

table main:

xxx.xxx.92.67 dev ppp0  proto kernel  scope link  src xxx.xxx.94.117

192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.1

127.0.0.0/8 via 127.0.0.1 dev lo

default via xxx.xxx.92.67 dev ppp0

table 1:

xxx.xxx.116.70 dev ppp1  proto kernel  scope link  src xxx.xxx.112.226

192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.1

127.0.0.0/8 via 127.0.0.1 dev lo

default via xxx.xxx.116.70 dev ppp1

table 2:

192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.1

127.0.0.0/8 via 127.0.0.1 dev lo

default via 192.168.2.2 dev eth0

The route from table main is always used for new connections :-/

Any advices?

----------

## truc

```
ip rule add fwmark 1 table 1

ip rule add fwmark 2 table 2 
```

set a priority to your rules! This will save you a lot of headaches for the future, this might even be your problem here, since I don't see something wrong with your configuration and I use something similar on my side (and it's working), I'll check tomorow

EDIT: I've checked and we're doing the same, so, did you try explicitly setting a priority?

----------

## JSheridan

well the priority should be correct:

0:      from all lookup local

97:     from all fwmark 0x3 lookup 103

98:     from all fwmark 0x2 lookup 102

99:     from all fwmark 0x1 lookup 101

32766:  from all lookup main

32767:  from all lookup default

Still if I mark in the output mangle table all packets use the default route over ppp0. Then they're getting mangled and rerouted to eth0 still using the ip of the ppp connection. I tried masquerading them but somehow it didn't really help. I got replies but they weren't delivered to the application.

What worked was binding another ip to the eth device and force user applications to use this ip and then directly add a ip rule using the src selector.

So what happens looks like:

application puts a packet into output

mangle / output: 

  of=ppp0 / src_ip=x.x.x.x

  set mark to 0x3

nat / output:

  of=ppp0 / src_ip=x.x.x.x

filter / output:

  of=ppp0 / src_ip=x.x.x.x

mangle / postrouting:

  of=eth0 / src_ip=x.x.x.x

nat / postrouting:

  masquerading packet resulting in

  of=eth0 / src_ip=192.168.2.1

response:

mangle / prerouting:

  if=eh0 / dst_ip=192.168.2.1

nat/ prerouting:

  packet not visible anymore

I don't know what happens with the packet. 

What am I missing?

Edit:

If my default route contains a route consisting of two combined routes via nexthop then it works the way it should. if it contains just one or two default routes with different priorities, it does not work.

The response / input packets are probably dropped after "demasquerading". I am probably just missing a routing entry, but which one?

Oh and without the nexthop default route i get martian sources like:

martian source 217.xxx.xxx.xxx from 193.99.144.85, on dev eth0

the source is the ip of the ppp device before being rerouted so basically my assumption seems to be correct in a way right? But why does this work when after replacing the default route with a route with nexthop?

----------

