# Activate two connections simultaneously

## orion777

Good day!

I try to activate two connections simultaneously. Both of them are cellular modems. 

The first one is recognized as wwan0, the second one is usb0.

The problem is that if only wwan0 is activated, then it is able to accept ssh sessions, as well as reply on ping requests, etc.

Next, if I manually activate the usb0, then wwan0 drops ssh session, does not respond on ping requests, whereas usb0 will gets its IP and will accept ssh and ping requests. In this situation BOTH interfaces will have their IP addresses. ALSO wwan0 will be able to send ping requests, but will not respond to ping/ssh requests from the network.

```
pi64 ~ # ifconfig

eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

        ether b8:27:eb:28:f1:db  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 0  bytes 0 (0.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

        inet 127.0.0.1  netmask 255.0.0.0

        inet6 ::1  prefixlen 128  scopeid 0x10<host>

        loop  txqueuelen 1000  (Local Loopback)

        RX packets 8  bytes 576 (576.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 8  bytes 576 (576.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet 213.100.165.92  netmask 255.255.255.248  broadcast 213.100.165.95

        ether 02:1e:10:1f:00:00  txqueuelen 1000  (Ethernet)

        RX packets 73  bytes 6400 (6.2 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 74  bytes 10677 (10.4 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wwan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet 213.100.164.15  netmask 255.255.255.224  broadcast 213.100.164.31

        ether 00:1e:10:1f:00:00  txqueuelen 1000  (Ethernet)

        RX packets 254  bytes 18468 (18.0 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 224  bytes 31469 (30.7 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

```

Next, if I deactivate usb0, then the wwan0 starts to respond on ping and ssh requests..

In my setup I would like to use wwan0 for the ssh session, whereas usb0 should to be used to accept connections from wan.

(Yes, similar situation occurs if eth0 and usb0 are used. Also I was playing with starting priorities in NetworkManager GUI; also I was working without X server and its graphical NetworkManager, but the result is similar).

```

pi64 ~ # iptables -S

-P INPUT ACCEPT

-P FORWARD ACCEPT

-P OUTPUT ACCEPT
```

Please suggest some ideas..

----------

## NeddySeagoon

orion777,

Tell us what you want to achieve rather than asking for help with your perceived solution.

I suspect your routing table will explain the observed behaviour but not offer any help for a fix.

----------

## Hu

How are you activating the connections?  What is the output of ip route in each of the three permutations?  What is the output of iptables-save, which is more detailed than iptables -S?

----------

## orion777

Good evening!

I would like like to use the first 3G/LTE Huawei 3372h cellular dongle to establish ssh sessions for the remote control. This first dongle is called wwan0 (or eth1, depending on its internal configuration, but the result is similar).

wwan0 activates automatically when the system boots up (I was not doing here anything).

The second dongle is more advanced: Huawei ME906; in the system it us called usb0. I would like to use it to test cellular connection depending on some kind of mobility, different types of antennas, with and without receiver diversity, etc. To do this, I need that usb0 should to reply on ping request from wan side, as well as to accept iperf, netperf, ... incomming connections from the wan, whereas ssh sessions should to remain in stable wwan0.

However, usb0 won't activates automatically (even with priority +1) unless at least one connection is already active. So I activate it manually thru the ssh session via the nmtui or nmcli -t con up id "Tele2 connection". After this it gets its IP address, it will be able to ping something eg ping -I usb0 8.8.8.8, but it will not respond on any requests from the wan (e.g. ping or iperf from wan side).

However, sometimes it starts to accept all requests from the wan, but in this case wwan0 will drop ssh session...

So next, if I deactivate wwan0, then usb0 will automatically starts to accept ssh, ping requests without my intervention..

Example: raspberry is just booted

```
pi64 ~ # ifconfig

eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

        ether b8:27:eb:28:f1:db  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 0  bytes 0 (0.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

        inet 127.0.0.1  netmask 255.0.0.0

        inet6 ::1  prefixlen 128  scopeid 0x10<host>

        loop  txqueuelen 1000  (Local Loopback)

        RX packets 8  bytes 576 (576.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 8  bytes 576 (576.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

usb0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

        ether 02:1e:10:1f:00:00  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 0  bytes 0 (0.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wwan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet 213.100.164.15  netmask 255.255.255.224  broadcast 213.100.164.31

        ether 00:1e:10:1f:00:00  txqueuelen 1000  (Ethernet)

        RX packets 97  bytes 8717 (8.5 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 92  bytes 9985 (9.7 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

pi64 ~ # ip route

default via 213.100.164.1 dev wwan0 proto static metric 700

213.100.164.0/27 dev wwan0 proto kernel scope link src 213.100.164.15 metric 700

pi64 ~ # iptables-save

pi64 ~ # nmcli -t con up id "Tele2 connection"
```

in this example after usb0 starting, wwan0 freeze ssh session, so I will establish new one thru usb0..

```
pi64 ~ # ifconfig

eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500

        ether b8:27:eb:28:f1:db  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 0  bytes 0 (0.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

        inet 127.0.0.1  netmask 255.0.0.0

        inet6 ::1  prefixlen 128  scopeid 0x10<host>

        loop  txqueuelen 1000  (Local Loopback)

        RX packets 8  bytes 576 (576.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 8  bytes 576 (576.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet 213.100.165.92  netmask 255.255.255.248  broadcast 213.100.165.95

        ether 02:1e:10:1f:00:00  txqueuelen 1000  (Ethernet)

        RX packets 74  bytes 6378 (6.2 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 81  bytes 11987 (11.7 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wwan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet 213.100.164.15  netmask 255.255.255.224  broadcast 213.100.164.31

        ether 00:1e:10:1f:00:00  txqueuelen 1000  (Ethernet)

        RX packets 239  bytes 18171 (17.7 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 184  bytes 21489 (20.9 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

pi64 ~ # ip route

default via 213.100.165.89 dev usb0 proto static metric 101

default via 213.100.164.1 dev wwan0 proto static metric 700

213.100.164.0/27 dev wwan0 proto kernel scope link src 213.100.164.15 metric 700

213.100.165.88/29 dev usb0 proto kernel scope link src 213.100.165.92 metric 101

pi64 ~ # iptables-save
```

So now only usb0 is pingable and accept ssh connections from the wan. In order to proove that both connections are still working by itself, I will run ping to the wan

```
pi64 ~ # ping -I usb0 8.8.8.8

PING 8.8.8.8 (8.8.8.8) from 213.100.165.92 usb0: 56(84) bytes of data.

64 bytes from 8.8.8.8: icmp_seq=1 ttl=47 time=53.1 ms

64 bytes from 8.8.8.8: icmp_seq=2 ttl=47 time=65.7 ms

64 bytes from 8.8.8.8: icmp_seq=3 ttl=47 time=63.8 ms

^C

--- 8.8.8.8 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2003ms

rtt min/avg/max/mdev = 53.092/60.870/65.709/5.554 ms

pi64 ~ # ping -I wwan0 8.8.8.8

PING 8.8.8.8 (8.8.8.8) from 213.100.164.15 wwan0: 56(84) bytes of data.

64 bytes from 8.8.8.8: icmp_seq=1 ttl=47 time=463 ms

64 bytes from 8.8.8.8: icmp_seq=2 ttl=47 time=206 ms

64 bytes from 8.8.8.8: icmp_seq=3 ttl=47 time=203 ms

^C

--- 8.8.8.8 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2000ms

rtt min/avg/max/mdev = 202.801/290.628/462.903/121.824 ms

```

----------

## NeddySeagoon

orion777,

Please post the output of route.

I suspect that you have two default routes when both interfaces are up. That can't work. Only the lowest one in the routing table can be used.

----------

## Hu

He showed ip route, which is a slightly different format, but shows the problem.  You are right.  He has two defaults, one through each device: *orion777 wrote:*   

> 
> 
> ```
> pi64 ~ # ip route
> 
> ...

 

----------

## orion777

Now I was replacing wwan0 with the same modem H3372, but which is working in a HiLink mode (the previous one was in Stick mode), so it runs its internal NAT server and is recognized as eth1.

So now I have reverse situation: eth1 has greatest priority (smaller metric), than usb0. USB0 still don't accept ssh sessions, as well as ping requests from the WAN side.

```
pi64 ~ # route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

default         hi.link         0.0.0.0         UG    101    0        0 eth1

default         static-213-100- 0.0.0.0         UG    700    0        0 usb0

192.168.8.0     0.0.0.0         255.255.255.0   U     101    0        0 eth1

static-213-100- 0.0.0.0         255.255.255.248 U     700    0        0 usb0

pi64 ~ # ip route

default via 192.168.8.1 dev eth1 proto dhcp metric 101

default via 213.100.165.89 dev usb0 proto static metric 700

192.168.8.0/24 dev eth1 proto kernel scope link src 192.168.8.10 metric 101

213.100.165.88/29 dev usb0 proto kernel scope link src 213.100.165.92 metric 700
```

So yes, as we can see, I have two default routes when both interfaces are up. But I was not playing with ip routes..

So what I have to do in this situation? Is it possible to make system to accept ssh sessions thru eth1, whereas usb0 will accept iperf, ping requests to/from the wan side?

----------

## NeddySeagoon

orion777,

When either interface is up alone, it works.

When you bring up the second interface and get 

```
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

default         hi.link         0.0.0.0         UG    101    0        0 eth1

default         static-213-100- 0.0.0.0         UG    700    0        0 usb0 
```

all of the external traffic is sent over usb0 as the top default route cannot be reached.

That breaks all the external connections on eth1. The top default route.

You must not have a second default route.  If that's difficult the script that brings up the second interface needs to delete the second default route.

Now it gets tricky as you cannot use the second interface to reach the outside world.

To fix that, you seed to assign a static route to the second interface. That's difficult if you have to use dhcp too. 

Policy routing may be able to help. I'm only aware of the existence of policy routing.

I've failed badly to make it work setting up my own VPN.

----------

## Anon-E-moose

 *NeddySeagoon wrote:*   

> orion777,
> 
> When either interface is up alone, it works.
> 
> When you bring up the second interface and get 
> ...

 

If whatever you're using to connect to the internet allows you to set the interface (like ping) then yes it will work with one default and one non default.

That's how I have my vpn set up, I pass the IP to use to my torrent app and it will use the vpn route. 

If it won't allow you to specify the interface/IP (like firefox) then you're out of luck

But there's no way to just tell some apps to use one and other apps to use the other (other than what I described above)

Actually I vaguely remember reading something once where Iptables was set up to do something like that, but it's not easy.

----------

## NeddySeagoon

Anon-E-moose,

You can filter TCP traffic by port number but that won't work for UDP, as there are no ports.

Whatever, two default routes cannot work.

----------

## Anon-E-moose

True you cannot have two default routes (kind of an oxymoron if you think about it  :Smile:  ).  

With my vpn it's not set up as a default route, just a route, therefore I can have selective applications use it.

----------

## orion777

In the given configuration eth1 obtains its IP by DHCP (just because it is slightly faster), but I can make settings both for usb0 and eth1 to force  them to use static IPs. Is it necessary?

So... What I have to do at first? This?

```
ip route del 213.100.165.88/29
```

* because I would like to establish ssh sessions thru the eth1 (or wwan0, they ase same modem with different configs)

Next, as I understand, I have to run iperf server with the specified interface's IP. It is should to be possible, yes. The ping replies, as I understood, will not work at all, right?

----------

## nativemad

 *Quote:*   

> True you cannot have two default routes

 

This is not quite right.

You can have two default gateways, but you will need iproute2 and some lookup-table for that!

In that table you can define a gateway per interface and tell it to answer requests on the appropriate interface.

New outgoing connections will always follow the "default" (or better the rules within the default table), or it could be defined on the application level like with ping -I, as already mentioned.

HTH, cheers

----------

## orion777

I'm not familiar with all these tricks.. Maybe You can specify some guidelines to read?

----------

## nativemad

 *orion777 wrote:*   

> I'm not familiar with all these tricks.. Maybe You can specify some guidelines to read?

 

These are my to google results for "iproute2 looup table":

http://linux-ip.net/html/routing-tables.html

http://www.allgoodbits.org/articles/view/24

And here is an older one from this forum:

https://forums.gentoo.org/viewtopic-t-857276-start-0.html

----------

## Anon-E-moose

You're right, each interface can have a default route assigned to it, but the system only ever considers one as default. The terminology is a little confusing. 

I hadn't messed with my vpn routing in a while, as I've had it working fine the way I have it set, but with your mentioning routing tables, I'm re-looking at my rules.

Thanks for the hint.

----------

## orion777

Good evening!

So, since I make all configurations of the modem from the GUI of NetworkManager, then:

edit connection settings -> IPv4settings -> Routes -> and select "use this connection only for resources on its network" and this will remove the interface from the "default".

Next, it is possible to select "connect automatically" and it will activates automatically during boot of the system (without manual activation via the nmtui), but this is optional.

However, I still is able to ping over two interfaces, whereas iperf don't connect thru the modem.. (as well as external connections are also impossible).

I can tolerate with no possibility to ping/ssh from the WAN side thru this interface, but iperf should to work over it... This is a problem   :Sad:   Any ideas?

```
pi64 ~ # route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

default         hi.link         0.0.0.0         UG    101    0        0 eth1

192.168.8.0     0.0.0.0         255.255.255.0   U     101    0        0 eth1

static-213-100- 0.0.0.0         255.255.255.248 U     102    0        0 usb0

pi64 ~ # ping 8.8.8.8 -I eth1

PING 8.8.8.8 (8.8.8.8) from 192.168.8.10 eth1: 56(84) bytes of data.

64 bytes from 8.8.8.8: icmp_seq=1 ttl=46 time=302 ms

^C

--- 8.8.8.8 ping statistics ---

2 packets transmitted, 1 received, 50% packet loss, time 1001ms

rtt min/avg/max/mdev = 302.226/302.226/302.226/0.000 ms

pi64 ~ # ping 8.8.8.8 -I usb0

PING 8.8.8.8 (8.8.8.8) from 213.100.165.92 usb0: 56(84) bytes of data.

64 bytes from 8.8.8.8: icmp_seq=1 ttl=47 time=137 ms

64 bytes from 8.8.8.8: icmp_seq=2 ttl=47 time=48.4 ms

64 bytes from 8.8.8.8: icmp_seq=3 ttl=47 time=67.1 ms

^C

--- 8.8.8.8 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2003ms

rtt min/avg/max/mdev = 48.360/84.188/137.069/38.170 ms

pi64 ~ # iperf -c 84.245.226.141 -B 192.168.8.10 -t 2

-bash: iperf: command not found

pi64 ~ # iperf3 -c 84.245.226.141 -B 192.168.8.10 -t 2

Connecting to host 84.245.226.141, port 5201

[  5] local 192.168.8.10 port 59639 connected to 84.245.226.141 port 5201

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd

[  5]   0.00-1.00   sec  77.0 KBytes   630 Kbits/sec    0   16.5 KBytes

[  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    0   22.0 KBytes

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval           Transfer     Bitrate         Retr

[  5]   0.00-2.00   sec  77.0 KBytes   315 Kbits/sec    0             sender

[  5]   0.00-2.00   sec  30.2 KBytes   124 Kbits/sec                  receiver

iperf Done.

pi64 ~ # iperf3 -c 84.245.226.141 -B 213.100.165.92 -t 2

^C- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval           Transfer     Bitrate         Retr

iperf3: interrupt - the client has terminated

pi64 ~ #

```

----------

## NeddySeagoon

orion777,

```
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface 

static-213-100- 0.0.0.0         255.255.255.248 U     102    0        0 usb0
```

is restricted to the 8 IP addresses in its  255.255.255.248 network. One of those will be a gateway.

You can add a static route over this gateway.

----------

## orion777

Okay, it seems to work with the following

```
ip route add 84.245.226.0/24 via 213.100.165.89
```

where I am asking to route everything from my IP address of the remote windows machine to go thru the usb0 gateway.

But You mention that it is possible to specify exact ports for a tcp connection. How it can be done? What are the key words for this task to search in google?

----------

## NeddySeagoon

orion777,

You need iptables and rules to filter TCP/IP traffic based on its source and/or destination port.

UDP does tot have ports, so this is limited to TCP/IP only.

----------

## orion777

So Okay, here what we have:

```
route

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

default         hi.link         0.0.0.0         UG    101    0        0 eth1

192.168.8.0     0.0.0.0         255.255.255.0   U     101    0        0 eth1

static-213-100- 0.0.0.0         255.255.255.248 U     700    0        0 usb0
```

But if I make 

```
iptables -A INPUT -i usb0 -p tcp --dport 5201 -j ACCEPT
```

 then the locally started iperf3 -s server still accept connections only from eth1, whereas such connections are not possible from usb0.

Whereas if I try to allow iperf3 -c clients packets to run to usb0, then I can't specify interface:

```
 iptables -A OUTPUT -i usb0 -p tcp --dport 5201 -j ACCEPT

iptables v1.8.3 (legacy): Can't use -i with OUTPUT

```

 maybe I have to use -s or -d, but actually I can't understand how to do this..  :Rolling Eyes: 

----------

## nativemad

It is not often that Neddy is wrong, but also UDP has port and port numbers.   :Wink: 

An iptables OUTPUT rule can't be set on an incoming interface, therefore it would need to be -o instead of -i!

But if you don't have any rule that denies traffic, you won't need explicit ACCEPT rules.

So if "iptables -L" is empty or doesn't have any deny rule, no access rule is needed.

I still think your problem can only be solved with ip route lookup tables.

There you tell that incoming connections should get answered through the interface where they came from.

Now your default route is on eth1. So you actually got 3 gateways to the internet!? That does not make it easier...

Look at the solution here https://forums.gentoo.org/viewtopic-t-857276-start-0.html.

He used iptables to set a mark from which mac/interface they came. He checks that mark on Output and uses the seperate routing table per interface to use the right gateway.

As you have separated interfaces, your iptables rules don't need to rely on mac adresses. You could use -i usb0 and so on.

HTH

----------

## NeddySeagoon

nativemad,

Thank you for the learning opportunity.

----------

## Hu

For completeness, though UDP has ports, it does not have connections the way that TCP does.  It is common to describe flows of related UDP packets as a "connection" because stateful firewalls often need to treat related packets as a group.  For example, a UDP packet out to a DNS server will usually elicit a UDP response back with the answer.  You want the firewall to recognize the incoming packet as solicited traffic and allow it through, but you may not want to allow all incoming unsolicited UDP to enter.  To handle this, the firewall needs to track the flow and recognize that this is an expected response.  Partly due to how TCP flows are handled, these UDP flows are often described as connections and often handled through similar tools in the firewall administration program(s).  However, while TCP has a specific set of steps to establish a connection, and a specific set to formally terminate a connection, UDP has neither.  Instead, firewalls infer UDP "connections" through inspection of the packets traversing the system.

The initiator may be designed not to expect any response at all from the receiver, which can be useful in protocols where the sender doesn't care whether the receiver is processing the full flow.  This could happen in a streaming audio/video system (such as for conference calls), where late/lost packets are useless, so there is no point in detecting or reporting them.  The receiver should attempt to keep going with user-perceptible gaps for the lost data, and resume when the stream does.

Similarly, with UDP, there is no explicit stop protocol.  When the sender has nothing more to send, it can just stop with no notice.  This is annoying for firewall administrators, since it would be nice to forget a flow when the participants declare it done, but since there is no standard for declaring termination, your choices are to rely on a timeout heuristic ("No traffic in the last 5 minutes, I guess they're done") or you need a protocol-aware helper to inspect the application data to detect when a termination occurs.  For TCP, the firewall can just watch for RST and FIN, and count on the standard requiring the peers to use those for a clean shutdown.

----------

