# Need help configuring system with two NICs

## pstickar

Hi

My problem in short: some apps (e.g. Firefox) execute extremely slow when two NICs in my computer are active.

If only one of the NICs is active, all apps behave as expected.

I tried a few things, that I find reasonable, but the problem persists.

May be someone here can give me some advice. Many thanks in advance!!

The same, much longer:

I'm trying to configure my system to use two devices (not the real names, but pretty clear):

wlan0  <-- Access to the outside world (internet)

eth0   <-- A private network of mine

When everything connects, the answer to command ifconfig gives:

```
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet AAA.AAA.AAA.184  netmask 255.255.255.0  broadcast AAA.AAA.AAA.255

        inet6 fe80::aaaa:aaaa:aaaa:aaaa  prefixlen 64  scopeid 0x20<link>

        ether a0:a0:a0:a0:a0:a0  txqueuelen 1000  (Ethernet)

        RX packets 28  bytes 2808 (2.7 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 153  bytes 12938 (12.6 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

        device interrupt 18

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

        inet 127.0.0.1  netmask 255.0.0.0

        inet6 ::1  prefixlen 128  scopeid 0x10<host>

        loop  txqueuelen 1000  (Lokale Schleife)

        RX packets 48  bytes 3024 (2.9 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 48  bytes 3024 (2.9 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

        inet BBB.BBB.BBB.4  netmask 255.255.255.0  broadcast BBB.BBB.BBB.255

        inet6 fe80::bbbb:bbbb:bbbb:bbbb  prefixlen 64  scopeid 0x20<link>

        inet6 2a02:cccc:cccc:ccc:cccc:cccc:ccc:cccc  prefixlen 64  scopeid 0x0<global>

        ether b0:b0:b0:b0:b0:b0  txqueuelen 1000  (Ethernet)

        RX packets 15992  bytes 16969059 (16.1 MiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 8236  bytes 1359486 (1.2 MiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
```

and the answer to command netstat -rn gives:

```
default via AAA.AAA.AAA.1 dev eth0 proto dhcp src AAA.AAA.AAA.184 metric 1002

default via BBB.BBB.BBB.1 dev wlan0 proto dhcp src BBB.BBB.BBB.4 metric 3003

BBB.BBB.BBB.0/24 dev wlan0 proto dhcp scope link src BBB.BBB.BBB.4 metric 3003

AAA.AAA.AAA.0/24 dev eth0 proto dhcp scope link src AAA.AAA.AAA.184 metric 1002
```

After some instructive readings, I decided to follow the advice in

https://serverfault.com/questions/123553/how-to-set-the-preferred-network-interface-in-linux

and coded my own script:

```
ip route get 8.8.8.8 | grep -q AAA.AAA.AAA.1

RETVAL=$?

case $RETVAL in

1)

echo "Nothing to change."

;;

0)

echo "Re-do the routes."

ip route del default

ip route add default via BBB.BBB.BBB.1 dev wlan0

ip route add AAA.AAA.AAA.0/24 via AAA.AAA.AAA.1 dev eth0

;;

esac
```

This works in the sense that I access both address-spaces (e.g. pinging works very well and is very responsive) but the usual apps in KDE do their job very, very slowly. For example, Firefox takes 25 seconds to start (usual is immediate), and quite a bit of additional time to open any new URL. If I run the script above manually, or as the exit-hook, it makes no difference.

If I disconnect the cable on eth0, then everything behaves normal, at once, in the sense that the apps do not suffer any delay (and if they were in the middle of a delay, they resume to normal at once).

Some context: I'm using dhcpcd. I'm neither using NetworkManager, nor Netifrc. The net.* scripts are not there anymore. The wireless connection establishes with the help of wpa_supplicant.

My dhcpcd.conf looks like this (intentionally left the comments out):

```
duid

persistent

vendorclassid

option domain_name_servers, domain_name, domain_search

option classless_static_routes

option interface_mtu

option host_name

option rapid_commit

require dhcp_server_identifier

slaac private

noipv4ll
```

Again, many thanks in advance for your help.

Regards,

Pablo

----------

## alamahant

 *Quote:*   

> 
> 
> ]default via AAA.AAA.AAA.1 dev eth0 proto dhcp src AAA.AAA.AAA.184 metric 1002
> 
> 

 

eth0 should have NO default gateway.

----------

## NeddySeagoon

pstickar,

You have obfuscated the information we need to help you.

I suspect that your IPv4 addresses are all in the private ranges and will go through NAT before they go out on the internet.

Certainly you will get a nastygram from your ISP if you send any packets like that to them for any length of time.

As they are not useful outside of you network, there is no need to hide them. 

IPv6 is slightly different but its covered there too.

All routable IPv6 addresses start with a 2, like your 

```
inet6 2a02:cccc:cccc:ccc:cccc:cccc:ccc:cccc  prefixlen 64  scopeid 0x0<global> 
```

I hope you have an IPv6 firewall, otherwise that interface os directly connected to the big bad internet.

```
default via AAA.AAA.AAA.1 dev eth0 proto dhcp src AAA.AAA.AAA.184 metric 1002

default via BBB.BBB.BBB.1 dev wlan0 proto dhcp src BBB.BBB.BBB.4 metric 3003 
```

That probably does not do what you want as eth0 has the lowest metric, it will always be used as the default route.

Please make your post again, sharing everything except public IPs.

----------

## kikko

Hi pstickar

in the end, you want an additional route for your local network via eth0, while forwarding everything else via wlan0, correct?

The slowness is surely weird, but as you also have figured out is related to network setup 

Have 2 default gateways is weird, moreover the script does not delete both default routes:

```
 ~ # ip route

default via 192.168.1.1 dev enp12s0 proto dhcp src 192.168.1.5 metric 202 

default via 192.168.1.1 dev wlp3s0 proto dhcp src 192.168.1.118 metric 303 

192.168.1.0/24 dev enp12s0 proto dhcp scope link src 192.168.1.5 metric 202 

192.168.1.0/24 dev wlp3s0 proto dhcp scope link src 192.168.1.118 metric 303 

 ~ # ip route del default

 ~ # ip route

default via 192.168.1.1 dev wlp3s0 proto dhcp src 192.168.1.118 metric 303 

192.168.1.0/24 dev enp12s0 proto dhcp scope link src 192.168.1.5 metric 202 

192.168.1.0/24 dev wlp3s0 proto dhcp scope link src 192.168.1.118 metric 303 

```

My suggestion is to exclude DHCP for eth0 (use the "denyinterfaces eth0" in dhcpcd.conf, see the `man dhcpcd.conf` for the details) in first place and then add a single static route for the AAA network 

If this works for you, next step could be configure dhcpcd to behave differently for the 2 interfaces (using the "interface" directive")

----------

## pstickar

Thank you guys for the answers. I need to do some homework before I can react.

@NeddySeagoon:

- it was not the intention to obfuscate anything, but to me AAA and BBB are clearer than 192, 169, or 172, which I tend to confuse every second time. Yes, they are two non-overlapping and private address ranges, as the two routers had been configured. If I understand you correctly, you are suggesting that I might be asking one router addresses in the range of the other one.

- Other than obfuscation I have now the feeling that my post mixed IPv4 and IPv6.

- With regard to IPv6 you gave me interesting information that I was completely oblivious of. I used to think that "someone" cared to give me an IPv4 address, and then I just start from there. I also believed that it was either IPv4 or IPv6, but not both as complementary. Now I see the risks of being a bit too innocent. There is a firewall at the router, already setup by the ISP ... and I'm reading about shorewall right now. I'm tempted to set the USE flags "lite4" and "lite6" for a start, because the options are too many for me to digest. I'm following your article on IPv6, which I find a little terse...

I'll be back soon.

Pablo

----------

## NeddySeagoon

pstickar,

I was concerned that you had two interfaces in the same subnet.  That's the most usual problem with users trying to set up two interfaces.

Its usually wifi and wired. 

192.168.x.x and 172.16.x.x are indeed separate. that just leaves you with too many or incorrect default routes.

Here you have

```
default via AAA.AAA.AAA.1 dev eth0 proto dhcp src AAA.AAA.AAA.184 metric 1002

default via BBB.BBB.BBB.1 dev wlan0 proto dhcp src BBB.BBB.BBB.4 metric 3003 
```

Which I will rewrite as

```
default via 192.168.x.1 dev eth0 proto dhcp src AAA.AAA.AAA.184 metric 1002

default via .172.16.x.1 dev wlan0 proto dhcp src BBB.BBB.BBB.4 metric 3003 
```

the sources don't matter.

Default means send any packets that remain, via ...

When two routes have the same destination, which route will be used is determined by the metric, also known as cost.

The one with the lowest cost is used.

In your case, that's eth0, which in not what you want an you said you want to use wlan0 to reach the internet.

Deleting the default route via eth0 is going to help. 

If you would like to improve that IPv6 page, I can move it to the main Wiki space, or you can use the Discussion page.

----------

## eccerr0r

You seem to have a DHCP server on your eth0 subnet.  If you didn't make one, it's on one of your devices, perhaps a router you're using as a switch.  You need to either disable this DHCP server, or more likely tell it to stop submitting the router's address as the network router -- or tell it to send the address of this computer with the wifi uplink instead.  You'll need to request help from whoever made that device's software.

As said you can "fix" it by deleting the default route related to it, but this isn't a fix, this would be a workaround.

----------

## pstickar

Thank you all for the hints.

Right now I have problems with my router (WLAN), so I have some (unexpected) extra homework here. 

I'll be back soon.

Pablo

----------

## pstickar

Once again I have to apologize for not being able to follow. Some unexpected private problems keep me way too busy.

@NeddySeagoon: I cannot improve your text, actually I have a hard time following it. But I think it should be mentioned in the Gentoo-Firewall section (and not as a footnote). Furthermore, I think it should be in the section that discusses whether a Firewall is needed or not. There, you can challenge the reader to check that they understand what is going on.

@eccerr0r: I like your solution a lot. However, if I disable the dhcp server at that router, it won't be able to assign an address to the other devices. They should all be static. In this case, being a very private network, it should be ok. I will try your solution as soon as I can. I'm not sure if that router can do that. Budget played a role when purchasing it.

I hope I can come back very soon.

Best,

Pablo

----------

## NeddySeagoon

pstickar,

Your private LAN only needs a switch, not a router, unless your privale LAN has a gateway to something else.

You may be able to cheat a little. The router on your private LAN can probably be connected to use your PC with the two NICs as its WAN or upstream port.

It would then run DHCP for the private LAN but not its connection to your dual NIC PC.

There is a complication. It would put your private LAN behind NAT from your PC, so you would need to do port forwarding to reach systems on the private LAN.

That may or may not be a problem,  depending on your use case. It a bit of a hack.

eccerr0r has the correct solution. Make the router know that its not the gateway on the private LAN and tell its dhcp users who is.

For a small number of systems, a static setup is not a maintenance burden.

----------

## pstickar

Hello NeddySeagon,

you comment  *Quote:*   

> Your private LAN only needs a switch, not a router

  sounds very interesting to me. I already have a switch. But if I take the router out, how do I generate the addresses for the private stuff?

One day in the past my idea was:

```
Router/WLAN ---------------  wlan/PC/eth0 ------------------- Switch ----- private stuff.

192.168.x.x                                                   172.16.x.x 

```

and it did not work because I found no way to assign addreses that way. So I did:

```
Router/WLAN ---------------  wlan/PC/eth0 ------------------- Cheap-Router ----- private stuff.

192.168.x.x                                                   172.16.x.x 

```

which is possible, but brings me here asking questions.

I do not like having the private stuff connected to the big bad internet. I want to connect experimental things there, that will make errors.

Best,

Pablo

----------

## eccerr0r

You could set up a custom DHCP server on the machine with the wifi uplink.

Key issue is that DHCP server needs to be able to be configurable such that it does not advertise the wrong gateway.  Your current DHCP server on your eth0 is doing just that, apparently since it thinks it still is the uplink router when it is not.  Apparently my CPE router's DHCP server seems to not be able to set up a manually configured uplink router, so you may be out of luck with simply configuring it instead of disabling it.

So you want your eth0 connected devices to not have full network access, was that the plan?   Anyway when I was using DD-WRT on a WRT54G, one of the features was to have all 5 ethernet ports as LAN machines with full access to the internet, and the 802.11g wifi antenna's sole purpose was to uplink/WAN to another access point.  I think some wifi hardware on some routers you could even use the antenna to be both uplink and be an access point at the same time (though of course this burns bandwidth).  Pretty neat what you could do with that firmware.

----------

## NeddySeagoon

pstickar,

What eccerr0r said. 

The software that provides DHCP to the subnet is not the same software that does NAT to the private LAN can still have no access to to internet.

The DHCP server does need to run 24/7 though as leases need to be renewed as they expire. The systems will drop off the private LAN if DHCP leases are allowed to expire.

What router do you have on the private LAN? 

If I can find its instruction manual, I'll have a look to see if it can do what you need.

----------

## pstickar

Hi,

The router is a TP-Link TL-R480T+. I'm reading the user guide. I learn that I can configure the DHCP server, and also the routing options.

The section to configure the DHCP server has the options:

Starting IP Address...

Ending IP Address...

Lease Time...

Default Gateway: It is recommended to enter the IP address of the LAN port

Default Domain: Enter the domain name of your network.

Primary/Secondary DNS: Enter the DNS server address provided by your ISP. If you are not clear, please consult your ISP.

Option 60: Specify the option 60 for device identification. Mostly it is used under the scenario where the clients apply for different IP addresses from different servers according to the needs. By default, it is TP-LINK.

If a client requests option 60, the server will respond a packet containing the option 60 configured here. And then the client will compare the received option 60 with its own. If they are the same, the client will accept the IP address assigned by the server, otherwise the assigned IP address will not be accepted.

Option 138: Specify the option 138, which can be configured as the management IP address of an AC (Access Controller) device. If the APs in the local network request this option, the server will respond a packet containing this option to inform the APs of the AC’s IP address.

The section on static routing configurations has the vocabulary that you were using before:

Name  Enter a name for the static route entry.

Destination IP Specify the destination IP address the route leads to.

Subnet Mask Specify the subnet mask of the destination network.

Next Hop Specify the IP address to which the packet should be sent next.

Interface Specify the physical network interface through which this route is accessible.

Metric Define the priority of the route. A smaller value means a higher priority. The default value is 0. It is recommended to keep the default value.

Description ...

Status ...

I cannot experiment right now, otherwise I would try a few shots in the dark before writing here:

One thing I would try is the option 138, which sounds to me like the place to write the IP of the other router, the one that leads to the outside world. Another possibility is to write a static route with a very large metric, to make it look like a slow route.

Best,

Pablo

----------

## eccerr0r

You lucked out: you only need to change the "default gateway".

If you can clear it, that would be good.  But do you want the rest of the machines on your network to access the internet through the PC with the wifi uplink?  Then you should set it to its (STATIC) address.

Problem may still exist, though I never tried to see what happens if a PC has a default route of itself, whether it will honor that or silently discard it...

----------

## NeddySeagoon

pstickar,

Change Default Gateway: It is recommended to enter the IP address of the LAN port.

Point that to the LAN port on your dual NIC PC.

Any packets on the private LAN that are destined for elsewhere will be send to your PC.

There in a minor complication.

The private LAN interface in your dual NIC PC needs a fixed IP address. 

Fixed does not mean static here. Fixed means never changes, static is one method of assigning such an IP address.

DHCP servers can also assign IP addresses by MAC address, so the address never changes but its still dynamically assigned.

One more step ...

Your dual NIC PC must not accept the default route assigned on the private LAN as its default route.

I think there is an option for that in the net file.

Is that enough pointers?

----------

## eccerr0r

I don't know if this is a valid test but I tried adding a second default route with a higher metric to my own computer.  It seems to still work, though this is not an apples-to-apples situation.  Kind of makes sense sort of... Unresolvable routing loops are bad.

----------

## NeddySeagoon

eccerr0r,

That would work but its a hack.

When the real default route goes down, you will get confused.

----------

## eccerr0r

Actually it wouldn't be a "hack" but rather was thinking out loud what would happen if the DHCP server told all clients the default gateway is one machine that also picks up a DHCP lease (oh boy I would hope this lease would be static either by a static entry or hardcoded).  If it were a static entry in the DHCP server, the dhcp client would pick it up and probably still set the default route which incidentally is itself.  All ethernet leases apparently will automatically get higher metric numbers as this tends to be the usual case where people would prefer to use wire over wireless -- except in this case the machine that is the actual gateway router would get itself with a higher metric.

This would be a routing loop, but apparently the kernel seems to notice that it already traversed itself (despite it has the higher metric) and apparently won't try the same machine again - and go to the other default route, and things work.

This is different than the initial case where the default route had no uplink at all despite having a higher metric.

----------

## NeddySeagoon

eccerr0r,

```
default via AAA.AAA.AAA.1 dev eth0 proto dhcp src AAA.AAA.AAA.184 metric 1002

default via BBB.BBB.BBB.1 dev wlan0 proto dhcp src BBB.BBB.BBB.4 metric 3003 
```

Here wired has a lower metric than wifi. It would work the other way round. Lower metrics preferred.

----------

## eccerr0r

ah got the numbers  mixed up, never clear which is which, I mean, swapfiles use higher = preferred...

I guess in this case as a quick test, the default route to itself still needs to be removed, so it looks like hacks are still needed if the default gateway can't be simply removed from the DHCP server.  Looks like the best solution is running the dhcp server directly on the machine with the uplink though it means it needs to be on more often, else a hack (deleting the improper default route, whether it to be itself or not) needs to be put in.

----------

## Ralphred

This is all overly complicated for a router that has an optional 'default gateway' in the DHCP server config, and reserved IP leases by MAC.

Easiest solution: set a static IP outside of the DHCP servers pool on eth0, and no gateway

Difficult solution*: write an if-up script that deletes the gateway on eth0 when you are in network aaa.aaa.aaa.0

Easy solution with access to router: Delete the gateway from the DHCP config.

Difficult solution with access to router: Set gateway to eth0 address, set eth0 address to pre-defined, throw a masquerade/overload rule at iptables nat postrouting and set 'net.ipv4.ip_forward = 1' in /etc/sysctl.conf (your PC is now a router)

*Even this is easier than messing about with metrics in a use case they are not suited for.

----------

## eccerr0r

Well, metrics were never suggested as a solution to the problem, they were used to analyze and explain the problem...

In any case, it's clear now there is no ideal solution here, at least without the detail - do the other machines on the ethernet network need access to the internet or not?

IMHO the 'proper' 'general' solution right now is for the router to use its wifi as the uplink and forget about the wifi in the PC.  But this has its caveats as well, including whether or not the router can actually be set up this way.

BTW: If the wifi router has a "mode" as "client" instead of the usual "Managed Access point" or "ad hoc" the router should have the software available to do this mode.

advantages:

- lease and internet available to all machines all the time

- no static assignments needed

disadvantages:

- None of the PCs have LAN address of uplink network, which could prove to be problem for services and NAT

- requires router firmware support

- if the wifi on the router is a bottleneck, it will remain to be a bottleneck... (doesn't support uplink wifi modes like 802.11a, wireless-ac, etc.?)

Keeping with using the wifi as uplink on the one PC

advantages:

- no network topology changes needed

- no special router firmware needed

- pc has network address on uplink network for services and nat

disadvantages:

- pc needs to be powered on to route packets out of the network

- needs special configs to remove incorrect route on the PC

- pc needs static ip if it needs to route packets to the internet

need a decision...

----------

## pstickar

Hi all,

many many thanks for all the helpful comments. I'm learning quite a bit here.

I do not need/want the elements in the private network to connect outside.

So I'm going to try the following:

- In the router connected to eth0, I'll instruct its dhcp server to assign a fixed IP address to my computer (mac-address ot eth0).

- I'll write that fixed IP address in the field "Default gateway" of that same router.

I did not know that such a thing can work. It looks like an infinite loop to me.

Regarding the hint  *Quote:*   

> Your dual NIC PC must not accept the default route assigned on the private LAN as its default route

  from NeddySeagoon, I'm not sure how to do that. It comes to my mind to add the following line to /etc/dhcpd.conf:

```
denyinterfaces eth0
```

Regarding the firewal, it will be instructed to assume that eth0 is green. If one day I take the PC to another place, without WiFi, and I connect a cable to eth0, accessing the big bad internet, it might be a nightmare. I have to very very careful here, and if one day WiFi does not work, I have to resist the temptation to connect the box through a cable.

Best,

Pablo

----------

## NeddySeagoon

pstickar,

In your dual NIC system, set the IP address statically for the wired interface.

Then if you don't define any routes, you don't get any routes

Ahhhh ... I knew I had seen it somewhere.

Read /usr/share/doc/netifrc-0.7.3/net.example.bz2 or whatever your version is. Its all good reading but it will mostly make your eyes glaze over. :)

```
# GENERIC DHCP OPTIONS

# Set generic DHCP options like so

#dhcp_eth0="release nodns nontp nonis nogateway nosendhost"

# This tells the dhcp client to release its lease when it stops, not to

# overwrite dns, ntp and nis settings, not to set a default route and not to

# send the current hostname to the dhcp server and when it starts.

# You can use any combination of the above options - the default is not to

# use any of them.
```

so clients can tell the dhcp server what they need. 

That starts at line 531 in my version.

----------

## eccerr0r

 *pstickar wrote:*   

> 
> 
> - In the router connected to eth0, I'll instruct its dhcp server to assign a fixed IP address to my computer (mac-address ot eth0).
> 
> - I'll write that fixed IP address in the field "Default gateway" of that same router.
> ...

 

After seeing what your network intent was (your LAN machines should NOT have internet access), now a better idea of what you need to do can be devised.

It indeed is a infinite loop if you set it that way, so actually what you want to do is to leave that field blank.  If you can't leave that field blank (e.g. the router/switch insists you put something there), then you have to set your PC to delete that route and NeddySeagoon has a clean way of doing it if you're using netifrc to configure your ethernet device.

----------

## Ralphred

 *eccerr0r wrote:*   

> what you want to do is to leave that field blank.

 

That fulfils all the criteria, and saves time and energy on the private LAN machines trying to get outside, if they don't have a gateway of last resort they'll just report no route to host, as opposed to waiting for a timeout on a bunk gateway.

 *eccerr0r wrote:*   

> If you can't leave that field blank (e.g. the router/switch insists you put something there)

 

You can with that router, it's one of TP-Links better offerings, will most likely work when the warranty expires too  :Smile: .

 *pstickar wrote:*   

> If one day I take the PC to another place, without WiFi, and I connect a cable to eth0, accessing the big bad internet, it might be a nightmare.

 

If that's a possible use case, don't alter the PC config, and just delete the "default gateway" from the routers DHCP settings

The Unix philosophy of "Do one job, do it well" works really nicely if you are trying to design resilient networks, a [modem]-[router]-[switch]-[AP] will be much easier to set-up and far more robust than a [8 port-wifi-modem-router], I even prefer midspans to PoE switches if I don't need it on the majority of ports.

----------

