# <SOLVED> Multiple NIC Routing Help

## eusanpe

Hello all,

I have a server with two nics and want to be able to get to my Dell MD3600i backup server.

This is the IP Info:

```

External

eth0---> IP: 192.168.20.100 ---> Router (Gateway: 192.168.20.1) ---> Cable Modem to internet

Internal

eth1---> IP: 192.168.3.40 ---> Cisco Managed Switch ---> Router (Gateway: 192.168.3.1)

Dell MD3600i Backup Server---> 192.168.3.113
```

```
# netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         192.168.20.1    0.0.0.0         UG        0 0          0 eth0

127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo

192.168.3.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1

192.168.20.0    0.0.0.0         255.255.255.0   U         0 0          0 eth0
```

I want to be able to reach my Dell MD3600i backup server.

Can anyone give me some pointers for setting up routing for eth1?

Thanks,

TonyLast edited by eusanpe on Sat Feb 23, 2019 11:08 am; edited 1 time in total

----------

## Syl20

You don't ? Your routing table seems correct.

Are you sure all the netmasks (on your server, on the Dell server, and on the router) are 255.255.255.0 for the network 192.168.3.0 ?

Oh, wait... do you want to reach your Dell server _from_ your server, or from another computer, _through_ your server ?

----------

## eusanpe

Good morning,

Yes everything is correct. Here is the situation...

I set up some space on the backup server. My laptop ip address is 192.168.20.120. I have no issues on the 192.168.20.0/24 network.

I can ssh into the server with the dual nics. After that I cannot reach the 192.168.3.1 gateway or anything else behind it.

Thanks,

Tony

----------

## NeddySeagoon

eusanpe,

Where is the route from the 192.168.20.0/24 network to the 192.168.3.0/24 network and back again?

When you ssh into the server with two nics, your ssh session (remote shell) should be able to reach both networks but not your laptop.

You either need to route traffic over a ssh tunnel from the laptop, or you need a route between networks, or possibly a firewall to do NAT to join the two networks.

----------

## Syl20

So, I suppose you have :

```

                      Cable modem                                             Router

                      192.168.20.1                                          192.168.3.1

                           |                                                     |

   laptop         ------ [USW] -------      eth0     server     eth1      ---- [CSW] ----     backup server

192.168.20.120                        192.168.20.100         192.168.3.40                    192.168.3.113 

```

where [CSW] is the "Cisco Managed switch", and [USW] is an unknown switch.

Is the Cisco switch well configured ? Is the port "no-shut", and so on ?

Can you make a tcpdump capture on eth1 when trying to reach the backup server, to be sure the requests packets go the right way ? Is there a firewall on the servers ?

----------

## eusanpe

 *Quote:*   

> where [CSW] is the "Cisco Managed switch", and [USW] is an unknown switch.
> 
> Is the Cisco switch well configured ? Is the port "no-shut", and so on ?
> 
> Can you make a tcpdump capture on eth1 when trying to reach the backup server, to be sure the requests packets go the right way ? Is there a firewall on the servers ?

 

Syl20,

 Here is better diagram.

```
                                 Cable modem                                                                        Cisco Router (Firewall)

                                      |                                                                             192.168.3.1

                  Wifi                |               eth0: 192.168.20.100                   eth1: 192.168.3.40          | 

laptop ----------------------  Asus Wireless Router ------------------------- Dell Server ---------------------------- [CSW] ---------- Dell Backup Server

192.168.20.120                   192.168.20.1                                                                                             192.168.3.113

```

The Dell server was a standalone server with the address 192.168.3.40. I used to VPN to the Cisco Router and access the Dell server. This worked great. I decided to add another NIC so I can access the

Dell server from my internal network directly instead of connecting via VPN. I can ssh to the Dell server using ssh 192.168.20.120 just fine. I am just trying to figure out route setup to the 192.168.3.0/24 network

from that point since the default gateway is 192.168.20.1.

Thank you,

Tony

----------

## NeddySeagoon

eusanpe,

On the laptop, you need a static route that says to reach 192.168.3.0/24 use 192.168.20.100 as the gateway.

On the backup server. you a static route that says to reach 192.168.20.0/24 use 192.168.3.40 as the gateway.

That's a route each way.

As the server is now a router. you need to turn on IPv4 forwarding too.

----------

## eusanpe

```
On the laptop, you need a static route that says to reach 192.168.3.0/24 use 192.168.20.100 as the gateway.

On the backup server. you a static route that says to reach 192.168.20.0/24 use 192.168.3.40 as the gateway.

That's a route each way.

As the server is now a router. you need to turn on IPv4 forwarding too.
```

This is my routing table

```
# netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         192.168.20.1    0.0.0.0         UG        0 0          0 eth0

127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo

192.168.3.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1

192.168.20.0    0.0.0.0         255.255.255.0   U         0 0          0 eth0

```

If I try ip route I get the following message

```

# ip route add 192.168.3.0/24  via 192.168.20.100

RTNETLINK answers: File exists

# ip route add 192.168.20.0/24 via 192.168.3.40

RTNETLINK answers: File exists

```

Tony

----------

## NeddySeagoon

eusanpe,

There are changes there to all three systems, not just the server with two NICs. 

On the laptop, you need a static route that says to reach 192.168.3.0/24 use 192.168.20.100 as the gateway.

```
ip route add 192.168.3.0/24  via 192.168.20.100
```

On the backup server. you a static route that says to reach 192.168.20.0/24 use 192.168.3.40 as the gateway.

```
ip route add 192.168.20.0/24 via 192.168.3.40 
```

On the server, with the two NICS,  

Edit  /etc/sysctl.conf

```
# Disables packet forwarding

net.ipv4.ip_forward = 1
```

```
cat /proc/sys/net/ipv4/ip_forward
```

will tell the current state of forwarding.

Forwarding allows packets received on one interface to be sent out of the other.

----------

## eusanpe

I have some errands to run. Will update you when I get back home.

Thank you for all your help!

Tony

----------

## eusanpe

NeddySeagoon,

I set the route on the laptop.

```
ip route add 192.168.3.0/24  via 192.168.20.100

```

I can ping the 192.168.3.40 ip on the server from the laptop but cannot ping 192.168.3.1 on the other side. On the server side,

I can ping 192.168.3.1.

Ping to 192.168.3.1 from the laptop returns

```
% ping 192.168.3.1

PING 192.168.3.1 (192.168.3.1) 56(84) bytes of data.

From 192.168.20.100: icmp_seq=1 Destination Host Unreachable

From 192.168.20.100: icmp_seq=2 Destination Host Unreachable

From 192.168.20.100: icmp_seq=3 Destination Host Unreachable

```

I will have to see how to set a route, if possible, on the Dell MD3600i backup server.

I have port forwarding set on the Dell Server.

Thanks again for your help!

Tony

----------

## papas

have you, on  dell server, add gateway for the 192.168.3.0/24 net ??

 :Embarassed: 

i just repeat your original question

Sorry guys ...Last edited by papas on Fri Feb 22, 2019 4:00 pm; edited 1 time in total

----------

## eusanpe

papas,

No I do not. Here is the routing table..

```
# netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         192.168.20.1    0.0.0.0         UG        0 0          0 eth0

127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo

192.168.3.0     0.0.0.0         255.255.255.0   U         0 0          0 eth1

192.168.20.0    0.0.0.0         255.255.255.0   U         0 0          0 eth0
```

Tony

----------

## papas

I'm thinking that you need the next hop for the 192.168.3.0 network so, try to add, in your dell server, one route via 192.168.3.1, for all the other destinations there is the default gateway 192.168.20.1

----------

## NeddySeagoon

papas,

Turning forwarding on should do that.

eusanpe,

Adding the route to your laptop, lets it reach the server.

Turning forwarding on in the sever allows packets to go from 192.168.20.0/24 to 192.168.3.0/24 (and back).

Adding the route to your backup server allows packets to be sent the other way along the route.

It won't work until you make all three changes at the same time.

----------

## eusanpe

papas,

Thank you for your insight.

NeddySeagoon,

I have Port Forwarding set on the Dell Server and the route set up on the laptop. I was going to figure out the Dell Backup server afterwards. I shouldn't need to set up the backup server just to 

reach the ip address 192.168.3.1 which is the Cisco Router. Once I can reach that then I can work on figuring out the backup server. 

Laptop Routing Table

```
% netstat -rn

Kernel IP routing table

Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface

0.0.0.0         192.168.20.1    0.0.0.0         UG        0 0          0 wlan0

127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo

192.168.3.0     192.168.20.100  255.255.255.0   UG        0 0          0 wlan0

192.168.20.0    0.0.0.0         255.255.255.0   U         0 0          0 wlan0

```

IP Forwarding enabled on Dell Server

```
# cat /proc/sys/net/ipv4/ip_forward

1

```

Do I need to do anything to the ASUS wireless router?

Thanks again,

Tony

----------

## NeddySeagoon

eusanpe,

Run wireshark or tcpdump on the Dell Server recording all traffic on eth1.

While that's runnig, ping 192.168.3.1 from the laptop.

You should see ICMP requests from the laptop. 

They will have 192.168.20.120 as the source address.

How does the Cisco Router know to use 192.168.3.40 as the gateway to reach the laptop?

You need to set up a static route there.

As the Cisco Router, Dell Backup Server and Dell Server eth1 are all in the 192.168.3.0/24 subnet, the Cisco Router should not be keeping them apart.

That is, traffic is not being firewalled there, the Cisco Router is behaving as a hub/switch.

In short, I suspect you are reaching the Cisco Router but you are not getting any replies due to the missing return route.

----------

## eusanpe

NeddySeagoon,

Adding a static route to my Cisco router solved all the issues. I have ISCSI set up and everything is working now.

Thank you very much for you knowledge and assistance.

Sincerely,

Tony

----------

## NeddySeagoon

eusanpe,

With the static route only on the Cisco router, traffic from one subnet to the other will have an extra hop.

Its not wrong to do that and its the path of minimum maintenance, which is, in general, good.

If you put the static rout everywhere, the latency through the Cisco router is avoided.

It won't (or shouldn't) change the data rate you get.

Its a nuance to consider if low latency is important to your application.

----------

