# Question About DHCP ICS Server With 2 Routers...

## finalturismo

So i have a DHCP server running on my Linux server, the cable comes directly in from the ONT. 

Great setup for fast network response btw...

Anyway i have some 10GB SFP fiber cards.

And i want to add another router to my DHCP ICS configuration. 

The problem is when i activate another router i lose internet connection. 

The router portion works fine as they both dish out IP addresses but only 1 seems to work at a time with the nat to the wan port.....

I checked my IP Tables setup and it looks fine aswell....

Now i know its possible to have 2 routers on the same subnet so what am i doing wrong... they even  give examples of multi router setups with DHCP ICS...

Anyway if anyone can give me a pointer would be great.

```
#allow booting;

#allow bootp;

subnet 77.77.77.0 netmask 255.255.255.0

{

range 77.77.77.2 77.77.77.253;

option routers 77.77.77.1;

option routers 77.77.77.254;

option domain-name FIXAPC;

option domain-name-servers 8.8.8.8, 1.1.1.1, 77.77.77.1, 77.77.77.254;

default-lease-time -1;

max-lease-time -1;

}
```

Here is my net conf

```

config_eth0="dhcp"

config_eth1="77.77.77.1/24"   (1GB RJ45 NIC)

config_eth5="77.77.77.254/24"    (10GB SFP NIC)

```

Here is my IP Tables

```
# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*nat

:PREROUTING ACCEPT [228:14571]

:INPUT ACCEPT [57:3283]

:OUTPUT ACCEPT [2:116]

:POSTROUTING ACCEPT [3:164]

-A PREROUTING -i eth0 -p tcp -m tcp --dport 2 -j DNAT --to-destination 77.77.77.81:22

-A PREROUTING -i eth0 -p tcp -m tcp --dport 8000 -j DNAT --to-destination 77.77.77.81

-A PREROUTING -i eth0 -p tcp -m tcp --dport 4040 -j DNAT --to-destination 77.77.77.81

-A POSTROUTING -o eth0 -j MASQUERADE

COMMIT

# Completed on Sat Aug  1 22:06:03 2020

# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*mangle

:PREROUTING ACCEPT [8717:5199059]

:INPUT ACCEPT [1952:159636]

:FORWARD ACCEPT [6764:5039223]

:OUTPUT ACCEPT [1817:163787]

:POSTROUTING ACCEPT [8581:5203010]

COMMIT

# Completed on Sat Aug  1 22:06:03 2020

# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*filter

:INPUT ACCEPT [24:1299]

:FORWARD DROP [0:0]

:OUTPUT ACCEPT [1817:163787]

-A INPUT -i lo -j ACCEPT

-A INPUT -i eth1 -j ACCEPT

-A INPUT -i eth5 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 80 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 3000 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 443 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 22 -j ACCEPT

-A FORWARD -s 77.77.77.0/24 -i eth1 -j ACCEPT

-A FORWARD -s 77.77.77.0/24 -i eth5 -j ACCEPT

-A FORWARD -d 77.77.77.0/24 -i eth0 -j ACCEPT

COMMIT

# Completed on Sat Aug  1 22:06:03 2020

```

no clue what iam doing wrong, i even did 2 entries on the dhcp file instead of 1.....

They both work without any edit to iptables setup but only either eth1 can be the router or eth5.... not at the same time... any ideas?Last edited by finalturismo on Mon Sep 28, 2020 3:18 pm; edited 6 times in total

----------

## finalturismo

bump

----------

## pietinger

I dont understand these rules:

 *finalturismo wrote:*   

> 
> 
> ```
> # Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020
> 
> ...

 

I think it should be "77.77.0.0/16" or "77.77.77.0/24"

----------

## Hu

 *pietinger wrote:*   

> I dont understand these rules:
> 
>  *finalturismo wrote:*   
> 
> ```
> ...

 77.77.0.0/24 is a valid mask, and represents all 256 entries that start 77.77.0.x, just as your proposed replacement represents all 256 entries that start 77.77.77.x.  I suspect that OP has replaced the real addresses with the 77.77.0.0 placeholder for privacy reasons.

----------

## pietinger

 *Hu wrote:*   

> 77.77.0.0/24 is a valid mask, and represents all 256 entries that start 77.77.0.x, just as your proposed replacement represents all 256 entries that start 77.77.77.x.  I suspect that OP has replaced the real addresses with the 77.77.0.0 placeholder for privacy reasons.

 

Hu, thanks. Yes, I know subnetting and supernetting ... (as network Ing.) ... when I write "I dont understand" its a replacement for "I think its wrong".

I dont thought about wrong adresses because giving us an iptables output with replaced adresses would be complete senseless ... (then I dont need any output from iptables). But let us wait for an answer from him.

----------

## finalturismo

 *pietinger wrote:*   

> I dont understand these rules:
> 
>  *finalturismo wrote:*   
> 
> ```
> ...

 

I run an expanded subnet sometimes because i like to keep a dhcp server with an unlimited lease time so that i dont have ip changing for devices.

I think of it like an auto static ip dhcp server. It helps alot...

but i edited the subnet for /24 because i was going to change it. Now that you pointed this out let me try 77.77.77.0/24 or 77.77.0.0/16.

I do see a problem here but i think i might of tried this already when it was correct.

I will make the change and try again.

Edit: ok i was able to narrow down the problem but i have no clue what is causing it.

IT IS NOT Iptables or the DHCP config....

i have removed eth1 from both of my iptables and dhcp config file and only have 1 router setup at the moment. 

BUT start both services for each network adapter i lose internet. I still get an IP but i cannot ping the router or the internet....

If i simply disable 1 adapter in my /etc/conf.d/net everything starts working

as soon as i enable both of them, connection goes down.

This is completely seperate from ip tables and the dhcp server as i have removed that adapter completely from both.

Is it the fact that both adapters are sym linked to the loop adapter? 

I even went as far as to check that the DNS servers from that adapter are removed in /etc/resolv.conf....

Something is causing this and its not DNS, IPtables or the DHCP server configuration....

and i have no clue what....

If i even attempt to add another static ip to the same subnet it takes down network. Regardless of what ip i set inside the same subnet.

----------

## pietinger

finalturismo, why do you think it is not iptables ?

Do you have configured bonding on your switch ?

What arp-filters did you set ?

Did you set rp_filters to 2 on all interfaces (net.ipv4.conf.ethX.rp_filter = 2) ?

I really would like to see an "iptables -L -v -n" because there are other things I dont understand.

I think also you should configure two rules to forbid forwarding between eth1 and eth5 (for example: "-A FORWARD -i eth1 -o eth5 -j DROP" and reverse) to get not a loop (this supersedes some other rules you have).

You said you have read some articles; do you know these ones ?

https://andersbrownworth.com/cms/258

(http://linux-ip.net/html/ether-bonding.html

https://lxadm.com/Routing_with_multiple_network_cards

https://access.redhat.com/solutions/30564 )

EDIT: Do you have the correct kernel config ? (e.g. CONFIG_IP_ADVANCED_ROUTER)

----------

## finalturismo

 *pietinger wrote:*   

> finalturismo, why do you think it is not iptables ?
> 
> Do you have configured bonding on your switch ?
> 
> What arp-filters did you set ?
> ...

 

I think you are right in the 2 interfaces forwarding to each other.  But in this instance it is not the case unless my iptables-restore < firewall settings is not actually doing the command. 

I did the entire home routing guide correctly(ive set up a few Gentoo routers, but never 2 static ips or routers on same subnet) and my kernel does have forwarding and advanced routing enabled. 

I completely removed eth 5 from both my iptables and dhcp configuration. As soon as i add ANY IP(with the same subnet) to my conf.d net file it causes the connection to not be able to ping anymore. 

I can add any of the following to my net config file and it will cause that adapter to not be able to ping. 

```
config_net=eth6=77.77.77.88/24 (Does not work)

config_net=eth6=77.77.76.88/24 (Works)

config_net=eth5=77.77.77.88/24 (Does not work)

config_net=eth5=77.77.76.88/24 (Works)
```

 (if i add even a new adapter with an ip of the same subnet, it still causes that adapter to not be able to ping, but as soon as i turn off eth1 and switch configs for eth 1 and 5 it works fine...)

But your making me question my self now... i think your right maybe it is IP tables and what you said sounded like it made sense, what doesn't make sense is why is it doing this when i add a completely new adapter?

here is the output you requested with my complete configuration of DHCP and IPTables.

keep in mind that my eth5 adapter service is not running. (So i didnt lose connection.)

```
Chain INPUT (policy ACCEPT 115K packets, 9083K bytes)

 pkts bytes target     prot opt in     out     source               destination

 311K   51M ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0

23401 1734K ACCEPT     all  --  eth1   *       0.0.0.0/0            0.0.0.0/0

  678 61282 ACCEPT     all  --  eth5   *       0.0.0.0/0            0.0.0.0/0

  558 49384 ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80

   69  5037 ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:3000

26542 2843K ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443

 3667  510K ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22

Chain FORWARD (policy DROP 0 packets, 0 bytes)

 pkts bytes target     prot opt in     out     source               destination

 147K   17M ACCEPT     all  --  eth1   *       77.77.77.0/24        0.0.0.0/0

 559K 1301M ACCEPT     all  --  eth5   *       77.77.77.0/24        0.0.0.0/0

1198K 3183M ACCEPT     all  --  eth0   *       0.0.0.0/0            77.77.77.0/24
```

Here is also an updated iptables config, the last 1 i posted wasn't in the best of shap because i was testing random things. 

```
# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*nat

:PREROUTING ACCEPT [228:14571]

:INPUT ACCEPT [57:3283]

:OUTPUT ACCEPT [2:116]

:POSTROUTING ACCEPT [3:164]

-A PREROUTING -i eth0 -p tcp -m tcp --dport 2 -j DNAT --to-destination 77.77.77.81:22

-A PREROUTING -i eth0 -p tcp -m tcp --dport 8000 -j DNAT --to-destination 77.77.77.81

-A PREROUTING -i eth0 -p tcp -m tcp --dport 4040 -j DNAT --to-destination 77.77.77.81

-A POSTROUTING -o eth0 -j MASQUERADE

COMMIT

# Completed on Sat Aug  1 22:06:03 2020

# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*mangle

:PREROUTING ACCEPT [8717:5199059]

:INPUT ACCEPT [1952:159636]

:FORWARD ACCEPT [6764:5039223]

:OUTPUT ACCEPT [1817:163787]

:POSTROUTING ACCEPT [8581:5203010]

COMMIT

# Completed on Sat Aug  1 22:06:03 2020

# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*filter

:INPUT ACCEPT [24:1299]

:FORWARD DROP [0:0]

:OUTPUT ACCEPT [1817:163787]

-A INPUT -i lo -j ACCEPT

-A INPUT -i eth1 -j ACCEPT

-A INPUT -i eth5 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 80 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 3000 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 443 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 22 -j ACCEPT

-A FORWARD -s 77.77.77.0/24 -i eth1 -j ACCEPT

-A FORWARD -s 77.77.77.0/24 -i eth5 -j ACCEPT

-A FORWARD -d 77.77.77.0/24 -i eth0 -j ACCEPT

COMMIT

# Completed on Sat Aug  1 22:06:03 2020
```

 *Quote:*   

> Did you set rp_filters to 2 on all interfaces (net.ipv4.conf.ethX.rp_filter = 2) ?

 

Wait what? to 2? I can go ahead and give that a try too when i get on launch along with the forwarding rule.

----------

## ct85711

Ok, when you said once you start both, the connection dies, check to your configured routes (simply compare the results from ip route from before and after).  It is seeming like something is screwing up and interfering your default route.  The network card shouldn't have lost it's ip address, but if the default route for the network card goes away/points somewhere else; that would kill your internet (since it isn't sending the packets to the correct location).

----------

## finalturismo

 *ct85711 wrote:*   

> Ok, when you said once you start both, the connection dies, check to your configured routes (simply compare the results from ip route from before and after).  It is seeming like something is screwing up and interfering your default route.  The network card shouldn't have lost it's ip address, but if the default route for the network card goes away/points somewhere else; that would kill your internet (since it isn't sending the packets to the correct location).

 

It just killed the internal lan, it even gets an ip address from the 2nd router. But i cant even ping the router that assigned it , its IP.

It just says destination host unreachable....

This really confuses the hell out of me, cant wait to get on launch and try out those recommendations.... 

```
0.0.0.0         47.183.193.1    0.0.0.0         UG    202    0        0 eth0

0.0.0.0         77.77.77.1      0.0.0.0         UG    203    0        0 eth1

47.183.193.0    0.0.0.0         255.255.255.0   U     202    0        0 eth0

77.77.77.0      0.0.0.0         255.255.255.0   U     203    0        0 eth1

127.0.0.0       127.0.0.1       255.0.0.0       UG    0      0        0 lo
```

This is before and i did try this too, from what i remember it was just cloned entries of eth1. But it just said eth5. I dont think i saw anything fishy, but iam not the best when it comes to IP routing either....

----------

## finalturismo

 *ct85711 wrote:*   

> Ok, when you said once you start both, the connection dies, check to your configured routes (simply compare the results from ip route from before and after).  It is seeming like something is screwing up and interfering your default route.  The network card shouldn't have lost it's ip address, but if the default route for the network card goes away/points somewhere else; that would kill your internet (since it isn't sending the packets to the correct location).

 

It just killed the internal lan, it even gets an ip address from the 2nd router. But i cant even ping the router that assigned it , its IP.

It just says destination host unreachable....

This really confuses the hell out of me, cant wait to get on launch and try out those recommendations.... 

BEFORE

```
0.0.0.0         47.183.193.1    0.0.0.0         UG    202    0        0 eth0

0.0.0.0         77.77.77.1      0.0.0.0         UG    203    0        0 eth1

47.183.193.0    0.0.0.0         255.255.255.0   U     202    0        0 eth0

77.77.77.0      0.0.0.0         255.255.255.0   U     203    0        0 eth1

127.0.0.0       127.0.0.1       255.0.0.0       UG    0      0        0 lo
```

AFTER both adapters enabled

```

0.0.0.0         77.77.77.254     0.0.0.0        UG      78    0        0 eth 5

0.0.0.0         47.183.193.1    0.0.0.0         UG    202    0        0 eth0

0.0.0.0         77.77.77.1      0.0.0.0         UG    203    0        0 eth1

47.183.193.0    0.0.0.0         255.255.255.0   U     202    0        0 eth0

77.77.77.0      0.0.0.0         255.255.255.0   U     0    0        0 eth5

77.77.77.0      0.0.0.0         255.255.255.0   U     203    0        0 eth1

127.0.0.0       127.0.0.1       255.0.0.0       UG    0      0        0 lo

```

After Applying suggested Iptables recommendations by pietinger

```

Chain INPUT (policy ACCEPT 445 packets, 87171 bytes)

 pkts bytes target     prot opt in     out     source               destination         

 4996  807K ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           

  581 43616 ACCEPT     all  --  eth1   *       0.0.0.0/0            0.0.0.0/0           

   18  6585 ACCEPT     all  --  eth5   *       0.0.0.0/0            0.0.0.0/0           

   10  1123 ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80

    0     0 ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:3000

  657 66841 ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443

  107 13607 ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22

Chain FORWARD (policy DROP 0 packets, 0 bytes)

 pkts bytes target     prot opt in     out     source               destination         

  139 42177 DROP       all  --  eth1   eth5    0.0.0.0/0            0.0.0.0/0           

    0     0 DROP       all  --  eth5   eth1    0.0.0.0/0            0.0.0.0/0           

 1232  286K ACCEPT     all  --  eth1   *       77.77.77.0/24        0.0.0.0/0           

    0     0 ACCEPT     all  --  eth5   *       77.77.77.0/24        0.0.0.0/0           

 1231 1108K ACCEPT     all  --  eth0   *       0.0.0.0/0            77.77.77.0/24       

Chain OUTPUT (policy ACCEPT 6834 packets, 1967K bytes)

 pkts bytes target     prot opt in     out     source               destination     

```

Correct me if iam wrong but wouldnt those blocking rules block data attempt to move between eth1 and eth5? or can you further break down the meaning of those 2 rules?

```
*nat

:PREROUTING ACCEPT [228:14571]

:INPUT ACCEPT [57:3283]

:OUTPUT ACCEPT [2:116]

:POSTROUTING ACCEPT [3:164]

-A PREROUTING -i eth0 -p tcp -m tcp --dport 2 -j DNAT --to-destination 77.77.77.81:22

-A PREROUTING -i eth0 -p tcp -m tcp --dport 8000 -j DNAT --to-destination 77.77.77.81

-A PREROUTING -i eth0 -p tcp -m tcp --dport 4040 -j DNAT --to-destination 77.77.77.81

-A POSTROUTING -o eth0 -j MASQUERADE

COMMIT

# Completed on Sat Aug  1 22:06:03 2020

# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*mangle

:PREROUTING ACCEPT [8717:5199059]

:INPUT ACCEPT [1952:159636]

:FORWARD ACCEPT [6764:5039223]

:OUTPUT ACCEPT [1817:163787]

:POSTROUTING ACCEPT [8581:5203010]

COMMIT

# Completed on Sat Aug  1 22:06:03 2020

# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*filter

:INPUT ACCEPT [24:1299]

:FORWARD DROP [0:0]

:OUTPUT ACCEPT [1817:163787]

-A INPUT -i lo -j ACCEPT

-A INPUT -i eth1 -j ACCEPT

-A INPUT -i eth5 -j ACCEPT

-A FORWARD -i eth1 -o eth5 -j DROP

-A FORWARD -i eth5 -o eth1 -j DROP

-A INPUT -i eth0 -p tcp -m tcp --dport 80 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 3000 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 443 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 22 -j ACCEPT

-A FORWARD -s 77.77.77.0/24 -i eth1 -j ACCEPT

-A FORWARD -s 77.77.77.0/24 -i eth5 -j ACCEPT

-A FORWARD -d 77.77.77.0/24 -i eth0 -j ACCEPT

COMMIT

# Completed on Sat Aug  1 22:06:03 2020
```

MY /etc/sysctl.conf file

```
 

net.ipv4.ip_forward = 1

net.ipv4.ip_dynaddr = 1

net.ipv4.conf.default.rp_filter = 2

net.ipv4.conf.all.rp_filter = 2

```

----------

## ct85711

```
0.0.0.0         77.77.77.254     0.0.0.0        UG      78    0        0 eth 5

0.0.0.0         47.183.193.1    0.0.0.0         UG    202    0        0 eth0

0.0.0.0         77.77.77.1      0.0.0.0         UG    203    0        0 eth1 
```

Now, going by the routing table, after it adds another default route (the top line).  The other 2 lines are effectively an fallback default route each.  Now the way the routing works, is that it uses the default route, if it doesn't know how to get to the destination; usually picking the route that has the least cost.  The 78/202/203 are the route metric/weights.  Think of it as the cost of using that route (the system wants the cheapest path); so the system will naturally want the top path, since it has a metric that is over 1/2 the cost of the other 2.

Now as for the firewall...

```

Chain INPUT (policy ACCEPT 445 packets, 87171 bytes)

 pkts bytes target     prot opt in     out     source               destination         

 4996  807K ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           

  581 43616 ACCEPT     all  --  eth1   *       0.0.0.0/0            0.0.0.0/0           

   18  6585 ACCEPT     all  --  eth5   *       0.0.0.0/0            0.0.0.0/0           

   10  1123 ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80

    0     0 ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:3000

  657 66841 ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443

  107 13607 ACCEPT     tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22 
```

The first line saying "Chain INPUT (policy ACCEPT..." is the input table, with a default policy to accept any packets.  What this is effectively is saying, if the packet from any of the rule in that table doesn't deal with the incoming packet, it will accept it.  This is the same as having NO rules in that table, since it will accept everything anyways.

```
Chain FORWARD (policy DROP 0 packets, 0 bytes)

 pkts bytes target     prot opt in     out     source               destination         

  139 42177 DROP       all  --  eth1   eth5    0.0.0.0/0            0.0.0.0/0           

    0     0 DROP       all  --  eth5   eth1    0.0.0.0/0            0.0.0.0/0           

 1232  286K ACCEPT     all  --  eth1   *       77.77.77.0/24        0.0.0.0/0           

    0     0 ACCEPT     all  --  eth5   *       77.77.77.0/24        0.0.0.0/0           

 1231 1108K ACCEPT     all  --  eth0   *       0.0.0.0/0            77.77.77.0/24  
```

Now the FORWARD table is interesting.  The default policy being DROP, means that it is actually filtering any packets that are forwarded.  The first 2 rules/lines are saying any packet coming in from eth1/eth5 and going out the other side (eth5/eth1 respectively) to straight out kill/drop.  Then you have any packets coming in eth1/eth5 from your internal network to accept (it doesn't care where it is going).  Assuming the 77.77.77.0/24 network is your internal network.

Now the thing to keep in mind, the system goes through the applicable table from top down, stopping after it hits the first rule that matches.  If it doesn't match any it uses the default policy as a catch all of what it does.

My thoughts is that the first 2 rules in the FORWARD table/chain may be interfering with the ones below it.  For the INPUT table/chain as it is right now makes all the rules pointless, but it ok for troubleshooting/testing but don't leave the default to accept after your issue is fixed.

A suggestion for later, it is recommended you organize the rules so that the ones that you expect to receive a lot of traffic towards the top. As each rule it attempts, adds more time.  For a small amount of rules, it is negligible, but if you get to having several (100-1k or more rules) it adds up quickly if it has go through every all of the rules (if the correct rule is at the bottom).  Otherwise, the order the rules are in means everything.

----------

## pietinger

finalturismo,

first of all: You know it is better / more secure to use the private network adress range "10.0.0.0" for you LAN instead a real IP adress range from Iran ?

IMHO you should do your network and routing config first, then filtering and dhcp at last. So lets start:

In your sysctl.conf I am missing something. Try this =>

```
net.ipv4.ip_forward = 1

net.ipv4.ip_dynaddr = 1

net.ipv4.conf.default.rp_filter = 2

net.ipv4.conf.all.rp_filter = 2

net.ipv4.conf.all.arp_ignore = 1

net.ipv4.conf.all.arp_announce = 2
```

 *finalturismo wrote:*   

> Correct me if iam wrong but wouldnt those blocking rules block data attempt to move between eth1 and eth5?

 

Yes, its only to be save against a loop (if network config is wrong).

I dont know if your current iptables rules are only for testing. If not you have two problems:

1) It makes no sense to allow some incoming traffic, when allowing ALL traffic with your default policy =>

```
Chain INPUT (policy ACCEPT 445 packets, 87171 bytes)
```

You should set the default policy to DROP (best with logging at last rule).

2) At the moment you allow ALL traffic between your LAN and your ISP without any filtering =>

```
Chain FORWARD (policy DROP 0 packets, 0 bytes)

[...]          

 1232  286K ACCEPT     all  --  eth1   *       77.77.77.0/24        0.0.0.0/0           

[...]         

 1231 1108K ACCEPT     all  --  eth0   *       0.0.0.0/0            77.77.77.0/24       
```

But at the moment this config is allright for testing.

If it is only for testing and you will later add some rules to OUTPUT (and then with default policy DROP), you should add "-A OUTPUT -i lo -j ACCEPT", because at the moment this is allowed by the default policy. You also should filter the traffic between your LAN and your ISP; maybe you want read my suggestions in https://forums.gentoo.org/viewtopic-t-1114432.html

Did you read the links I gave you ?

----------

## finalturismo

 *pietinger wrote:*   

> finalturismo,
> 
> first of all: You know it is better / more secure to use the private network adress range "10.0.0.0" for you LAN instead a real IP adress range from Iran ?
> 
> IMHO you should do your network and routing config first, then filtering and dhcp at last. So lets start:
> ...

 

Ok i did all the sysctl.conf stuff and rebooted, still no dice, same issue.  The 77.77.77.0 is all LAN IP i just like the number, i use it for good luck. Apparently that isnt working out here XD. 

I did start reading 1 of your links when i was at working. Iam going to try reading the other links now and mess around with some stuff. I usually figure stuff out fairly quickly unless its stuttering in a Windows KVM or dual routers on a dhcp server setup in Linux XD. I got a feeling this might go on for a while before i get a solution.  Been working on this for about 1-2 days now 0.o....

This is my latest ip tables, if anyone wants to just send me an altered version of my Iptables setup so i can at least get a basic 2 router setup going that would be great XD. 

Seeing as everything thinks my tables setup is not correct. Keep in mind iam still learning IPtables, i know the basics and how to open ports. But this multi-router setup has me stumped.  I mean sure my IP-tables dont look perfect but i dont see any reason to stop eth5 as soon as i have both routers enabled.

```
# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*nat

:PREROUTING ACCEPT [228:14571]

:INPUT ACCEPT [57:3283]

:OUTPUT ACCEPT [2:116]

:POSTROUTING ACCEPT [3:164]

-A PREROUTING -i eth0 -p tcp -m tcp --dport 2 -j DNAT --to-destination 77.77.77.81:22

-A PREROUTING -i eth0 -p tcp -m tcp --dport 8000 -j DNAT --to-destination 77.77.77.81

-A PREROUTING -i eth0 -p tcp -m tcp --dport 4040 -j DNAT --to-destination 77.77.77.81

-A POSTROUTING -o eth0 -j MASQUERADE

COMMIT

# Completed on Sat Aug  1 22:06:03 2020

# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*mangle

:PREROUTING ACCEPT [8717:5199059]

:INPUT ACCEPT [1952:159636]

:FORWARD ACCEPT [6764:5039223]

:OUTPUT ACCEPT [1817:163787]

:POSTROUTING ACCEPT [8581:5203010]

COMMIT

# Completed on Sat Aug  1 22:06:03 2020

# Generated by iptables-save v1.6.1 on Sat Aug  1 22:06:03 2020

*filter

:INPUT ACCEPT [24:1299]

:FORWARD DROP [0:0]

:OUTPUT ACCEPT [1817:163787]

-A INPUT -i lo -j ACCEPT

-A INPUT -i eth1 -j ACCEPT

-A INPUT -i eth5 -j ACCEPT

-A FORWARD -i eth1 -o eth5 -j DROP

-A FORWARD -i eth5 -o eth1 -j DROP

-A INPUT -i eth0 -p tcp -m tcp --dport 80 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 3000 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 443 -j ACCEPT

-A INPUT -i eth0 -p tcp -m tcp --dport 22 -j ACCEPT

-A FORWARD -s 77.77.77.0/24 -i eth1 -j ACCEPT

-A FORWARD -s 77.77.77.0/24 -i eth5 -j ACCEPT

-A FORWARD -d 77.77.77.0/24 -i eth0 -j ACCEPT

COMMIT

# Completed on Sat Aug  1 22:06:03 2020
```

I tried removing the LO accept all, still same result...  :Sad: 

The only thing iam finding questionable is when i run route -n on my server i see

```
77.77.77.0  0.0.0.0   eth1

77.77.77.0  0.0.0.0   eth5
```

Wouldnt this cause some sort of subnet conflict if not filtered correctly?

Also what about doing a net conf of

```
config_eth5="77.77.77.254/24"

routes_eth5="default via 77.77.77.1"
```

Would this have any effect or am i totally not thinking right?

I went through my entire kernel config all my advanced routing and net-filter stuff is enabled.... and confirmed working.. to my knowledge. 

grrrrrrr

Update @ 9:26pm Central:

So i was doing some reading on the post you sent and did some searching. Specifically on the one with using 2 routers, than i stopped for a bit because its kind hard for me to get into that much detail unless i had to. Before i go off messing with alot of routing stuff i did a search and found this

https://access.redhat.com/solutions/305303

 *Quote:*   

> On any RHEL system, when using two or more IP addresses within the same subnet with the same default gateway, only one of the interfaces is able to pass traffic beyond the gateway while the other interfaces are limited to their local subnet.

 

So it seems this is a general brick wall many people run into when doing 2 interfaces on a Linux system using the same subnet....

Now to figure out how to get over the wall? i think iama take a quick nap and reset my brain before i move on, i would still like to see a tightened up IPtables config from any of you pros using my posted config. I would gladly use and edit it.  Quick power nap time!

Watching some tutorials about bonding right now according to the links i saw. 

Ok so i found a random post from someone saying to create a bridge and than assign an IP address to that bridge. That sounds easy enough (Thank God)

ZOMG bonding is just what i need and this looks EASSSSSSSYYYY yay!!!!!!! wish me luck guys here i go.

----------

## ct85711

Ok, I will explain what the routes mean to help clear up the confusion...

```

0.0.0.0         77.77.77.254     0.0.0.0        UG      78    0        0 eth 5

0.0.0.0         47.183.193.1    0.0.0.0         UG    202    0        0 eth0

0.0.0.0         77.77.77.1      0.0.0.0         UG    203    0        0 eth1

47.183.193.0    0.0.0.0         255.255.255.0   U     202    0        0 eth0

77.77.77.0      0.0.0.0         255.255.255.0   U     0    0        0 eth5

77.77.77.0      0.0.0.0         255.255.255.0   U     203    0        0 eth1

127.0.0.0       127.0.0.1       255.0.0.0       UG    0      0        0 lo 
```

Now the first 3 lines are considered default routes and is the important ones to be concerned about.

0.0.0.0         77.77.77.254     0.0.0.0        UG      78    0        0 eth 5

This is saying simply saying any ip address can be reached through 77.77.77.254 on eth5 with a metric of 78.

77.77.77.0      0.0.0.0         255.255.255.0   U     0    0        0 eth5

This line is saying the 77.77.77.x network can be reached directly through eth5.

In short, the first column is saying the desired ip add/network.  The 2nd column, says the next hop ip address.  The 3rd column is the netmask applied to the first column (the 0.0.0.0 means you don't care about any of the address bits).  The 5th column, is the metric/path's cost.  Lastly, the last column says the interface to use.

Unlike firewall rules that go through the rules in a simplistic fashion of top to bottom, the routing table is more intelligent in preferring the route that most closely matches what it wants.

When troubleshooting a firewall, a easy method of identifying the problematic line is to start off as a blank slate and add each line in and see where it fails.

Now, a firewall setup with INPUT/FORWARD/OUTPUT with a default policy of ACCEPT with no other rules, is the same as no firewall.  This base state is the first part to make sure everything works without the firewall interfering with anything.  This is the most important part, if this doesn't than your problem is NOT related to the firewall. If it works, then add each rule and test to see when it breaks.

Note: Having the rule that accepts when the default is to ACCEPT will have NO effect either way, and to be ignored.  Also, it is very important when you ping to test to see if connection works, ALWAYS do it to an ip address, means DO NOT ping to www.google.com or like that; do something like ping 8.8.8.8 or ping 1.1.1.1.  The reason is because, by pinging a url address adds in more complexity, as the computer needs to first send a dns lookup to convert the www.google.com to an ip address then send a ping to that address; where doing to an ip address directly eliminates the lookup step and makes sure you can communicate to the internet.  After that, if pinging 8.8.8.8 works and www.google.com doesn't, this means you can communicate with the internet; you simply have an issue with resolving the www.google.com address to 216.58.194.36.

----------

## pietinger

finalturismo,

yes, bonding is the best you can do in this case ... IF ... your switch is able for it (therefore I have asked in my previous post).

Your current iptables rules allows all we need and is NOT the problem. The statements for sysctl.conf should have solved your problem, and at the moment I dont know why it doesnt work. But I only know installations (like you want) with (all) static IP addressses. Maybe DHCP does some weird stuff (because you do NATting on eth0 also). It would be interesting what happens, if you work with a fixed ip address for eth0. Do before "ip -d -N route" and "ip -d address". If you want you can try this net.conf (check the ip address of your ISP before; if it the same):

```
config_eth0="47.183.193.X netmask 255.255.255.0 brd 47.183.193.255"

config_eth1="77.77.77.1/24"

config_eth5="77.77.77.254/24"

routes_eth0="default via 47.183.193.1"
```

and check your network again with ip -d -N route" and "ip -d address".

Check also your runlevel with "rc-update": sysctl must be in boot, and no dhcp please.

EDIT: I forgot: Do an "ip -d neighbor" also (before and after) !

----------

## finalturismo

 *ct85711 wrote:*   

> Ok, I will explain what the routes mean to help clear up the confusion...
> 
> ```
> 
> 0.0.0.0         77.77.77.254     0.0.0.0        UG      78    0        0 eth 5
> ...

 

Well i understand the metrics and more about the routing table now thanks to you XD. I was aware of the extra complexity for DNS already, iam not that much of a networking n00b. Just a little rusty on IPtables because i dont understand some of the concepts but iam getting there slowly but surly.

So i couldn't do the bonding right away because my kernel didnt have the built in module and using Modprobe still didnt work.  I checked the kernel config and it was compiling the module with M but it didnt have a star next to it for built in. Anyone know what i have to select to be able to directly build the module into the kernel instead of loading it? i know this is the problem because on my T7610 workstation it already had bond0 on interface list and i was able to setup bonding in less than 5 minutes. 

Sometimes i use genkernel all --btrfs or i do genkernel all --kernel-config=  and sometimes i do make and install and do it manually.... 

I know there is a way to have the module directly built into the kernel but i dont know what to select in the make menuconfig to enable that. 

When i ran make menuconfig on my T7610 it showed a star next to bonding. 

Iam really suprised i didnt know that speed stacked in bonding or i would have been doing this all the time. Thanks to Openrc, bonding is stupidly simple on Gentoo  :Smile: 

Love the old init system and openrc layer. Cant beat it, systemD sucks and its one of the main reasons i choose Gentoo for my production environments. 

I run our main office server on Gentoo and my personal server at the house.

Also if i do Modprobe bonding on my r720XD it shows the interface and gets an IP. But it cannot transmit any packets 0.o... also i notice the T7610 it has more initialization steps when i do rc-service bond0 start. So i know this is because i dont have the module directly built into the kernel.Last edited by finalturismo on Tue Sep 29, 2020 8:02 pm; edited 1 time in total

----------

## pietinger

 *finalturismo wrote:*   

> Anyone know what i have to select to be able to directly build the module into the kernel instead of loading it?

 

Just press "space" two times ... its a toggle function [ ] -> [M] -> [*] -> [ ]

----------

## finalturismo

 *pietinger wrote:*   

>  *finalturismo wrote:*   Anyone know what i have to select to be able to directly build the module into the kernel instead of loading it? 
> 
> Just press "space" two times ... its a toggle function [ ] -> [M] -> [*] -> [ ]

 

Ya i know that, but on some items it will not let you build it in directly until you select something else in the kernel config menu. I just dont know what else i have to select.

Like on my T7610 iam able to toggle it to a built in modules because i have another item selected somewhere. I just dont want to enable a bunch of unneeded stuff to get the job done.

----------

## pietinger

 *finalturismo wrote:*   

> [...] but on some items it will not let you build it in directly until you select something else in the kernel config menu. I just dont know what else i have to select.

 

Sorry for my misunderstanding.

When your are in "make menuconfig", go onto the item and select "help". Here you will find at the end two informations:

"Depends on" and "Selected by"

If some of these modules are built as modules, you can only build depending stuff also as module. You have to set these modules as built-in before.

----------

## finalturismo

 *pietinger wrote:*   

>  *finalturismo wrote:*   [...] but on some items it will not let you build it in directly until you select something else in the kernel config menu. I just dont know what else i have to select. 
> 
> Sorry for my misunderstanding.
> 
> When your are in "make menuconfig", go onto the item and select "help". Here you will find at the end two informations:
> ...

 

Thanks, just what i needed to know!

----------

