# Network Update...Need advice

## dpshak

As the subject says, I've upgraded my network.   

We are in the process of dropping DishNetwork in favor of Comcast with a Tivo box.  I'm also going to build a media server to work in conjunction with the Tivo.  Because all of my hardwired boxes are capable of 10/100/1000 networking and all of my wireless devices are 802.11n capable, I decided to replace my old Linksys WRT54G router with a NetGear WNR3500L to take advantage of the higher network speeds for media transfer.  I also have an old, unused, P4 box with a couple of small hard drives that I thought I would turn into a firewall box with local rsync server and ntp server, etc.  

This works, after a fashion, but NOT quite how I intended!   :Confused:   (That seems to be the way these things always work for me...  :Rolling Eyes:  )

This is the way it looks: Cable Modem(WAN) -> Firewall_In(eth0) -> Firewall_Out(eth1) -> WNR3500L -> LAN (both wired and wireless)

DHCP, through the cable modem, provides the Public IP address to Firewall_In.  Firewall_Out has a fixed (192.168.1.2) address.  The router has a fixed (192.168.1.3) address.  Finally, I use the router to assign addresses to the various devices based on their MAC address - 192.168.1.1##.  Everything on the LAN can talk amongst themselves, and they can all access the internet.  What they CANNOT do, is see the firewall box!   :Shocked: 

This is where I run into the proverbial wall.  I don't know where to start to fix this!  At the time I was putting all this together, it seemed like the perfect solution.  A firewall box right at the 'fence' to protect the whole local network, a router that provided both wired and wireless networking, and oh by the way, let's put that 'left-over' hardware in the firewall box to use by hosting a local rsync server.  This way, certain devices on the LAN can be blocked from any WAN access because anything they need is available on the LAN.

So the question is: Can I make this work?  (If so, how?) or am I out of luck?

----------

## DawgG

probably a firewall-rule on Firewall:eth1, often a default DROP policy is set. check it with 

```
iptables -L
```

.

if that is not obvious you can put a log-rule in front of all firewall-rules and check each packet.

if packets get forwarded thru/by that host but not "answered" by it the first place to look would be its INPUT/OUTPUT-chains.

GOOD LUCK!

----------

## Hu

Please post the output of iptables-save -c as seen by the firewall box.

----------

## dpshak

I THINK DawgG, you may have something here... I just have to wait until the rest of the LAN users go to bed to fix and restart the LAN...  Apparently, I copied the info from Gentoo Router too closely and failed to allow for MY LAN addresses...

HU:  this is the result...laboriously hand copied from that machine...   :Smile: 

```
# Generated by iptables-save v1.4.20 on Tue Dec 17 16:16:27 2013

*nat

:PREROUTING ACCEPT [303804:22393592]

:INPUT ACCEPT [100170:6563977]

:OUTPUT ACCEPT [117330:7554789]

:POSTROUTING ACCEPT [2:656}

[308526:19242943] -A POSTROUTING -o enp2s1 -j MASQUERADE

COMMIT

# Completed on Tue Dec 17 16:16:27 2013

# Generated by iptables-save v1.4.20 on Tue Dec 17 16:16:27 2013

*mangle

:PREROUTING ACCEPT [7193308:4747973669]

:INPUT ACCEPT [267882:61008957]

:FORWARD ACCEPT [6925212:4586944250]

:OUTPUT ACCEPT [266278:25521084]

:POSTROUTING ACCEPT [7190926:4712429102]

COMMIT

# Completed on Tue Dec 17 16:16:27 2013

# Generated by iptables-save v1.4.20 on Tue Dec 17 16:16:27 2013

*filter

:INPUT ACCEPT [157540:50278021]

:FORWARD DROP [0:0]

:OUTPUT ACCEPT [265988:25491947]

[72:7146] -A INPUT -i lo -j ACCEPT

[97415:6345222] -A INPUT -i enp3s4 -j ACCEPT

[0:0] -A INPUT ! -i enp3s4 -p udp -m udp --dport 67 -j REJECT --reject-with icmp-port-unreachable

[47:3108] -A INPUT ! -i enp3s4 -p udp -m udp --dport 53 -j REJECT --reject-with icmp-port-unreachable

[227:12472] -A INPUT ! -i enp3s4 -p tcp -m tcp --dport 0:1023 -j DROP

[3277713:407148620] -A FORWARD -s 192.168.0.0/16 -i enp3s4 -j ACCEPT

[0:0] -A FORWARD -d 192.168.0.0/16 -i enp3s4 -j DROP

[0:0] -A FORWARD -s 192.168.0.0/16 -i enp3s4 -j ACCEPT

[3646935:4199759398] -A FORWARD -d 192.168.0.0/16 -i enp2s1 -j ACCEPT

COMMIT

# completed on Tue Dec 17 16:16:27 2013
```

(enp2s1=Firewall_In and enp3s4=Firewall_Out)

and THANKS for the assist!!

----------

## Hu

Your network rules need work, but nothing in there explains the symptoms that prompted you to post the thread.

 *dpshak wrote:*   

> 
> 
> HU:  this is the result...laboriously hand copied from that machine...  
> 
> 

 Why not just POST it directly from that machine to the Internet?

 *dpshak wrote:*   

> 
> 
> ```
> 
> *mangle
> ...

 This table is unused.  You could remove it from the kernel to reduce overhead.

 *dpshak wrote:*   

> 
> 
> ```
> 
> [97415:6345222] -A INPUT -i enp3s4 -j ACCEPT
> ...

 Traffic inbound to the gateway from the LAN is unconditionally allowed, so the problem reported in your first post is not the fault of a packet filter rule.

Please explain in more detail the symptoms that led you to say "What they CANNOT do, is see the firewall box!"

 *dpshak wrote:*   

> 
> 
> ```
> 
> [0:0] -A INPUT ! -i enp3s4 -p udp -m udp --dport 67 -j REJECT --reject-with icmp-port-unreachable
> ...

 Why are you sending any response at all to unsolicited traffic from the Internet?

 *dpshak wrote:*   

> 
> 
> ```
> 
> [227:12472] -A INPUT ! -i enp3s4 -p tcp -m tcp --dport 0:1023 -j DROP
> ...

 This breaks authentication based on use of privileged ports.  Also, by omission, you allow incoming connections to high numbered ports.  You also permit incoming UDP datagrams to any port other than the two blacklisted above.  None of these results are desirable.

 *dpshak wrote:*   

> 
> 
> ```
> 
> [3277713:407148620] -A FORWARD -s 192.168.0.0/16 -i enp3s4 -j ACCEPT
> ...

 This third rule is a duplicate of the first one.

 *dpshak wrote:*   

> 
> 
> ```
> 
> [3646935:4199759398] -A FORWARD -d 192.168.0.0/16 -i enp2s1 -j ACCEPT
> ...

 This rule permits any WAN-sourced traffic destined to the LAN, even if such traffic is unsolicited.  This is probably not what you want.  Depending on your ISP, it may be possible for users on the same segment as your cable modem to send you traffic addressed to an internal IP address.  Your present rules would forward such traffic to the internal machine, effectively avoiding the firewall filter.

Overall, the referenced page seems to give poor advice with regard to iptables rules.

----------

## dpshak

HU: Thanks again for taking the time!

Like a number of other folks, iptables gives me fits!   :Embarassed: 

My goal, in using that link referenced above, was to get the system up and running.  Once everything was running the way I wanted, I planned on installing Shorewall and using it to write proper network rules.

As I said in the original post, one of the things I wanted to do was to create a local rsync server on the firewall box.  To test the connection:

```
Main linux # ping -c3 192.168.1.2

PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.

From 192.168.1.100: icmp_seq=1 Destination Host Unreachable

From 192.168.1.100: icmp_seq=2 Destination Host Unreachable

From 192.168.1.100: icmp_seq=3 Destination Host Unreachable

--- 192.168.1.2 ping statistics ---

3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 1999ms

pipe 3
```

  BTW: I have pared down the rules to just the NAT segment.

I get identical results when I try to ping the 'Main' box from the firewall box.

Interestingly enough, when I (ping the router from the firewall):

```
FIREWALL iptables # ping -c3 192.168.1.3

PING 192.168.1.3 (192.168.1.3) 56(124) bytes of data.

^c

--- 192.168.1.3 ping statistics ---

3 packets transmitted, 0 received, 100% packet loss, time 1999ms
```

Maybe my problem is in the iptables rules in the router?!?  

Here are results from the router; 1st to the Firewall box and 2nd, to the Main box:

```
root@NetGear-3500Lv2:/tmp/home/root# ping 192.168.1.2

PING 192.168.1.2 (192.168.1.2): 56 data bytes

64 bytes from 192.168.1.2: seq=0 ttl=64 time=0.503 ms

64 bytes from 192.168.1.2: seq=1 ttl=64 time=0.441 ms

64 bytes from 192.168.1.2: seq=2 ttl=64 time=0.433 ms

--- 192.168.1.2 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.433/0.459/0.503 ms

root@NetGear-3500Lv2:/tmp/home/root# ping 192.168.1.100

PING 192.168.1.100 (192.168.1.100): 56 data bytes

64 bytes from 192.168.1.100: seq=0 ttl=64 time=0.621 ms

64 bytes from 192.168.1.100: seq=1 ttl=64 time=0.531 ms

--- 192.168.1.100 ping statistics ---

2 packets transmitted, 2 packets received, 0% packet loss

round-trip min/avg/max = 0.531/0.576/0.621 ms

```

Finally:

```
root@NetGear-3500Lv2:/tmp/home/root# iptables-save -c

# Generated by iptables-save v1.3.8 on Wed Dec 18 04:54:19 2013

*nat

:PREROUTING ACCEPT [130260:9062624]

:POSTROUTING ACCEPT [13:906]

:OUTPUT ACCEPT [44743:2895093]

:WANPREROUTING - [0:0]

[23:2408] -A PREROUTING -d 192.168.1.5 -j WANPREROUTING 

[47:4544] -A PREROUTING -d 192.168.1.0/255.255.255.0 -i vlan2 -j DROP 

[854:51240] -A PREROUTING -s 192.168.1.0/255.255.255.0 -d ! 192.168.1.0/255.255.255.0 -p udp -m udp --dport 53 -j DNAT --to-destination 192.168.1.3 

[125043:7751980] -A POSTROUTING -o vlan2 -j MASQUERADE 

[23:7590] -A POSTROUTING -s 192.168.1.0/255.255.255.0 -d 192.168.1.0/255.255.255.0 -o br0 -j SNAT --to-source 192.168.1.3 

[0:0] -A WANPREROUTING -p icmp -j DNAT --to-destination 192.168.1.3 

COMMIT

# Completed on Wed Dec 18 04:54:19 2013

# Generated by iptables-save v1.3.8 on Wed Dec 18 04:54:19 2013

*mangle

:PREROUTING ACCEPT [3092334:1857967431]

:INPUT ACCEPT [129709:16659564]

:FORWARD ACCEPT [2956502:1840363402]

:OUTPUT ACCEPT [127909:19784855]

:POSTROUTING ACCEPT [3083941:1860114351]

COMMIT

# Completed on Wed Dec 18 04:54:19 2013

# Generated by iptables-save v1.3.8 on Wed Dec 18 04:54:19 2013

*filter

:INPUT DROP [0:0]

:FORWARD DROP [0:0]

:OUTPUT ACCEPT [127909:19784855]

:shlimit - [0:0]

:wanin - [0:0]

:wanout - [0:0]

[43:2954] -A INPUT -m state --state INVALID -j DROP 

[64566:11980030] -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT 

[4:320] -A INPUT -p tcp -m tcp --dport 22 -m state --state NEW -j shlimit 

[13:906] -A INPUT -i lo -j ACCEPT 

[65087:4675674] -A INPUT -i br0 -j ACCEPT 

[0:0] -A INPUT -p udp -m udp --sport 67 --dport 68 -j ACCEPT 

[2956502:1840363402] -A FORWARD -m account --aaddr 192.168.1.0/255.255.255.0 --aname lan 

[0:0] -A FORWARD -i br0 -o br0 -j ACCEPT 

[470:33910] -A FORWARD -m state --state INVALID -j DROP 

[163329:9626988] -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu 

[2872012:1835119719] -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT 

[0:0] -A FORWARD -i vlan2 -j wanin 

[84020:5209777] -A FORWARD -o vlan2 -j wanout 

[84020:5209777] -A FORWARD -i br0 -j ACCEPT 

[4:320] -A shlimit -m recent --set --name shlimit --rsource 

[0:0] -A shlimit -m recent --update --seconds 60 --hitcount 4 --name shlimit --rsource -j DROP 

COMMIT

# Completed on Wed Dec 18 04:54:19 2013
```

Maybe you can see something in there that is causing my problem?

----------

## Hu

 *dpshak wrote:*   

> 
> 
> My goal, in using that link referenced above, was to get the system up and running.  Once everything was running the way I wanted, I planned on installing Shorewall and using it to write proper network rules.
> 
> 

 In my experience, shorewall-generated configurations are ugly.  Once you use one, you must continue to use shorewall to maintain it or throw it all out and write a maintainable one by hand.

 *dpshak wrote:*   

> 
> 
> ```
> Main linux # ping -c3 192.168.1.2
> 
> ...

 Weird.  That looks like there is no ARP response.  Looking at your topology, I wonder if there is a setup problem here.  I initially wrote off that you put the WNR3500L between Firewall_Out and the LAN, which would be fine if WNR3500L is just a switch or is in bridging mode.  Your output in the preceding post looks like it is acting as a bridge.  If you have the network ports for it, I would remove the P4 from firewall service, switch the WNR3500L into NAT mode with firewalling, and move the P4 into the LAN.  You would then have this topology:

Cable Modem -> WNR3500L

WNR3500L -> wireless LAN

WNR3500L -> wired LAN (including P4)

 *dpshak wrote:*   

> 
> 
> I get identical results when I try to ping the 'Main' box from the firewall box.
> 
> 

 Those results are different.  One is a failure to find a route to the target.  The other is a failure to receive a response from the target.

 *dpshak wrote:*   

> 
> 
> Maybe my problem is in the iptables rules in the router?!?  
> 
> 

 Mixing bridging and NAT on a single host is usually not what you want, but your iptables rules on the WNR3500L appear to do this.  Please provide the output of ip route ; ip addr ; brctl show as run by the WNR3500L.

----------

## dpshak

You are correct, in that I have the the WNR3500L AFTER the Firewall_Out and is acting as a router.  It's running the TomatoUSB firmware instead of the OEM firmware.  Other than assigning IP addresses to MAC addresses and configuring some WiFi settings, I haven't messed with any bridging, or iptables or the like.  

ip route from the WNR3500L:

```
root@NetGear-3500Lv2:/tmp/home/root# ip route

192.168.1.2 dev vlan2  scope link 

192.168.1.0/24 dev br0  proto kernel  scope link  src 192.168.1.3 

192.168.1.0/24 dev vlan2  proto kernel  scope link  src 192.168.1.5 

127.0.0.0/8 dev lo  scope link 

default via 192.168.1.2 dev vlan2
```

ip addr from the WNR3500L:

```
root@NetGear-3500Lv2:/tmp/home/root# ip addr

1: lo: <LOOPBACK,MULTICAST,UP,10000> mtu 16436 qdisc noqueue 

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo

    inet6 ::1/128 scope host 

       valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000

    link/ether 04:a1:51:19:92:22 brd ff:ff:ff:ff:ff:ff

3: eth1: <BROADCAST,MULTICAST,ALLMULTI,UP,10000> mtu 1500 qdisc pfifo_fast qlen 1000

    link/ether 04:a1:51:19:92:24 brd ff:ff:ff:ff:ff:ff

4: vlan1@eth0: <BROADCAST,MULTICAST,ALLMULTI,UP,10000> mtu 1500 qdisc noqueue 

    link/ether 04:a1:51:19:92:22 brd ff:ff:ff:ff:ff:ff

5: vlan2@eth0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc noqueue 

    link/ether 04:a1:51:19:92:23 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.5/24 brd 192.168.1.255 scope global vlan2

6: br0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc noqueue 

    link/ether 04:a1:51:19:92:22 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.3/24 brd 192.168.1.255 scope global br0

7: imq0: <NOARP> mtu 1500 qdisc noop qlen 30

    link/void 

8: imq1: <NOARP> mtu 1500 qdisc noop qlen 30

    link/void
```

brctl show from WNR3500L:

```
root@NetGear-3500Lv2:/tmp/home/root# brctl show

bridge name     bridge id               STP enabled     interfaces

br0             8000.04a151199222       no              vlan1

                                                        eth1
```

IF I use the WNR3500L as a router/firewall, I will probably retire the P4 - as the LAN will be protected by the router's firewall and I really don't have the ports to spare.  There are other boxes on the LAN that could fulfill the rest of the services I was going to put on the P4....

Would it make more sense to do it that way and add more/better rules to the router?  I'm only using roughly 10% of the available memory in the router, at present, so it would seem to me that some could be added without taxing it too much.  If I don't sound very knowledgeable here, it's because I'm NOT!  This was/is going to be a learning experience - pretty much the same way I've learned my way around Gentoo!  Read a bunch, try some stuff, if it works, GREAT!  If it doesn't work, read some more, try some more, until it does!

Thanks for your time!

----------

## Hu

That is one of the odder network configurations I have seen.  You may not have configured bridging, but someone or something did.

You have a redundant route which is wrong (#3), but it is shadowed by one which is right (#2).  You have the P4 assigned an address that makes it appear to be on the same side of the router as the other machines, but it is not.  There are some other aspects that are not wrong, but probably not what you really want.  The simple way to keep the P4 in service would be to change the addressing so that its internal interface is on a separate subnet from the LAN-side devices.  Your vlan configuration seems to indicate that you are using a single switch to connect both the LAN devices to the Netgear and the Netgear to the P4.  The earlier ping failures suggest that this has not breached isolation, but it looks strange.

----------

## dpshak

I ran across a post, somewhere, where Shibby (the author of the firmware in the Netgear) talked about bridging in that router... unfortunately I haven't been able to find it again.  

I think, in the interests of efficiency, I'm going to pull the P4 out of the lineup for now.  When I have the time to learn more about networking in general, routing, bridging, vlans, etc. (and iptables)  I may add it back in.

I DO appreciate your time and efforts HU!  THANKS!   :Smile: 

----------

## BobWya

 *dpshak wrote:*   

> ...
> 
> I think, in the interests of efficiency, I'm going to pull the P4 out of the lineup for now.  When I have the time to learn more about networking in general, routing, bridging, vlans, etc. (and iptables)  I may add it back in.
> 
> ...
> ...

 

You might want to check out the docs and ML at...

Linux Advanced Routing & Traffic Control

It's where I learned all my iptables foo from  :Smile: 

Although I gather iptables is going to ripped out and replaced with something much better soon... Nftables: a new packet filtering engine  :Rolling Eyes: 

Bob

----------

## dpshak

Thanks for the info BobWya!    :Smile: 

As I told Hu above, I REALLY don't have enough ports in the Netgear router!  :Confused:  (Piss, poor planning on my part!) 

I'm planning on investing in a switch in the next week or so.  Once I have that on hand, I'm going to (try to) use the P4 box as a router to feed the Switch and connect the hardwired PC's to the switch, along with the Netgear (for WiFi access) and see how that works.

From what I've seen of Hu, across this forum, he KNOWS HIS STUFF!  If he says Shorewall isn't the way to go, I guess I'll have to buckle down and figure out how to do iptables (or its replacement) manually!

----------

## Hu

Shorewall may produce a functional configuration.  My chief complaint with it is that, like with many machine-generated outputs, the output is messy and often suboptimal.  Feel free to use Shorewall to create a first firewall if you need something working quickly.  Just be aware that if you ever want to do something not supported by the Shorewall frontend, which includes getting iptables help from people who do not know Shorewall, you may end up doing a lot of duplicate work.

----------

## BobWya

 *dpshak wrote:*   

> 
> 
> From what I've seen of Hu, across this forum, he KNOWS HIS STUFF!  If he says Shorewall isn't the way to go, I guess I'll have to buckle down and figure out how to do iptables (or its replacement) manually!

 

I'm pretty lazy these days so I tend to just to use UFW on my Desktops/Laptop these days and just use the facilities in my SOHO router...

Shorewall is pretty useless really. Just as much work as handcrafting a BASH script or iptables directly. Of course the other option is to just cheap out using an appliance-type Linux distro or PFSense... But in the latter case it's worth knowing what makes them tick - for troubleshooting purposes!!   :Cool: 

Also should mention Arno's Projects - which is linked to from the Arch wiki page on Firewalls... It's useful to see other peoples work to reverse engineer it for your own purposes after all...   :Cool: 

Glad to be of service...

Bob

----------

