# [quite solved] Tiny iptables-problem

## Jimini

Hi there,

I have a really small, but annoying problem: I've set up an IRCD on a local machine and want to route all incoming traffic on port 9999 to that machine on port 6667. That's it.

My rules:

```
iptables -I FORWARD -p tcp --dport 9999 -j ACCEPT

iptables -t nat -I PREROUTING -p tcp --dport 9999 -j DNAT --to-destination 10.0.0.2
```

iptables -L:

```
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:9999
```

iptables -t nat -L:

```
DNAT       tcp  --  anywhere             anywhere            tcp dpt:9999 to:10.0.0.2:6667 
```

The server on 10.0.0.2 is really running:

```
tcp6       0      0 :::9999                 :::*                    LISTEN      112        1016793    21684/unrealircd
```

If I connect to the local ip, everything works fine. Just if I try to connect from the outside, the connection times out. So I assume, that my iptables-configuration must be incorrect.

I would be really glad for some helpful hints, perhaps I'm just too tired to figure this out.

Best regards,

Jimini

----------

## twelfthnight

Are you redirecting to 10.0.0.2:6667 while your server listens to port 9999 ?

----------

## Yuu

Hi,

@twelfthnight : I've understanded that he wants to redirect 127.0.0.1:9999 to  10.0.0.2:6667, but maybe I'm wrong.

@Jimini : Maybe you should try to modify the end of your second iptable rule : --to-destination 10.0.0.2 10.0.0.2:6667

If the problem persists, maybe you should try disabling rp_filter (be carefull) with :

 *Quote:*   

> for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do echo 0 > $i; done

 

But I'm not really sure for rp_filter (see here for details), so please use it with caution; or just set-it to 0 for testing, and revert-it to 1.

Good luck.

----------

## Hu

The rp filter should not be an issue here.  Jimini, please post the output of iptables-save on the edge device and clarify what you want.  You mention port 6667 in your opening text and it is shown in the iptables output, but the iptables commands you give and the netstat output indicate that you want everything to be on port 9999 at every stage.

----------

## Jimini

At first, thanks for your replies and sorry for my faulty post - of course, the output of netstat I posted above is wrong - the server listens on 6667. 

- UnrealIRCD is running on port 6667 on 10.0.0.2

- iptables runs on the router (10.0.0.1)

- all traffic from the outside arrives on 10.0.0.1:9999 and should be[/code] routed to 10.0.0.2:6667

The output of iptables-save is really huge, so I put it here. I am a little astonished about ppp0 appearing here - I removed this interface about one week ago, since I do not neet ppp anymore to connect.

Best regards,

Jimini

----------

## Hu

Interfaces in iptables are just strings.  At match time, the kernel compares those strings to the interface name using a defined set of rules.  Therefore, it is legal, though not often useful, to have a rule which names an interface that does not exist.

Looking at the pastebin, I am surprised at how many junk MARK rules you have.  That is not normal, especially to have so many of them that do the same thing.  You should probably investigate whatever daemons you allowed to manage your firewall.  One of them seems to be malfunctioning for it to have added useless rules like that.

For your immediate problem, there are a few tweaks.  Your third PREROUTING rule is sufficient to do what you requested, but is a bit too broad in my opinion.  It will affect any TCP traffic that crosses the router bound for port 9999, regardless of which interface it entered on or which machine it was meant to reach.  Your fourth FORWARD rule is apparently meant to be used here, but is incorrect.  The TCP port specified in the FORWARD rule is the one which will appear on the forwarded packet if it is actually sent.  Thus, at that stage in the process, you should be matching on port 6667, not 9999.  Also, remember to enable IP forwarding if you have not already done so.

Your sixth FORWARD rule looks useless.  Traffic with a source of localhost should not be traversing real interfaces.

----------

## Jimini

 *Hu wrote:*   

> Interfaces in iptables are just strings.  At match time, the kernel compares those strings to the interface name using a defined set of rules.  Therefore, it is legal, though not often useful, to have a rule which names an interface that does not exist.
> 
> Looking at the pastebin, I am surprised at how many junk MARK rules you have.  That is not normal, especially to have so many of them that do the same thing.  You should probably investigate whatever daemons you allowed to manage your firewall.  One of them seems to be malfunctioning for it to have added useless rules like that.

 

Ok, thanks for your explanation. I found a really dumb mistake in my iptables-script: iptables -t mangle -F was missing. The clean output of iptables-save is here:

```
# Generated by iptables-save v1.4.6 on Wed Sep  8 08:09:52 2010

*security

:INPUT ACCEPT [155661525:40601829279]

:FORWARD ACCEPT [1250391432:669595520464]

:OUTPUT ACCEPT [230304678:265014507546]

COMMIT

# Completed on Wed Sep  8 08:09:52 2010

# Generated by iptables-save v1.4.6 on Wed Sep  8 08:09:52 2010

*raw

:PREROUTING ACCEPT [1138882265:588695864098]

:OUTPUT ACCEPT [131825923:168833968562]

COMMIT

# Completed on Wed Sep  8 08:09:52 2010

# Generated by iptables-save v1.4.6 on Wed Sep  8 08:09:52 2010

*nat

:PREROUTING ACCEPT [69:10150]

:POSTROUTING ACCEPT [6:459]

:OUTPUT ACCEPT [8:1078]

-A PREROUTING -i eth0 -p udp -m udp --dport 20534 -j DNAT --to-destination 10.0.0.2:20534 

-A PREROUTING -i eth0 -p tcp -m tcp --dport 20530 -j DNAT --to-destination 10.0.0.2:20530 

-A PREROUTING -i eth0 -p tcp -m tcp --dport 9999 -j DNAT --to-destination 10.0.0.2:9999 

-A PREROUTING -i eth0 -p tcp -m tcp --dport 51413 -j DNAT --to-destination 10.0.0.2:51413 

-A PREROUTING -s 10.0.0.0/24 -d 10.0.0.1/32 -p tcp -m tcp --dport 22 -j DNAT --to-destination 10.0.0.1:10101 

-A PREROUTING -s 134.76.8.5/32 -p tcp -m tcp --dport 22 -j DNAT --to-destination 10.0.0.1:10101 

-A PREROUTING -s 134.76.184.67/32 -p tcp -m tcp --dport 22 -j DNAT --to-destination 10.0.0.1:10101 

-A POSTROUTING -o eth0 -j MASQUERADE 

COMMIT

# Completed on Wed Sep  8 08:09:52 2010

# Generated by iptables-save v1.4.6 on Wed Sep  8 08:09:52 2010

*mangle

:PREROUTING ACCEPT [2124:1178699]

:INPUT ACCEPT [34:3786]

:FORWARD ACCEPT [2090:1174913]

:OUTPUT ACCEPT [22:2464]

:POSTROUTING ACCEPT [2102:1166355]

COMMIT

# Completed on Wed Sep  8 08:09:52 2010

# Generated by iptables-save v1.4.6 on Wed Sep  8 08:09:52 2010

*filter

:INPUT DROP [12:2014]

:FORWARD DROP [0:0]

:OUTPUT DROP [2:544]

:fail2ban-BadBots - [0:0]

:fail2ban-SSH - [0:0]

-A INPUT -s 89.238.160.30/32 -j DROP 

-A INPUT -s 10.0.0.0/24 -i eth1 -m state --state NEW -j ACCEPT 

-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT 

-A INPUT -s 10.0.0.0/24 -i eth1 -p udp -m udp --dport 631 -m state --state NEW -j ACCEPT 

-A INPUT -s 10.0.0.0/24 -i eth1 -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT 

-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT 

-A INPUT -p icmp -m icmp --icmp-type 8 -m state --state NEW -j ACCEPT 

-A INPUT -s 10.0.0.0/24 -i eth1 -p tcp -m tcp --dport 5001 -j ACCEPT 

-A INPUT -p tcp -m tcp --dport 6667 -m state --state NEW -j ACCEPT 

-A INPUT -p tcp -m tcp --dport 6668 -m state --state NEW -j ACCEPT 

-A INPUT -p tcp -m tcp --dport 8033 -m state --state NEW -j ACCEPT 

-A INPUT -i lo -p tcp -m tcp --dport 3306 -j ACCEPT 

-A INPUT -p tcp -m tcp --dport 3000 -m state --state NEW -j ACCEPT 

-A INPUT -p tcp -m tcp --dport 10101 -m state --state NEW -j ACCEPT 

-A FORWARD -d 89.238.160.30/32 -j DROP 

-A FORWARD -p udp -m udp --dport 20534 -j ACCEPT 

-A FORWARD -p tcp -m tcp --dport 20530 -j ACCEPT 

-A FORWARD -p tcp -m tcp --dport 9999 -j ACCEPT 

-A FORWARD -p tcp -m tcp --dport 51413 -j ACCEPT 

-A FORWARD -s 10.0.0.0/24 -i eth1 -o eth0 -m state --state NEW -j ACCEPT 

-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT 

-A FORWARD -p icmp -m icmp --icmp-type 8 -m state --state NEW -j ACCEPT 

-A OUTPUT -d 89.238.160.30/32 -j DROP 

-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT 

-A OUTPUT -d 10.0.0.0/24 -o eth1 -p udp -m udp --dport 631 -m state --state NEW -j ACCEPT 

-A OUTPUT -d 10.0.0.0/24 -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT 

-A OUTPUT -d 62.141.58.13/32 -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT 

-A OUTPUT -d 57.118.100.175/32 -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT 

-A OUTPUT -d 213.73.91.35/32 -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT 

-A OUTPUT -p tcp -m tcp --dport 21 -m state --state NEW -j ACCEPT 

-A OUTPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT 

-A OUTPUT -p icmp -m icmp --icmp-type 8 -m state --state NEW -j ACCEPT 

-A OUTPUT -d 10.0.0.0/24 -o eth1 -p tcp -m tcp --dport 5001 -j ACCEPT 

-A OUTPUT -p tcp -m tcp --dport 6667 -m state --state NEW -j ACCEPT 

-A OUTPUT -p tcp -m tcp --dport 6668 -m state --state NEW -j ACCEPT 

-A OUTPUT -o lo -p tcp -m tcp --dport 3306 -j ACCEPT 

-A OUTPUT -d 134.76.249.201/32 -p udp -m udp --dport 123 -m state --state NEW -j ACCEPT 

-A OUTPUT -d 134.76.249.102/32 -p udp -m udp --dport 123 -m state --state NEW -j ACCEPT 

-A OUTPUT -d 134.76.10.46/32 -p udp -m udp --dport 123 -m state --state NEW -j ACCEPT 

-A OUTPUT -d 137.226.34.228/32 -o eth0 -p tcp -m tcp --dport 873 -m state --state NEW -j ACCEPT 

-A OUTPUT -o eth0 -p tcp -m tcp --dport 25 -m state --state NEW -j ACCEPT 

-A OUTPUT -d 10.0.0.0/24 -p tcp -m tcp --dport 22 -m state --state NEW -j ACCEPT 

COMMIT

# Completed on Wed Sep  8 08:09:52 2010
```

 *Quote:*   

> For your immediate problem, there are a few tweaks.  Your third PREROUTING rule is sufficient to do what you requested, but is a bit too broad in my opinion.  It will affect any TCP traffic that crosses the router bound for port 9999, regardless of which interface it entered on or which machine it was meant to reach.  Your fourth FORWARD rule is apparently meant to be used here, but is incorrect.  The TCP port specified in the FORWARD rule is the one which will appear on the forwarded packet if it is actually sent.  Thus, at that stage in the process, you should be matching on port 6667, not 9999.

 

Okay, I also run an IRCD on my router (10.0.0.1) which uses port 6667, so I can't route all traffic that arrives on port 6667 to 10.0.0.2. However, I changed the listening port of my IRCD on 10.0.0.2 from 6667 to 9999 and modified my rules, they now look like this:

```
iptables -I FORWARD -p tcp --dport 9999 -j ACCEPT

iptables -t nat -I PREROUTING -i $wan -p tcp --dport 9999 -j DNAT --to-destination 10.0.0.2:9999
```

Of course I have enabled ip forwarding and the IRCD on 10.0.0.2 is still reachable from the inside of my network - not from the outside. I'm a little confused by editing iptables-rules at 7 o'clock in the morning, but I hope I got your posting right.

Best regards,

Jimini

----------

## Hu

 *Jimini wrote:*   

> Okay, I also run an IRCD on my router (10.0.0.1) which uses port 6667, so I can't route all traffic that arrives on port 6667 to 10.0.0.2. However, I changed the listening port of my IRCD on 10.0.0.2 from 6667 to 9999 and modified my rules, they now look like this:
> 
> ```
> iptables -I FORWARD -p tcp --dport 9999 -j ACCEPT
> 
> ...

 Your rules look correct for what I think you have built.  At this point, we will need to dig deeper.  Please emerge net-analyzer/tcpdump on the router and use it to monitor both the internal and external interfaces of the router when an external user connects to $public_IP:9999.  We are looking to confirm that:A TCP SYN from the external user reaches the router (no ISP interference).

A TCP SYN with the same source information, but a rewritten destination, is emitted from the internal interface to the internal server.

A TCP SYN|ACK from the internal server is sent back through the router to the external host.

A TCP ACK is sent from the external host to and through the router.

Actual data flows between the two.Since you say that the internal system is not reachable, at least one of those list items will fail.  Once we know which one fails, we will have a better understanding of why the connection does not work.

----------

## Jimini

Hm, I set up tcpdump and let a friend of mine try to connect to the server - it worked and I don't know why. I isolated the problem in that way, that connecting to port 9999 from the outside works fine. From the inside it doesn't - I can only reach the IRCD by connecting to 10.0.0.2 directly. 

WAN => 10.0.0.1:9999 => is routed to 10.0.0.2:9999

LAN => 10.0.0.1:9999 => failure

LAN => 10.0.0.2:9999 => works

However, I executed tcpdump port 9999 (someone connects from the outside):

```
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on eth0, link-type EN10MB (Ethernet), capture size 68 bytes

16:53:58.148833 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: S 1860954387:1860954387(0) win 65535 <mss 1452,[|tcp]>

16:53:58.149111 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: S 1827854999:1827854999(0) ack 1860954388 win 5792 <mss 1460,[|tcp]>

16:53:58.190130 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: . ack 1 win 65535 <nop,nop,timestamp[|tcp]>

16:53:58.192157 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: P 1:19(18) ack 1 win 65535 <nop,nop,timestamp[|tcp]>

16:53:58.192309 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: . ack 19 win 91 <nop,nop,timestamp[|tcp]>

16:53:58.238121 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: P 19:46(27) ack 1 win 65535 <nop,nop,timestamp[|tcp]>

16:53:58.238470 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: . ack 46 win 91 <nop,nop,timestamp[|tcp]>

16:53:58.243073 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: P 1:220(219) ack 46 win 91 <nop,nop,timestamp[|tcp]>

16:53:58.288115 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: . ack 220 win 65535 <nop,nop,timestamp[|tcp]>

16:54:08.919040 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: P 220:301(81) ack 46 win 91 <nop,nop,timestamp[|tcp]>

16:54:08.960694 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: . ack 301 win 65535 <nop,nop,timestamp[|tcp]>

16:54:08.960803 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: P 301:317(16) ack 46 win 91 <nop,nop,timestamp[|tcp]>

16:54:09.002144 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: . ack 317 win 65535 <nop,nop,timestamp[|tcp]>

16:54:09.004455 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: P 46:62(16) ack 317 win 65535 <nop,nop,timestamp[|tcp]>

16:54:09.030294 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: P 317:1371(1054) ack 62 win 91 <nop,nop,timestamp[|tcp]>

16:54:09.098131 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: . ack 1371 win 65535 <nop,nop,timestamp[|tcp]>

16:54:09.098278 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: P 1371:2035(664) ack 62 win 91 <nop,nop,timestamp[|tcp]>

16:54:09.157142 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: . ack 2035 win 65535 <nop,nop,timestamp[|tcp]>

16:54:09.161878 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: P 62:82(20) ack 2035 win 65535 <nop,nop,timestamp[|tcp]>

16:54:09.162087 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: P 2035:2353(318) ack 82 win 91 <nop,nop,timestamp[|tcp]>

16:54:09.205234 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: P 82:105(23) ack 2353 win 65535 <nop,nop,timestamp[|tcp]>

16:54:09.205403 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: P 2353:2429(76) ack 105 win 91 <nop,nop,timestamp[|tcp]>

16:54:09.259801 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: . ack 2429 win 65535 <nop,nop,timestamp[|tcp]>

16:54:09.601165 IP 188.108.139.205.51936 > 91-65-144-59-dynip.superkabel.de.9999: P 105:126(21) ack 2429 win 65535 <nop,nop,timestamp[|tcp]>

16:54:09.640948 IP 91-65-144-59-dynip.superkabel.de.9999 > 188.108.139.205.51936: . ack 126 win 91 <nop,nop,timestamp[|tcp]>
```

Seems to work fine, doesn't it?

But if I try to connect from the inside of my network, tcpdump shows nothing. So I assume, that there is just a little mistake in my iptables-script or something similar. I hope I didn't get something totally wrong with my idea of connecting to 10.0.0.1 and being routed to 10.0.0.2?

Again, my rules:

```
iptables -I FORWARD -p tcp --dport 9999 -j ACCEPT

iptables -t nat -I PREROUTING -i $wan -p tcp --dport 9999 -j DNAT --to-destination 10.0.0.2:9999

iptables -t nat -I PREROUTING -i $lan -p tcp --dport 9999 -j DNAT --to-destination 10.0.0.2:9999
```

Best regards,

Jimini

----------

## Hu

 *Jimini wrote:*   

> Hm, I set up tcpdump and let a friend of mine try to connect to the server - it worked and I don't know why. I isolated the problem in that way, that connecting to port 9999 from the outside works fine. From the inside it doesn't - I can only reach the IRCD by connecting to 10.0.0.2 directly.

 This is expected!  Per my earlier comments, you changed the line that would have caused the router to NAT internal traffic back to an internal server.  Performing such a translation produces a suboptimal network path for no good reason.  You might be able to enable that functionality with some additional rewrite rules, or some modifications to the existing one.  However, unless you specifically need an internal application to follow this path, I suggest you just use the proper destination address when you are inside.

----------

## Jimini

Seems as this will be the only practicable solution. Thank you for your helpful hints!

Best regards,

Jimini

----------

