# What did screw my l7-filter + iptables + tc the last week?

## dj_farid

What did change a week ago or so that messed up my traffic shaping?

I have been running a script that shapes my traffic and prioritizes it as I want it to be. Some days ago it just stopped to prioritize my packets.

What did change and cause this problem?

Here is my script that I have not changed since it worked:

```
#!/bin/bash

TC='/sbin/tc'

echo "Doing some traffic shaping"

# Clear

tc qdisc del dev eth0 root    # Clear any previous stuff

# Shaping

CEIL=904

tc qdisc add dev eth0 root handle 1: htb default 10

tc class add dev eth0 parent 1: classid 1:1 htb rate ${CEIL}kbit

tc class add dev eth0 parent 1:1 classid 1:10 htb rate 820kbit ceil ${CEIL}kbit prio 1

tc class add dev eth0 parent 1:1 classid 1:20 htb rate 80kbit ceil 900kbit quantum 1514 prio 2

tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10

tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10

##Restrict BitTorrent upload

iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark

iptables -t mangle -A OUTPUT -m mark ! --mark 0 -j ACCEPT

iptables -t mangle -A OUTPUT -m layer7 --l7proto bittorrent -j MARK --set-mark 2

#iptables -t mangle -A OUTPUT -m ipp2p --bit -j MARK --set-mark 2

iptables -t mangle -A OUTPUT -j CONNMARK --save-mark

tc filter add dev eth0 protocol ip parent 1:0 handle 2 fw flowid 1:20
```

There are not error messages or anything. The problem is that if I have bittorrent running while I transfer with FTP to somewhere, the ftp transfers are very very slow. When the script was working it was the other way around, FTP went full speed and bittorrent slowed down.

This is what I have emerged since the time that I know that it worked:

```

1162362537:  === (2 of 2) Compiling/Merging (sys-apps/kbd-1.12-r8::/usr/portage/sys-apps/kbd/kbd-1.12-r8.ebuild)

1162362604: === Unmerging... (sys-apps/kbd-1.12-r7)

1162362076:  === (1 of 2) Compiling/Merging (net-misc/openssh-4.4_p1-r5::/usr/portage/net-misc/openssh/openssh-4.4_p1-r5.ebuild)

1162362525: === Unmerging... (net-misc/openssh-4.3_p2-r5)

1162458743:  === (1 of 3) Cleaning (sys-devel/autoconf-wrapper-3.2-r2::/usr/portage/sys-devel/autoconf-wrapper/autoconf-wrapper-3.2-r2.ebuild)

1162458758: === Unmerging... (sys-devel/autoconf-wrapper-3.2)

1162458762:  === (2 of 3) Compiling/Merging (sys-devel/autoconf-2.60::/usr/portage/sys-devel/autoconf/autoconf-2.60.ebuild)

1162458798: === Unmerging... (sys-devel/autoconf-2.59-r7)

1162458805:  === (3 of 3) Compiling/Merging (sys-apps/coreutils-6.4::/usr/portage/sys-apps/coreutils/coreutils-6.4.ebuild)

1162459725: === Unmerging... (sys-apps/coreutils-5.94-r1)

1162544264:  === (1 of 1) Compiling/Merging (sys-apps/baselayout-1.12.6::/usr/portage/sys-apps/baselayout/baselayout-1.12.6.ebuild)

1162544320: === Unmerging... (sys-apps/baselayout-1.12.5-r2)

```

There is one bug that looks like mine: https://bugs.gentoo.org/show_bug.cgi?id=148544

But it seems kind of unlikely that it's the baselayout that did the change. I downgdraded to the baselayout version that is supposed to work according to that bugreport. Did not help me.

I have upgraded net-misc/l7-filter and net-misc/l7-protocols to the ~x86 versions in portage. Did not help.

Using vanilla kernel 2.6.17.13

----------

## dj_farid

A small update. I really hope that someone could shed some light on my problem here...

My network looks like this:

___________

|The Internet|

----------------

      |

-----------------

|gentoo router|

-----------------

      |

------------------

|gentoo/WinXP |

------------------

On the gentoo router I am running rTorrent among other things.

I want the torrent traffic not to affect the rest of my traffic.

As I said in the previous post, it was working with the script I had. Then all of a sudden it stopped working.

I changed all the "OUTPUT"s to "PREROUTING" in the script. Now it is almost working. I thought that it would not work, since I thought that the packets are not going through the PREROUTING. But I was wrong.

The strange thing now:

If I have rTorrent running on the router, no matter how much/little it seeds.

Then I boot into Windows XP and start a FTP session from there. I am not able to download at faster speeds than very slow. It is enough that rTorrent is running for it to affect the download speed to my XP.

Downloading to the router is not affected by rTorrent in any way.

Why does it do like this?

What is the difference in having the marking done in PREROUTING vs. OUTPUT?

----------

## MorpheuS.Ibis

i have this in my mangle table. 

as i dont really get the difference between PREROUTING and FORWARD or anything i put it into OUTPUT, FORWARD and PREROUTING.

```
[0:0] -A OUTPUT -o eth0 -m mark --mark 0x0 -j MARK --set-mark 0x3

[0:0] -A OUTPUT -p icmp -j MARK --set-mark 0x1

[0:0] -A OUTPUT -p tcp -m tcp --dport 22 -j MARK --set-mark 0x1

[0:0] -A OUTPUT -p ! tcp -j MARK --set-mark 0x1

[0:0] -A OUTPUT -p tcp -m tcp --dport 80 -j MARK --set-mark 0x3

[0:0] -A OUTPUT -p tcp -m tcp --dport 443 -j MARK --set-mark 0x3

[0:0] -A OUTPUT -p tcp -m tcp --dport 25 -j MARK --set-mark 0x3

[0:0] -A OUTPUT -o eth0 -p tcp -m ipp2p --ipp2p -j MARK --set-mark 0x4

[0:0] -A OUTPUT -o eth0 -p tcp -m tcp --sport 6881:7000 -j MARK --set-mark 0x4

[0:0] -A OUTPUT -o eth0 -p tcp -m tcp --dport 6881:7000 -j MARK --set-mark 0x4

[0:0] -A OUTPUT -o eth0 -m layer7 --l7proto bittorrent -j MARK --set-mark 0x4

[0:0] -A OUTPUT -o eth0 -m layer7 --l7proto directconnect -j MARK --set-mark 0x4

[0:0] -A OUTPUT -o eth0 -m layer7 --l7proto edonkey -j MARK --set-mark 0x4

[0:0] -A OUTPUT -o eth0 -m layer7 --l7proto gnutella -j MARK --set-mark 0x4

[0:0] -A OUTPUT -o eth0 -m layer7 --l7proto fasttrack -j MARK --set-mark 0x4

```

my logic says that the OUTPUT/FORWARD/PREROUTING scheme is like this...

the packet is about to leave your router, PREROUTING marks it,

the packet is locally generated and leaves, OUTPUT marks it,

the packet is forwarded from LAN, FORWARD marks it.

according to it, the PREROUTING chain would do that all but what does 10 lines in iptables config means if it does work...

----------

## dj_farid

Anyone knows?

----------

## MorpheuS.Ibis

have you tried telling ipp2p to mark all p2p data? and did you think about giving a single port to you windows torrent and marking that port as well? and maybe seting larger difference in tc scripts can help, if the traffic goes thru it...

EDIT: and havent the actual speed of your line drroped? ISPs do that quite often, drop your speed just a bit

----------

## dj_farid

The problem is not that the traffic is not being marked. The problem is that the prioritising does not seem to work as before.

I have changed the speed settings but still the weird prioritising...

The speed is still exactly the same.

----------

## dj_farid

I have played around trying to get things working.

I noticed that if I set the speed for the priority 2, which is rtorrent traffic, really really slow, it works like expected for a moment:

```
 tc class add dev eth0 parent 1:1 classid 1:20 htb rate 1kbit ceil 80kbit quantum 1514 prio 2
```

Then 30 seconds later the rtorrent traffic chokes my whole connection.

Is there some kind of limit of how much CONNMARK can handle?

Does rtorrent fool CONNMARK somehow that it looses track of the connection?

EDIT:

I got it working better now, but it is still bad.

I removed the line that does not mark packets that are already marked:

```
#!/bin/bash

TC='/sbin/tc'

# Clear

tc qdisc del dev eth0 root    # Clear any previous stuff

# Shaping

CEIL=904

tc qdisc add dev eth0 root handle 1: htb default 10

tc class add dev eth0 parent 1: classid 1:1 htb rate ${CEIL}kbit

tc class add dev eth0 parent 1:1 classid 1:10 htb rate 820kbit ceil ${CEIL}kbit prio 1

tc class add dev eth0 parent 1:1 classid 1:20 htb rate 8kbit ceil 850kbit quantum 1514 prio 2

tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10

tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10

##Restrict BitTorrent upload

iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark

#iptables -t mangle -A OUTPUT -m mark ! --mark 0 -j ACCEPT

iptables -t mangle -A OUTPUT -m layer7 --l7proto bittorrent -j MARK --set-mark 2

#iptables -t mangle -A OUTPUT -m ipp2p --bit -j MARK --set-mark 2

iptables -t mangle -A OUTPUT -j CONNMARK --save-mark

tc filter add dev eth0 protocol ip parent 1:0 handle 2 fw flowid 1:20

```

My conclusion is that l7-protocol stops marking packests after a while for some reason, or rtorrent is too aggressive sending packets and eventually chokes everything else.

Could also be that sfq (stochastic fairness queuing) is too fair. I will look into something less fair.

Any suggestions or thoughts?

----------

## dj_farid

I found this that probably explains what is going on on my system:

 *Quote:*   

> Unfortunately, some clever software (e.g. Kazaa and eMule among others) obliterate the benefit of this attempt at fair queuing by opening as many TCP sessions (flows) as can be sustained. In many networks, with well-behaved users, SFQ can adequately distribute the network resources to the contending flows, but other measures may be called for when obnoxious applications have invaded the network.

 

Taken from here: http://www.linux.com/howtos/Traffic-Control-HOWTO/classless-qdiscs.shtml

----------

## MorpheuS.Ibis

so....what happens if you omit the SFQ filter? does HTB work on its own?

----------

## dj_farid

 *MorpheuS.Ibis wrote:*   

> so....what happens if you omit the SFQ filter? does HTB work on its own?

 

Disaster strikes if I just remove the lines that define the SFQ.   :Shocked: 

The connection goes unusable if my upload is maxed out.

Does anyone know what happened with the ESFQ that the page I refered to earlier mentions? It does not exist in the current kernel.

It is too bad that most (all I have found) documetation on the net about traffic shaping is quite old and in many parts out-dated. There seems to be something in every guide that does not work the same way today as it did when the guide was written.

----------

## dj_farid

Well now I got things working so that I can have rtorrent running without it bothering my upload almost nothing.

It is not the best solution. I see it as a workaround until me or someone can come up with something better.

Anyways, this is how I got it now:

Changed ~/.rtorrent.rc 

```
max_open_sockets = 4
```

This makes rtorrent less aggressive. It does not download and upload as fast as before with this setting, but still better than nothing.

The script looks like this (nothing new):

```
#!/bin/bash

TC='/sbin/tc'

echo "Doing some traffic shaping"

# Clear

tc qdisc del dev eth0 root    # Clear any previous stuff

# Shaping

CEIL=904

tc qdisc add dev eth0 root handle 1: htb default 10

tc class add dev eth0 parent 1: classid 1:1 htb rate ${CEIL}kbit

tc class add dev eth0 parent 1:1 classid 1:10 htb rate 820kbit ceil ${CEIL}kbit prio 1

tc class add dev eth0 parent 1:1 classid 1:20 htb rate 8kbit ceil 850kbit quantum 1514 prio 2

tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10

tc qdisc add dev eth0 parent 1:20 handle 20: sfq perturb 10

##Restrict BitTorrent upload

iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark

#iptables -t mangle -A PREROUTING -m mark ! --mark 0 -j ACCEPT

iptables -t mangle -A OUTPUT -m layer7 --l7proto bittorrent -j MARK --set-mark 2

#iptables -t mangle -A OUTPUT -m ipp2p --bit -j MARK --set-mark 2

iptables -t mangle -A OUTPUT -j CONNMARK --save-mark

tc filter add dev eth0 protocol ip parent 1:0 handle 2 fw flowid 1:20

```

The first commented line, if uncommented will not mark already marked packets. Uncommenting this line is not good for some reason.

The second commented line is if using ipp2p instead of l7proto. I can't get it working for some reason. I suspect new versions of iptables though: https://forums.gentoo.org/viewtopic-t-519355-postdays-0-postorder-asc-start-25.html

----------

