# Traffic shaping for P2P

## mani001

Hi,

I'm trying to set up some traffic shaping in my Linux box (mostly, set low priority for P2P)...and geez...you need a Ph.D. or something to get this working   :Confused:   I tried:

```

TC=$(which tc)

# device

DEVICE=eth0

TOP_PRIORITY_CLASS="1:10"

NORMAL_PRIORITY_CLASS="1:20"

LOW_PRIORITY_CLASS="1:30"

$TC qdisc del dev $DEVICE root 2> /dev/null > /dev/null

$TC qdisc add dev $DEVICE root       handle 1:    htb default 20

$TC class add dev $DEVICE parent 1:  classid 1:1  htb rate 1000kbit

$TC class add dev $DEVICE parent 1:1 classid $TOP_PRIORITY_CLASS htb rate 700kbit #

$TC class add dev $DEVICE parent 1:1 classid $NORMAL_PRIORITY_CLASS htb rate 200kbit #

$TC class add dev $DEVICE parent 1:1 classid $LOW_PRIORITY_CLASS htb rate 100kbit #

```

In order to mark the packets I used iptables:

```

root@$iptables -t mangle -L -v -x 

Chain PREROUTING (policy ACCEPT 105175 packets, 52173770 bytes)                                                                                                                                                                              

    pkts      bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 105175 packets, 52173770 bytes)

    pkts      bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)

    pkts      bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 165585 packets, 95803752 bytes)

    pkts      bytes target     prot opt in     out     source               destination         

      10     1932 CLASSIFY   icmp --  any    any     anywhere             anywhere             CLASSIFY set 1:10

       0        0 CLASSIFY   tcp  --  any    any     anywhere             anywhere             tcp dpt:ssh CLASSIFY set 1:10

    8726   771489 CLASSIFY   tcp  --  any    any     anywhere             anywhere             tcp dpt:http CLASSIFY set 1:20

    1403   259528 CLASSIFY   tcp  --  any    any     anywhere             anywhere             tcp dpt:https CLASSIFY set 1:20

       0        0 CLASSIFY   tcp  --  any    any     anywhere             anywhere             tcp dpt:smtp CLASSIFY set 1:20

  152542 94603213 CLASSIFY   all  --  any    any     anywhere             anywhere             owner UID match transmission CLASSIFY set 1:30

  165585 95803752 CLASSIFY   all  --  any    any     anywhere             anywhere             CLASSIFY set 1:20

Chain POSTROUTING (policy ACCEPT 165585 packets, 95803752 bytes)

    pkts      bytes target     prot opt in     out     source               destination         

root@ $

```

Of course, it doesn't work and if I run the corresponding script the network starts going terribly slow until I run "tc qdisc del dev eth0 root"...

Is there something fundamentally wrong with this setup?

Greetings!!

----------

## PaulBredbury

 *mani001 wrote:*   

> htb rate 100kbit

 

If your network is otherwise idle, then you're potentially wasting 90% of it (assuming 1,000 kbit max)! If I understand that right, anyway.

Here's what I use.

----------

## mani001

Hi PaulBredbury,

...and thank you very much for your reply

 *Quote:*   

> 
> 
> If your network is otherwise idle, then you're potentially wasting 90% of it (assuming 1,000 kbit max)! If I understand that right, anyway. 
> 
> 

 

According to

http://wiki.openwrt.org/doc/howto/packet.scheduler/packet.scheduler.example2

the bandwitdth one class doesn't use should be distributed among the others...but it doesn't seem to work that way   :Confused: 

I have a couple of questions regarding your script:

- what is the point of

```

iptables -t mangle -A POSTROUTING -o $iface -p tcp --tcp-flags ALL ACK -m length --length \ 

            $[($i+1)*($i+1)*10]:$[($i+2)*($i+2)*10-1] -j CLASSIFY --set-class 1:$[$i*10] 

```

? Does that check the size of ACK (and only ACK) packets? Aren't they supposed to all have the same size? (I'm just wondering...I don't know that much about TCP/IP)

- The

```

# time-critical traffic 

CLA=10 

iptables -t mangle -A POSTROUTING -o $iface -p tcp --tcp-flags ALL FIN,ACK -j CLASSIFY --set-class 1:$CLA 

iptables -t mangle -A POSTROUTING -o $iface -p tcp --tcp-flags ALL SYN,ACK -j CLASSIFY --set-class 1:$CLA 

iptables -t mangle -A POSTROUTING -o $iface -p tcp --tcp-flags ALL RST,ACK -j CLASSIFY --set-class 1:$CLA 

iptables -t mangle -A POSTROUTING -o $iface -p tcp --tcp-flags ALL RST -j CLASSIFY --set-class 1:$CLA 

iptables -t mangle -A POSTROUTING -o $iface -p tcp --syn -j CLASSIFY --set-class 1:$CLA 

iptables -t mangle -A POSTROUTING -o $iface -p udp -j CLASSIFY --set-class 1:$CLA 

```

part assigns a high priority to the ACK, SYN and stuff like that packets regardless of the application. Then if I intend to set a low priority for P2P I should skip this part because I want the ACK packets coming from P2P to have a low priority, am I right?

- Why do you "mangle" POSTROUTING? I'm "mangling" OUTPUT. Does that matter?

From you script, I made a new customed script that looks like this

```

#!/bin/bash

IPT=$(which iptables)

TC=$(which tc)

# device

DEVICE=eth0

MAX_RATE="1000"

SSH_PORT=22

HTTP_PORT=80

HTTPS_PORT=443

SMTP_PORT=25

# clear the mangle iptables rules

iptables -t mangle -F 2> /dev/null > /dev/null

# return the traffic control to default

$TC qdisc del dev $DEVICE root 2> /dev/null > /dev/null

$TC qdisc add dev $DEVICE root       handle 1:    htb default 30

$TC class add dev $DEVICE parent 1:  classid 1:1  htb rate ${MAX_RATE}kbit

CLASSES=(10 20 30)

for i in "${CLASSES[@]}" ; do

   tc class add dev $DEVICE parent 1:1 classid 1:${i} htb rate $[$MAX_RATE/${#CLASSES[@]}]kbit ceil ${MAX_RATE}kbit prio $[$i/10]

   tc qdisc add dev $DEVICE parent 1:${i} handle ${i}: sfq perturb 10

done 

# ssh

iptables -t mangle -A OUTPUT -p tcp --dport $SSH_PORT -j CLASSIFY --set-class 1:${CLASSES[0]}

# http

iptables -t mangle -A OUTPUT -p tcp --dport $HTTP_PORT -j CLASSIFY --set-class 1:${CLASSES[1]}

# https

iptables -t mangle -A OUTPUT -p tcp --dport $HTTPS_PORT -j CLASSIFY --set-class 1:${CLASSES[1]}

# smtp

iptables -t mangle -A OUTPUT -p tcp --dport $SMTP_PORT -j CLASSIFY --set-class 1:${CLASSES[1]}

# icmp

iptables -t mangle -A OUTPUT -p icmp -j CLASSIFY --set-class 1:${CLASSES[2]}

# transmission

iptables -t mangle -A OUTPUT -m owner --uid-owner transmission -j CLASSIFY --set-class 1:${CLASSES[2]}

# remaining packets...

iptables -t mangle -A OUTPUT -j CLASSIFY --set-class 1:${CLASSES[2]}

```

Even though it's working better than before, it still alows transmission (P2P) to choke my connection    :Mad: 

Greetings.

----------

## PaulBredbury

 *mani001 wrote:*   

> Does that check the size of ACK (and only ACK) packets? Aren't they supposed to all have the same size?

 

Good question! A quick googling shows that this is probably more reasonable, and seems common:

```
--length 0:128
```

You can probably notice that I'm not properly answering, because I haven't googled sufficiently yet  :Smile: 

 *Quote:*   

> if I intend to set a low priority for P2P I should skip this part

 

Don't skip it. At least, not without trying it first. I don't think it has that much of an effect, only being the handshaking.

 *Quote:*   

> I'm "mangling" OUTPUT. Does that matter?

 

It's what everyone seems to use. OUTPUT is from the way-old days of kernel 2.4, when POSTROUTING wasn't available. Depends on what other rules are set up, of course.

 *Quote:*   

> # remaining packets...
> 
> iptables -t mangle -A OUTPUT -j CLASSIFY --set-class 1:${CLASSES[2]}

 

That's the same class as with your transmission entry. Transmission should be lowest-priority.

 *Quote:*   

> choke my connection

 

You *must* set MAX_RATE to be lower than your output speed, otherwise the whole thing won't work. Try 800 to be sure, then 900.

----------

## Hu

 *mani001 wrote:*   

> ? Does that check the size of ACK (and only ACK) packets? Aren't they supposed to all have the same size? (I'm just wondering...I don't know that much about TCP/IP)

 No.  ACK is a flag in the packet header.  It is possible for the peer to send you a single packet which is both an acknowledgement of past data you sent to it and a payload of new data from the remote application to you.  When circumstances allow this to be done, this reduces the number of packets on the network, which is a Good Thing.  As to why the rule tests the size, I suspect it is so that a locally generated packet which is both ACK+large data is shaped just the way plain large data would be shaped, but a locally generated packet which is only an ACK or is an ACK+small data is expedited.  If you do not send ACKs in a timely manner, your downloads will stall because the remote peer thinks you have fallen behind.  This is a common cause for large uploads causing your download performance to suffer.  The large data packets from your upload cause the small ACK packets to be delayed, which in turn tricks the remote peer into sending you far less traffic than you can handle.

----------

## mani001

I tried setting MAX_RATE to 400 (and I know for sure that my upload speed is greater than 400kbps  :Smile:  ) and it doesn't make a difference (transmission gets to upload at maximum speed and chromium has to wait a few seconds to get a response from a web server). I also tried "mangling" POSTROUTING instead of "OUTPUT"...and nothing changed...so, there is still probably something wrong.

Hu, you convinced me about the ACK stuff (I even think there was a time when I knew that...there was a time  :Smile:  ) but still it didn't help.

I'll tinker with this a little bit more...I just hope I don't have to read the the whole "official" HOWTO

http://lartc.org/lartc.html#LARTC.QDISC

 :Shocked: 

----------

## PaulBredbury

This is fascinating stuff. I'm editing my script a lot, right now.

Perhaps you need e.g.:

```
iptables -t mangle -A POSTROUTING -o eth0 -m conntrack --ctorigdstport 80 -j CLASSIFY --set-class 1:20

iptables -t mangle -A POSTROUTING -o eth0 -m conntrack --ctorigdstport 443 -j CLASSIFY --set-class 1:20
```

Since 80 = HTTP, and 443 = HTTPS.

As you mention, this is great, to check:

```
watch "iptables -t mangle -L -v"
```

----------

## PaulBredbury

This is what I use now:

```
   # Flush existing rules

   iptables -F -t mangle

   for iface in eth0 ppp0 wlan0 ; do

      mgl="iptables -t mangle -A POSTROUTING -o $iface"

      if [[ -e /sys/class/net/$iface ]] ; then

         MAX=900

         if [[ $iface == ppp0 ]] ; then MAX=33 ; fi

         if [[ $iface == wlan0 ]] ; then MAX=5000 ; fi

         tc qdisc del dev $iface root 2>/dev/null

         tc qdisc add dev $iface root handle 1: htb default 70

         tc class add dev $iface parent 1: classid 1:1 htb rate ${MAX}kbit

         for i in {1..7} ; do

            tc class add dev $iface parent 1:1 classid 1:$[$i*10] htb rate $[$MAX/7]kbit ceil ${MAX}kbit prio $[$i-1]

            tc qdisc add dev $iface parent 1:$[$i*10] handle $[$i*10]: pfifo

         done

         # Finish classifying

         # http://lartc.org/howto/lartc.cookbook.fullnat.intro.html

         ret="-j RETURN"

         # My HTTP server - 10

         cls="-j CLASSIFY --set-class 1:10"

         # 40 is www group

         $mgl -m owner --gid-owner 40 $cls

         $mgl -m owner --gid-owner 40 $ret

         # Very important, but with low bandwidth requirement - 20

         cls="-j CLASSIFY --set-class 1:20"

         # Jabber

         $mgl -m conntrack --ctorigdstport 5222 $cls

         $mgl -m conntrack --ctorigdstport 5222 $ret

         # 35 = bind group, for DNS, in /etc/group

         $mgl -m owner --gid-owner 35 $cls

         $mgl -m owner --gid-owner 35 $ret

         # Time-critical traffic - 30

         cls="-j CLASSIFY --set-class 1:30"

         est="-m state --state ESTABLISHED"

         $mgl -m length --length 0:68 $est $cls

         $mgl -m length --length 0:68 $est $ret

         # High-priority interactive traffic - 40

         cls="-j CLASSIFY --set-class 1:40"

         # 2150 is my SSH port. 22 = normal SSH. 123 = NTP. 53 = DNS.

         $mgl -p tcp -m multiport --dport 22,2150 $cls

         $mgl -p tcp -m multiport --dport 22,2150 $ret

         $mgl -p tcp -m multiport --sport 22,2150 $cls

         $mgl -p tcp -m multiport --sport 22,2150 $ret

         # 44 = ntp group, 24 = ntp user - but matching on them doesn't work! Maybe ntpd doesn't drop permissions fast enough?

         #$mgl -m owner --uid-owner 24 -j CLASSIFY --set-class 1:40

         $mgl -m conntrack --ctorigdstport 123 $cls

         $mgl -m conntrack --ctorigdstport 123 $ret

         # Low-priority interactive traffic - 50

         cls="-j CLASSIFY --set-class 1:50"

         for p in 80 443 25 110 20 21 194 ; do

            $mgl -m conntrack --ctorigdstport $p $cls

            $mgl -m conntrack --ctorigdstport $p $ret

         done

         # Non-critical traffic - 60

         cls="-j CLASSIFY --set-class 1:60"

         $mgl -p icmp $cls

         $mgl -p icmp $ret

         # Default - 70 - don't add entries here

         # Is default anyway. Specified here just so it can be seen in the iptables stats.

         cls="-j CLASSIFY --set-class 1:70"

         $mgl $cls

      fi

   done

fi
```

So no need for --tcp-flags checking, I reckon.

Edit: Added "-j RETURN", and changed 64 to the more common 68.

Edit2: Removed optimization for packet length 69-128 - it doesn't help.

Edit3: Switched from sfq perturb 10 to pfifo.Last edited by PaulBredbury on Thu Jun 21, 2012 4:14 am; edited 4 times in total

----------

## mani001

Thanks for sharing. I'll try to draw inspiration from your script   :Very Happy:   I'm still struggling with this thing...for me it seems that no traffic control at all works better than anything else   :Shocked: 

----------

## PaulBredbury

Move your priority traffic to have a higher priority.

There's no perfect solution.

----------

## Mad Merlin

I have a silly suggestion, but do you have control over the client? If so, there are many BitTorrent clients which can rate limit themselves (if Transmission cannot, try rtorrent).

----------

## PaulBredbury

 *Mad Merlin wrote:*   

> rate limit

 

We don't want an artificial speed limit.

----------

## danielhilst

PaulBredbury, I see that yu use sfq instead of default pfifo.. why?

I'm not at home now, so I will try your config as soon as I get home.. 

```
tc qdisc add dev $iface parent 1:$[$i*10] handle $[$i*10]: sfq perturb 10 
```

Cheers!

----------

## PaulBredbury

Only because it was in most of the docs  :Confused: 

I'm trying pfifo instead of sfq perturb 10 now. Seems OK.

----------

## PaulBredbury

This is much neater, creating new mangle chains:

```
# Flush existing rules

iptables -t mangle -F

iptables -t mangle -X

imgl="iptables -t mangle"

cls="-j CLASSIFY --set-class"

acc="-j ACCEPT"

# Create new chains

p=10

for c in server dns interactive other torrent ; do

   $imgl -N $c

   $imgl -A $c $cls 1:${p}

   $imgl -A $c $acc

   p=$(($p+10))

done

iface=eth0

# This MUST be less than your upstream speed, otherwise it's all pointless

MAX=900

mgl="iptables -t mangle -A POSTROUTING -o $iface"

tc qdisc del dev $iface root 2>/dev/null

tc qdisc add dev $iface root handle 1: htb default 40

tc class add dev $iface parent 1: classid 1:1 htb rate ${MAX}kbit

# prio 5 is for torrents, gets serviced last

# Based on Arno's firewall script: 60traffic-shaper.plugin

# http://rocky.eld.leidenuniv.nl/joomla/

b="1452"  # Rough guess, under MTU

# IMPORTANT: The rate sums up to exactly $MAX

tc class add dev $iface parent 1:1 classid 1:10 htb ceil ${MAX}kbit rate 300kbit burst $b prio 1

tc class add dev $iface parent 1:1 classid 1:20 htb ceil ${MAX}kbit rate 300kbit burst $b prio 2

tc class add dev $iface parent 1:1 classid 1:30 htb ceil ${MAX}kbit rate 100kbit burst $b prio 3

tc class add dev $iface parent 1:1 classid 1:40 htb ceil ${MAX}kbit rate 100kbit prio 4

tc class add dev $iface parent 1:1 classid 1:50 htb ceil ${MAX}kbit rate 100kbit prio 5

tc qdisc add dev $iface parent 1:10 handle 10: pfifo

tc qdisc add dev $iface parent 1:20 handle 20: pfifo

tc qdisc add dev $iface parent 1:30 handle 30: pfifo

tc qdisc add dev $iface parent 1:40 handle 40: pfifo

tc qdisc add dev $iface parent 1:50 handle 50: pfifo

# http://lartc.org/howto/lartc.cookbook.fullnat.intro.html

ret="-j RETURN"

est="-m conntrack --ctstate ESTABLISHED"

# BitTorrent

cls="-j torrent"

# For when running transmission as separate user

$mgl -m owner --uid-owner dl $cls

# For when running transmission as same user but "torrent" group

$mgl -m owner --gid-owner torrent $cls

# Steam

$mgl -m owner --gid-owner steam -j other

# Server

$mgl -m owner --gid-owner www -j server

$mgl -m owner --gid-owner ddclient -j server

$mgl -m owner --gid-owner bind -j dns

$mgl -m owner --gid-owner skype -j interactive

$mgl -m owner --gid-owner pidgin -j interactive

# Dropbox

$mgl -m owner --gid-owner dropbox -j other

$mgl -m iprange --dst-range 199.47.216.0-199.47.219.255 -j other

# Interactive traffic

cls="-j interactive"

$mgl -m owner --uid-owner privoxy $cls

for p in 80 443 25 110 20 21 194 ; do

   $mgl -m conntrack --ctorigdstport $p $cls

done

cls="-j other"

# Specified, so it appears in the iptables stats

$mgl $cls
```

Also see iptables matching on group, a nice trick for matching on apps.

Edit: Added "burst", based on Arno's firewall script.

----------

