# Home Gateway Traffic Shaping?

## brownjl

Hi,

Can anyone give me any hints on whether the following is possible and what programs I take a look at...

I gentoo box acting as a home adsl gateway for our shared student house, it has 3 interfaces, eth0 is the external interface and eth1-2 are internal. I've used IPTables to forward traffic between the internal subnets and to NAT/Masq the connections onto the external interface.

One of our house mates downloads a lot from binary newsgroups and I was hoping to do some traffic shaping such that the newsgroup flows have least priority over say HTTP or online gaming, can any one give me any hints on how I can do this??  Would it effect data flows between the two internal subnets?

Any help is greatly appricated..

Thanks

JAmes

----------

## think4urs11

have a look at

http://www.tldp.org/HOWTO/ADSL-Bandwidth-Management-HOWTO/implementation.html

or

http://lartc.org/wondershaper/

HTH

T.

----------

## Xepher

Below is my own code based upon the wondershaper script. I modified it so that there are 5 tiers of priority (instead of 3) and by DEFAULT dumps unknown ports to the lowest. I did this because my roommates always used a lot of P2P programs, many of which operated over a whole range of  ports. I also ran a small server from the firewall, and wanted it to have it to have priority over p2p stuff, but below games and normal webbrowsing/aim/ftp/etc. 

The script should be pretty self-explanatory. All you need to set it your outgoing NIC and uplink bandwidth. You may want to toy with the percentages for each category (they look like "$[5*UPLINK/20]" which is 5/20 of the total uplink speed) Note there is usually a normal speed setting and a "ceiling" which it is never allowed to go over.

Make sure you set the max uplink to slightly _less_ than your real uplink speed. The linux filtering only works on established connections. The initial handshake process needs some free space to play in before it starts being monitored by the kernel filters.

Also, you'll need to have the TC (traffic control) options enabled in the kernel, specifically the htb and sfq filters.

```
#!/bin/bash

UPLINK=300

DEV=eth0

#priority source ports

LIKEPORTSRC="123"

#priority destination ports

LIKEPORTDST="123"

#normal source ports

NORMALPORTSRC="53 14567 14690 23000 51234 8767"

#normal destination ports

NORMALPORTDST="53 80 20 21 993 14534"

#server source ports

SERVERPORTSRC="20 21 25 80 110 143 3000"

#server destination ports

SERVERPORTDST=""

# download source ports

DOWNLOADPORTSRC=""

# download destination ports

DOWNLOADPORTDST=""

# Status

if [ "$1" = "status" ]

then

   tc -s qdisc ls dev $DEV

   tc -s class ls dev $DEV

   exit

fi

# clean existing down- and uplink qdiscs, hide errors

tc qdisc del dev $DEV root    2> /dev/null > /dev/null

tc qdisc del dev $DEV ingress 2> /dev/null > /dev/null

if [ "$1" = "stop" ] 

then 

   exit

fi

###### uplink

# install root HTB, point default traffic to 1:50:

tc qdisc add dev $DEV root handle 1: htb default 50

# shape everything at $UPLINK speed - this prevents huge queues in your

# DSL modem which destroy latency:

tc class add dev $DEV parent 1: classid 1:1 htb rate ${UPLINK}kbit 

# mission critical class 1:10:

tc class add dev $DEV parent 1:1 classid 1:10 htb rate $[5*UPLINK/20]kbit \

   ceil $[UPLINK]kbit prio 1 

# normal class 1:20 

tc class add dev $DEV parent 1:1 classid 1:20 htb rate $[10*UPLINK/20]kbit \

   ceil $[18*UPLINK/20]kbit prio 2 

# server class 1:30

tc class add dev $DEV parent 1:1 classid 1:30 htb rate $[3*UPLINK/20]kbit \

   ceil $[18*UPLINK/20]kbit prio 3 

# download class 1:40

tc class add dev $DEV parent 1:1 classid 1:40 htb rate $[1*UPLINK/20]kbit \

   ceil $[5*UPLINK/20]kbit prio 4 

# hate class 1:50

tc class add dev $DEV parent 1:1 classid 1:50 htb rate 1kbit \

   prio 5 ceil $[1*UPLINK/20]kbit 

# all get Stochastic Fairness:

tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10

tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10

tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10

tc qdisc add dev $DEV parent 1:40 handle 40: sfq perturb 10

tc qdisc add dev $DEV parent 1:50 handle 50: sfq perturb 10

# TOS Minimum Delay (ssh, NOT scp) in 1:10:

tc filter add dev $DEV parent 1:0 protocol ip prio 1 u32 \

      match ip tos 0x10 0xff  flowid 1:10

# ICMP (ip protocol 1) in the interactive class 1:10 so we 

# can do measurements & impress our friends:

tc filter add dev $DEV parent 1:0 protocol ip prio 1 u32 \

        match ip protocol 1 0xff flowid 1:10

# To speed up downloads while an upload is going on, put ACK packets in

# the interactive class:

tc filter add dev $DEV parent 1: protocol ip prio 1 u32 \

   match ip protocol 6 0xff \

   match u8 0x05 0x0f at 0 \

   match u16 0x0000 0xffc0 at 2 \

   match u8 0x10 0xff at 33 \

   flowid 1:20

#TCP

#traffic we like

for a in $LIKEPORTSRC

do

tc filter add dev $DEV parent 1: protocol ip prio 14 u32 \

           match ip sport $a 0xffff flowid 1:10

done

for a in $LIKEPORTDST

do

tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \

           match ip dport $a 0xffff flowid 1:10

done

#normal traffic

for a in $NORMALPORTSRC

do

tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \

           match ip sport $a 0xffff flowid 1:20

done

for a in $NORMALPORTDST

do

tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \

           match ip dport $a 0xffff flowid 1:30

done

# Server traffic

for a in $SERVERPORTSRC

do

tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \

           match ip sport $a 0xffff flowid 1:30

done

for a in $SERVERPORTDST

do

tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \

           match ip dport $a 0xffff flowid 1:30

done

# some traffic however suffers a worse fate

for a in $DOWNLOADPORTDST

do

tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \

           match ip dport $a 0xffff flowid 1:40

done

for a in $DOWNLOADPORTSRC

do

tc filter add dev $DEV parent 1: protocol ip prio 15 u32 \

           match ip sport $a 0xffff flowid 1:40

done

# rest is 'non-interactive' ie 'bulk' and ends up in 1:50
```

----------

## mekong

Wondershaper is nice but it doesn't shape downlink, only uplink. In his situation, his house mate download alot from newsgroup. I would suggest to use IMQ patch from http://www.linuximq.net/ and modify the wondershaper script a bit.

----------

## brownjl

Thanks for you help so far, I had a go with Xepher script and that seemed to work but as mekong said IMQ maybe more suitable since it also effects the downlink..

I gave IMQ a go Patched the gentoo-sources kernel and iptables however when I modprobe imq I get the following error

```
/lib/modules/2.4.25-gentoo-r1/kernel/drivers/net/imq.o: init_module: Device or resource busy

Hint: insmod errors can be caused by incorrect module parameters, including invalid IO or IRQ parameters.

      You may find more information in syslog or the output from dmesg

/lib/modules/2.4.25-gentoo-r1/kernel/drivers/net/imq.o: insmod /lib/modules/2.4.25-gentoo-r1/kernel/drivers/net/imq.o failed

/lib/modules/2.4.25-gentoo-r1/kernel/drivers/net/imq.o: insmod imq failed
```

I have had a google but cant seem to find anyone with a similar error can anyone help?

James

----------

## brownjl

Solved!!!!

I discoved that the current IMQ needs to be compiled into the Kernel and not as a module.

I found the script here for traffic shaping most useful,

http://www.tldp.org/HOWTO/ADSL-Bandwidth-Management-HOWTO/implementation.html

I just altered it to suit my needs, I found that I had to Compile into the kernel the two IMQ modules RED, SFQ and HTB amongst other to make this work.

```
#!/bin/bash

#

DEV=eth0

RATEUP=230

RATEDN=475

# 

# End Configuration Options

#

if [ "$1" = "status" ]

then

        echo "[qdisc]"

        tc -s qdisc show dev $DEV

        tc -s qdisc show dev imq0

        echo "[class]"

        tc -s class show dev $DEV

        tc -s class show dev imq0

        echo "[filter]"

        tc -s filter show dev $DEV

        tc -s filter show dev imq0

        echo "[iptables]"

        iptables -t mangle -L MYSHAPER-OUT -v -x 2> /dev/null

        iptables -t mangle -L MYSHAPER-IN -v -x 2> /dev/null

        exit

fi

# Reset everything to a known state (cleared)

tc qdisc del dev $DEV root    2> /dev/null > /dev/null

tc qdisc del dev imq0 root 2> /dev/null > /dev/null

iptables -t mangle -D POSTROUTING -o $DEV -j MYSHAPER-OUT 2> /dev/null > /dev/null

iptables -t mangle -F MYSHAPER-OUT 2> /dev/null > /dev/null

iptables -t mangle -X MYSHAPER-OUT 2> /dev/null > /dev/null

iptables -t mangle -D PREROUTING -i $DEV -j MYSHAPER-IN 2> /dev/null > /dev/null

iptables -t mangle -F MYSHAPER-IN 2> /dev/null > /dev/null

iptables -t mangle -X MYSHAPER-IN 2> /dev/null > /dev/null

ip link set imq0 down 2> /dev/null > /dev/null

#rmmod imq 2> /dev/null > /dev/null

if [ "$1" = "stop" ] 

then 

        echo "Shaping removed on $DEV."

        exit

fi

###########################################################

#

# Outbound Shaping (limits total bandwidth to RATEUP)

# set queue size to give latency of about 2 seconds on low-prio packets

ip link set dev $DEV qlen 30

# changes mtu on the outbound device.  Lowering the mtu will result

# in lower latency but will also cause slightly lower throughput due 

# to IP and TCP protocol overhead.

ip link set dev $DEV mtu 1000

# add HTB root qdisc

tc qdisc add dev $DEV root handle 1: htb default 26

# add main rate limit classes

tc class add dev $DEV parent 1: classid 1:1 htb rate ${RATEUP}kbit

# add leaf classes - We grant each class at LEAST it's "fair share" of bandwidth.

#                    this way no class will ever be starved by another class.  Each

#                    class is also permitted to consume all of the available bandwidth

#                    if no other classes are in use.

tc class add dev $DEV parent 1:1 classid 1:20 htb rate $[$RATEUP/7]kbit ceil ${RATEUP}kbit prio 0

tc class add dev $DEV parent 1:1 classid 1:21 htb rate $[$RATEUP/7]kbit ceil ${RATEUP}kbit prio 1

tc class add dev $DEV parent 1:1 classid 1:22 htb rate $[$RATEUP/7]kbit ceil ${RATEUP}kbit prio 2

tc class add dev $DEV parent 1:1 classid 1:23 htb rate $[$RATEUP/7]kbit ceil ${RATEUP}kbit prio 3

tc class add dev $DEV parent 1:1 classid 1:24 htb rate $[$RATEUP/7]kbit ceil ${RATEUP}kbit prio 4

tc class add dev $DEV parent 1:1 classid 1:25 htb rate $[$RATEUP/7]kbit ceil ${RATEUP}kbit prio 5

tc class add dev $DEV parent 1:1 classid 1:26 htb rate $[$RATEUP/7]kbit ceil ${RATEUP}kbit prio 6

# attach qdisc to leaf classes - here we at SFQ to each priority class.  SFQ insures that

#                                within each class connections will be treated (almost) fairly.

tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10

tc qdisc add dev $DEV parent 1:21 handle 21: sfq perturb 10

tc qdisc add dev $DEV parent 1:22 handle 22: sfq perturb 10

tc qdisc add dev $DEV parent 1:23 handle 23: sfq perturb 10

tc qdisc add dev $DEV parent 1:24 handle 24: sfq perturb 10

tc qdisc add dev $DEV parent 1:25 handle 25: sfq perturb 10

tc qdisc add dev $DEV parent 1:26 handle 26: sfq perturb 10

# filter traffic into classes by fwmark - here we direct traffic into priority class according to

#                                         the fwmark set on the packet (we set fwmark with iptables

#                                         later).  Note that above we've set the default priority

#                                         class to 1:26 so unmarked packets (or packets marked with

#                                         unfamiliar IDs) will be defaulted to the lowest priority

#                                         class.

tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 20 fw flowid 1:20

tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 21 fw flowid 1:21

tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 22 fw flowid 1:22

tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 23 fw flowid 1:23

tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 24 fw flowid 1:24

tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 25 fw flowid 1:25

tc filter add dev $DEV parent 1:0 prio 0 protocol ip handle 26 fw flowid 1:26

# add MYSHAPER-OUT chain to the mangle table in iptables - this sets up the table we'll use

#                                                      to filter and mark packets.

iptables -t mangle -N MYSHAPER-OUT

iptables -t mangle -I POSTROUTING -o $DEV -j MYSHAPER-OUT

# add fwmark entries to classify different types of traffic - Set fwmark from 20-26 according to

#                                                             desired class. 20 is highest prio.

iptables -t mangle -A MYSHAPER-OUT -p udp -j MARK --set-mark 20                # UDP - DNS etc

iptables -t mangle -A MYSHAPER-OUT -p icmp -j MARK --set-mark 20               # ICMP (ping)

iptables -t mangle -A MYSHAPER-OUT -p tcp --dport 3389 -j MARK --set-mark 21   # RDP

iptables -t mangle -A MYSHAPER-OUT -p tcp --dport 5389 -j MARK --set-mark 21   # RDP

iptables -t mangle -A MYSHAPER-OUT -p tcp -m length --length :64 -j MARK --set-mark 21 # small packets (probably just ACKs)

iptables -t mangle -A MYSHAPER-OUT -p tcp --dport ssh -j MARK --set-mark 22    # secure shell

iptables -t mangle -A MYSHAPER-OUT -p tcp --sport ssh -j MARK --set-mark 22    # secure shell

iptables -t mangle -A MYSHAPER-OUT -p tcp --dport telnet -j MARK --set-mark 22 # telnet

iptables -t mangle -A MYSHAPER-OUT -p tcp --sport telnet -j MARK --set-mark 22 # telnet 

iptables -t mangle -A MYSHAPER-OUT -p tcp --dport 1863 -j MARK --set-mark 24   # MSN

iptables -t mangle -A MYSHAPER-OUT -p udp --dport 1863 -j MARK --set-mark 24   # MSN      

iptables -t mangle -A MYSHAPER-OUT -p tcp --dport http -j MARK --set-mark 23   # HTTP

iptables -t mangle -A MYSHAPER-OUT -p tcp --dport 110 -j MARK --set-mark 23    # Pop3 Email

iptables -t mangle -A MYSHAPER-OUT -p tcp --dport 119 -j MARK --set-mark 26    # News Port  

iptables -t mangle -A MYSHAPER-OUT -p tcp --dport 20 -j MARK --set-mark 26     # ftp-data port, low prio

iptables -t mangle -A MYSHAPER-OUT -m mark --mark 0 -j MARK --set-mark 25      # redundant- mark any unmarked packets as 25

# Done with outbound shaping

#

####################################################

echo "Outbound shaping added to $DEV.  Rate: ${RATEUP}Kbit/sec."

####################################################

#

# Inbound Shaping (limits total bandwidth to RATEDN)

ip link set imq0 up

# add qdisc - default low-prio class 1:21

tc qdisc add dev imq0 handle 1: root htb default 21

# add main rate limit classes

tc class add dev imq0 parent 1: classid 1:1 htb rate ${RATEDN}kbit

# add leaf classes - TCP traffic in 21, non TCP traffic in 20

#

tc class add dev imq0 parent 1:1 classid 1:20 htb rate $[$RATEDN/2]kbit ceil ${RATEDN}kbit prio 0

tc class add dev imq0 parent 1:1 classid 1:21 htb rate $[$RATEDN/2]kbit ceil ${RATEDN}kbit prio 1

# attach qdisc to leaf classes - here we at SFQ to each priority class.  SFQ insures that

#                                within each class connections will be treated (almost) fairly.

tc qdisc add dev imq0 parent 1:20 handle 20: sfq perturb 10

tc qdisc add dev imq0 parent 1:21 handle 21: red limit 1000000 min 5000 max 100000 avpkt 1000 burst 50

# filter traffic into classes by fwmark 

tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 20 fw flowid 1:20

tc filter add dev imq0 parent 1:0 prio 0 protocol ip handle 21 fw flowid 1:21

# add MYSHAPER-IN chain to the mangle table in iptables - this sets up the table we'll use

#                                                         to filter and mark packets.

iptables -t mangle -N MYSHAPER-IN

iptables -t mangle -I PREROUTING -i $DEV -j MYSHAPER-IN

# add fwmark entries to classify different types of traffic - Set fwmark from 20-21 according to

#    desired class. 20 is highest prio.

iptables -t mangle -A MYSHAPER-IN -p ! tcp -j MARK --set-mark 20              # Set non-tcp packets to highest priority

iptables -t mangle -A MYSHAPER-IN -p tcp -m length --length :64 -j MARK --set-mark 20 # short TCP packets are probably ACKs

iptables -t mangle -A MYSHAPER-IN -p tcp --dport ssh -j MARK --set-mark 20    # secure shell

iptables -t mangle -A MYSHAPER-IN -p tcp --sport ssh -j MARK --set-mark 20    # secure shell

iptables -t mangle -A MYSHAPER-IN -p tcp --dport telnet -j MARK --set-mark 20 # telnet 

iptables -t mangle -A MYSHAPER-IN -p tcp --sport telnet -j MARK --set-mark 20 # telnet

iptables -t mangle -A MYSHAPER-IN -m mark --mark 0 -j MARK --set-mark 21              

# finally, instruct these packets to go through the imq0 we set up above

iptables -t mangle -A MYSHAPER-IN -j IMQ

# Done with inbound shaping 

#

####################################################

echo "Inbound shaping added to $DEV.  Rate: ${RATEDN}Kbit/sec."

```

Thanks 

James

----------

## Xepher

For the record, downlink shaping is a rather ugly hack. All you can do is drop packets if they come in "too fast" and hope that your ISP is smart enough to start shaping/slowing things at their end. Mine never was and I just ended up with a still-flooded downstream and a lot of resent packets, making the overall problem _worse._ The best kludge I could find was to get an estimate of the tcp/ip uplink overhead for a download... every connection has to send ACK packets before more downstream data gets sent. If you can figure out the overhead to a reasonable accuracy, you can shape the outgoing ACK packets, and that will effectively slow downstream as well. In other words slow down how fast your roommie's computer can say "I got it!" and the overall transfer will slow down.

----------

## flakeme

My situation:

3 win xp pc's

1 linux gateway

adsl 800kbit down 128kbit up

My (completly normal) problem:

If I play a latancy critical game (like cs) and someone else opens a web page my ping goes up to 600 shortly.

If someone starts a dl the ping stays at 600.

What i have tried:

reading lartc.org documentation but it's really much and I don't understand everything to the last bit.

Using http://lartc.org/wondershaper/ with

# low priority source ports

NOPRIOPORTSRC=80

# low priority destination ports

NOPRIOPORTDST=80

Why it doesn't satisfy me:

I have to limit overall down bandwith to something like 500kbit loosing 40% of dl speed, to be able to play, BUT still having little lags.

More worse are the short bursts when someone dl AND someone opens a webpage, causing short ms jumps to 250.

What I think about the problem:

my "game traffic" is also shaped and sqeezed into the 500kbit limit.

My theoretical solutions:

||Number one:||

If packets from source port 80 are comming in and bandwith limit of 70kbyte/s is reached hold back ACK packets with destination port 80.

||Number two:||

dynamic bandwith partioning just looking at IP layer ignoring ports.

Excample in my LAN setting:

Fairly devided every pc would get ~27kb bandwith

At pc1 I am playing using 5kb

27 -> 5

27 -> 0

27 -> 0

then on pc2 someone starts dl with 50kb

27 -> 5

27 -> 50

27 -> 0

as a result the shaper grants pc2 more bandwith because pc3 uses nothing of it

27 -> 5

50 -> 50

4 -> 0

(you could also take some of pc1 but thats not the point now)

BUT suddenly pc3 also wants to dl with 10kb

27 -> 5

50 -> 50

3 -> 10 <-- now there are packets dropped

As a result all bandwiths over 27kb are limited back

27 -> 5

43 -> 40

10 -> 10

and everyone lived happily ever after...

Number one MUST be possible to realise, but i dont know how (for gods sacke just give me a script full of tc commands, i can't write it myself (lazy+dumb))

Number two is a dream, and probably thats why I never found something lilke that.

If some one finds or knows someone who knows someone who.... has something like that. gimmigimmi

----------

## drkstorm

i am very much interested in limiting d/l and u/l rates, i have dsl and whenever someone is downloading a file, my games lag out, i have a script that "marks" packets using iptables.

What can i use to "Read" those marks and limit packets accordingly?

here is the relevant part of the script:

```

# mark "mail/news" traffic to "3"

MAIL="25 110 119 143 993 995"

for marked in $MAIL

        do

        $IPTABLES -A PREROUTING -i $EXTERNAL -t mangle -p tcp --dport $marked -j MARK --set-mark 3

        $IPTABLES -A PREROUTING -i $EXTERNAL -t mangle -p udp --dport $marked -j MARK --set-mark 3

        done

# mark "interactive" traffic to "4"

LOGIN="22 23 6667"

for interactive in $LOGIN

        do

        $IPTABLES -A PREROUTING -i $EXTERNAL -t mangle -p tcp --dport $interactive -j MARK --set-mark 4

        $IPTABLES -A PREROUTING -i $EXTERNAL -t mangle -p udp --dport $interactive -j MARK --set-mark 4

        done

# mark "web" traffic to "2"

WEB="80 443"

for web in $WEB

        do

        $IPTABLES -A PREROUTING -i $EXTERNAL -t mangle -p tcp --dport $web -j MARK --set-mark 2

        $IPTABLES -A PREROUTING -i $EXTERNAL -t mangle -p udp --dport $web -j MARK --set-mark 2

done

# mark "game" traffic to "1"

GAMES="27910:27980 7777 22450 26000 26950 27015 27020 27500 28000:28008 28910"

for games in $GAMES

        do

        $IPTABLES -A PREROUTING -i $EXTERNAL -t mangle -p tcp --dport $games -j MARK --set-mark 1

        $IPTABLES -A PREROUTING -i $EXTERNAL -t mangle -p udp --dport $games -j MARK --set-mark 1

        done

```

----------

