# To traffic shaping gurus: Tweaking ingress policing?

## bigfunkymo

I've made a traffic shaping script that is based on the wondershaper for my home internet's WAN interface of my router.  In a nutshell, it shapes upstream traffic into 3 HTB classes based on the DiffServ field of IP packet headers (the firewall mangles packets appropriately), and polices the downstream to significantly less than what my cable modem is capable of (to keep downstream queueing at the ISP to a minimum).

My question today is about the downstream policing.  What I have is very simple.  It simply drops ALL packets over 3Mbps:

```
# Enable ingress policing

tc qdisc add dev $INTERFACE handle ffff: ingress

# Drop packets coming in faster than RXRATE

tc filter add dev $INTERFACE parent ffff: protocol ip prio 50 u32 \

   match ip src 0.0.0.0/0 \

   police rate ${RXRATE}kbit \

   burst 20k drop \

   flowid :1

```

This is fine if the only thing that gets dropped are TCP packets for file transfers and such.  What is not fine is having any ICMP, UDP, or interactive TCP traffic getting dropped.  Is it possible to specify that ICMP and UDP are never dropped and that certain kinds of TCP packets are never dropped?  Will my firewall have the opportunity to mangle incoming packets before they are dropped by the policer (enabling me to keep using the firewall to tag packets for policing)?

----------

## Jazz

cool, can u post the script for the outgoing connection ?

I've setup a ftp server, and it drowns my downloads !!

Thanx

----------

## bigfunkymo

The previous code was just a snippet to keep the thread on topic, but I'll go ahead and post my whole script for you here:

```

#!/bin/bash

# Traffic shaper for external interface

# Based on WonderShaper 1.1a (HTB)

RXRATE=3000

TXRATE=340

INTERFACE=eth0

## End of configuration

if [ "$1" = "status" ]

then

        tc -s qdisc ls dev $INTERFACE

        tc -s class ls dev $INTERFACE

        exit

fi

# Erase all existing TC settings, suppress output.

tc qdisc del dev $INTERFACE root    2> /dev/null > /dev/null

tc qdisc del dev $INTERFACE ingress 2> /dev/null > /dev/null

if [ "$1" = "stop" ]

then

        exit

fi

### Egress shaping ###

# Create root HTB, default to 1:20

tc qdisc add dev $INTERFACE root handle 1: htb default 20

# Limit egress speed to TXRATE

tc class add dev $INTERFACE parent 1: classid 1:1 htb \

   rate ${TXRATE}kbit \

   burst 10k

# Create high priority class 1:10

tc class add dev $INTERFACE parent 1:1 classid 1:10 htb \

   rate ${TXRATE}kbit \

   burst 10k prio 1

# Create default class 1:20

tc class add dev $INTERFACE parent 1:1 classid 1:20 htb \

   rate $[9*$TXRATE/10]kbit \

   burst 10k prio 2

# Create low priority class 1:30

tc class add dev $INTERFACE parent 1:1 classid 1:30 htb \

   rate $[8*$TXRATE/10]kbit \

   burst 10k prio 2

# Enable stochastic fairness for each class

tc qdisc add dev $INTERFACE parent 1:10 handle 10: sfq perturb 10

tc qdisc add dev $INTERFACE parent 1:20 handle 20: sfq perturb 10

tc qdisc add dev $INTERFACE parent 1:30 handle 30: sfq perturb 10

# Packets with Diffserv/ToS "Minimum Delay" flag get high priority

# Set this flag using packet mangling. SSH sets this flag itself.

tc filter add dev $INTERFACE parent 1:0 protocol ip prio 10 u32 \

   match ip tos 0x10 0xff \

   flowid 1:10

# ICMP packets get high priority

tc filter add dev $INTERFACE parent 1:0 protocol ip prio 10 u32 \

   match ip protocol 1 0xff \

   flowid 1:10

# ACK packets get high priority

tc filter add dev $INTERFACE parent 1: protocol ip prio 10 u32 \

   match ip protocol 6 0xff \

   match u8 0x05 0x0f at 0 \

   match u16 0x0000 0xffc0 at 2 \

   match u8 0x10 0xff at 33 \

   flowid 1:10

# Packets with Diffserv/ToS "Maximum Throughput" flag get low priority

# Set this flag using packet mangling.

tc filter add dev $INTERFACE parent 1:0 protocol ip prio 10 u32 \

   match ip tos 0x08 0xff \

   flowid 1:30

# All other traffic gets normal priority

tc filter add dev $INTERFACE parent 1: protocol ip prio 18 u32 \

   match ip dst 0.0.0.0/0 \

   flowid 1:20

### Ingress policing ###

# Enable ingress policing

tc qdisc add dev $INTERFACE handle ffff: ingress

# Drop packets coming in faster than RXRATE

tc filter add dev $INTERFACE parent ffff: protocol ip prio 50 u32 \

   match ip src 0.0.0.0/0 \

   police rate ${RXRATE}kbit \

   burst 20k drop \

   flowid :1

```

By the way, I'm still searching for a way to make the policer never drop certain types of traffic  :Wink: 

----------

## plut0

Yes it is possible.  Easiest way to do it is with iptables.  Heres an example with dropping SYN packets and ftp.

Mark the packet:

```
iptables -t mangle -A PREROUTING -p tcp --dport 21 -j MARK --set-mark X
```

And for traffic control:

```
tc filter add dev eth0 parent 1: protocol ip prio handle X fw classid 1:Y police rate 144 burst 144 mtu 1500 action drop/continue
```

X = same for your marked packet in iptables and handle in tc

Y = classid for htb

rate 144 = 3 SYN packets, so 3 SYNs/second

Btw, might want to take a look at IMQ.  It helps setup ingress/egress better.

----------

## bigfunkymo

I thought it would probably be something that simple.  I was having one hell of a time with the TC documentation though.  I'm trying to stay away from using MARK and impliment this via DiffServ alone--which is pretty much the same concept of using the firewall to tag packets for the shaper.

What I'm trying to do more specifically is to police ingress to about 3mbps, but never allow dropping of ICMP, UDP, and some TCP traffic that I will appropriately mangle via firewall, even if that means the interface accepts packets faster than 3mbps.

... googling IMQ right now...

----------

## plut0

Whats wrong with marking packets?  Btw, iptables supports DSCP.

I have another suggestion for your setup and that is using the RED algorithm for ingress.  You can setup a max limit for downloading and when it is exceeded it will start dropping packets.  The other nice thing about it is it supports ECN.  If the remote side and routers between support it, when ECN is sent the remote side will queue packets instead of just dropping them so you can cut down on your packet loss.

----------

## bigfunkymo

I didn't say anything was wrong with marking packets.  I just mean that I've already decided on using DiffServ.

I'm not familiar with the DSCP acronym... what does that mean?

----------

## plut0

dscp = Differentiated Services Code Point

----------

## bigfunkymo

heh, yea I know iptables supports DSCP.  I'm using shorewall to configure iptables and to set up the packet mangling.

----------

## bigfunkymo

I've modified my ingress policing section like this:

```

### Ingress policing ###

# Enable ingress qdisc

tc qdisc add dev $INTERFACE handle ffff: ingress

# ICMP packets are never dropped

tc filter add dev $INTERFACE parent ffff: protocol ip prio 40 u32 \

   match ip protocol 1 0xff \

   flowid :1

# UDP packets are never dropped

tc filter add dev $INTERFACE parent ffff: protocol ip prio 40 u32 \

   match ip protocol 17 0xff \

   flowid :1

# Packets marked for "Minimum Delay" are never dropped

# THIS PART IS BROKEN

tc filter add dev $INTERFACE parent ffff: protocol ip prio 40 u32 \

   match ip tos 0x10 0x10 \

   flowid :1

# Drop all other packets coming in faster than RXRATE

tc filter add dev $INTERFACE parent ffff: protocol ip prio 50 u32 \

   match ip src 0.0.0.0/0 \

   police rate ${RXRATE}kbit \

   burst 20k \

   drop \

   flowid :1

```

I've done some testing with tcpdump and such and come up with the following:

This script will police the ingress to the specified rate and it will never drop ICMP or UDP packets.  Beware that if you have enough UDP traffic (that is not being policed) or UDP + TCP (tcp is being policed) then you can defeat the whole purpose  of having ingress policing in the first place--that is you will find your ISP queueing your downstream packets and thus causing latency in your interactive traffic.  However, I would argue that this is still the desired effect because dropping UDP packets is just silly--by the time it reaches your external interface, it's already taken up some of your bandwidth anyway and it won't be retransmitted. No sense in creating packet loss for connectionless protocols.  Also, if you saturate your line with UDP/ICMP on the downstream, then you're probably better off having the ISP queue it up a little.

It is also worth noting that the filter that checks for the ToS field (the one that says that part is broken) will never actually do anything unless the packets come marked that way from your ISP.  I have yet to capture a packet that came that way through my cable modem.  I'm guessing that ISP's and backbone providers probably clear the ToS field out at their borders so they can classify packets themselves.  If a packet is dropped due to ingress policing, netfilter will never get a chance to set the ToS flags.  In order for me to protect TCP traffic from being dropped, I will have to make rules for them here.

----------

