# [SOLVED] iptables; mark and connmark targets and modules?

## Sadako

I'm playing around with iptables, and I have a few questions about these, the main one being what's the real difference between the two?

You can apparently set the mark with -j CONNMARK, but most things I've come across use -j MARK, any particular reason?

Especially when it looks like you have to use the connmark module (to save and restore the mark in the mangle chain), but you could concievably do without the mark module?

And is there any issue with using --state matches within the mangle table?

Also, I'm a little confused as to what table/chain to set the marks.

At present I'm setting them in the PREROUTING and OUTPUT chains of the mangle table, for filtering within the filter table.

Seeing as how this isn't on a router or anything, ie no forwarding, all packets are either originating from or addressed to the box in question, are these the most appropriate choices?

For example, when would/should you use the INPUT chain instead of PREROUTING?

Thx in advance for anything you can make clearer to me.

----------

## Bones McCracker

If it's not a router, it doesn't matter whether you use input or prerouting, because everything goes through both.  That's not true on a router, where not everything goes through input.

Mark marks a packet.  Connmark marks every subsequent packet which is part of the same connection (per the conntrack tables).  Connmark is a way to avoid having iptables do redundant work that conntrack is already doing.  Intelligent use of connmark can dramatically reduce the iptables workload (and misuse can compromise the security of your machine).

If the matching task is such that once you have matched the connection with your iptables rule (probably based upon the syn packet, a single other packet, or some heuristic using a series of packets), you can simply rely upon conntrack's matching (and every packet over the connection should also be a match), then use connmark.

If the filtering task is such that not all of the packets that comprise the connection (as judged by conntrack) should be matched, then don't use connmark, instead use mark. Also, mark is more general in nature.  For example, it is useful for traffic control.  You might want to queue or re-route a packet because of congestion, but not necessarily queue or reroute the entire connection.

Where you put them in your tables/chains depends entirely upon what you are doing with them.  They are typically found in multiple locations.  Although there are some pretty standard constructs, you almost have to think of it as a programming language (or a framework) that allows great flexibility.  Another complicating factor is functional overlap between packet filtering, routing, traffic control, and general network stack.  There are many tasks you can do with iptables, something else, or the two integrated.

It's hard to find documentation that is correct, complete, and understandable (looking back at my bookmarks, I apparently felt that 14 different references were worth bookmarking).  And even once you do, it's just enough to enable you to experiment - which is the only way you'll gain an understanding.  I recommend building a router.

----------

## Sadako

Hey boney, long time no see (other than on bgo).

 *BoneKracker wrote:*   

> If it's not a router, it doesn't matter whether you use input or prerouting, because everything goes through both.  That's not true on a router, where not everything goes through input.

 Ah, that clears that part up quite a bit.

So for this I'll use INPUT, as that's precisely what it's there for, and having the rules in PREROUTING instead does nothing different.

 *Quote:*   

> Mark marks a packet.  Connmark marks every subsequent packet which is part of the same connection (per the conntrack tables).  Connmark is a way to avoid having iptables do redundant work that conntrack is already doing.  Intelligent use of connmark can dramatically reduce the iptables workload (and misuse can compromise the security of your machine).

 Any guidance wrt when to use mark and when to use connmark?

In what situations could using connmark over mark present a security risk?

 *Quote:*   

> If the matching task is such that once you have matched the connection with your iptables rule (probably based upon the syn packet, a single other packet, or some heuristic using a series of packets), you can simply rely upon conntrack's matching (and every packet over the connection should also be a match), then use connmark.
> 
> If the filtering task is such that not all of the packets that comprise the connection (as judged by conntrack) should be matched, then don't use connmark, instead use mark. Also, mark is more general in nature.  For example, it is useful for traffic control.  You might want to queue or re-route a packet because of congestion, but not necessarily queue or reroute the entire connection.
> 
> Where you put them in your tables/chains depends entirely upon what you are doing with them.  They are typically found in multiple locations.  Although there are some pretty standard constructs, you almost have to think of it as a programming language (or a framework) that allows great flexibility.  Another complicating factor is functional overlap between packet filtering, routing, traffic control, and general network stack.  There are many tasks you can do with iptables, something else, or the two integrated.

 ATM, I'm just setting the mark based on the incoming/outgoing ports, and doing all the actual filtering in the filter table/chains, so connmark would seem to be the appropriate choice.

Just a couple of more questions, to see if I'm getting this clearly;

Are the "-j CONNMARK --restore-mark" and "-j CONNMARK --save-mark" rules still required with "-j CONNMARK --set-mark", or only with "-j MARK --set-mark"?

The impression I'm getting at the moment is that "-j CONNMARK --save-mark" is to mark the entire connection when "-j MARK --set-mark" marks one packet within that connection, is this accurate?

And will "-j CONNMARK --set-mark" only match "NEW" packets, ie if I have a series of "-j CONNMARK --set-mark" rules would it be more efficient to have them in their own chain, referenced with "--state NEW -j CONNMARK_CHAIN"?

 *Quote:*   

> It's hard to find documentation that is correct, complete, and understandable (looking back at my bookmarks, I apparently felt that 14 different references were worth bookmarking).

 I know, I've been reading up on it, but pretty much everything I find is either too vague, contradicts something else, or is a little out of date...

 *Quote:*   

> And even once you do, it's just enough to enable you to experiment - which is the only way you'll gain an understanding.  I recommend building a router.

 That's what I intend on doing (I really want to get one of these, together with one of these and this).

Need to rob a bank or something first, though...  :Confused: 

Anyways, thx boney, You're missed in otw (kinda  :Razz: ).

----------

## Bones McCracker

 *Hopeless wrote:*   

> Hey boney, long time no see (other than on bgo).

 

Yes, it seems I have been permanently banned from otw.  I was given no explanation whatsoever, and I am left to assume one or more admins/mods have allowed their somewhat extremist political views an their own prejudices to interfere with their roles as unbiased arbiters.  But that's not the subject of this forum.  On to your questions:

 *Hopeless wrote:*   

> Any guidance wrt when to use mark and when to use connmark?
> 
> In what situations could using connmark over mark present a security risk?

 

In most cases, all you need is simple state matching.  The connection-tracking "state" (new, related, established, etc.) is all handled by conntrack automatically, and all you need to do is match against that (you don't need to manipulate that state as a "target").  Connmark allows you the flexibility to establish similar connection-wide characteristics (ones that you define) and to mark the whole connection accordingly.  I use it to mark connections as being in one of five classes of services (i.e. priority levels), and I later put packets into separate queues based on those markings, so that the higher priority traffic never gets blocked by the lower priority traffic.

No real guidance on when to use mark vs. connmark, other than the fact that they serve different purposes.  You can almost always use "mark" to get the same thing done, but "mark" requires each new packet to be processed by the iptables rule, whereas connmark doesn't.  In general, if you logic applies to an entire connection, use connmark.

 *Hopeless wrote:*   

> 
> 
> Just a couple of more questions, to see if I'm getting this clearly;
> 
> Are the "-j CONNMARK --restore-mark" and "-j CONNMARK --save-mark" rules still required with "-j CONNMARK --set-mark", or only with "-j MARK --set-mark"?
> ...

 

It's been a long time since I looked at this in detail.  Without looking it up, I want to say "yes, and yes".

One important aspect of this (and I think you've grasped this already) is that connmark and mark are not two different levels of the same thing.  Connmark does not literally "mark" each packet in a connection (it puts a different kind of mark that is not a "mark" per-se).  They shouldn't have named them "mark" and "connmark"; they should have named them "packetflag" and "connectionlabel" or something.  For example, iproute2 can read a "mark" from a packet, but it can't read a "connmark" (at least, it couldn't when I last fiddled with this).

What I did was take a regular low-end PC (like one that's good for nothing), throw in some cheap interface cards, and start hacking.

In reality, unless you intend to do some fancy routing, it is likely you will not need to use either mark or connmark (and even then, they can be avoided because there are other ways to do it).

----------

## Sadako

Thanks boney, I have one more question about it, but I think I've got a good enough handle on it to mark this as solved.

Here is what I'm using within the mangle table now (via `iptables --line-numbers -vnL -t mangle`, with the empty chains omitted);

```
Chain INPUT (policy ACCEPT 3296K packets, 3754M bytes)

num   pkts bytes target     prot opt in     out     source               destination         

1    33443 2846K TORR_IN   all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           state NEW

Chain OUTPUT (policy ACCEPT 3207K packets, 2836M bytes)

num   pkts bytes target     prot opt in     out     source               destination         

1     146K   11M TORR_OUT  all  --  *      eth0    0.0.0.0/0            0.0.0.0/0           state NEW owner socket exists owner UID match 1156

Chain TORR_IN (1 references)

num   pkts bytes target     prot opt in     out     source               destination         

1      231 45146 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           connmark match !0x0 

2    20919 1162K CONNMARK   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:23725 CONNMARK xset 0x20/0xffffffff 

3    12521 1663K CONNMARK   udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp dpt:13646 ipp2p v0.8.2 --bit CONNMARK xset 0x30/0xffffffff 

4        0     0 CONNMARK   udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp dpt:11170 ipp2p v0.8.2 --bit CONNMARK xset 0x40/0xffffffff 

Chain TORR_OUT (1 references)

num   pkts bytes target     prot opt in     out     source               destination         

1    47163 2897K RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           connmark match !0x0 

2    58339 3501K CONNMARK   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp spts:49152:61000 CONNMARK xset 0x10/0xffffffff 

3        4  1846 CONNMARK   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp spt:23725 CONNMARK xset 0x20/0xffffffff 

4    41575 4698K CONNMARK   udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp spt:13646 ipp2p v0.8.2 --bit CONNMARK xset 0x30/0xffffffff 

5        0     0 CONNMARK   udp  --  *      *       0.0.0.0/0            0.0.0.0/0           udp spt:11170 ipp2p v0.8.2 --bit CONNMARK xset 0x40/0xffffffff
```

This appears to work fine, and within the filter table I'm only accepting packets which have been marked (and I'm not seeing any undesirable affects of this), however I don't quite understand why packets are matching the "-m connmark ! --mark 0x0 -j RETURN" rules in the mangle table.

I mean, the only packets to even enter that table are NEW packets, which I would have thought should not have any marking, and the only way I can see them being marked is if those packets matching ! --mark 0x0 have already been through that chain...

So what am I missing?

----------

## Bones McCracker

Isn't that a test to see if it's not marked?

It's been too long since I had my fingers deep in this.

Look into the use of 0 as the mark.  I may be thinking of something else, but I think if you test for 0, you're testing to see if it's marked (and if you test for !0, you're testing to see if it's not marked).

----------

## Sadako

I kind of got the impression that 0 was just the "default" mark if no other mark is set, although a quick google isn't turning up anything that states anything remotely like this...

Anyways, I added three simple rules to the start of the INPUT chain, just to see what happens with the match counters;

```
-A INPUT 

-A INPUT -m connmark ! --mark 0x0 

-A INPUT -m connmark --mark 0x0
```

The sum of the counters for the second and third rules equals the counter for the first, ie all packets going through that chain.

This implies all packets are either marked 0 or something else, and as I haven't marked anything as 0 myself, that kinda suggests it is the default (and it does make sense to me that every connection tracked by conntrack would have some mark, even a default, and there'd be no such thing as an "unmarked" connection)...

Maybe this is an example of a difference between the conntrack marks and the netfilter packet marks, as it does make perfect sense to me that the latter would simply be unmarked by default, unlike the connection marks?

----------

## Bones McCracker

 *Hopeless wrote:*   

> I kind of got the impression that 0 was just the "default" mark if no other mark is set, although a quick google isn't turning up anything that states anything remotely like this...
> 
> Anyways, I added three simple rules to the start of the INPUT chain, just to see what happens with the match counters;
> 
> ```
> ...

 

Yes, that's correct.  Every packet will test for either 0 or !0.  What I vaguely recall was wrestling with whether a packet matching 0 is one that has not been otherwise marked (which would make sense intuitively), or whether a packet matching !0 is one that has not been otherwise marked (which is what I vaguely recall, although it would seem otherwise by common sense).

----------

