# [solved] "time exceeded in-transit"

## eqxro

I've got a router with 2 ISPs, one LAN, resulting in a 3 NIC setup. NICs eth0, eth1 are the two ISPs and eth2 is the LAN. I let the users select their preffered gateway (I created a new routing table for each ISP, all identical but the default route). The problem is that if the system's default gateway is, let's say, through eth1, I can access the net from the router, but none of the users going out on eth1 have access to the internet. However, if one chooses the eth0 route, they get access to the internet. If I set the router's default gateway, the behavior is the same. eth1 can't access the internet from the LAN no matter what I try.

Wierd thing is it worked yesterday... I've got gentoo and shorewall with the wonder shaper trafic shaping tcstart script installed on the router. Here are my custom routes:

```
Reboot shorewall # ip route show

192.168.0.0/24 dev eth2  proto kernel  scope link  src 192.168.0.1

82.76.28.0/24 dev eth0  proto kernel  scope link  src 82.76.28.129

10.0.252.0/22 dev eth1  proto kernel  scope link  src 10.0.253.42

127.0.0.0/8 dev lo  scope link

default via 10.0.255.1 dev eth1

Reboot shorewall # ip rule show

0:      from all lookup local

32759:  from 192.168.0.4 lookup T1

32760:  from 192.168.0.3 lookup T2

32762:  from 192.168.0.2 lookup T2

32766:  from all lookup main

32767:  from all lookup default

Reboot shorewall # ip route show table T1

192.168.0.0/24 dev eth2  proto kernel  scope link  src 192.168.0.1

82.76.28.0/24 dev eth0  proto kernel  scope link  src 82.76.28.129

10.0.252.0/22 dev eth1  proto kernel  scope link  src 10.0.253.42

127.0.0.0/8 dev lo  scope link

default via 82.76.28.1 dev eth0

Reboot shorewall # ip route show table T2

192.168.0.0/24 dev eth2  proto kernel  scope link  src 192.168.0.1

82.76.28.0/24 dev eth0  proto kernel  scope link  src 82.76.28.129

10.0.252.0/22 dev eth1  proto kernel  scope link  src 10.0.253.42

127.0.0.0/8 dev lo  scope link

default via 10.0.255.1 dev eth1

```

If I try pinging the DNS server from the LAN, I get:

```
01:52:41.412056 IP (tos 0x10, ttl 127, id 6, offset 0, flags [DF], length: 84) localhost > thorin.mediasat.ro: icmp 64: echo request seq 7

01:52:41.435962 IP (tos 0x10, ttl   1, id 30343, offset 0, flags [none], length: 84) thorin.mediasat.ro > localhost: icmp 64: echo reply seq 7

01:52:41.436081 IP (tos 0xd0, ttl  64, id 38011, offset 0, flags [none], length: 112) localhost > thorin.mediasat.ro: icmp 92: time exceeded in-transit for IP (tos 0x10, ttl   1, id 30343, offset 0, flags [none], length: 84) thorin.mediasat.ro > localhost: icmp 64: echo reply seq 7

```

Last edited by eqxro on Mon Mar 14, 2005 6:23 pm; edited 1 time in total

----------

## eqxro

Some more info, from tethereal:

```

################################# PING send

Frame 18 (98 bytes on wire, 98 bytes captured)

    Arrival Time: Mar 14, 2005 00:55:08.632890000

    Time delta from previous packet: 0.996462000 seconds

    Time since reference or first frame: 4.998961000 seconds

    Frame Number: 18

    Packet Length: 98 bytes

    Capture Length: 98 bytes

    Protocols in frame: eth:ip:icmp:data

Ethernet II, Src: 4c:00:10:3a:a9:8f, Dst: 00:d0:b7:51:1b:cf

    Destination: 00:d0:b7:51:1b:cf (Intel_51:1b:cf)

    Source: 4c:00:10:3a:a9:8f (4c:00:10:3a:a9:8f)

    Type: IP (0x0800)

Internet Protocol, Src Addr: 10.0.253.42 (10.0.253.42), Dst Addr: 193.231.169.2 (193.231.169.2)

    Version: 4

    Header length: 20 bytes

    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

        0000 00.. = Differentiated Services Codepoint: Default (0x00)

        .... ..0. = ECN-Capable Transport (ECT): 0

        .... ...0 = ECN-CE: 0

    Total Length: 84

    Identification: 0x00c7 (199)

    Flags: 0x04 (Don't Fragment)

        0... = Reserved bit: Not set

        .1.. = Don't fragment: Set

        ..0. = More fragments: Not set

    Fragment offset: 0

    Time to live: 63

    Protocol: ICMP (0x01)

    Header checksum: 0xc8cd (correct)

    Source: 10.0.253.42 (10.0.253.42)

    Destination: 193.231.169.2 (193.231.169.2)

Internet Control Message Protocol

    Type: 8 (Echo (ping) request)

    Code: 0

    Checksum: 0xb001 (correct)

    Identifier: 0xa822

    Sequence number: 0x00c8

    Data (56 bytes)

0000  9d c3 34 42 00 00 00 00 00 3b 0e 00 00 00 00 00   ..4B.....;......

0010  10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f   ................

0020  20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f    !"#$%&'()*+,-./

0030  30 31 32 33 34 35 36 37                           01234567

################################# PING reply

Frame 19 (98 bytes on wire, 98 bytes captured)

    Arrival Time: Mar 14, 2005 00:55:08.634872000

    Time delta from previous packet: 0.001982000 seconds

    Time since reference or first frame: 5.000943000 seconds

    Frame Number: 19

    Packet Length: 98 bytes

    Capture Length: 98 bytes

    Protocols in frame: eth:ip:icmp:data

Ethernet II, Src: 00:d0:b7:51:1b:cf, Dst: 4c:00:10:3a:a9:8f

    Destination: 4c:00:10:3a:a9:8f (4c:00:10:3a:a9:8f)

    Source: 00:d0:b7:51:1b:cf (Intel_51:1b:cf)

    Type: IP (0x0800)

Internet Protocol, Src Addr: 193.231.169.2 (193.231.169.2), Dst Addr: 10.0.253.42 (10.0.253.42)

    Version: 4

    Header length: 20 bytes

    Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

        0000 00.. = Differentiated Services Codepoint: Default (0x00)

        .... ..0. = ECN-Capable Transport (ECT): 0

        .... ...0 = ECN-CE: 0

    Total Length: 84

    Identification: 0x3445 (13381)

    Flags: 0x00

        0... = Reserved bit: Not set

        .0.. = Don't fragment: Not set

        ..0. = More fragments: Not set

    Fragment offset: 0

    Time to live: 1

    Protocol: ICMP (0x01)

    Header checksum: 0x1350 (correct)

    Source: 193.231.169.2 (193.231.169.2)

    Destination: 10.0.253.42 (10.0.253.42)

Internet Control Message Protocol

    Type: 0 (Echo (ping) reply)

    Code: 0

    Checksum: 0xb801 (correct)

    Identifier: 0xa822

    Sequence number: 0x00c8

    Data (56 bytes)

0000  9d c3 34 42 00 00 00 00 00 3b 0e 00 00 00 00 00   ..4B.....;......

0010  10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f   ................

0020  20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f    !"#$%&'()*+,-./

0030  30 31 32 33 34 35 36 37                           01234567

################################# PING time exceeded in-transit

Frame 20 (126 bytes on wire, 126 bytes captured)

    Arrival Time: Mar 14, 2005 00:55:08.635070000

    Time delta from previous packet: 0.000198000 seconds

    Time since reference or first frame: 5.001141000 seconds

    Frame Number: 20

    Packet Length: 126 bytes

    Capture Length: 126 bytes

    Protocols in frame: eth:ip:icmp:ip:icmp:data

Ethernet II, Src: 4c:00:10:3a:a9:8f, Dst: 00:d0:b7:51:1b:cf

    Destination: 00:d0:b7:51:1b:cf (Intel_51:1b:cf)

    Source: 4c:00:10:3a:a9:8f (4c:00:10:3a:a9:8f)

    Type: IP (0x0800)

Internet Protocol, Src Addr: 10.0.253.42 (10.0.253.42), Dst Addr: 193.231.169.2 (193.231.169.2)

    Version: 4

    Header length: 20 bytes

    Differentiated Services Field: 0xc0 (DSCP 0x30: Class Selector 6; ECN: 0x00)

        1100 00.. = Differentiated Services Codepoint: Class Selector 6 (0x30)

        .... ..0. = ECN-Capable Transport (ECT): 0

        .... ...0 = ECN-CE: 0

    Total Length: 112

    Identification: 0x606b (24683)

    Flags: 0x00

        0... = Reserved bit: Not set

        .0.. = Don't fragment: Not set

        ..0. = More fragments: Not set

    Fragment offset: 0

    Time to live: 64

    Protocol: ICMP (0x01)

    Header checksum: 0xa74d (correct)

    Source: 10.0.253.42 (10.0.253.42)

    Destination: 193.231.169.2 (193.231.169.2)

Internet Control Message Protocol

    Type: 11 (Time-to-live exceeded)

    Code: 0 (Time to live exceeded in transit)

    Checksum: 0xf4ff (correct)

    Internet Protocol, Src Addr: 193.231.169.2 (193.231.169.2), Dst Addr: 10.0.253.42 (10.0.253.42)

        Version: 4

        Header length: 20 bytes

        Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

            0000 00.. = Differentiated Services Codepoint: Default (0x00)

            .... ..0. = ECN-Capable Transport (ECT): 0

            .... ...0 = ECN-CE: 0

        Total Length: 84

        Identification: 0x3445 (13381)

        Flags: 0x00

            0... = Reserved bit: Not set

            .0.. = Don't fragment: Not set

            ..0. = More fragments: Not set

        Fragment offset: 0

        Time to live: 1

        Protocol: ICMP (0x01)

        Header checksum: 0x1350 (correct)

        Source: 193.231.169.2 (193.231.169.2)

        Destination: 10.0.253.42 (10.0.253.42)

    Internet Control Message Protocol

        Type: 0 (Echo (ping) reply)

        Code: 0

        Checksum: 0xb801 (correct)

        Identifier: 0xa822

        Sequence number: 0x00c8

        Data (56 bytes)

0000  9d c3 34 42 00 00 00 00 00 3b 0e 00 00 00 00 00   ..4B.....;......

0010  10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f   ................

0020  20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f    !"#$%&'()*+,-./

0030  30 31 32 33 34 35 36 37                           01234567

```

----------

## eqxro

Okay, I solved this, it seems my ISP sent back al the packets with TTL=1 and they couldn't be forwarded anymore from my router (it would die on the server). I had to patch my kernel to be able to do something like iptables -t mangle -A PREROUTING -i eth1 -j TTL --ttl-set 64. The patch is patch-o-matic-ng, the TTL part only.

----------

