# [solved] Routing using iproute2 fwmark to Squid machine

## maiku

I'm trying to attempt to set up a Squid proxy doing transparent proxying (interception) on a machine different from the router.

I have attempted two guides http://wiki.squid-cache.org/ConfigExamples/Intercept/IptablesPolicyRoute and http://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.cookbook.squid.html which essentially say the same thing.

The issue I'm having is that I see the packets being marked on my router but the browser just times out. What could I be missing in the chain? The iptables on the Squid machine never counts up.

Router:

 *Quote:*   

> # ip rule show
> 
> 0:      from all lookup local
> 
> 32765:  from all fwmark 0x2 lookup www
> ...

 

Squid machine:

 *Quote:*   

> # iptables -v -L -t nat -n
> 
> Chain PREROUTING (policy ACCEPT 811 packets, 161K bytes)
> 
>  pkts bytes target     prot opt in     out     source               destination
> ...

 

Where br0 is the internal network interface on the router, the network is 10.0.0.0/24, and 10.0.0.137 is the IP of the Squid machine.

Thanks for the help!

----------

## papahuhn

Is /proc/sys/net/ipv4/ip_forward enabled on the squid machine?

----------

## maiku

Yes.

 *Quote:*   

> # cat /proc/sys/net/ipv4/ip_forward
> 
> 1

 

----------

## papahuhn

Could you tcpdump port 80 on your squid machine? Do packets arrive at it? Does squid send synacks as a response which are not acked back?

----------

## maiku

Here is what happens *Quote:*   

> # tcpdump port 80
> 
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> 
> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
> ...

 

So it doesn't see any packets coming in.

What's a good filter to check to see the other things that you wanted the output for?

----------

## papahuhn

The other output is not relevant, if there are no incoming packets. Something's wrong on the router, but I don't see it. Two questions:

1) Why do you use eth0 as output-interface for the MASQUERADE if br0 is the internet-connected interface?

2) Could you tcpdump port 80 on the router and see what it does with packets coming from a client (non-squid)?

----------

## maiku

 *Quote:*   

> 1) Why do you use eth0 as output-interface for the MASQUERADE if br0 is the internet-connected interface? 

 Thanks for pointing this out. br0 is the internal network interface. It's bonded with eth1 and tap0

/etc/conf.d/net *Quote:*   

> modules=( "iproute2" )
> 
> config_eth0=( "dhcp" )
> 
> config_eth1=( "null" )
> ...

 eth0 is the Internet interface. *Quote:*   

> 2) Could you tcpdump port 80 on the router and see what it does with packets coming from a client (non-squid)?

 Of course. Here is what it says: *Quote:*   

> # tcpdump -n -i br0 port 80
> 
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> 
> listening on br0, link-type EN10MB (Ethernet), capture size 68 bytes
> ...

 I hope this helps.

----------

## papahuhn

The lines in your last dump come always in pairs, so packets are routed out on the same interface, as one should expect. If you do that dump again with "tcpdump -nei br0 port 80", what destination MAC address do you see for the second packet of every pair? It should be the squid's, but then squid's tcpdump should show something ...

----------

## maiku

Aha. There's the problem then. *Quote:*   

> # tcpdump -nei br0 port 80
> 
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> 
> listening on br0, link-type EN10MB (Ethernet), capture size 68 bytes
> ...

 00:23:14:52:9f:18 is the MAC of the laptop making the request. 00:1a:4d:74:fc:b8 is br0. 00:0c:29:83:56:64 which is the MAC of the Squid machine never appears in the dump.

It continues like this before timing out. So the issue is on the routing end then?

----------

## papahuhn

What the heck...

How does the router's arp cache look like? Does the router know that it is using the laptop as next hop? Do you happen to have any ebtables rules?

----------

## maiku

Ah. Details I forgot to mention: *Quote:*   

> # arp
> 
> Address                  HWtype  HWaddress           Flags Mask            Iface
> 
> 10.0.0.114               ether   00:23:14:52:9f:18   C                     br0
> ...

 10.0.0.114 is the laptop, 10.0.0.137 is the Squid machine and it is also a virtual machine on the laptop. I guess that explains the mystery there of why it's sending packets back to the laptop.

Manually adding the MAC address of the virtual machine does make it unreachable. I guess that's to be expected. Is there anything that I can do in this circumstance to get around this?

----------

## papahuhn

Virtual machines have their own macs if they are bridged to the host's network interface. Which virtualization software do you use? You should try to get an individual mac for your VM first.

----------

## maiku

I'm using VMware player on windows 7. I am using a bridged interface and the system believes that it is using a different MAC than the host system. The VM shows a different MAC. In fact, the router also sees that MAC when assigning DHCP and shows it in the logs.

Any ideas on that one?

----------

## papahuhn

Not really an idea, no. I would flush the arp cache and tcpdump for an event which causes the mac address reassignment. As long as the router uses the VM's mac address, your setup should work. You could also try to set the arp entry permanently and see how long that works. What happens if you arping the VM, btw?

----------

## maiku

Wow. This is such an amazing mystery.

The host system (Windows) see the correct MAC address *Quote:*   

> 10.0.0.137            00-50-56-25-58-2d     dynamic

 arping from the router shows the wrong info *Quote:*   

> # arping -I br0 10.0.0.137
> 
> ARPING 10.0.0.137 from 10.0.0.1 br0
> 
> Unicast reply from 10.0.0.137 [00:23:14:52:9F:18]  2.538ms
> ...

 Other machines on the network recognize it as the wrong MAC address (as the same MAC as the host which is 00:23:14:52:9F:18).

Then the router is also told the wrong MAC *Quote:*   

> 15:09:25.440473 00:1a:4d:74:fc:b8 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: arp who-has 10.0.0.137 tell 10.0.0.1
> 
> 15:09:25.442510 00:23:14:52:9f:18 > 00:1a:4d:74:fc:b8, ethertype ARP (0x0806), length 60: arp reply 10.0.0.137 is-at 00:23:14:52:9f:18

 Is VMWare just being garbage?

Setting the MAC manually just causes it to become unreachable. *Quote:*   

> # arp
> 
> Address                  HWtype  HWaddress           Flags Mask            Iface
> 
> 10.0.0.137               ether   00:50:56:25:58:2d   CM                    br0
> ...

 This is ridiculous.

----------

## papahuhn

Is your laptop online via WiFi? There might be a problem that the mac address cannot be changed for a WLAN interface.

----------

## maiku

 *papahuhn wrote:*   

> Is your laptop online via WiFi? There might be a problem that the mac address cannot be changed for a WLAN interface.

 This was a very intuitive answer. You are very smart.

You're right. That is the issue. When switching to hard wire it works fine. I mean EVERYTHING works fine. This is solved.

Thanks so much.

----------

## papahuhn

This is nice to hear. 

So long.

----------

