# squid and squidguard - some https sites work, some don't

## bendy

Hi,

A few weeks ago I updated a firewall/proxy at work with gentoo installed.  It had not been updated for a while and an updated version of squid was installed.  We have started having problems as trying to access some https urls from within our lan result in an error message in the browser:

```
The requested URL could not be retrieved

While trying to retrieve the URL: http:443

The following error was encountered:

    Unable to determine IP address from host name for 

The dnsserver returned:

    Name Error: The domain name does not exist. 

This means that:

 The cache was not able to resolve the hostname presented in the URL. 

 Check if the address is correct. 

Your cache administrator is root. 
```

However, some other https urls work fine.

I initially upgraded to the stable squid-2.7.3 package, and when I noticed the problems I tried updating to the ~arch masked squid-3.0.8, which made no difference.  I have also tried emerging both with and without the ssl USE flag.

An example of an https site that doesn't work is my ebuyer account page https://accounts.ebuyer.com/customer/account/index.html?action=xxxxxxx=

This shows up in /var/log/squid/access.log as:

```
1223558160.845      4 192.168.8.109 TCP_MISS/404 0 CONNECT accounts.ebuyer.com:443 - DIRECT/- -
```

However, visiting my ISPs webmail site at https://webmail.blueyonder.co.uk/ gives the following in the log:

```
1223558462.781      4 192.168.8.109 TCP_MISS/404 0 CONNECT www.virginmedia.com:443 - DIRECT/- -

1223558462.797      4 192.168.8.109 TCP_MISS/404 0 CONNECT www.virginmedia.com:443 - DIRECT/- -

1223558462.805      0 192.168.8.109 TCP_MISS/404 0 CONNECT www.virginmedia.com:443 - DIRECT/- -

1223558462.813      4 192.168.8.109 TCP_MISS/404 0 CONNECT www.virginmedia.com:443 - DIRECT/- -

1223558462.853      4 192.168.8.109 TCP_MISS/404 0 CONNECT www.virginmedia.com:443 - DIRECT/- -

1223558462.853      4 192.168.8.109 TCP_MISS/404 0 CONNECT www.virginmedia.com:443 - DIRECT/- -

1223558465.761   3132 192.168.8.109 TCP_MISS/200 8929 CONNECT webmail.blueyonder.co.uk:443 - DIRECT/195.188.53.48 -
```

In this case, the https site works, although some content from virginmedia.com (who operate the blueyonder service) might be missing.

What seems to be happening is that some https urls are getting rewritten to nothing, hence the "DIRECT/- -" in the logs.  However, I can't find any pattern in which urls work, and which don't.  I am not using squid in transparent mode - all clients have the proxy details set manually.

I am using squidguard-1.3-r3, and my squid.conf (minus all the comments) is:

```
http_port 192.168.8.5:3128

hierarchy_stoplist cgi-bin ?

acl QUERY urlpath_regex cgi-bin \?

cache deny QUERY

acl apache rep_header Server ^Apache

broken_vary_encoding allow apache

cache_mem 128 MB

cache_dir ufs /cache 3000 16 256

url_rewrite_program /usr/bin/squidGuard

url_rewrite_children 10

refresh_pattern ^ftp:      1440   20%   10080

refresh_pattern ^gopher:   1440   0%   1440

refresh_pattern .      0   20%   4320

acl manager proto cache_object

acl localhost src 127.0.0.1/255.255.255.255

acl to_localhost dst 127.0.0.0/8

acl SSL_ports port 443 563

acl Safe_ports port 80      # http

acl Safe_ports port 21      # ftp

acl Safe_ports port 443 563   # https, snews

acl Safe_ports port 70      # gopher

acl Safe_ports port 210      # wais

acl Safe_ports port 1025-65535   # unregistered ports

acl Safe_ports port 280      # http-mgmt

acl Safe_ports port 488      # gss-http

acl Safe_ports port 591      # filemaker

acl Safe_ports port 777      # multiling http

acl Safe_ports port 901      # SWAT

acl purge method PURGE

acl CONNECT method CONNECT

http_access allow manager localhost

http_access deny manager

http_access allow purge localhost

http_access deny purge

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

acl our_networks src 192.168.8.0/24

http_access allow our_networks

http_access allow localhost

http_access deny all

http_reply_access allow all

icp_access allow all

visible_hostname proxy

coredump_dir /var/cache/squid
```

Before this update, I had squid-2.6.19 which did not have this problem, and I have not changed the squid.conf configuration (I don't think).

Anyone got any suggestions?

----------

## bendy

Bump

Any squid gurus out there?

----------

## think4urs11

Did you check your squidGuard config already?

Seems as if the URL gets mangled somewhere as the error message states:

```
The requested URL could not be retrieved

While trying to retrieve the URL: http:443
```

which is of course something which isn't resolvable. Normally this should read as https://whatever.somewhere.foo/... so i'd check first where and why this is changed to http:443

----------

## bendy

Hi,

I've been using the same squidGuard.conf file for years, and certainly haven't made any changes to it since way before I started having these problems.

The problems started when I upgraded squid itself, and it's only some https URLs that get mangled.  Others, and none-SSL sites work fine.

The squidGuard.conf file doesn't seem to allow any changes to how the URLs get re-written.

----------

## think4urs11

what happens when you remove squidGuard from squid.conf?

----------

## bendy

 *Think4UrS11 wrote:*   

> what happens when you remove squidGuard from squid.conf?

 

If I comment our the squidguard redirector line in squid.conf then everything works fine.  Squidguard seems to be re-writing https URL's in a way that used to work in squid-2.6 but doesn't work in later versions.

----------

## richard.scott

I've been having the same problem....

I figured that squidGuard and Squid just couldn't monitor https pages   :Shocked: 

----------

## DawgG

 *Quote:*   

> I figured that squidGuard and Squid just couldn't monitor https pages 

 

that's why you use end-to-end-encryption: only client and server can read the data. squid does a http-CONNECT for https-connections, ie connects client and server directly and does not cache any data. anyting else could be considered some kind of mitm  :wink: 

 *Quote:*   

> If I comment our the squidguard redirector line in squid.conf then everything works fine. Squidguard seems to be re-writing https URL's

 

i have never used squidguard but from the logs you posted i guess since squidguard cannot "read" anything inside the https-connections it just disallows them and kills them by removing part of the url. that's very uncool since it's always better to use an encrypted connection (caching issues aside). you should configure squidguard to ignore https-connections and not do any redirections so squid can do the required CONNECT directly.

GOOD LUCK!

----------

