# mystery reverse DNS lookups

## madlcd

I have a small network (three machines running Gentoo) all configured with static IPs. I have an old Pentium 2 box running IPcop as a firewall/router that is configured to connect using dial on demand to a PPPOE ISP connection. I point everything to the IPcop box's IP in /etc/resolve.conf so that DNS requests automatically go to the ISP's DNS. The connection is supposed to disconnect after 15 minutes of inactivity.  This has worked fine for years, but a few months ago I noticed that the PPPOE connection was staying up all the time. I ran Wireshark and found that two of the three machines issue a reverse DNS lookup every minute or so for its own unrouteable local IP (e.g. 192.168.1.101).  This keeps the PPPOE connection up continuously.  The third machine is used mostly as a backup server and has a fairly minimal set of applications installed. It does not issue any reverse DNS lookup requests.

My question is what application/service could be making these reverse DNS lookups? 

Or, how can I track down which application/service is doing this?

Thanks

----------

## causality

Unfortunately this won't tell you which program is requesting reverse DNS lookups.  However, I think the way to deal with this is to edit the /etc/hosts file on each machine to assign some kind of name (FQDN) to each 192.168.x.x IP address you are using.  Then it won't even try to use DNS if it needs that information.

The actual cause of the DNS activity could be nearly anything.  You might be using Samba or NFS or some other server to allow your local machines to share files within the network and/or with that backup server.  Other servers such as SSHD might issue reverse DNS lookups as part of their operation; i.e. SSHD will try this to make sure that the forward and reverse DNS entries match for an incoming IP address.

----------

## desultory

If you already have a DNS server running on your own network, why not add an entry for the offending machine? If you use some otherwise unused designation it should be fairly trivial to find what has been running the queries.

----------

## luispa

I agree, add a dns server in one of your boxes or fill all local hosts files with local ip's. 

Luis

----------

## desultory

The IPCop box should already have DNS service running.

----------

## madlcd

OK, It looks like I'm closing in on it.  Each machine did not have it's own IP address in the /etc/hosts file, only the loop back.  I fixed that, and now I get one less request from each machine that makes these reverse lookups. Originally one machine would make two DNS requests and the other would make four.

I had only really looked at the first request in each cluster of DNS requests, and it was an IP version 4 reverse lookup.  That request has now been eliminated.  The remaining requests appear to be IP version 6 reverse lookups!

To try to find what was generating these, I used a process of elimination and began shutting down services one at a time.  It turns out CUPS is the problem.  If I shutdown CUPS on the machine that does not physically have the printer attached, it stops making the queries.  If I leave CUPS running on that machine, but shut CUPS down on the machine with the printer, then no more reverse DNS queries from either machine.  I may have done something to my CUPS configuration a few months back when I was attempting to get my laptop to print through the IPcop firewall.  I will try looking at this.

Thanks

----------

## causality

Sorry if some of this is rather basic and already familiar to you, but with all the problems that can occur with networking, I wanted to be somewhat thorough in this response.

You mentioned that there are three machines on your network.  Generally speaking, the /etc/hosts file of any one machine should have entries for all three machines.  All machines should have their own hostname and a common domain name.  This is total guesswork that I am pulling out of thin air, but it would look something like this:

```

#/etc/hosts

127.0.0.1       localhost.yourdomain localhost

192.168.1.1     router.yourdomain      router

192.168.1.2     machine1.yourdomain machine1

192.168.1.3     machine2.yourdomain machine2

192.168.1.4     machine3.yourdomain machine3

```

Of course the hostnames are arbitrary and you can call them anything but they should have a consistent domain.  Assuming that my router's address is 192.168.1.1, my netmask is 255.255.255.0, and that there are three Linux machines on the network, an exact copy of this /etc/hosts file should be present on all three machines.

With a hosts file like that, all three of these statements will do the exact same thing:

```
ping 192.168.1.3

ping machine2

ping machine2.yourdomain
```

None of those would trigger a single DNS lookup assuming that your system is configured to try the /etc/hosts file first and only fallback to DNS if it needs information that is not in the /etc/hosts file (such as when you resolve something like forums.gentoo.org).  This is a default setting, but if you want to check for this, look at your /etc/nsswitch.conf file.  You should see a line like this:

```
hosts:       files dns
```

That is configured to refer to /etc/hosts and fallback to DNS.  If it were "dns files" it would be the reverse, but that's undesirable in a setup like this.

The principle here is that when a machine on your LAN looks up the address of another machine on your LAN, this transaction should begin and end with your LAN and should not try to route packets out of your network.  Your dial-on-demand is being triggered because a machine on your LAN should have this information but doesn't, so in a last-ditch effort to get the info, it wants to query a remote DNS server (probably your ISP's).  Since getting to a remote DNS server requires an Internet connection, it's triggering your dialer to provide that Internet connection.

----------

## madlcd

Thanks Casualty.

It never hurts to be thorough.  You've reached the same conclusion I had.  I was missing the "machine1" entry from machine1's /etc/hosts and the "machine2" entry from machine2's /etc/hosts etc. 

Correcting that has solved part of the problem.

For some reason CUPS started needing this information, so it would try to look up the host name of the machine it was running on.  The remaining reverse lookups, on closer examination are the mac addresses of the network cards. Machine1 has eth0 and eth1 bridged. This may be why the cluster of lookups from it are: mac address of eth0, mac address of eth1, mac address of eth0.  The lookup from machine2 is for the mac address of its network interface.  Machine2 is actually plugged into eth1 of machine1.  This is why I can see the traffic from both simply by looking at br0 on machine1. I'm guessing machine3 is probably also doing this, but I don't see its traffic.

The question now, I guess, is why is CUPS trying to find the name of the machine it's running on?  My network configuration has not changed in years, and this only started a few months ago.

----------

## madlcd

Oops, sorry Causality not Casualty.

Dyslexics of the world untie!!!

----------

## causality

My /etc/cups/cupsd.conf file has these three lines in it:

```
# Only listen for connections from the local machine.

Listen localhost:631

Listen /var/run/cups/cups.sock

```

This instructs CUPS to only ever use the loopback interface and a (file-based) Unix socket.  That means it won't listen for connections coming from any other machine, ever, not even from other machines on my LAN.  That seems to be a default setting, and it should ensure that all CUPS traffic is local.  Such local traffic would never invoke DNS or even the hosts file.  Of course, if you are running a print server (so that machines which have no printer can connect to a machine that has a printer in order to print documents) then your setup would have to be different.  If you are not running such a print server, then your CUPS daemon should look like the above (local traffic only) unless you have a need to configure it otherwise.

It sounds very odd to me that CUPS would even know or care about MAC addresses.  It does use TCP/IP but, per the OSI Model, IP traffic is Layer 3, the network layer, while Ethernet (and Ethernet structures such as MAC addresses) is Layer 2, the transport layer.  Normally, an application (such as Web browsers, Web servers, FTP clients, etc) would never concern itself with anything below IP.  This leads me to doubt that CUPS is doing this itself.  It does make me wonder if this is an issue related to your network configuration that CUPS just happens to be making you aware of.

Could you show me the data that indicates CUPS is doing this?  I don't know if you determined that with a packet sniffer, or the netstat command, etc. but if possible, can you cut+paste the complete output?  This sounds unusual and I'd like to see it if you can do that.

----------

## madlcd

The only indication I have that CUPS is doing this is that when I stop the CUPS daemon, the packet sniffer no longer sees any DNS queries. It could be something that CUPS is using. I do agree that it is very unusual. I do have one box, "machine1" in our little examples that serves as the print server for the network. If I stop CUPS on it then the DNS queries cease.

I did find that I had inadvertently checked the box "Share published printers connected to this system" on the CUPS administration panel for "machine2", which has no printers physically attached to it, once I unchecked this, it stopped making the DNS queries. "machine1" however still makes these queries.

----------

## desultory

http://www.cups.org/documentation.php/doc-1.4/man-cups-lpd.html

 *Quote:*   

> -n
> 
>     Disables reverse address lookups; normally cups-lpd will try to discover the hostname of the client via a reverse DNS lookup. 

 

----------

## causality

Thanks Desultory.  I'm still a little confused though.  This is how I think it works (please correct me if I have it wrong):  CUPS wants to do a reverse DNS so it asks the resolver library.  The resolver library (DNS client) checks the contents of /etc/nsswitch.conf and would normally see "hosts:  files dns".  So it checks the /etc/hosts file, finds the information it needs for either forward or reverse lookups of any of those three machines on his LAN, and never has to query a DNS server.  This alone should never generate any TCP/UDP traffic on port 53.

Here, I assume two things.  One, that his /etc/hosts files are correctly configured on all three machines.  Two, that CUPS is not an Internet-facing server; that is, it listens on either localhost or a non-routable address like 192.168.x.x, and the router is not doing any port forwarding for it.  On the other hand, if it is accepting connections from remote hosts over the Internet, then that would break my assumptions.  

I don't know.  I'm far more familiar with general networking than I am with CUPS as a specific application.  Maybe it really is doing something unusual, but it just seems to me that there should be no observable DNS traffic if it needed to perform a reverse lookup for machines listed in /etc/hosts.    :Confused: 

----------

## madlcd

OK, now I'm really confused.  The DNS queries do seem to be IPV6 addresses that are based on the MAC addresses of the network cards with "f.f.f" in the middle and a bunch of zeros and "f.e.8" at the end (beginning? since it is reversed).  I notice that the IPV6 kernel module was loaded so I black listed it and rebooted.  The problem appears to be gone now.  I wonder if this may be a CUPS bug where the default behavior of no reverse lookups is broken for IPV6. I tried setting "HostNameLookups Off" in the cups configuration file, but that did not work.

Anyway the symptom is cured, but I still don't know what the underlying issue is.

----------

## causality

Now it makes sense.  It seems that your /etc/hosts file has only IPv4 addresses and does not also specify the IPv6 equivalents.  If your kernel supports IPv6 and loads its modules or has its drivers built-in, then IPv6-aware programs will try to use it.  When they don't find the IPv6 addresses in your /etc/hosts file they will look for them via DNS.

IPv6 has a much larger address space than IPv4.  Specifically, IPv4 uses 32-bit addresses while IPv6 uses 128-bit addresses.  IPv4 addresses are written in the dotted-decimal format like 192.168.1.1, while IPv6 addresses usually use hexadecimal numbers separated by colons like 2001:db8:85a3::8a2e:370:7334.  In that way they do look a lot like MAC addresses.  The IPv6 address used is based on or influenced by the hardware MAC address, not because it couldn't be done differently, but because the MAC address is a unique identifier for the host possessing the address.

----------

## xtz

Based on experience with other suh cases: If you do not intend to use ipv6 for now, it is STROOOONGLY advised to put -ipv6 in the USE flags in /etc/make.conf. If something is build with -ipv6, I don't think it will try to use ipv6 even if you have support for it in the kernel.

----------

## desultory

 *xtz wrote:*   

> Based on experience with other suh cases: If you do not intend to use ipv6 for now, it is STROOOONGLY advised to put -ipv6 in the USE flags in /etc/make.conf. If something is build with -ipv6, I don't think it will try to use ipv6 even if you have support for it in the kernel.

 Why so vehement? The transition to IPv6 is essentially inevitable. Why remove support for it now when it can be, quite harmlessly, in place and ready for use when the infrastructure catches up?

----------

## cach0rr0

 *desultory wrote:*   

>  *xtz wrote:*   Based on experience with other suh cases: If you do not intend to use ipv6 for now, it is STROOOONGLY advised to put -ipv6 in the USE flags in /etc/make.conf. If something is build with -ipv6, I don't think it will try to use ipv6 even if you have support for it in the kernel. Why so vehement? The transition to IPv6 is essentially inevitable. Why remove support for it now when it can be, quite harmlessly, in place and ready for use when the infrastructure catches up?

 

Too many issues like this one have kept me from wanting to be an early adopter. 

Surely ipv6 is well on its way, no contention there, but I'm waiting 'til it becomes closer to imperative, as I can be a bit more sure niggling issues like that will be worked out.

----------

## xtz

desultory, as I said (and cach0rr0 also) there have been too many issues, caused by that. I'm not denying IPv6. I even encourage people to migrate to it (when possible, of course). But if you don't have plans to switch to IPv6 in the near future, I think it's causing too much headaches compared to no advantages at all, until you actually switch. Plus, it can be enabled at any time.

----------

