# Hacked or paranoid? [SOLVED: paranoid]

## nihilo

I see the following in my logs:

```
Jan 28 23:59:27 [sshd] Invalid user wwwweb from 58.147.10.178

Jan 28 22:51:17 [sshd] input_userauth_request: invalid user wwwweb

Jan 28 22:51:11 [sshd] warning: /etc/hosts.deny, line 108: can't verify hostname: gethostbyname(mx-ll-58.147.10-178.tttmaxnet.com) failed

Jan 28 22:51:17 [sshd] reverse mapping checking getaddrinfo for mx-ll-58.147.10-178.tttmaxnet.com [58.147.10.178] failed - POSSIBLE BREAK-IN ATTEMPT!

Jan 28 22:51:17 [sshd] Invalid user wwwweb from 58.147.10.178

Jan 28 22:51:17 [sshd] input_userauth_request: invalid user wwwweb

Jan 28 22:51:21 [sshd] warning: /etc/hosts.deny, line 108: can't verify hostname: gethostbyname(mx-ll-58.147.10-178.tttmaxnet.com) failed

Jan 28 22:51:32 [sshd] Connection closed by 58.147.10.178
```

The reverse mapping messages I've seen frequently, and am not worried about. The "Connection closed by" is one that I haven't seen before, and the fact that the timestamp on that message is 11 seconds after the previous message scares me. 

Does "connection closed" mean that a successful connection was made (I guess through an unpatched bug in openssh), or merely that the attempt took that long?

Looking at 'last' shows no session at that time, but if hacked, that would probably have been altered. The ssh logs don't say anything about a session being opened at that time either, but again, a smart hack would have altered the logs.

I'm running the latest stable openssh, and all other software is up to date. Root login is disabled. Only specific users are enabled (of which there are very few). Authentication is via public key only. I don't run any security problems like PHP apps. 

I should be safe, right? Can anybody confirm that the "Connection closed" message definitely does not mean that a successful login occurred?

[EDIT] Am 99.9% sure that this was me being paranoid. There's been no anomalous network activity, and no more than usual.Last edited by nihilo on Wed Mar 19, 2008 9:28 pm; edited 1 time in total

----------

## downer

Hi,

I would read it as your box tries to do a reverse lookup of the client (which may be spoofing), but fails and after 10 or so seconds the client gets tired of waiting and hangs up. There is no indication that the client gained access.

//D

----------

## nihilo

 *downer wrote:*   

> Hi,
> 
> I would read it as your box tries to do a reverse lookup of the client (which may be spoofing), but fails and after 10 or so seconds the client gets tired of waiting and hangs up. There is no indication that the client gained access.
> 
> //D

 

Thanks for the fast response. The only thing that still has me a bit worried is that I get tons of messages everyday about the reverse lookup failure, but none of them except this one has the "Connection closed by" message -- and 10 seconds seems a nice amount of time to install a rootkit and change logs, etc.

The lookup failure message does not actually cause the authentication to fail in any way. It's just from 'denyhosts' which periodically adds suspicious ip addresses to a blacklist so that they can't even connect later. The client would not have to wait around at all, and could have tried to authenticate immediately.

Is it possible that clients usually try to authenticate and fail, but this one time, the client opened the connection but didn't finish trying to authenticate and then prematurely closed the connection after 10 seconds without authenticating? Perhaps that's what you were suggesting.

----------

## nihilo

Hmmm. One more tidbit of information:

I just tried to ssh in without using pubkey authentication, and then disconnected before trying to authenticate, and also tried the same with a 'kill -9' to the ssh client process before giving a password, and in neither case did the 'Connection closed by' message appear.

----------

## downer

sshd says that the user wwweb is invalid so that he should have been granted access is very unlikely (as in the death-of-openssh-type-bug). 

I don't know how you have you box set up but usually hosts not matching reverse lookup should be blocked (i know my debian host does this for instance).

Yes, I suggested that the client (most likely running a script) hit a timeout and simply gave up. On the other hand you would probably see this more in your logs in that case.

What do you get in the logs if you just lets it hang until it times out?

//D

----------

## nihilo

 *downer wrote:*   

> sshd says that the user wwweb is invalid so that he should have been granted access is very unlikely (as in the death-of-openssh-type-bug). 

 

Yeah, it would have to be a remote exploit that achieves root access (to change logs) using an invalid user when password authentication is not allowed. Seems pretty unlikely, and if so, then I guess tens of thousands of other people are probably getting hit by the same exploit this evening before it gets patched. 

 *downer wrote:*   

> I don't know how you have you box set up but usually hosts not matching reverse lookup should be blocked (i know my debian host does this for instance).

 

It's been a while since I set it up, but I think it checks once a minute or once every few minutes and then blocks the IP, so it's not realtime. I think I read somewhere that it wasn't safe to require reverse lookup to succeed because sometimes a legit client does not succeed for some reason. I could be misremembering that though.

 *downer wrote:*   

> Yes, I suggested that the client (most likely running a script) hit a timeout and simply gave up. On the other hand you would probably see this more in your logs in that case.
> 
> What do you get in the logs if you just lets it hang until it times out?
> 
> 

 

I only have the last 5 days or so of logs available, but there have been tons of reverse lookup messages in that time, with only that one "connection closed" message.

If I connect and let it hang, after the grace period expires (20 seconds), I see:

```
[sshd] error: ssh_msg_send: write
```

I don't know how to make a reverse lookup fail, otherwise I would try that and see if that gives that message.

----------

## nihilo

I'm off to bed. Thanks for the help, downer. I'll see if there was a massive hack overnight when I get up tomorrow, and otherwise will assume I'm safe.

Thanks again...

----------

## downer

 *nihilo wrote:*   

> Yeah, it would have to be a remote exploit that achieves root access (to change logs) using an invalid user when password authentication is not allowed. Seems pretty unlikely, and if so, then I guess tens of thousands of other people are probably getting hit by the same exploit this evening before it gets patched.

 

Yeah, if that were the case someone would have pulled the emergency break by now I guess. 

 *nihilo wrote:*   

> It's been a while since I set it up, but I think it checks once a minute or once every few minutes and then blocks the IP, so it's not realtime. I think I read somewhere that it wasn't safe to require reverse lookup to succeed because sometimes a legit client does not succeed for some reason. I could be misremembering that though.

 

I guess you can have the occational false-positive so to speak, but it has never happened to me.. But yeah, when I come to think about it, the option is called "PARANOID"  :Wink: 

 *nihilo wrote:*   

> I only have the last 5 days or so of logs available, but there have been tons of reverse lookup messages in that time, with only that one "connection closed" message.
> 
> If I connect and let it hang, after the grace period expires (20 seconds), I see:
> 
> ```
> ...

 

Ok, well I guess it can be client dependat also. I have seen some strange things when debugging ws_ftp or tectia ssh clients talking to openssh for instance, but assuming it is the same machine(s) hasseling you it probably would appear more. One option could be to change sshd port to something non standard. But that is probably only realistic if you don't have many users connecting to it... When I did it 99.95% av all evil connection attempts stopped. Ah, the silence...  :Smile: 

----------

## Akkara

How busy is your ethernet traffic normally?  Perhaps running a tcpdump overnight and then watching for things might help you isolate the cause.

----------

## Hu

Some networks lack proper reverse lookup records, even when they do not otherwise host malicious machines.  Therefore, you are correct to allow reverse lookups to fail.  However, if you know that all legitimate users of the server have proper reverse lookup records, it should be fine to enable that requirement.

You said denyhosts only bans peers every few minutes.  Perhaps denyhosts banned that client between the time the TCP connection completed (causing the reverse lookup attempt) and the time the client sent an authentication request (which would have been denied)?  If so, the firewall could have dropped the authentication request, leading to the apparent delay.  This would also explain the rarity of the event, since it becomes a race between the malicious client and the denyhosts ban loop.

----------

