# Log message: "Your system is part of a botnet."

## wilburpan

I have a basic Gentoo box used as a home server that I use ssh to log into remotely.  I've gotten used to messages in the logs about probable break in attempts, but as far as I can tell, there hasn't been a successful break in so far.

Today I looked at /var/log/message, and found this:

```
Jan 21 07:55:55 gentoobox sshd[22665]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116

Jan 21 09:51:00 gentoobox sshd[22825]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116

Jan 21 09:51:01 gentoobox sshd[22826]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116

Jan 21 09:51:01 gentoobox sshd[22827]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116

Jan 21 09:51:01 gentoobox sshd[22828]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116

Jan 21 09:51:02 gentoobox sshd[22829]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116

Jan 21 09:51:02 gentoobox sshd[22830]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116

Jan 21 09:51:02 gentoobox sshd[22831]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116

Jan 21 09:51:02 gentoobox sshd[22832]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116

Jan 21 09:51:03 gentoobox sshd[22833]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116

Jan 21 09:51:03 gentoobox sshd[22834]: Bad protocol version identification 'Your system is part of a botnet. Fix your SHIT!!!' from 88.79.164.116
```

All the other messages that I've seen before have some indication of a break in attempt, such as "invalid user", "pam_tally(sshd:auth): pam_get_uid; no such user", or ""reverse mapping... failed - POSSIBLE BREAK-IN ATTEMPT!"  But this is the first message I've seen that seems like it's coming back from an outside server.

The other thing is that yesterday my wife got a call from Comcast, which is my ISP, saying that they detected activity on our internet that made them suspect a virus on our computers, and their oh-so useful advice was to install and run Norton Anti-Virus, not realizing that we don't have any Windows computers in our house.  We just have the Gentoo Linux server and two MacBook Pros.  Just to be sure, I ran clamav on all three computers, and other than some potential phish emails, clamav didn't pick up anything.

Scanning /var/log/messages doesn't show anything obvious.

So what should I do?  I'm guessing that the call from Comcast and my finding this log message might just be a coincidence, but of course I'd like to be sure.  Any suggestions would be welcome.  Thanks in advance!

----------

## Jaglover

http://linux.slashdot.org/story/09/09/12/1413246/First-Botnet-of-Linux-Web-Servers-Discovered?from=rss

First thing to do is to take this box off line, your ISPs head up is probably not false positive.

----------

## NeddySeagoon

wilburpan,

Your uninvited guest may not have got in through ssh. Other exposed services can have vunerabilities.

Neither is root access required for many things.

Take the box off line and rebuild it. You can't patch it up.

----------

## EzInKy

 *NeddySeagoon wrote:*   

> wilburpan,
> 
> Your uninvited guest may not have got in through ssh. Other exposed services can have vunerabilities.
> 
> Neither is root access required for many things.
> ...

 

Still could have been through ssh though, I've been seeing distributed hammering in my logs:

```

Jan 21 14:08:13 *** sshd[7066]: error: PAM: Authentication failure for illegal user dragon from bzq-82-80-239-106.cablep.bezeqint.net

Jan 21 14:08:58 *** sshd[7073]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=190.144.143.204 

Jan 21 14:09:00 *** sshd[7071]: error: PAM: Authentication failure for illegal user dromero from 190.144.143.204

Jan 21 14:09:14 *** sshd[7076]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=vostokmv.ru 

Jan 21 14:09:16 *** sshd[7074]: error: PAM: Authentication failure for illegal user drop from vostokmv.ru

Jan 21 14:12:00 *** sshd[7098]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=80.227.150.70 

Jan 21 14:12:02 *** sshd[7096]: error: PAM: Authentication failure for illegal user dsmith from 80.227.150.70

Jan 21 14:12:04 *** sshd[7101]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=jxeelab.iasi.rdsnet.ro 

Jan 21 14:12:06 *** sshd[7099]: error: PAM: Authentication failure for illegal user dsp from jxeelab.iasi.rdsnet.ro

Jan 21 14:12:40 *** sshd[7104]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=noodle.natsci.csulb.edu 

Jan 21 14:12:41 *** sshd[7102]: error: PAM: Authentication failure for illegal user dspam from noodle.natsci.csulb.edu

Jan 21 14:12:46 *** sshd[7107]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=ipplan.inreach.com 

Jan 21 14:12:48 *** sshd[7105]: error: PAM: Authentication failure for illegal user dt from ipplan.inreach.com

Jan 21 14:13:55 *** sshd[7111]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=201-217-215-66-host.ifx.net.co 

Jan 21 14:13:57 *** sshd[7108]: error: PAM: Authentication failure for illegal user duane from 201-217-215-66-host.ifx.net.co

Jan 21 14:14:45 *** sshd[7117]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=bsn-77-81-11.dsl.siol.net 

Jan 21 14:14:47 *** sshd[7115]: error: PAM: Authentication failure for illegal user dummy from bsn-77-81-11.dsl.siol.net

Jan 21 14:16:13 *** sshd[7121]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=193.111.10.146 

Jan 21 14:16:15 *** sshd[7119]: error: PAM: Authentication failure for illegal user eacadm from 193.111.10.146

Jan 21 14:16:44 *** sshd[7124]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=gate.city-bild-werbung.de 

Jan 21 14:16:46 *** sshd[7122]: error: PAM: Authentication failure for illegal user eagle from gate.city-bild-werbung.de

Jan 21 14:17:02 *** sshd[7127]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=adsl-066-156-064-237.sip.asm.bellso

```

----------

## wilburpan

Rebuilding the box is not too much of a problem for me -- I've done it before.

What I could use some help with is how to go about identifying what was compromised so I can do a better job of protecting this box.  Basically, the only services that I had set up that could access this box was ssh and afp, and from reviewing logs I can't figure out how or what was compromised.  There are records of my ssh logins, cron jobs running, connections to the server via afp from the other computers in the house, and a ton of ssh attempts from hackers, all unsuccessful.

Otherwise, I'll probably wind up with this box set up just like it was before, which means that it will be compromised again, if that's what actually happened.

----------

## aCOSwt

 *Jaglover wrote:*   

> your ISPs head up is probably not false positive.

 

Most probably but is it a reason for being so trivialistically rude ?

After all, the uninvited guest came through the ISP's services, did'nt it ?

What is this shitty ISP ?

----------

## timeBandit

 *aCOSwt wrote:*   

>  *Jaglover wrote:*   your ISPs head up is probably not false positive. 
> 
> Most probably but is it a reason for being so trivialistically rude ?
> 
> After all, the uninvited guest came through the ISP's services, did'nt it ?
> ...

 Read the original post with a bit more care. The ISP (Comcast) was never stated to be rude. (Having some experience with Comcast, I never found them rude, only somewhat clueless or obtuse at worst.  :Rolling Eyes: )

The message in the OP's log came from a node somewhere in Germany (where Comcast doesn't have much of a presence  :Wink: ) that presumably got hammered by the compromised box, whose owner then had a look at the source of the unwanted traffic, found an SSH server there and decided to respond in kind. The phone call from Comcast, the ISP, was a separate event triggered by their independent detection of suspicious traffic to/from the OP's box.

----------

## NeddySeagoon

aCOSwt,

My ISP doesn't filter, block, virus scan, traffic shape or do anything nasty like that.

They just provide a pipe, so my securiy is my problem

wilburpan,

Look in /tmp for things that shouldn't be there, every user except nobody has write access there.

Look in every /home/<username> too.  

When you find something, the file or dir name will probably start with a dot, so use ls -a

Its unlikely intruders will leave anything in .bash_history but you never know.

chkrootkit and tools like it are useful sometimes.

You can make an image of the drive and do forensics later if youhave the space.

----------

## ppurka

Another option is to emerge rkhunter and chkrootkit and run them. It may (or may not) find some compromised binaries.

Hopefully, you will gain some information if either of those come up with some kind of positive result.

----------

## EzInKy

Good analysis, timeBandit B-)

----------

## EzInKy

 *NeddySeagoon wrote:*   

> aCOSwt,
> 
> My ISP doesn't filter, block, virus scan, traffic shape or do anything nasty like that.
> 
> 

 

Neither does mine, and I put up with slower connection than those that do just for the "privilege" of controlling my own connection.

----------

## Jaglover

 *wilburpan wrote:*   

> Rebuilding the box is not too much of a problem for me -- I've done it before.
> 
> What I could use some help with is how to go about identifying what was compromised so I can do a better job of protecting this box.  Basically, the only services that I had set up that could access this box was ssh and afp ...

 

I take this box is exposed directly to the internet? Disabling password authentication for SSH is a good measure. Don't know about AFP, was this service open to the internet, too? You can run portscan on this box to see what ports are open.

----------

## eccerr0r

I'm glad there are people that actually look at their logs even if they run machines that get 0wn3d.

Whoever's at 88.79.164.116 I have to give props for.  I know the chances of them reading this board is unlikely but I have to give thanks for his/her effort in helping taking 0wn3d machines off the net.  I may have to start doing this as well despite the fact that the reason machines get 0wn3d is because they don't update, and likely they don't check logs either... normally I have to go straight to the ISP to get them cut off as the ISP more likely has the capability...

Also keep in mind virus (detected by clamav) and worms (typically not detected) are different.  But at least worms tend to be accompanied by real "root kits" which tend to be detectable at least now.

Don't take the ISP warning lightly, this is likely real.  Botnets strive to maintain a low profile to make sure their rootkit is NOT detected, and most certainly does NOT want to try to trip off your alarm to go looking for one.

For curiosity's sake, how often do you sync and update your box, and reboot (or at least restart the updated applications)?  Recently there's been a lot of bots randomly ssh'ing into my machines, need to make sure nobody has weak passwords... or do away with passwords (key based, vpn, etc.) if possible...

AFP = NetATalk? (appletalk fileserver?!??!) oh boy...

----------

## cach0rr0

 *ppurka wrote:*   

> Another option is to emerge rkhunter and chkrootkit and run them. It may (or may not) find some compromised binaries.
> 
> Hopefully, you will gain some information if either of those come up with some kind of positive result.

 

I wouldn't trust 'em. The second I become compromised, I no longer trust any application that's on my system - even ones whose purpose is identifying a breach - nor do I trust my running kernel. 

Would recommend he boots from a livecd that has the requisite tools, and perform any scans from there; if his kernel is compromised, scanning from within the compromised environment is bordering on pointless. 

@OP

the ssh 'hail mary' attacks as one poster put it, have been off the charts over the past few days. On the 17th alone I was hit well over 1000 times, from a few hundred different IP's - and i like yourself just run a little home server on my residential Comcast account. 

(full log here - http://whitehathouston.com/naughty.log )

(unique IP list here - http://whitehathouston.com/ip_naughty.log )

Would highly recommend you do the following

-take the machine off the network

-boot from a livecd that's capable of mounting all partitions on your machine (so if e.g. you use LVM, make sure the CD supports LVM)

-booted into the livecd environment, snag an external hard drive, and back up any important files you plan to keep to the external HDD

-tar up your old system, compress it, and copy the resultant archive to your external HDD so that you can review it later and try to determine how they got in

-blow away your server install completely. Reformat every partition and so forth. 

-rebuild your home server per the "Hardened Gentoo" route, e.g. go hardened-sources, enable the grsec/PaX options, and take the time to set up grsec RBAC

-disable keyboard-interactive authentication on SSH, use only public-key auth (carry your keys on a thumb drive on your car keychain!)

-do a careful audit of what all services you're running. If it's only SSH and AFP, you're already done the necessary bits to tighten down SSH by moving to key-based auth, so it's time to take great care in tightening down AFP (I have no idea what that is?) per whatever doc you can find. Again, RBAC can do a tremendous amount here, ensuring that app only has access to the files and directories that are truly needed.

-set up a glsa-check to run regularly as a cron job, and mail you with the results (ex: glsa-check -ql affected )

----------

## ppurka

 *cach0rr0 wrote:*   

>  *ppurka wrote:*   Another option is to emerge rkhunter and chkrootkit and run them. It may (or may not) find some compromised binaries.
> 
> Hopefully, you will gain some information if either of those come up with some kind of positive result. 
> 
> I wouldn't trust 'em. The second I become compromised, I no longer trust any application that's on my system - even ones whose purpose is identifying a breach - nor do I trust my running kernel. 

 That is why I told the OP to *emerge* them. If you are insecure then you can unmerge and then re-emerge them.

----------

## wilburpan

Thanks for the input.  To address some of the questions and comments:

-- Nothing in /tmp or /home that shouldn't be there, even hidden files are accounted for.

-- I do have ssh with password authentication.  But if someone was able to hack into my box by figuring out the password, shouldn't there be a log message accounting for that?  I'm the only one that should be using ssh to log into this box, and I don't see any successful logins that weren't from me.

-- AFP is Apple File Protocol, from the netatalk package, and I use it for mounting the server as a network drive from my Macs.  It's the easiest way I've found since I use Macs.

-- My box is connected to the internet via sshd and afpd.  The first to log into to do stuff, and the second so I can mount my home server as a network drive on my MacBook Pro when I am away from home.

-- My box is pretty up to date.  I'm running ~x86, and I emerge --sync && emerge -DNUv world about twice a week.

I tend to reboot with kernel upgrades (duh) and with changes to what seem to me to be major Gentoo system upgrades, like coreutils.

Some more questions before I continue:

So of course I'll need to back up the contents of /home before shutting down my box.  How will I know that if I just clone my /home directory, I'm not also just cloning whatever rootkit/worm/virus that has gotten in?

Also, just for argument's sake, the phone call that we got from Comcast notwithstanding, how do I know that the "Bad protocol version identification" message I got isn't in response to some botnet attack from another compromised box out there that has spoofed my IP address, and the box in Germany being attacked erroneously thinks that it's my box doing it?  I've been looking through lots of logs now looking for the break in, and I haven't found it yet, and I started thinking that this might be pretty slim evidence of a break in to warrant rebuilding my box.

----------

## Jaglover

 *Quote:*   

> -- Nothing in /tmp or /home that shouldn't be there, even hidden files are accounted for. 

 

You can be sure of this only if you look at this system from a LiveCD, no tool in your box can be trusted.

Is this box behind a NAT router with only two ports forwarded?

----------

## wilburpan

Yes, only those two ports are forwarded, and I have a NAT router.  I'm using a different public TCP port than the private one for AFP, but I didn't do the same for ssh.    :Embarassed:   I just changed the public TCP port for ssh from 22 to something else for the time being.

----------

## eccerr0r

 *wilburpan wrote:*   

> Also, just for argument's sake, the phone call that we got from Comcast notwithstanding, how do I know that the "Bad protocol version identification" message I got isn't in response to some botnet attack from another compromised box out there that has spoofed my IP address, and the box in Germany being attacked erroneously thinks that it's my box doing it?  I've been looking through lots of logs now looking for the break in, and I haven't found it yet, and I started thinking that this might be pretty slim evidence of a break in to warrant rebuilding my box.

 

If I saw that message in my logs I would at least do due diligence and at least check if I do or not.  You should check your machine carefully and make sure it was NOT compromised, and if it wasn't, simply ignore the warning, but I don't think anyone really would spend the effort to do such warning unless there really was an issue with your box.  This coupled with the ISP warning just seems too real (the only other possible scenario is if you had dynamic IP that *just* changed and someone else had your IP and was infected.... but your ISP should know when the handoff occured and not make this mistake).  The message the person did was too specific to what the problem is, which is why I think it's real.

Remember breakins were meant to be stealthy, they DON'T want you to know so that you DON'T remove it, and they can have free access to YOUR CPU and YOUR network.  They can remove your lastlog or purge entries, or possible not even show an entry if they don't use a wtmp library call to get in.

One thing you should look at is whether your lastlog's first entry is when _you_ last deleted or created it... if there's a discrepancy, that's a point of alarm (though no apparent modification is not a certain sign.)

You mention you're using a NAT box... maybe it's not your linux box and some other box you have behind your NAT?  Then that could be another explanation and Linux is proven safe once again!  Remember that the other person who's complaining about you has no idea you're running NAT and only thing he/she can tell is that you're running sshd... and can't tell if the sshd machine is the same as the attacker machine.

----------

## Jaglover

wilburpan,

using non-standard ports is deceiving yourself. It can stop script-kiddies, but they are not the real threat. Furthermore, logs can be cleaned up - and they will be in case of pro. You can look at bash history of root, but if you find nothing then it is no proof your box is not compromised.

----------

## wilburpan

 *eccerr0r wrote:*   

> You mention you're using a NAT box... maybe it's not your linux box and some other box you have behind your NAT?  Then that could be another explanation and Linux is proven safe once again!  Remember that the other person who's complaining about you has no idea you're running NAT and only thing he/she can tell is that you're running sshd... and can't tell if the sshd machine is the same as the attacker machine.

 

I'm doubtful that it's another box.  The other two computers we have are Macbook Pros, which also have the same *nix underpinnings.  Neither one of them have any type of sharing or access turned on within OS X, and the two ports my router has open both forward to my Linux box.

----------

## wilburpan

So going back to my other question, I'll need to back up the contents of /home before shutting down my box. I'll also save /etc and my kernel config, to save me some time in reconstructing my settings.

So how will I know that if I just clone/backup my /home and /etc directories, I'm not also just cloning whatever rootkit/worm/virus that has gotten in, and when I copy the contents of /home to the rebuilt server, I won't be reinstalling the offending agent?

----------

## cach0rr0

 *wilburpan wrote:*   

> So going back to my other question, I'll need to back up the contents of /home before shutting down my box. I'll also save /etc and my kernel config, to save me some time in reconstructing my settings.
> 
> So how will I know that if I just clone/backup my /home and /etc directories, I'm not also just cloning whatever rootkit/worm/virus that has gotten in, and when I copy the contents of /home to the rebuilt server, I won't be reinstalling the offending agent?

 

/home is very unlikely to contain a culprit 

in the unlikely event it has anything useful, doing the usual requisite scans from a different kernel, with a different OS install (or livecd) would likely be sufficient. 

Or the better move, identify specifically what you want to save from /home; your MP3's are not going to be the culprit, for example. Back up "only what you need", as it were.

----------

## eccerr0r

 *wilburpan wrote:*   

> I'm doubtful that it's another box.  The other two computers we have are Macbook Pros, which also have the same *nix underpinnings.  Neither one of them have any type of sharing or access turned on within OS X, and the two ports my router has open both forward to my Linux box.

 

NO!!! Please don't make assumptions like that!  EVERY machine, including the router (!!!), is suspect till we know which one was at fault (if any, of course we aren't sure!).  Please check ALL your boxen. You can't rule out anything till you at least looked at it.

Some routers can be rooted by a public web address.  Do you have that enabled?

Did you check the Macbooks for viruses?  Are you sure the intrusion mechanism wasn't from malware versus an automated attempt (worm)?

Remember that there are more OSX machines out there than Linux already.  Hackers will target OSX first due to market share.

But yes, favorite places to put rootkits on Linux (OSX too) is in /dev and /usr/lib, because they have so much mumbojumbo in them to disguise it...

----------

## Jaglover

Returning to the main question - how they did it - in case it is your Gentoo box after all.

I'd say it was brute-force attack on SSH or vulnerability in ASF.

As I said before, disable password-based authentication for SSH. Don't know much about ASF, but the filesystem shared should be mounted with noexec option to start with, I'm sure Google can tell more about securing AFS.

----------

## eccerr0r

Since netatalk/afp was on a nonstandard port (or it seems to be implied to be the case) the chances that it was found is rare unless one of your remote users divulged it.

I'd be very hesitant on putting netatalk on a public port because I don't see it getting much review as sshd, etc., as it's not nearly a popular package as something like samba, but samba's a different story...

BTW: The "Hail Mary" wasn't my coining, check out slashdot:

http://linux.slashdot.org/story/09/11/15/1653228/The-Hail-Mary-Cloud-Is-Growing

----------

## Jaglover

 *Quote:*   

> Since netatalk/afp was on a nonstandard port (or it seems to be implied to be the case) the chances that it was found is rare ...

 

Sorry, cannot agree with this. Pros do full portsacans and tools they use react to response not to port #.

----------

## wilburpan

 *eccerr0r wrote:*   

> NO!!! Please don't make assumptions like that!  EVERY machine, including the router (!!!), is suspect till we know which one was at fault (if any, of course we aren't sure!).  Please check ALL your boxen. You can't rule out anything till you at least looked at it.

 

Actually, I did that already.

 *eccerr0r wrote:*   

> Some routers can be rooted by a public web address.  Do you have that enabled?

 

No.

 *eccerr0r wrote:*   

> Did you check the Macbooks for viruses?  Are you sure the intrusion mechanism wasn't from malware versus an automated attempt (worm)?

 

Yes, for viruses.  Since I still don't know what the intrusion mechanism was, I can't really answer the second question.

----------

## wilburpan

 *Jaglover wrote:*   

> Returning to the main question - how they did it - in case it is your Gentoo box after all.
> 
> I'd say it was brute-force attack on SSH or vulnerability in ASF.
> 
> As I said before, disable password-based authentication for SSH. Don't know much about ASF, but the filesystem shared should be mounted with noexec option to start with, I'm sure Google can tell more about securing AFS.

 

The noexec option is in place for afs.  I'm reading over the documentation for disabling password-based authentication now.

----------

## eccerr0r

 *Jaglover wrote:*   

>  *Quote:*   Since netatalk/afp was on a nonstandard port (or it seems to be implied to be the case) the chances that it was found is rare ... 
> 
> Sorry, cannot agree with this. Pros do full portsacans and tools they use react to response not to port #.

 

While it is true it is NOT security to move ports, people don't portscan unless they have reason to want to portscan that machine.  Which means if it was the entry point, the perpetrator likely knew the owner of the system....  Portscans are time costly especially remotely, and also very easy to detect/block versus a hapless user trying to guess a forgotten password.

I do not get portscanned regularly, that would show up in my logs.

----------

## timeBandit

 *wilburpan wrote:*   

> I do have ssh with password authentication.  But if someone was able to hack into my box by figuring out the password, shouldn't there be a log message accounting for that?  I'm the only one that should be using ssh to log into this box, and I don't see any successful logins that weren't from me.

 The logs question has already been addressed, accompanied by the solid recommendation to switch to PK-only authentication after you rebuild the box. To that I will add my strong recommendation to tunnel AFP over SSH. There is no reason to expose any public port other than SSH when you're the sole remote user. Block AFP at your router, tell your SSH client to forward it to your client machine when you connect and mount your remote volume through the tunnel. If you experience unacceptably slow performance due to the encryption overhead, you can experiment with different ciphers (also SSH client options). Some of the available ciphers are much faster than others.

I'd also suggest having a look at this tip I posted a few years back, which explains how to have your machine send you a text message whenever you connect. My defense plan if I ever received an unexpected alert was to log in ASAP and run /etc/init.d/net.eth0 stop--even over SSH.  :Mr. Green: 

----------

## wilburpan

So I finally had a chance to call Comcast to get more details about what they saw.  (My wife got the initial call from them.)  The only information I could get is that Comcast received a report from a third party service, which they would not identify.  The report said that a connection from my IP address was made to a botnet command center.  There's no information on when that happened, how many times it happened, what port was used, what the IP address of the command center was, or any other information that might be useful to help me track this down.

The Comcast rep did agree with me that this amount of information was basically like telling me that someone said they smelled smoke briefly coming from my house and then running away without saying anything more about it.   :Rolling Eyes:  He also said that what I've done so far with virus and rootkit scans (all negative) was probably a good enough effort from their standpoint.  So I don't know how helpful this was in the end

So for the time being I've shut down the ssh and afp services at my router, and I'll have to find some time to rebuild my Gentoo server box.  I'll keep all the suggestions in mind.

If someone could point me to a howto on tunnelling services through ssh, that would be great.

----------

## Jaglover

 *eccerr0r wrote:*   

> I do not get portscanned regularly, that would show up in my logs.

 

I do not have definite answer to this. I do know when I move my alternate http port I'll get this next day:

```
222.215.230.49 - - [21/Jan/2010:01:34:21 +0000] "GET /prx1.php?hash=CD4698206B83BA6C40ED4D651F405A2E5566E36F1A05 HTTP/1.0" 404 203

222.215.230.49 - - [21/Jan/2010:01:34:22 +0000] "GET /prx1.php?hash=CD4698206B83BA6C40ED4D651F405A2E5566E36F1A05 HTTP/1.0" 404 203

125.65.112.161 - - [21/Jan/2010:03:34:25 +0000] "GET /prx1.php?hash=CD4698206B83BA6C40ED4D651F405A2E5566E36F1A05 HTTP/1.0" 404 203

125.65.112.161 - - [21/Jan/2010:03:34:26 +0000] "GET /prx1.php?hash=CD4698206B83BA6C40ED4D651F405A2E5566E36F1A05 HTTP/1.0" 404 203

125.65.112.168 - - [21/Jan/2010:05:19:31 +0000] "GET /prx1.php HTTP/1.0" 404 203

125.65.112.168 - - [21/Jan/2010:05:19:31 +0000] "GET /prx1.php HTTP/1.0" 404 203

125.65.112.168 - - [21/Jan/2010:05:53:59 +0000] "GET /prx1.php HTTP/1.0" 404 203

125.65.112.168 - - [21/Jan/2010:05:53:59 +0000] "GET /prx1.php HTTP/1.0" 404 203

125.65.112.168 - - [21/Jan/2010:07:47:57 +0000] "GET /prx1.php HTTP/1.0" 404 203

```

I'd say the bottom line is the capability of bad guys should not be underestimated. They have botnets at their disposal, after all.   :Rolling Eyes: 

----------

## Jaglover

 *wilburpan wrote:*   

> If someone could point me to a howto on tunnelling services through ssh, that would be great.

 

It's not hard, but why recite what is already there:

http://www.google.com/linux?hl=en&q=ssh+tunneling&btnG=Search

 :Smile: 

----------

## eccerr0r

 *Jaglover wrote:*   

> I do not have definite answer to this. I do know when I move my alternate http port I'll get this next day:
> 
> 

 

What port did you end up using?  Hope it wasn't some x000 number or some fairly common port used for some service...

It seems a lot of programs these days are using http-like syntax for commands (look at CUPS? Wingate? squid?), so it might just be possible they were looking for some other proxy program that usually used the port number you chose.  Proxies are loved by hackers to hide their tracks.

I'm not disagreeing that port moves is not security, ... but it does tend to slow them down if they have to search more possibilities (it's like adding another byte to your password).  Hackers want easy targets first.

----------

## Shining Arcanine

Here is a good guide on X11 tunnelling:

http://souptonuts.sourceforge.net/sshtips.htm

By the way, if you want to move a service to an uncommonly used port, I suggest picking one that is above 5000.

----------

## Jaglover

 *Quote:*   

> What port did you end up using? Hope it wasn't some x000 number or some fairly common port used for some service...

 It was, in fact, 8008. 

I'm getting new ISP this Sunday, let's see how long it takes until they find me.

Being not a blackhat myself it's hard to tell how good the tools are they use. Surfing the net one stumbles across interesting stuff sometimes, I'm pretty sure they scan whole subnets in a glimpse, process is fully automated, everything goes on while the blackhat himself is sitting in Aruba, smoking best stuff, surrounded by concubines.   :Evil or Very Mad: 

----------

## eccerr0r

ssh tends to have a higher value if exploited, so that's more of a service to protect.  Plus it's "well known" and almost "ubiquitous" so it's a good target.

It looks like the attacks (prx1.php) is a gateway type exploit.  They're exploiting a gateway software instead of Apache -- which would never work.

I wouldn't be surprised you get hit with something on the first day you get a new IP address.  I have a static /29 and several of the IP addresses I don't have machines allocated and when I do hook one up, I get an attack, indicating they are either scanning sequentially or via algorithm.

Nmap seems to be a favorite tool for portscanning.  This tool does portscans fairly quickly, but it takes a while to scan each host individually.

----------

## cach0rr0

 *eccerr0r wrote:*   

> 
> 
> Nmap seems to be a favorite tool for portscanning.  This tool does portscans fairly quickly, but it takes a while to scan each host individually.

 

nmap is still a tried and true badass tool for scanning and enumeration. 

And if they have thousands of bots at their disposal, with each bot only scanning a limited set of ports with -sV, not terribly difficult to figure out who has SSH daemons running on alternate ports. 

True enough you remove a large chunk if you move ssh off of -p22, but you'll still have a non-trivial amount of hits - enough to be cause for alarm. 

Your sshd will be found eventually. So while there is some limited merit in the whole 'security through obscurity', I'm much happier leaving ssh on 22, and making for damn sure that's not going to be an attack vector no matter how long you sit and brute it. 

Plus the usual precautious, e.g. mounting /tmp as noexec,nosuid, RBAC, hardened-sources, disabling file uploads in PHP, requiring SSL+Basic or client certs to be able to touch any of my potentially exploitable webapps, limited privs for every mysql user.

----------

