# SSH with 2 NICs - not working right.

## Shirow

OK,

I'm having a weird SSH problem.

I have 2 NICs in my box - one going to a router (which is connected to a cable modem) and one going directly to a cable modem.

Here's the pertinent info:

eth0      Link encap:Ethernet  HWaddr xxxx

          inet addr:MYPUBLICIPADDESS  Bcast:255.255.255.255  Mask:255.255.240.0

          UP BROADCAST NOTRAILERS RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:24548539 errors:2390 dropped:1738 overruns:146 frame:4780

          TX packets:11200002 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:100

          RX bytes:1511072948 (1441.0 Mb)  TX bytes:2745757280 (2618.5 Mb)

          Interrupt:11

eth1      Link encap:Ethernet  HWaddr xxxx

          inet addr:192.168.0.128  Bcast:192.168.0.255  Mask:255.255.255.0

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:169102 errors:0 dropped:0 overruns:0 frame:0

          TX packets:97618 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:100

          RX bytes:216485027 (206.4 Mb)  TX bytes:15431013 (14.7 Mb)

          Interrupt:11 Base address:0x2000

lo        Link encap:Local Loopback

          inet addr:127.0.0.1  Mask:255.0.0.0

          UP LOOPBACK RUNNING  MTU:16436  Metric:1

          RX packets:13 errors:0 dropped:0 overruns:0 frame:0

          TX packets:13 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:0

          RX bytes:1020 (1020.0 b)  TX bytes:1020 (1020.0 b)

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

192.168.0.0     *               255.255.255.0   U     0      0        0 eth1

mypublicgateway *               255.255.240.0   U     0      0        0 eth0

loopback        localhost       255.0.0.0       UG    0      0        0 lo

default         mypublicgateway 0.0.0.0         UG    0      0        0 eth0

I want internet traffic to go out the NIC connected to the cable modem - I have the second NIC connected to my LAN for internal SSH access and file transfers.

From inside the LAN, I can SSH into my box 100% of the time on it's private IP address.

From outside the LAN, SSH randomly works and doesn't work. Sometimes I get in, sometimes it hangs at the password prompt, sometimes it refuses connection.

I've restarted SSH and nothing changes.

eth0 is DHCP, eth1 has a statically assigned private IP and a static route pointing to my router (see route statement above.)

Any ideas?

----------

## Shirow

just bumping, hoping someone can help!

----------

## appetitus

Your information is somewhat useless.  You don't give enough details about the actual network setup and the little you give is confusing (at the top, both NICs goto modems, at the bottom one is now your LAN).  Try a diagram, or a better word picture.  Firewalls, DHCP and sshd configs are all places that could be misconfigured.  Tools such as 'netstat' and 'nmap' are your friends, or the GUI ethereal.  It's pretty simple, on the external eth, you need to see port 22 LISTENING and then your rules about ssh connects need to be satisfied.

----------

## Shirow

Hello,

The first NIC isn't going to a modem directly - it's going to a router which is connected to a modem. I thought, by my route entry, it wouldn't be using this router for outbound traffic (as the modem that the second NIC is directly connected to is set as the default gw.)

I've been through nmap etc, and port 22 is listening on the NIC directly connected to a modem. No firewall is running. However, SSH still only works intermittently on the directly connected NIC.

What info would be useful for people to help troubleshoot this problem? Here's a (crappy!) diagram of the network: black lines are cat5 and red lines are the way traffic is flowing.

http://www.digitalgunfire.com/images/network.gif

----------

## viperlin

i think your having trouble using ssh from like outside (internet)

if you use https://grc.com/x/ne.dll?bh0bkyd2

(or if that fails, grc.com shields up portscan test, test all service ports)

check that port 22 is open.

(it's still a bit confusing   :Twisted Evil:  )

try putting an "internet" square on that image so we can tel which modem is going to the internet, and a green line saying where the ssh path you want to happen goes.

----------

## Shirow

Hi,

Thanks for the reply. Both modems go to the Internet - I want to use the modem behind the router for the Windows boxes (which is working) and the second modem which is not behind a router for the Gentoo box (this is also working.)

Port 22 is up - I CAN actually connect to my sshd - I often get to username and/or password prompt - sometimes I can even login. Othertimes, it times out at the password prompt.

----------

## viperlin

 *Shirow wrote:*   

> Hi,
> 
> Thanks for the reply. Both modems go to the Internet - I want to use the modem behind the router for the Windows boxes (which is working) and the second modem which is not behind a router for the Gentoo box (this is also working.)
> 
> Port 22 is up - I CAN actually connect to my sshd - I often get to username and/or password prompt - sometimes I can even login. Othertimes, it times out at the password prompt.

 

then if the router is the dhcp server then it will be telling the gentoo box to use it as a gateway making it go through the windows modem

set it to a static IP address (the gentoo box) and set DNS and Gateway to the cablemodem you want it to use (DNS may need to be your ISP's DNS server if the modem does not support DNS forwarding)

----------

## Shirow

I don't have eth0 (the NIC connected to the router) set to DHCP - I statically assigned a private IP address and setup a static route from the gentoo box to the router.

The second nic (eth1) is set to DHCP and is getting a gateway from the second cable modem.

Just to clarify - I AM going out through the second modem - that part is working fine. The only problem is the intermittent inbound SSH. But it DOES work randomly.

----------

## viperlin

IP address clash?   :Shocked:  (dhcp is crap IMHO, never has worked.)

----------

## Shirow

Nope, that part seems to be working fine.

I'm not 100% sure it's a routing problem even. The only thing I can think is that for some reason, SSH is trying to go out over the router as well as through the modem - but why would it randomly choose one over the other - and even this seems unlikely, as I only have one default gateway.

The other thing I notice is that, if I try and login a bunch of times remotely, then login over my LAN, I'll have a ton of sshd processes. So it looks as though some of them just aren't getting responses and aren't terminating.

The strangest thing is.. I get to username prompt. I get the RSA key. I get password prompt.. THEN it stops responding.

----------

## nevynxxx

Unless you pin it down, ssh should accept connections over each and every interface you have, this is how it works for me.

Internal network->eth0---router---eth1->cablemodem->internet

as I understand it you have

internet->eth0->---gentoo box---eth1->hub<-Internal network

                                                               <-cablemodem -> internet

i.e. both the internal network and the cable modem coming off the hub.

Assuming all this works, which simple pings from the apropriate boxes will prove, make sure there are no iptables rules stopping port 22 on either ethx and make sure ssh is not bound to any ethx, could be it is bound to one ethx and only gets through when the (wrong) cable modem has the (coincidently) correct settings.

----------

## Shirow

Yeah, all the routing works.

How can I check the SSH binding?

The reason it doesn't work inbound from outside the router (e.g. inbound to eth1 outside the network) is that the cable modem is the default gateway, so the responses try to go out through that modem.

At least, that's what I think..  :Wink: 

----------

## grimshaw

The ssh problem is not related to your NICs.  Your network setup looks fine.  A tad unconventional, but it is fine.

Lets try this two ways...

OUTBOUND:

On the gentoo box, ssh to someplace else  (lan or Internet).  

Does this work repeatedly?  Any errors?

INBOUND:

When you ssh to your gentoo box and you have problems, are you coming from the same node?  Or different nodes?

If it is from the same node, does this node have any trouble connecting via ssh to other machines?

I wonder if the problem is not with ssh on your gentoo box, but instead with the installation on the remote machine.

Run ssh with some verbosity when doing your tests.

ssh -vvv -l user ip.ofgentoo.com

- John

----------

## Shirow

 *Quote:*   

> OUTBOUND: 
> 
> On the gentoo box, ssh to someplace else (lan or Internet). 
> 
> Does this work repeatedly? Any errors? 
> ...

 

I'll try this when I get home - I'm pretty sure it works but I'd have to retest it to be 100% sure.

 *Quote:*   

> NBOUND: 
> 
> When you ssh to your gentoo box and you have problems, are you coming from the same node? Or different nodes? 
> 
> If it is from the same node, does this node have any trouble connecting via ssh to other machines? 
> ...

 

Different nodes. These nodes can all SSH fine to other boxes.

I'll try the outbound and let you know for sure. I'll try the verbose option too. Thanks!

----------

## Shirow

Here's some data..

OpenSSH_3.6.1p1+CAN-2003-0693, SSH protocols 1.5/2.0, OpenSSL 0x0090702f

debug1: Reading configuration data /etc/ssh_config

debug1: Rhosts Authentication disabled, originating port will not be trusted.

debug2: ssh_connect: needpriv 0

debug1: Connecting to mybox [myip] port 22.

debug1: Connection established.

debug1: identity file /Users/admin/.ssh/identity type -1

debug1: identity file /Users/admin/.ssh/id_rsa type -1

debug1: identity file /Users/admin/.ssh/id_dsa type -1

debug1: Remote protocol version 1.99, remote software version OpenSSH_3.7.1p2

debug1: match: OpenSSH_3.7.1p2 pat OpenSSH*

debug1: Enabling compatibility mode for protocol 2.0

debug1: Local version string SSH-2.0-OpenSSH_3.6.1p1+CAN-2003-0693

debug3: Trying to reverse map address MYIPADDRESS.

debug1: Miscellaneous failure

No credentials cache found

debug1: Miscellaneous failure

No credentials cache found

debug1: SSH2_MSG_KEXINIT sent

debug1: SSH2_MSG_KEXINIT received

debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1

debug2: kex_parse_kexinit: ssh-rsa,ssh-dss

debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se

debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se

debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96

debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96

debug2: kex_parse_kexinit: none,zlib

debug2: kex_parse_kexinit: none,zlib

debug2: kex_parse_kexinit: 

debug2: kex_parse_kexinit: 

debug2: kex_parse_kexinit: first_kex_follows 0 

debug2: kex_parse_kexinit: reserved 0 

debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1

debug2: kex_parse_kexinit: ssh-rsa,ssh-dss

debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr

debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour,aes192-cbc,aes256-cbc,rijndael-cbc@lysator.liu.se,aes128-ctr,aes192-ctr,aes256-ctr

debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96

debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96

debug2: kex_parse_kexinit: none,zlib

debug2: kex_parse_kexinit: none,zlib

debug2: kex_parse_kexinit: 

debug2: kex_parse_kexinit: 

debug2: kex_parse_kexinit: first_kex_follows 0 

debug2: kex_parse_kexinit: reserved 0 

debug2: mac_init: found hmac-md5

debug1: kex: server->client aes128-cbc hmac-md5 none

debug2: mac_init: found hmac-md5

debug1: kex: client->server aes128-cbc hmac-md5 none

debug1: SSH2_MSG_KEX_DH_GEX_REQUEST sent

debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP

debug2: dh_gen_key: priv key bits set: 118/256

debug2: bits set: 1557/3191

debug1: SSH2_MSG_KEX_DH_GEX_INIT sent

debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY

Now I'm getting a timeout there. I saw the reverse lookup failure. Interesting. The IP does resolve to a valid name. I'm not sure if that is a big issue or not?

Looks like it's not getting a reply on some query, whatever 

debug1: SSH2_MSG_KEX_DH_GEX_INIT sent

debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY

is.

----------

## nevynxxx

 *Shirow wrote:*   

> How can I check the SSH binding?
> 
> 

 

It would be in /etc/ssh/sshd_config soemwhere I think.

----------

## grimshaw

What is the authentication mechanism on the gentoo box?  Are you using kerberos?

- John

----------

## Shirow

Nope. Whatever the SSH default is.

----------

## grimshaw

 *Quote:*   

> 1 debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
> 
> 2 debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
> 
> 3 debug3: check_host_in_hostfile: filename /home/username/.ssh/known_hosts
> ...

 

I added the line numbers for reference.  This is an exceprt of my own login.  You login script is failing at line two.  As you will see on line three it should be consulting your known hosts file.  Check the permissions for readability and if the file exists.

- John

----------

## Shirow

Would there be a different hosts file for different NICs? I would assume it's the same.. and this works fine from inside my LAN.

I'll do the -vvv from inside my LAN and see what it tells me.

The only thing is... I HAVE got in from outside my LAN before.. it just randomly works sometimes and not others.

----------

## grimshaw

Not really.  The .known_hosts file is per user (one file in the user's home directory), not per interface.  It will, however, have entries of public keys from machines that have connected before.  

If a key changes on the remote host (if ssh is updated there and the key is regenerated), ssh will usually complain and tell you the keys don't matcj.  Then just delete the corresponding line in your .known_hosts file and let it regenerate.

- John

----------

## Shirow

Doesn't sound like that's it then, since it works 100% over the LAN.

*clutches at straws*

----------

## grimshaw

It occurs to me that I'm thinking in reverse.  The known_hosts file is stored on the connecting machine.  Check it on a machine that is having trouble reaching the gentoo box.  If you don't have write access to the file, you might be getting this type of error.  Admittedly, it is something of a longshot.

- John

----------

## Shirow

Yeah, long shot  :Smile:  I can connect to other boxes just fine from this machine - the hosts file is good too.

 :Sad: 

----------

## grimshaw

Just to make sure, by "hosts" file you mean "/home/user/.ssh/known_hosts" ?

OK.  New tactic.  Can you bring the gentoo box down for some testing?

I'm thinking of the following to eliminate any configuration wonkiness and get us to a sterile environment.

Put a gentoo build CD in the box.  Reboot and start the ssh daemon.  Set a password for root.  Go to a node that couldn't connect and try again.

- John

----------

## Shirow

Yes, known_hosts looks to me fine.

I *could* - rather not (it's streaming to a shoutcast server and it'd disconnect all the clients.)

I could setup a backup to stream and take it down after that.

Just needs some shuffling around - can't get to the box other than by SSH (I just SSH into it over my LAN) so I'll have to drag a keyboard/mouse to it.

----------

## grimshaw

I hated to ask about a reboot (I covet my uptime), but I'm a bit stumped and could use a sanity check that eliminates bot configuration and software issues on the gentoo box.

Perhaps this can wait until a maintenance window?

- John

----------

## Shirow

Yeah, it makes sense and it's what I was getting close to doing.

Just was hoping someone would say 'oh yeah this happened to me last week.. this is how you fix it'

 :Smile: 

----------

## Shirow

So, switching the NIC ended up fixing the problem.

Weird, eh?

----------

