# Can no longer 'ssh' .. [SOLVED]

## dufeu

edit: original topic "Can no longer 'ssh' into my PCs since last @world emerge"

Ever since I last did an 'emerge @world' on all my local PCs, I can no longer 'ssh' from any local PC into any local PC.

For eash PC, /var/log/emerge.log shows {representative sample created by 'grep openssh /var/log/emerge.log'} something similar to this:

```
1437073416:  >>> emerge (160 of 178) net-misc/openssh-6.9_p1-r2 to /

1437073416:  === (160 of 178) Cleaning (net-misc/openssh-6.9_p1-r2::/usr/portage/net-misc/openssh/openssh-6.9_p1-r2.ebuild)

1437073417:  === (160 of 178) Compiling/Packaging (net-misc/openssh-6.9_p1-r2::/usr/portage/net-misc/openssh/openssh-6.9_p1-r2.ebuild)

1437073547:  === (160 of 178) Merging (net-misc/openssh-6.9_p1-r2::/usr/portage/net-misc/openssh/openssh-6.9_p1-r2.ebuild)

1437073557:  >>> AUTOCLEAN: net-misc/openssh:0

1437073557:  === Unmerging... (net-misc/openssh-6.9_p1-r1)

1437073563:  >>> unmerge success: net-misc/openssh-6.9_p1-r1

1437073570:  === (160 of 178) Post-Build Cleaning (net-misc/openssh-6.9_p1-r2::/usr/portage/net-misc/openssh/openssh-6.9_p1-r2.ebuild)

1437073570:  ::: completed emerge (160 of 178) net-misc/openssh-6.9_p1-r2 to /

1440308983:  >>> emerge (18 of 68) net-misc/openssh-7.1_p1 to /

1440308983:  === (18 of 68) Cleaning (net-misc/openssh-7.1_p1::/usr/portage/net-misc/openssh/openssh-7.1_p1.ebuild)

1440308984:  === (18 of 68) Compiling/Packaging (net-misc/openssh-7.1_p1::/usr/portage/net-misc/openssh/openssh-7.1_p1.ebuild)

1440309115:  === (18 of 68) Merging (net-misc/openssh-7.1_p1::/usr/portage/net-misc/openssh/openssh-7.1_p1.ebuild)

1440309124:  >>> AUTOCLEAN: net-misc/openssh:0

1440309124:  === Unmerging... (net-misc/openssh-6.9_p1-r2)

1440309130:  >>> unmerge success: net-misc/openssh-6.9_p1-r2

1440309138:  === (18 of 68) Post-Build Cleaning (net-misc/openssh-7.1_p1::/usr/portage/net-misc/openssh/openssh-7.1_p1.ebuild)

1440309138:  ::: completed emerge (18 of 68) net-misc/openssh-7.1_p1 to /

1440380243:  *** emerge --quiet-build=y --config net-misc/openssh-7.1_p1

1440380254:  *** emerge --quiet-build=y --config =net-misc/openssh-7.1_p1
```

I've tried re-initializing the various ssh_host keys using ssh-keygen and everything else I've thought of trying hasn't worked so far.

This is a typical listing of host_keys prior to using ssh-keygen:

```
-rw-r--r-- 1 root root 300324 Aug 23 01:51 moduli

-rw-r--r-- 1 root root   1637 Aug 23 01:51 ssh_config

-rw------- 1 root root    672 Jul 29  2007 ssh_host_dsa_key

-rw-r--r-- 1 root root    602 Jul 29  2007 ssh_host_dsa_key.pub

-rw------- 1 root root    227 Jan 26  2011 ssh_host_ecdsa_key

-rw-r--r-- 1 root root    175 Jan 26  2011 ssh_host_ecdsa_key.pub

-rw------- 1 root root    399 Mar 30  2014 ssh_host_ed25519_key

-rw-r--r-- 1 root root     95 Mar 30  2014 ssh_host_ed25519_key.pub

-rw------- 1 root root    527 Jul 29  2007 ssh_host_key

-rw-r--r-- 1 root root    331 Jul 29  2007 ssh_host_key.pub

-rw------- 1 root root   1671 Jul 29  2007 ssh_host_rsa_key

-rw-r--r-- 1 root root    394 Jul 29  2007 ssh_host_rsa_key.pub

-rw------- 1 root root   4045 Aug 23 01:51 sshd_config
```

After using 'ssh-keygen', all the host_keys show new date I created them so I know 'ssh-keygen' did something.

When I add the '-v' option, the 'ssh' dialog looks like this:

```
OpenSSH_7.1p1-hpn14v5, OpenSSL 1.0.2d 9 Jul 2015

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: Connecting to 192.168.1.200 [192.168.1.200] port 22.

debug1: Connection established.

debug1: key_load_public: No such file or directory

debug1: identity file /home/guynonet/.ssh/id_rsa type -1

debug1: key_load_public: No such file or directory

debug1: identity file /home/guynonet/.ssh/id_rsa-cert type -1

debug1: key_load_public: No such file or directory

debug1: identity file /home/guynonet/.ssh/id_dsa type -1

debug1: key_load_public: No such file or directory

debug1: identity file /home/guynonet/.ssh/id_dsa-cert type -1

debug1: key_load_public: No such file or directory

debug1: identity file /home/guynonet/.ssh/id_ecdsa type -1

debug1: key_load_public: No such file or directory

debug1: identity file /home/guynonet/.ssh/id_ecdsa-cert type -1

debug1: key_load_public: No such file or directory                                                                                                         

debug1: identity file /home/guynonet/.ssh/id_ed25519 type -1

debug1: key_load_public: No such file or directory

debug1: identity file /home/guynonet/.ssh/id_ed25519-cert type -1

debug1: Enabling compatibility mode for protocol 2.0

debug1: Local version string SSH-2.0-OpenSSH_7.1p1-hpn14v5

debug1: Remote protocol version 2.0, remote software version OpenSSH_7.1p1-hpn14v5

debug1: match: OpenSSH_7.1p1-hpn14v5 pat OpenSSH* compat 0x04000000

debug1: Authenticating to 192.168.1.200:22 as 'root'

debug1: SSH2_MSG_KEXINIT sent

debug1: SSH2_MSG_KEXINIT received

debug1: AUTH STATE IS 0

debug1: REQUESTED ENC.NAME is 'chacha20-poly1305@openssh.com'

debug1: kex: server->client chacha20-poly1305@openssh.com <implicit> none

debug1: REQUESTED ENC.NAME is 'chacha20-poly1305@openssh.com'

debug1: kex: client->server chacha20-poly1305@openssh.com <implicit> none

debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

debug1: Server host key: ecdsa-sha2-nistp256 SHA256:yRZx2hw8SFKCvWhwwhkUBeBB4SzHuAZRq2eKmTQniZE

debug1: Host '192.168.1.200' is known and matches the ECDSA host key.

debug1: Found key in /home/guynonet/.ssh/known_hosts:1

debug1: SSH2_MSG_NEWKEYS sent

debug1: expecting SSH2_MSG_NEWKEYS

debug1: SSH2_MSG_NEWKEYS received

debug1: Roaming not allowed by server

debug1: SSH2_MSG_SERVICE_REQUEST sent

debug1: SSH2_MSG_SERVICE_ACCEPT received

debug1: Authentications that can continue: publickey,keyboard-interactive

debug1: Next authentication method: publickey

debug1: Trying private key: /home/guynonet/.ssh/id_rsa

debug1: Trying private key: /home/guynonet/.ssh/id_dsa

debug1: Trying private key: /home/guynonet/.ssh/id_ecdsa

debug1: Trying private key: /home/guynonet/.ssh/id_ed25519

debug1: Next authentication method: keyboard-interactive

Password: 

debug1: Authentications that can continue: publickey,keyboard-interactive

Password: 

debug1: Authentications that can continue: publickey,keyboard-interactive

Password: 

debug1: Authentications that can continue: publickey,keyboard-interactive

debug1: No more authentication methods to try.

Permission denied (publickey,keyboard-interactive)
```

I can locally log in as normal on all my local PCs so I don't think this is an issue with 'pam'. 

I haven't yet tried going back to the previous version of 'openssh'. I would rather get this new upgrade resolved than go back.

Some advice would sure be appreciated. It's very inconveinent to go from PC to PC to get things done. And yes, because of this issue with 'ssh', I also can't do local rsyncs between any of my local PCs as well. I've been doing needed file copies/transfers through existing 'nfs' files shares.

Final note: If I boot a local PC using 'sysrescd', I can 'ssh' into that 'sysrescd' intance from any of my other local PCs. I'm hoping that there is some simple 'sshd_config' change I need to make that I missed.

----------

## Tony0945

Did you read the eselect news of  2015-08-13  titled "OpenSSH 7.0 disables ssh-dss keys by default"?

Could that be your problem?

----------

## Buffoon

Probably remote root login is not allowed.

----------

## krinn

I'm sorry i see nothing wrong with your log, you must be clearer.

Here's what i see:

```
debug1: Authenticating to 192.168.1.200:22 as 'root' 
```

You are logging to your server as root

```
debug1: Found key in /home/guynonet/.ssh/known_hosts:1 
```

But your are login with user guynonet (let's call it guynonet@localhost to clear things later); so you must be doing ssh -l root server or ssh root@server

```
debug1: Trying private key: /home/guynonet/.ssh/id_rsa 
```

client is sending guynonet keys, that's normal you are log as this user, so in order for this to work, your server must have guynonet keys in its authorized_keys, if the server do have the keys for root@localhost it will not work, you are not root@localhost, you are guynonet@localhost trying to log as root@server, so guynonet keys must be in the server root authorized_keys.

If you forget to add guygnonet keys in your authorized_keys on your server, you can do "su" and be root, and when you will ssh, you will be root@localhost login to root@server, and this time, it will sent the root keys and it will works.

the server side authorized_keys:

if you are guynonet@localhost login to guynonet@server, the keys must be in server:/home/guynonet/.ssh/authorized_keys, but you are doing a guynonet@localhost login as root, so the guynonet keys must be in server:/root/.ssh/authorized_keys for this to work. Got it client is guynonet and it will send guynonet keys to server, but you want log as root on server, so any keys sent to the server will be compare in the root account, so the server:/root/.ssh/authorized_keys.

guynonet@localhost>ssh server = login as guynonet on server, keys of guynonet must be in server /home/guynonet/.ssh/authorized_keys

guynonet@localhost>ssh root@server = login as root on server, keys of guynonet must be in server /root/.ssh/authorized_keys

root@localhost>ssh server = login as root on server, keys of root@localhost must be in server /root/.ssh/authorized_keys

guynonet@localhost>su>root@localhost>ssh server = same as upper

```
Password:

debug1: Authentications that can continue: publickey,keyboard-interactive

```

just the 3 logins attempts, again, nothing bad from what i see

when the server is asking password, if you type guynonet password, you are wrong again, you must type the root password of the server, even you are guynonet you are login to the server as root, so it's the root password of the server that is asked.

If what i said seems clear and you think you did that and it doesn't work, you must explain a bit better what you do and what

Per exemple, you are showing us a list of ssh-keygen keys as some kind of proof, but where are these keys from??? we can see they are own by root:root, so you must be showing your root keys, fine, but the root@localhost or root@server keys?

For the reason i explain upper, if you are showing us root@localhost keys, it doesn't matter if they are correct or not, your log show guynonet@localhost is trying to log as root@server, so it is guynonet keys that would be revelant, and root@localhost keys are not use at all.

The only disturbing part of your post is the emerge.log of your grep ssh, i'm still trying to figure out why you provide this  :Very Happy: 

----------

## grosgood

```

1440309138:  ::: completed emerge (18 of 68) net-misc/openssh-7.1_p1 to /

```

and then...

```

$ python -c "import datetime as dt; print dt.datetime.fromtimestamp(1440309138)"

2015-08-23 01:52:18

```

and then...

```

-rw-r--r-- 1 root root   1637 Aug 23 01:51 ssh_config 

<stuff and feathers>....

-rw------- 1 root root   4045 Aug 23 01:51 sshd_config

```

leads me to think that the OP has had a configuration change happen whilst unawares, as the time stamp on these

configuration files are within a second or so of the completion of the merge. Probably the out-of-the-box settings now prevail, rather than what the OP has in mind. If that should be the case, then from man sshd_config:

 *Quote:*   

> 
> 
>  PermitRootLogin
> 
>              Specifies whether root can log in using ssh(1).  The argument
> ...

 

So Buffoon is pretty close, methinks, and so is krinn -- account/credentials mismatch. Would have been interesting to see what was typed in at the command line. Was it ssh <account_name>@host or just ssh host?

Hope the old configs were backed away somewhere...

----------

## dufeu

All these answers are interesting on a first quick read {I just got home}

I see though that I should have also included my 'process'.

I have my main PC set as my primary workstation where I'm usually sitting. I normally log into this PC with my normal user login. This is where I control/do all my work on the other PCs on my network. I do this by setting up a 'konsole' window for each subsiduary PC. Each 'konsole' window has 6-8 tabs open. In each tab, I normally do "ssh root@192.168.1.xxx" to get to the root login for that PC. This mean I usually have 6-8 'root' logins per PC.

I have 3 PCs set up as 'binary package servers'. I have one each for 'amd64', 'i686', and 'atom'. Therefore, I normally log in as root on eash of these in order to perform @world emerge etc. They're basically used for our systems infrastructure maintenance. Since they're not used 'interactively', I normally just set up emerges with some side use as fileservers/testbeds. {For example. I've been testing 'btrfs' over 'luks' on one}.

So, I normally log into my main PC with my usual user account. I then open a 'konsole'. In the console, I'll open 6 tabs. In each tab, I'll 'ssh' into the subsiduary PC that I'm using that 'konsole' window for.

I've done my 'portage' related maintenance/work across PCs for years this way and it has always 'just worked'.

As I read your replies, there have been changes in what constitutes default security settings/configuration/expectations. 

I'm not sure yet what I should be reading in order to understand this better.

If my explanation of my set up makes things clearer in terms of what I need to now know, I'd appreciate further guidance!

And thanks HUGELY for your current replies. It will take me a while to digest them, but at least I have some starting points to work from.

BTW - I've always done this work using 'out of the box/default' settings. Something to keep in mind.  :Wink: 

----------

## dufeu

 *grosgood wrote:*   

> 
> 
> ```
> 
> 1440309138:  ::: completed emerge (18 of 68) net-misc/openssh-7.1_p1 to /
> ...

 

You are correct and this is the problem.

I've change sshd_config 'PermitRootLogin' and can now log in.

This is an occasion where I would have expected to be prompted/warned via 'etc-update'. I was not prompted on 5 different PCs. Should I report this as a bug in the 'openssh' ebuild? I'm really thinking that the emerge behavior here does not follow gentoo standard treatment of configuration files.

Thoughts?

----------

## grosgood

@dufeu

CONFIG_PROTECT is your friend. man emerge and scroll down to "CONFIGURATION FILES." See also "Configuration Protection". Be aware of how CONFIG_PROTECT_MASK and FEATURES="...config-protect-if-modified..." are set too. Ebuild authors are not allowed to touch CONFIG_PROTECT and company; it belongs to the user to use fairly, poorly or not at all. I see no basis for bug reporting.

```

$ emerge --info | grep CONFIG_PROTECT

```

shows how these environment variables are set on your system(s), or if they have been set at all (which would be surprising).

Example: Here is how it is set on my box:

```

 $ emerge --info | grep CONFIG_PROTECT

CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/gnupg/qualified.txt"

CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"

```

/etc is generally protected by the first setting, but particular directories have been "unprotected" via the MASK setting.

These may be set/overwritten in a variety of places. Your profile establishes base settings; these may be amended/overridden in places like /etc/env.d, /etc/portage/make.conf, your shell configuration... see the provided links for details. 

FEATURES="...config-protect-if-modified..." unprotects configuration files that would otherwise be protected if they have never been modified after installation. This means, if you have never touched /etc/ssh/sshd_config, then emerge will ignore CONFIG_PROTECT because (presumably) you have never modified the file with site-specific settings. This reduces the amount of configuration merging one has to do after an update, at the cost of "surprises" that occur when one has been relying on default behaviors of never-touched configuration files that upstream authors then change. This may be what had occurred in your case, but there is a bit of mystery there. PermitRootLogin has been defaulting to "no" for about a million years. Maybe a little less. So it is not clear to me, if you've never touched these files, how you've been going in as root via ssh all this time. That aside, now that you have changed the file, the FEATURES setting no longer applies and sshd.config is now under the scope CONFIG_PROTECT, assuming that this environment variable includes /etc/ssh in the first place. So, brew yourself a pot of coffee and check that CONFIG_PROTECT is working the way that you would like. Have fun; take care.

POST EDIT: While poking around configuration files, you might consider HostbasedAuthentication, which conveniently precludes having to enter login passwords over ssh. Your session just starts, seemingly without a password. Behind the scenes, you're authenticated by means of public/private keypairs. Same mechanism supports the very paranoid -- the password becomes a strictly local affair between you and your ssh client as you are actually unlocking your private key during authentication. Keystrokes never go over the wire. These two related mechanisms entail distributing public/private key pairs to your various user accounts and ssh servers. Man ssh_config (no 'd') and man ssh-keygen for more background. -- G.

----------

