# Login not possible anymore

## Jimini

Hi folks,

from time to time I have the problem on my local file server, that logging in (locally and via SSH) is not possible anymore. Since rebooting it is not an acceptable solution for me, I'd like to dig into this problem :)

- Login via SSH does not work. I enter the password / key pass phrase and then the connection times out after ~2 minutes with "packet_write_wait: Connection to 10.0.0.2 port 22: Broken pipe". See output of ssh -vvv and my sshd_config attached under this posting.

- Local login does not work. I enter the password, see the time of the last login and nothing happens - no prompt or something like that.

- Remote logging does not work anymore. The system's syslog-ng is connected to my log server, but no data is transmitted.

- The problem occurred for about three times during the last six months. I can not correlate it with changes on the system.

- The system responds to ping and HTTPS requests.

- The system is monitored with Zabbix, which is complaining about network errors since the moment from which on no log data is transmitted anymore.

- According to Zabbix, enough disk space is available.

Of course I can not tell you about the exact software versions I use on this system, since I am unable to look them up right now. I use Gentoo with a 4.19 kernel and the latest stable version of syslog-ng and openssh.

ssh -vvv

```
OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n  7 Dec 2017

debug1: Reading configuration data /home/jimini/.ssh/config

debug1: /home/jimini/.ssh/config line 1: Applying options for 10.0.0.2

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: /etc/ssh/ssh_config line 19: Applying options for *

debug2: resolving "10.0.0.2" port 22

debug2: ssh_connect_direct: needpriv 0

debug1: Connecting to 10.0.0.2 [10.0.0.2] port 22.

debug1: Connection established.

debug1: identity file /home/jimini/.ssh/privkey type 0

debug1: key_load_public: No such file or directory

debug1: identity file /home/jimini/.ssh/privkey-cert type -1

debug1: Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3

debug1: Remote protocol version 2.0, remote software version OpenSSH_7.9

debug1: match: OpenSSH_7.9 pat OpenSSH* compat 0x04000000

debug2: fd 3 setting O_NONBLOCK

debug1: Authenticating to 10.0.0.2:22 as 'jimini'

debug3: hostkeys_foreach: reading file "/home/jimini/.ssh/known_hosts"

debug3: record_hostkey: found key type RSA in file /home/jimini/.ssh/known_hosts:25

debug3: load_hostkeys: loaded 1 keys from 10.0.0.2

debug3: order_hostkeyalgs: prefer hostkeyalgs: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa

debug3: send packet: type 20

debug1: SSH2_MSG_KEXINIT sent

debug3: receive packet: type 20

debug1: SSH2_MSG_KEXINIT received

debug2: local client KEXINIT proposal

debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c

debug2: host key algorithms: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519

debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com

debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com

debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1

debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1

debug2: compression ctos: none,zlib@openssh.com,zlib

debug2: compression stoc: none,zlib@openssh.com,zlib

debug2: languages ctos: 

debug2: languages stoc: 

debug2: first_kex_follows 0 

debug2: reserved 0 

debug2: peer server KEXINIT proposal

debug2: KEX algorithms: curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256

debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa

debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com

debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com

debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1

debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1

debug2: compression ctos: none,zlib@openssh.com

debug2: compression stoc: none,zlib@openssh.com

debug2: languages ctos: 

debug2: languages stoc: 

debug2: first_kex_follows 0 

debug2: reserved 0 

debug1: kex: algorithm: curve25519-sha256@libssh.org

debug1: kex: host key algorithm: rsa-sha2-512

debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none

debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none

debug3: send packet: type 30

debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

debug3: receive packet: type 31

debug1: Server host key: ssh-rsa SHA256:r7IFL7RaEBD79czOWMLRSHqfBeMzYtvDP2SvWZyZ3i8

debug3: hostkeys_foreach: reading file "/home/jimini/.ssh/known_hosts"

debug3: record_hostkey: found key type RSA in file /home/jimini/.ssh/known_hosts:25

debug3: load_hostkeys: loaded 1 keys from 10.0.0.2

debug1: Host '10.0.0.2' is known and matches the RSA host key.

debug1: Found key in /home/jimini/.ssh/known_hosts:25

debug3: send packet: type 21

debug2: set_newkeys: mode 1

debug1: rekey after 134217728 blocks

debug1: SSH2_MSG_NEWKEYS sent

debug1: expecting SSH2_MSG_NEWKEYS

debug3: receive packet: type 21

debug1: SSH2_MSG_NEWKEYS received

debug2: set_newkeys: mode 0

debug1: rekey after 134217728 blocks

debug2: key: /home/jimini/.ssh/privkey (0x556f0d54b200), explicit, agent

debug2: key: /home/jimini/.ssh/privkey2 (0x556f0d55b460), agent

debug2: key: jimini@Deimos (0x556f0d55b500), agent

debug2: key: jimini@Phobos (0x556f0d55bde0), agent

debug2: key: jimini@Phobos (0x556f0d55bf00), agent

debug3: send packet: type 5

debug3: receive packet: type 7

debug1: SSH2_MSG_EXT_INFO received

debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>

debug3: receive packet: type 6

debug2: service_accept: ssh-userauth

debug1: SSH2_MSG_SERVICE_ACCEPT received

debug3: send packet: type 50

debug3: receive packet: type 51

debug1: Authentications that can continue: publickey,password

debug3: start over, passed a different list publickey,password

debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password

debug3: authmethod_lookup publickey

debug3: remaining preferred: keyboard-interactive,password

debug3: authmethod_is_enabled publickey

debug1: Next authentication method: publickey

debug1: Offering public key: RSA SHA256:cJ2lDrsi2scaqqGcT9gi9iqYp+ImsPmFuRUsaKub6M4 /home/jimini/.ssh/privkey

debug3: send_pubkey_test

debug3: send packet: type 50

debug2: we sent a publickey packet, wait for reply

debug3: receive packet: type 60

debug1: Server accepts key: pkalg rsa-sha2-512 blen 279

debug2: input_userauth_pk_ok: fp SHA256:cJ2lDrsi2scaqqGcT9gi9iqYp+ImsPmFuRUsaKub6M4

debug3: sign_and_send_pubkey: RSA SHA256:cJ2lDrsi2scaqqGcT9gi9iqYp+ImsPmFuRUsaKub6M4

debug3: send packet: type 50

debug3: receive packet: type 52

debug1: Authentication succeeded (publickey).

Authenticated to 10.0.0.2 ([10.0.0.2]:22).

debug1: channel 0: new [client-session]

debug3: ssh_session2_open: channel_new: 0

debug2: channel 0: send open

debug3: send packet: type 90

debug1: Requesting no-more-sessions@openssh.com

debug3: send packet: type 80

debug1: Entering interactive session.

debug1: pledge: network

debug3: send packet: type 1

packet_write_wait: Connection to 10.0.0.2 port 22: Broken pipe
```

sshd_config

```
Port 12345

ListenAddress 10.0.0.2

Protocol 2

HostKey /etc/ssh/ssh_host_rsa_key

HostKey /etc/ssh/ssh_host_dsa_key

#HostKey /etc/ssh/ssh_host_ecdsa_key

#UsePrivilegeSeparation yes

#KeyRegenerationInterval 3600

#ServerKeyBits 768

SyslogFacility AUTH

LogLevel INFO

LoginGraceTime 120

PermitRootLogin no

StrictModes yes

PubkeyAuthentication yes

IgnoreRhosts yes

HostbasedAuthentication no

PermitEmptyPasswords no

ChallengeResponseAuthentication no

PasswordAuthentication no

X11Forwarding no

X11DisplayOffset 10

PrintMotd no

PrintLastLog yes

TCPKeepAlive no

MaxAuthTries 2

MaxSessions 2

AllowAgentForwarding NO

AllowTcpForwarding NO

KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256

KexAlgorithms curve25519-sha256@libssh.org,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1

# Allow client to pass locale environment variables

AcceptEnv LANG LC_*

UsePAM no

Subsystem sftp /usr/lib64/misc/sftp-server

ClientAliveInterval 600

ClientAliveCountMax 2

GatewayPorts yes

AllowTcpForwarding no
```

Any kind of assistance is welcome! :)

Kind regards,

Jimini

----------

## Jimini

I forgot to mention, that this system runs a LXC container with another Gentoo installation on it. This system also offers a SSH login, which does work normally.

King regards,

Jimini

----------

## alamahant

May be here is the problem?

```

TCPKeepAlive no 

```

May be try "yes" instead?

Furthermore 

```

UsePAM no 

```

It would be better if you used PAM with ssh.

From a centos sshd_config file:

 *Quote:*   

> 
> 
> # Set this to 'yes' to enable PAM authentication, account processing,
> 
> # and session processing. If this is enabled, PAM authentication will
> ...

 

----------

## Ant P.

Your sshd_config says port 12345 but you're trying to connect to port 22. You're probably not connecting to the sshd you think you are.

----------

## Jimini

Thank you for your replies. I assume, that the SSHd config should not be the problem, since local login does not work either. The connection to the SSHd seems to work, but for me it seems as if the system is unable to provide a shell.

Regarding the port: I have an iptables rule, which forwards tcp 22 to tcp 12345.

Kind regards,

Jimini

----------

## Ant P.

If the syslogger can't empty its buffers, then it's possible sshd is blocking when calling syslog(3). Is syslog-ng trying to print logs to its stdout? If that's the case and it has nowhere to write to, it'll eventually back up and the whole system will stop. I don't use syslog-ng, but I've had that happen with metalog.

----------

