# ssh fails from host Gentoo to kvm qemu FreeBSD guest

## lalebarde

Hi all,

I have set up a virtual machine based on FreeBSD with use of kvm qemu on my Gentoo. I don't manage to have a ssh connection working thought I digged and followed (I hope) the manuals. Please I need some help. 

Here is the information of interest (of course, I will post more on demand) :

HOST side :

```
chronos@JANUS ~/Documents $ kvm -hda freebsd-8.1-amd64.img -cdrom  FreeBSD-8.1-RELEASE-amd64-dvd1.iso -m 1000 -k fr -net nic,macaddr=02:5a:4b:3c:2d:10 -net tap,ifname=qtap0,script=no,downscript=no -redir tcp:22:192.168.99.66:22  -redir tcp:80:192.168.99.66:80 &
```

```
chronos@JANUS ~/Documents $ ping appolon

PING appolon (192.168.99.66) 56(84) bytes of data.

64 bytes from appolon (192.168.99.66): icmp_req=1 ttl=64 time=1.09 ms

64 bytes from appolon (192.168.99.66): icmp_req=2 ttl=64 time=0.159 ms

64 bytes from appolon (192.168.99.66): icmp_req=3 ttl=64 time=0.171 ms
```

```
chronos@JANUS ~/Documents $ ssh -v -p 42852 mercure@appolon

OpenSSH_5.8p1-hpn13v10lpk, OpenSSL 1.0.0d 8 Feb 2011

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: Connecting to appolon [192.168.99.66] port 42852.

debug1: Connection established.

debug1: identity file /home/chronos/.ssh/id_rsa type -1

debug1: identity file /home/chronos/.ssh/id_rsa-cert type -1

debug1: identity file /home/chronos/.ssh/id_dsa type 2

debug1: identity file /home/chronos/.ssh/id_dsa-cert type -1

debug1: identity file /home/chronos/.ssh/id_ecdsa type -1

debug1: identity file /home/chronos/.ssh/id_ecdsa-cert type -1

debug1: Remote protocol version 2.0, remote software version OpenSSH_5.4p1 --------------------------------

debug1: match: OpenSSH_5.4p1 -------------------------------- pat OpenSSH*

debug1: Remote is NON-HPN aware

debug1: Enabling compatibility mode for protocol 2.0

debug1: Local version string SSH-2.0-OpenSSH_5.8p1-hpn13v10lpk

debug1: SSH2_MSG_KEXINIT sent

debug1: SSH2_MSG_KEXINIT received

debug1: AUTH STATE IS 0

debug1: REQUESTED ENC.NAME is 'aes128-ctr'

debug1: kex: server->client aes128-ctr hmac-md5 none

debug1: REQUESTED ENC.NAME is 'aes128-ctr'

debug1: kex: client->server aes128-ctr hmac-md5 none

debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent

debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP

debug1: SSH2_MSG_KEX_DH_GEX_INIT sent

debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY

debug1: Server host key: DSA 7a:a5:44:d6:59:5e:2b:0f:03:a2:e0:f3:06:b1:75:7f

debug1: Host '[appolon]:42852' is known and matches the DSA host key.

debug1: Found key in /home/chronos/.ssh/known_hosts:2

debug1: ssh_dss_verify: signature correct

debug1: SSH2_MSG_NEWKEYS sent

debug1: expecting SSH2_MSG_NEWKEYS

debug1: SSH2_MSG_NEWKEYS received

debug1: Roaming not allowed by server

debug1: SSH2_MSG_SERVICE_REQUEST sent

debug1: SSH2_MSG_SERVICE_ACCEPT received

debug1: Authentications that can continue: publickey,password

debug1: Next authentication method: publickey

debug1: Trying private key: /home/chronos/.ssh/id_rsa

debug1: Offering DSA public key: /home/chronos/.ssh/id_dsa

debug1: Authentications that can continue: publickey,password

debug1: Trying private key: /home/chronos/.ssh/id_ecdsa

debug1: Next authentication method: password

mercure@appolon's password: 

debug1: Authentications that can continue: publickey,password

Permission denied, please try again.

mercure@appolon's password: 

debug1: Authentications that can continue: publickey,password

Permission denied, please try again.

mercure@appolon's password: 

debug1: Authentications that can continue: publickey,password

debug1: No more authentication methods to try.

Permission denied (publickey,password).
```

```
$ scp -P 42852 ~/.ssh/id_dsa.pub chronos@appolon:/home/chronos/.ssh/

chronos@appolon's password: 

Permission denied, please try again.

chronos@appolon's password: 

Permission denied, please try again.

chronos@appolon's password: 

Permission denied (publickey,password).

lost connection
```

```
$ scp -P 42852 ~/.ssh/id_dsa.pub mercure@appolon:/home/mercure/.ssh/

mercure@appolon's password: 

Permission denied, please try again.

mercure@appolon's password: 

Permission denied, please try again.

mercure@appolon's password: 

Permission denied (publickey,password).

lost connection
```

Both users mercure and appolon are declared in the guest (I added appolon with the same password than in the host thought it should be useless).

```
# uname -a

Linux JANUS 2.6.37-tuxonice #4 SMP PREEMPT Sun Aug 14 12:37:01 CEST 2011 x86_64 Intel(R) Core(TM) i7 CPU 860 @ 2.80GHz GenuineIntel GNU/Linux
```

```
# eix -AI ssh

[I] net-misc/openssh

     Installed versions:  5.8_p1-r1(08:26:24 05/06/2011)(X hpn kerberos ldap pam tcpd -X509 -libedit -selinux -skey -static)

[I] virtual/ssh

     Installed versions:  0(08:37:55 05/06/2011)(-minimal)
```

```
# eix -AI kvm

[U] app-emulation/qemu-kvm

     Installed versions:  0.14.1-r2!t[1](20:53:08 24/07/2011)(aio alsa bluetooth curl hardened jpeg ncurses png qemu_softmmu_targets_i386 qemu_softmmu_targets_x86_64 qemu_user_targets_i386 qemu_user_targets_x86_64 sdl ssl vhost-net 
```

```
# grep chronos /etc/group

audio:x:18:chronos

cdrom:x:19:haldaemon,chronos

video:x:27:root,chronos

usb:x:85:haldaemon,chronos

users:x:100:games,chronos

plugdev:x:1003:haldaemon,chronos

gdm:x:2006:chronos

kvm:x:102:chronos

chronos:x:4000:
```

```
# ifconfig -a

br0       Lien encap:Ethernet  HWaddr 02:5a:4b:3c:2d:1e  

          inet adr:192.168.99.1  Bcast:192.168.99.255  Masque:255.255.255.0

          adr inet6: fe80::5a:4bff:fe3c:2d1e/64 Scope:Lien

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:627 errors:0 dropped:0 overruns:0 frame:0

          TX packets:1130 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 lg file transmission:0 

          RX bytes:95126 (92.8 KiB)  TX bytes:142184 (138.8 KiB)

eth0      Lien encap:Ethernet  HWaddr a4:ba:db:e9:66:02  

          inet adr:192.168.0.10  Bcast:192.168.0.255  Masque:255.255.255.0

          adr inet6: fe80::a6ba:dbff:fee9:6602/64 Scope:Lien

          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

          RX packets:17184 errors:0 dropped:475 overruns:0 frame:0

          TX packets:16775 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 lg file transmission:1000 

          RX bytes:13195752 (12.5 MiB)  TX bytes:3687244 (3.5 MiB)

          Interruption:17 

lo        Lien encap:Boucle locale  

          inet adr:127.0.0.1  Masque:255.0.0.0

          adr inet6: ::1/128 Scope:Hôte

          UP LOOPBACK RUNNING  MTU:16436  Metric:1

          RX packets:57770 errors:0 dropped:0 overruns:0 frame:0

          TX packets:57770 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 lg file transmission:0 

          RX bytes:44585512 (42.5 MiB)  TX bytes:44585512 (42.5 MiB)

qtap0     Lien encap:Ethernet  HWaddr 02:5a:4b:3c:2d:1e  

          adr inet6: fe80::5a:4bff:fe3c:2d1e/64 Scope:Lien

          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1

          RX packets:627 errors:0 dropped:0 overruns:0 frame:0

          TX packets:1432 errors:0 dropped:4 overruns:0 carrier:0

          collisions:0 lg file transmission:500 

          RX bytes:103904 (101.4 KiB)  TX bytes:168424 (164.4 KiB)

sit0      Lien encap:IPv6-dans-IPv4  

          NOARP  MTU:1480  Metric:1

          RX packets:0 errors:0 dropped:0 overruns:0 frame:0

          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 lg file transmission:0 

          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
```

```
# rc-status | egrep -i "tap|kvm|br|eth|ssh"

 net.qtap0                                                         [  started  ]

 net.br0                                                           [  started  ]

 net.eth0                                                          [  started  ]

 bridge_forward                                                    [  started  ]

 kvm                                                               [  started  ]

 sshd                                                              [  started  ]

 net.eth0                                                          [  started  ]
```

```
# /sbin/iptables -L -v -t nat

Chain PREROUTING (policy ACCEPT 175 packets, 26385 bytes)

 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 92 packets, 20919 bytes)

 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 31221 packets, 1927K bytes)

 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 5941 packets, 362K bytes)

 pkts bytes target     prot opt in     out     source               destination         

25363 1570K MASQUERADE  all  --  any    eth0    anywhere             anywhere            

# /sbin/iptables -L -v

Chain INPUT (policy ACCEPT 288K packets, 219M bytes)

 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 177 packets, 17231 bytes)

 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 286K packets, 94M bytes)

 pkts bytes target     prot opt in     out     source               destination     
```

```
# dmesg | grep -i kvm

[37616.250903] kvm: enabling virtualization on CPU1

[37616.354704] kvm: enabling virtualization on CPU2

[37616.458507] kvm: enabling virtualization on CPU3

[37616.561311] kvm: enabling virtualization on CPU4

[37616.664111] kvm: enabling virtualization on CPU5

[37616.766916] kvm: enabling virtualization on CPU6

[37616.870719] kvm: enabling virtualization on CPU7

[    0.001999] Marking TSC unstable due to KVM discovered backwards TSC
```

GUEST side :

```
#       $OpenBSD: sshd_config,v 1.81 2009/10/08 14:03:41 markus Exp $

#       $FreeBSD: src/crypto/openssh/sshd_config,v 1.49.2.2.2.1 2010/06/14 02:09:06 kensmith Exp $ 

Port 42852

ListenAddress 0.0.0.0

Protocol 2

HostKey /etc/ssh/ssh_host_dsa_key 

HostKey /etc/ssh/ssh_host_rsa_key 

SyslogFacility AUTH

LoginGraceTime 2m

PermitRootLogin yes     #to be set to no !!!

StrictModes no               #to be set to yes !!!

MaxAuthTries 6

MaxSessions 10

RSAAuthentication yes  #not sure if I have to keep this one to yes

PubkeyAuthentication yes

AuthorizedKeysFile     .ssh/authorized_keys

PasswordAuthentication yes     #to be set to no !!!

PermitEmptyPasswords no 

ChallengeResponseAuthentication no

KeepAlive yes

Compression delayed

ClientAliveInterval 20

ClientAliveCountMax 6

Subsystem       sftp    /usr/libexec/sftp-server

AllowUsers mercure@192.168.99.1 chronos@192.168.99.1 mercure@192.168.0.10 chronos@192.168.0.10      #Eventually, I should keep only  mercure@192.168.99.1 which is the bridge br0 in the host and the gateway in the guest

AllowGroups sshusers wheel chronos mercure      #Eventually, suppress chronos and mercure
```

```
#ssh_config

   RSAAuthentication yes

   PasswordAuthentication yes

   HostbasedAuthentication no

   GSSAPIAuthentication no

   GSSAPIDelegateCredentials no

   Port 42852
```

```
# ifconfig -a

re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

   options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGINGVLAN_HWCSUM>

   ether 02:5a:4b:3c:2d:10

   inet 192.168.99.66 netmask 0xffffff00 broadcast 192.168.99.255 

   media: Ethernet autoselect (100baseTX <full-duplex>)

   status: active

pfsync0: flags=0<> metric 0 mtu 1460

   syncpeer: 224.0.0.240 maxupd: 128

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384

   options=3<RXCSUM,TXCSUM>

   inet6 ::1 prefixlen 128 

   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 

   inet 127.0.0.1 netmask 0xff000000 

   nd6 options=3<PERFORMNUD,ACCEPT_RTADV>

pflog0: flags=0<> metric 0 mtu 33152
```

```
# uname -a

FreeBSD APPOLON.com 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #2 : Fri Dec 3 18:49:15 CET 2010 root@APPOLON.com:/usr/obj/usr/src/sys/MonNoyau-2010-11-30 amd64
```

----------

## cach0rr0

what are the permissions of authorized_keys on the guest? 

the ssh logs on the guest should show clearly if sshd does not like the permissions. 

one other minor thing, it does not seem to affect you yet, but in addition to what you've done with iptables, make sure a)the mac addr on host and guest are different, which youve likely done, and b)ip forwarding is enabled in sysctl.conf

neither is relevant here since you've already proven a connection can be established. Actually since neither is relevant, why am i saying that? It is late, I need sleep. 

Anyway, yeah, permissions on the authorized_keys file == prime suspect

----------

## lalebarde

Thanks for your help cach0rr0.

The port that appear in the guest log is random and does not fit the host connexion request :

HOST :

```
chronos@JANUS ~/Documents $ ssh -p 42852 mercure@appolon

mercure@appolon's password: 

Permission denied, please try again.
```

GUEST :

```
# tail -f /var/log/auth.log

Sep 20 07:46:48 APPOLON sshd [3536] : Failed password for mercure from 192.168.99.1 port 52115 ssh2

Sep 20 07:50:15 APPOLON sshd [3554] : Failed password for mercure from 192.168.99.1 port 35645 ssh2
```

Other trials leads to other port numbers   :Shocked:  .

Concerning the permissions of authorized_keys on the guest, I have for user mercure : -rw-r--r--

But anyway, I could not manage to load the keys from the host with : scp -P 42852 ~/.ssh/id_dsa.pub mercure@appolon:/home/mercure/.ssh/

As far as I understand, scp uses an ssh connexion. So if ssh cannot connect for any reason, I cannot use keys. For that reason, I have had authorized temporarily password authentication in the guest sshd_config file.

Finally, concerning ip forwarding :

```
# /sbin/sysctl -a | grep forward

error: permission denied on key 'vm.compact_memory'

error: "Invalid argument" reading key "fs.binfmt_misc.register"

error: permission denied on key 'net.ipv4.route.flush'

net.ipv4.conf.all.forwarding = 1

net.ipv4.conf.all.mc_forwarding = 0

net.ipv4.conf.default.forwarding = 1

net.ipv4.conf.default.mc_forwarding = 0

net.ipv4.conf.lo.forwarding = 1

net.ipv4.conf.lo.mc_forwarding = 0

net.ipv4.conf.sit0.forwarding = 1

net.ipv4.conf.sit0.mc_forwarding = 0

net.ipv4.conf.eth0.forwarding = 1

net.ipv4.conf.eth0.mc_forwarding = 0

net.ipv4.conf.qtap0.forwarding = 1

net.ipv4.conf.qtap0.mc_forwarding = 0

net.ipv4.conf.br0.forwarding = 1

net.ipv4.conf.br0.mc_forwarding = 0

net.ipv4.ip_forward = 1

net.ipv6.conf.all.forwarding = 0

net.ipv6.conf.default.forwarding = 0

net.ipv6.conf.lo.forwarding = 0

net.ipv6.conf.sit0.forwarding = 0

net.ipv6.conf.eth0.forwarding = 0

error: permission denied on key 'net.ipv6.route.flush'

net.ipv6.conf.qtap0.forwarding = 0

net.ipv6.conf.br0.forwarding = 0
```

----------

## lalebarde

Sorry to bump, but these last 10 days I made no progress here   :Mad: 

----------

## NeddySeagoon

lalebarde,

The default NATed guest connection is firewalled to allow nothing in and anything out.

Can you ssh out from the guest to the host ?

On the guest, can you ssh to its own external IP?

(The IP address on eth0)

Other than a test, it not useful. If this fails, you can't ssh in from anywhere else either.

Are you the same username on the host and guest?

If not you must specifiy ssh <user>@<guest>

Its worth restarting sshd to force it to reread the config file, just so you are 100% sure its running the way you thing it is.

----------

## ebo

following up on cach0rr0's suggestion that it was a permissions problem I started poking around and discovered that if:

  drwx------ 8 hg users 4096 Oct 10 05:29 /home/hg

then I get the same behavior.  If I change the permissions to drwxr-xr-x for the ${HOME} and keep drwx------ for /home/hg/.ssh that everything works fine without setting sshd_config's StrictModes no

Thanks cach0rr0 for the pointer to the problem, but I wanted to document that the actual issue is one removed.

----------

