# [solved] question on ssh port forwarding

## Elleni

I setup monitorix to only listen to localhost, and connect to the server with

```
ssh -4 -p 1234 mysomain.com -L 9080:mydomain.com:9080
```

 and can then connect to monitorix site by opening http://localhost:9080/monitorix on my local browser. 

That works fine for a while. I can browse monitorix graphs and everything works as intended. 

```
debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 91

debug2: channel 2: open confirm rwindow 2097152 rmax 32768

debug3: receive packet: type 96

debug2: channel 2: rcvd eof

debug2: channel 2: output open -> drain

debug2: channel 2: obuf empty

debug2: channel 2: chan_shutdown_write (i0 o1 sock 8 wfd 8 efd -1 [closed])

debug2: channel 2: output drain -> closed

debug2: channel 2: read<=0 rfd 8 len 0

debug2: channel 2: read failed

debug2: channel 2: chan_shutdown_read (i0 o3 sock 8 wfd 8 efd -1 [closed])

debug2: channel 2: input open -> drain

debug2: channel 2: ibuf empty

debug2: channel 2: send eof

debug3: send packet: type 96

debug2: channel 2: input drain -> closed

debug2: channel 2: send close

debug3: send packet: type 97

debug3: channel 2: will not send data after close

debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 9 setting TCP_NODELAY

debug2: fd 9 setting O_NONBLOCK

debug3: fd 9 is O_NONBLOCK

debug1: channel 3: new [direct-tcpip]

debug3: send packet: type 90

debug3: channel 2: will not send data after close

debug3: channel 2: will not send data after close

debug3: receive packet: type 97

debug2: channel 2: rcvd close

debug3: channel 2: will not send data after close

debug2: channel 2: is dead

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com 9080, connect from 127.0.0.1 port 38380 to 127.0.0.1 port 9080, nchannels 4

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

  #2 direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 38380 to 127.0.0.1 port 9080 (t4 r1 i3/0 o3/0 e[closed]/0 fd 8/8/-1 sock 8 cc -1)

  #3 direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 38382 to 127.0.0.1 port 9080 (t3 nr0 i0/0 o0/0 e[closed]/0 fd 9/9/-1 sock 9 cc -1)

debug3: receive packet: type 91

debug2: channel 3: open confirm rwindow 2097152 rmax 32768

debug3: receive packet: type 96

debug2: channel 3: rcvd eof

debug2: channel 3: output open -> drain

debug2: channel 3: obuf empty

debug2: channel 3: chan_shutdown_write (i0 o1 sock 9 wfd 9 efd -1 [closed])

debug2: channel 3: output drain -> closed

debug2: channel 3: read<=0 rfd 9 len 0

debug2: channel 3: read failed

debug2: channel 3: chan_shutdown_read (i0 o3 sock 9 wfd 9 efd -1 [closed])

debug2: channel 3: input open -> drain

debug2: channel 3: ibuf empty

debug2: channel 3: send eof

debug3: send packet: type 96

debug2: channel 3: input drain -> closed

debug2: channel 3: send close

debug3: send packet: type 97

debug3: channel 3: will not send data after close

debug3: receive packet: type 97

debug2: channel 3: rcvd close

debug3: channel 3: will not send data after close

debug2: channel 3: is dead

debug2: channel 3: garbage collecting

debug1: channel 3: free: direct-tcpip: listening port 9080 for mydomain.comch port 9080, connect from 127.0.0.1 port 38382 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 3: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

  #3 direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 38382 to 127.0.0.1 port 9080 (t4 r1 i3/0 o3/0 e[closed]/0 fd 9/9/-1 sock 9 cc -1)
```

But all of a sudden, it fails with

```
debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 91

debug2: channel 2: open confirm rwindow 2097152 rmax 32768

debug3: receive packet: type 96

debug2: channel 2: rcvd eof

debug2: channel 2: output open -> drain

debug2: channel 2: obuf empty

debug2: channel 2: chan_shutdown_write (i0 o1 sock 8 wfd 8 efd -1 [closed])

debug2: channel 2: output drain -> closed

debug2: channel 2: read<=0 rfd 8 len 0

debug2: channel 2: read failed

debug2: channel 2: chan_shutdown_read (i0 o3 sock 8 wfd 8 efd -1 [closed])

debug2: channel 2: input open -> drain

debug2: channel 2: ibuf empty

debug2: channel 2: send eof

debug3: send packet: type 96

debug2: channel 2: input drain -> closed

debug2: channel 2: send close

debug3: send packet: type 97

debug3: channel 2: will not send data after close

debug3: receive packet: type 97

debug2: channel 2: rcvd close

debug3: channel 2: will not send data after close

debug2: channel 2: is dead

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39076 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

  #2 direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39076 to 127.0.0.1 port 9080 (t4 r1 i3/0 o3/0 e[closed]/0 fd 8/8/-1 sock 8 cc -1)

debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 92

channel 2: open failed: connect failed: Connection refused

debug2: channel 2: zombie

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 formydomain.com port 9080, connect from 127.0.0.1 port 39078 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

debug1: Connection to port 9080 forwarding tomydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 92

channel 2: open failed: connect failed: Connection refused

debug2: channel 2: zombie

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39080 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

debug1: Connection to port 9080 forwarding tomydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 92

channel 2: open failed: connect failed: Connection refused

debug2: channel 2: zombie

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39082 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 92

channel 2: open failed: connect failed: Connection refused

debug2: channel 2: zombie

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39084 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 92

channel 2: open failed: connect failed: Connection refused

debug2: channel 2: zombie

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39086 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 92

channel 2: open failed: connect failed: Connection refused

debug2: channel 2: zombie

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39088 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 92

channel 2: open failed: connect failed: Connection refused

debug2: channel 2: zombie

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39090 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 92

channel 2: open failed: connect failed: Connection refused

debug2: channel 2: zombie

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39092 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 92

channel 2: open failed: connect failed: Connection refused

debug2: channel 2: zombie

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39094 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

debug1: Connection to port 9080 forwarding to mydomain.com port 9080 requested.

debug2: fd 8 setting TCP_NODELAY

debug2: fd 8 setting O_NONBLOCK

debug3: fd 8 is O_NONBLOCK

debug1: channel 2: new [direct-tcpip]

debug3: send packet: type 90

debug3: receive packet: type 92

channel 2: open failed: connect failed: Connection refused

debug2: channel 2: zombie

debug2: channel 2: garbage collecting

debug1: channel 2: free: direct-tcpip: listening port 9080 for mydomain.com port 9080, connect from 127.0.0.1 port 39096 to 127.0.0.1 port 9080, nchannels 3

debug3: channel 2: status: The following connections are open:

  #1 client-session (t4 r0 i0/0 o0/0 e[write]/0 fd 5/6/7 sock -1 cc -1)

```

Do I need some change in ssh configuration or opening ssh session with some other options in order to avoid above error?

Or is it a problem with monitorix itself, as I realized that restarting monitorix lets me connect to the site again.Last edited by Elleni on Thu Dec 26, 2019 11:14 pm; edited 1 time in total

----------

## szatox

It's a guess, but I think it could help.

At the first glance it looks like a session timeout to me. Try enabling keepalive in sshd_config. (TCPKeepAlive, ClientAliveInterval, ClientAlliveCountMax)

----------

## mike155

You could try to enable keep-alive signals:

```
ServerAliveInterval 240

ServerAliveCountMax 5
```

----------

## Elleni

Hi guys, 

Thank you for your fast replies. What would be sane values to try? 

I tried to insert mentioned ServerAlive entries in /etc/ssh/sshd_config, but with them ssh refuses to restart. 

```
/etc/init.d/sshd restart

/etc/ssh/sshd_config: line 132: Bad configuration option: ServerAliveInterval

/etc/ssh/sshd_config: line 133: Bad configuration option: ServerAliveCountMax

/etc/ssh/sshd_config: terminating, 2 bad configuration options
```

I now tried with the values proposed by mike155 but on the Client.. variables proposed by szatox, but the problem persists. Also I realized that only a restart of monitorix service (using its own built-in webserver) re-enables the access to http://localhost:9080/monitorix, but restart of ssh service doesn't, so maybe some re-configuration in monitorix needed? 

Edit to add, that I went back to original settings, disabled TCPKeepAlive and ClientAliveInterval and ClientAliveCountMax, restarted ssh. It seems to take longer until connection fails again than with above settings. And as said, as soon as I restart monitorix service, connection works again for some time until the next failed connection, while a restart of ssh does not re-establish it.

----------

## mike155

```
ssh -4 -o ServerAliveInterval=240 -o ServerAliveCountMax=5 -p 1234 mysomain.com -L 9080:mydomain.com:9080
```

----------

## Elleni

Still breaks after a while, will try higer values to see if I get it stable. Thanks for clarifying where to put theses options.

----------

## mike155

It's better to decrease the ServerAliveInterval value until the connection is stable.

Try "ServerAliveInterval=60" if "ServerAliveInterval=240" doesn't work.

See: https://patrickmn.com/aside/how-to-keep-alive-ssh-sessions/Last edited by mike155 on Thu Dec 26, 2019 7:15 pm; edited 1 time in total

----------

## szatox

 *Quote:*   

>  /etc/ssh/sshd_config: line 132: Bad configuration option: ServerAliveInterval 

 This one is a client-side option and should go into ssh_config.

Options I posted are server-side so they go into sshd_config.

AFAIR *AliveCountMax says how lenient you are towards lost keepalives (like in: "consider connection broken after this many pings without a reply") and *AliveInterval is an interval between pings in seconds.

I use 6 and 15 respectively, resulting in a total timeout of 1min30 (after 6 ping attempts fail)

Longer intervals are annoyingly slow when it comes to detecting actual failures and you want to retry several times in case of a non-fatal network hiccup.

Shorter intervals increase network overhead, so you don't want to go too low there. How low exactly is too low depends on your particular use case.

----------

## Elleni

Thanks for all these information, giving me the opportunity to learn on keep alive ssh sessions. I got it stabler but eventually there were still some interrupts. Going to monitorix irc channel, I was told to try to disable authentication mechanism within monitorix, and that way the problem disappeared. There is a bug with autocheck responsiveness on the built-in webserver of monitorix, which will be solved with next release. 

Disabling auth within monitorix as having the connection limited to localhost and accessing via ssh anyway got my error solved.

----------

