# SOLVED: denyhosts and python not playing together

## gordonp

I recently noticed that my Denyhosts service wasn't running on my workstation; I purged every file and directory I could find with the name 'denyosts' in it, removed my former hosts.deny file, and started all over afresh -

I emerged denyhosts, went through the configuration, but still it gives the same error:

```
# systemctl status denyhosts.service

● denyhosts.service - SSH log watcher

   Loaded: loaded (/usr/lib64/systemd/system/denyhosts.service; enabled; vendor preset: disabled)

   Active: failed (Result: resources) since Mon 2016-08-29 09:51:15 PDT; 7s ago

  Process: 25852 ExecStart=/usr/bin/denyhosts.py --daemon --config=/etc/denyhosts.conf (code=exited, status=0/SUCCESS)

  Process: 25848 ExecStartPre=/bin/rm -f /var/run/denyhosts.pid (code=exited, status=0/SUCCESS)

Aug 29 09:51:15 pluto denyhosts.py[25852]: File "/usr/lib64/python3.4/site-packages/DenyHosts/deny_hosts.py", line 424, in process_log

Aug 29 09:51:15 pluto denyhosts.py[25852]: for line in fp:

Aug 29 09:51:15 pluto denyhosts.py[25852]: File "/usr/lib64/python3.4/codecs.py", line 319, in decode

Aug 29 09:51:15 pluto denyhosts.py[25852]: (result, consumed) = self._buffer_decode(data, self.errors, final)

Aug 29 09:51:15 pluto denyhosts.py[25852]: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x86 in position 5896: invalid start byte

Aug 29 09:51:15 pluto denyhosts.py[25852]: DenyHosts exited abnormally

Aug 29 09:51:15 pluto systemd[1]: denyhosts.service: PID file /var/run/denyhosts.pid not readable (yet?) after start: No such file or directory

Aug 29 09:51:15 pluto systemd[1]: Failed to start SSH log watcher.

Aug 29 09:51:15 pluto systemd[1]: denyhosts.service: Unit entered failed state.

Aug 29 09:51:15 pluto systemd[1]: denyhosts.service: Failed with result 'resources'.

```

I would appreciate any suggestions ...  THANKS in advance  :Smile: 

Other related info about my desktop/workstation (generally all "stable"):

```
# emerge -pv python denyhosts readline

These are the packages that would be merged, in order:

Calculating dependencies... done!

[ebuild   R    ] sys-libs/readline-6.3_p8-r2::gentoo  USE="-static-libs -utils" ABI_X86="32 (64) (-x32)" 0 KiB

[ebuild   R    ] dev-lang/python-3.4.3-r1:3.4::gentoo  USE="gdbm ipv6 ncurses readline sqlite ssl threads xml -build -examples -hardened -tk -wininst" 0 KiB

[ebuild   R    ] app-admin/denyhosts-3.0-r1::gentoo  PYTHON_TARGETS="python2_7 python3_4 -python3_3 (-python3_5)" 0 KiB

```

```
# eselect python list

Available Python interpreters:

  [1]   python2.7

  [2]   python3.3

  [3]   python3.4 *

```

and

```
# eselect profile list

Available profile symlink targets:

  [1]   default/linux/amd64/13.0

  [2]   default/linux/amd64/13.0/selinux

  [3]   default/linux/amd64/13.0/desktop

  [4]   default/linux/amd64/13.0/desktop/gnome

  [5]   default/linux/amd64/13.0/desktop/gnome/systemd *

  [6]   default/linux/amd64/13.0/desktop/kde

  [7]   default/linux/amd64/13.0/desktop/kde/systemd

  [8]   default/linux/amd64/13.0/desktop/plasma

  [9]   default/linux/amd64/13.0/desktop/plasma/systemd

  [10]  default/linux/amd64/13.0/developer

  [11]  default/linux/amd64/13.0/no-multilib

  [12]  default/linux/amd64/13.0/systemd

  [13]  default/linux/amd64/13.0/x32

  [14]  hardened/linux/amd64

  [15]  hardened/linux/amd64/selinux

  [16]  hardened/linux/amd64/no-multilib

  [17]  hardened/linux/amd64/no-multilib/selinux

  [18]  hardened/linux/amd64/x32

  [19]  hardened/linux/musl/amd64

  [20]  hardened/linux/musl/amd64/x32

  [21]  default/linux/uclibc/amd64

  [22]  hardened/linux/uclibc/amd64

```

Last edited by gordonp on Tue Aug 30, 2016 5:05 pm; edited 1 time in total

----------

## ian.au

 *Quote:*   

> UnicodeDecodeError: 'utf-8' codec can't decode byte 0x86 in position 5896: invalid start byte

 

I am going to guess that this is a result of the stricter encoding checks in python3x there are a few gotchas like this if selecting a 3.x default python

Do you have the same problem is you switch back to python 2.7 as the default interpreter?

 *Quote:*   

> Other related info about my desktop/workstation (generally all "stable"):

 

If running generally stable then make the default python interpreter 2.7 and comment out any python directives in /etc/portage/make.conf if you have them - that is the default at the moment. 

I misread an eselect news item back when 3.4 was made the stable *python3* implementation and dutifully made 3.4 my default interpreter for a few months, I started seeing these types of errors. They all went away after I re-read the python wiki and reverted to the defaults.

----------

## gordonp

Great suggestion, @ian.au

I set my python interpreter to 2.7 (which seems like a large step backwards, but maybe that's only my mis-perception), and IT WORKS!

----------

## ian.au

 *gordonp wrote:*   

> Great suggestion, @ian.au
> 
> I set my python interpreter to 2.7 (which seems like a large step backwards, but maybe that's only my mis-perception), and IT WORKS!

 

I don't see it a step backwards. I guess 2.7's the default because large chunks of portage are written with it and not all scripts have been refactored to 3.x. As long as you have a 3.x slotted you have access to a 3.x interpreter anyway when needed, and if you run in the default configuration, the devs sort that all out for you (or you can choose per package by setting some use flags).

----------

## Naib

Portage and associated scripts have been py3 compatible for quite some time, that is why there is a py3 USE flag. You can see how long ago that was added

----------

## ian.au

Still get corner cases when running 2.7 as default here, not portage related though probably - I did say that was a guess, thanks for the pointer.

----------

## jesnow

I had this same issue and fixed it just now only to discover that openssh has removed tcp wrapper support back in 2014, and somehow this was not propagated  to us. 

Denyhosts no longer denies anything. 

Jon.

----------

## Hu

While it could have been made a bit more prominent, there was (and still is) an effort to warn people who read their Portage log messages:

```

/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild:   # Make sure people who are using tcp wrappers are notified of its removal. #531156

/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild-   if grep -qs '^ *sshd *:' "${EROOT}"/etc/hosts.{allow,deny} ; then

/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild:      ewarn "Sorry, but openssh no longer supports tcp-wrappers, and it seems like"

/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild-      ewarn "you're trying to use it.  Update your ${EROOT}etc/hosts.{allow,deny} please."

/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild-   fi

/usr/portage/net-misc/openssh/openssh-7.3_p1-r7.ebuild-}

--

/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild:   # Make sure people who are using tcp wrappers are notified of its removal. #531156

/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild-   if grep -qs '^ *sshd *:' "${EROOT}"/etc/hosts.{allow,deny} ; then

/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild:      ewarn "Sorry, but openssh no longer supports tcp-wrappers, and it seems like"

/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild-      ewarn "you're trying to use it.  Update your ${EROOT}etc/hosts.{allow,deny} please."

/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild-   fi

/usr/portage/net-misc/openssh/openssh-7.3_p1-r8.ebuild-}

--

/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild:   # Make sure people who are using tcp wrappers are notified of its removal. #531156

/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild-   if grep -qs '^ *sshd *:' "${EROOT}"/etc/hosts.{allow,deny} ; then

/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild:      ewarn "Sorry, but openssh no longer supports tcp-wrappers, and it seems like"

/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild-      ewarn "you're trying to use it.  Update your ${EROOT}etc/hosts.{allow,deny} please."

/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild-   fi

/usr/portage/net-misc/openssh/openssh-7.4_p1.ebuild-}

--

/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild:   # Make sure people who are using tcp wrappers are notified of its removal. #531156

/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild-   if grep -qs '^ *sshd *:' "${EROOT}"/etc/hosts.{allow,deny} ; then

/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild:      ewarn "Sorry, but openssh no longer supports tcp-wrappers, and it seems like"

/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild-      ewarn "you're trying to use it.  Update your ${EROOT}etc/hosts.{allow,deny} please."

/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild-   fi

/usr/portage/net-misc/openssh/openssh-7.5_p1-r1.ebuild-}

--

/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild:   # Make sure people who are using tcp wrappers are notified of its removal. #531156

/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild-   if grep -qs '^ *sshd *:' "${EROOT}"/etc/hosts.{allow,deny} ; then

/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild:      ewarn "Sorry, but openssh no longer supports tcp-wrappers, and it seems like"

/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild-      ewarn "you're trying to use it.  Update your ${EROOT}etc/hosts.{allow,deny} please."

/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild-   fi

/usr/portage/net-misc/openssh/openssh-7.5_p1.ebuild-}

```

If this warning is not triggering for you, I suggest filing a bug to have the ebuild adjusted.

----------

## jesnow

Because it is very security related I would have thought a more prominent warning would be warranted. How am I going to see it in a big list of emerge messages? Secondly, I din't use TCP wrappers, I used denyhosts. I, like a lot of even savvy users, only have the vaguest idea what tcp-wrappers is, just that it's a dependency of some really core functionality. 

I haven't had any luck with itables, it is very complex software, and all the successor solutions like fail2ban etc are based on it. There is essentially no way of solving this problem without gong to a fully fledged firewall setup. Maybe that's good, one should not run telnetd or ftpd any more either. 

This is the achilles heel of FOSS, old solutions silently fail with no replacement. Or the replacement is poorly documented, because hey linux isn't for noobs. OK. 

Why couldn't somebody just fix TCP wrappers? That's not a gentoo problem, it seems like an everybody problem. There's also a world of out of date documentation out there telling you how to set up hosts.* to secure your system. And even googling hosts.deny doesn't get you a page that says: 

WARNING PEOPLE: hosts.deny NO LONGER WORKS !!! DENYHOSTS RUNS BUT CAN'T DENY ANYTHING!!!

I guess it does now. 

Cheers, 

Jon

[edit:] Thanks for your input by the way, it's very helpful.

----------

## ct85711

First off, tcp-wrappers is the hosts.deny/allow; so if you used hosts.deny you were using tcp-wrappers...

As far as stuff stopped working with tcp-wrappers, that decision was done by upstream developers, not Gentoo's devs.  I know for openssh, there were quite a few posts/articles about this and openssh devs said they won't support them.  Their reasoning is that they already have something internal doing the same thing so to save a few lines of code they stopped supporting tcp-wrappers...  There is quite a few people upset about this, but there is nothing we can do.  Simply put upstream has done a straight out their way or highway, and you don't have a choice of anything else.

----------

## Hu

With the right configuration, Portage would mail (or save to a special file) ewarn class messages.  Assuming (possibly unjustifiably) that Gentoo developers do not use ewarn for noise messages, these messages are always worth reading.  I agree that a more aggressive failure would have been nice.  I recall one recent openssh ebuild where a particular unsupported configuration would refuse to build.

Basic iptables is not that hard.  If you describe your problems with it, we may be able to help you.

Nobody restored the sshd/TCP wrappers integration because nobody with the skill to do so felt it was worth his/her time to do it.  I never used TCP wrappers for sshd because it seemed like a bad design to allow connections from hosts you know you don't want to service.

DenyHosts works fine for programs that still support TCP wrappers.  OpenSSH is not one of those programs.

----------

## jesnow

 *ct85711 wrote:*   

> First off, tcp-wrappers is the hosts.deny/allow; so if you used hosts.deny you were using tcp-wrappers...
> 
> 

 

Yes I was, but I didn't know it. 

 *Quote:*   

> 
> 
> As far as stuff stopped working with tcp-wrappers, that decision was done by upstream developers, not Gentoo's devs.  I know for openssh, there were quite a few posts/articles about this and openssh devs said they won't support them.  Their reasoning is that they already have something internal doing the same thing so to save a few lines of code they stopped supporting tcp-wrappers...  There is quite a few people upset about this, but there is nothing we can do.  Simply put upstream has done a straight out their way or highway, and you don't have a choice of anything else.

 

Yes, I understand that is indeed upstream's problem. I was exactly frustrated because openssh has no such functionality, and there is a russian server hammering me with a few thousand login requests a day, with no way to block it until I finally got iptables going. When you have a lot of kernel flags to set correctly and a lot lot more that you have to search through to find them it gets confusing. tcp-wrapper was very good about "just working" all these years.

I filed a bug asking for denyhosts to be dropped from portage because it no longer does anything in the gentoo context.

Could tcp-wrapper functionality be replaced entirely by a script that sets iptables rules? I have googled this a zillion ways and can't seem to find anything relevant. Or could denyhosts be modified to issue an iptables drop instead of writing to hosts.deny?

It seems like though, fail2ban is much more powerful and works with iptables out of the box, and there are other solutions as well. 

Thanks, 

Jon.

----------

## Hu

As I understand it, fail2ban is meant to be that script that uses iptables to enforce the denials.  According to its features page, it can use Netfilter or TCP wrappers (among other choices).  Not that the TCP wrappers support will do you much good in this case.

----------

## jesnow

Thats' right. IPtables took quite a bit of work to get going. Fail2ban is also a deep dive, very powerful. I'm just goign to ban attackers by hand when I notice them. It's not worth the work. Now I'm having trouble getting nomachine to play with iptables, but I'll have to start another thread for that. 

thanks, 

Jon.

----------

