# openldap - sasl authentication works only two times

## dialsc

hi all,

i’ve got openldap (2.3.39-r1) set up and running under gentoo 2007.0 (kernel 2.6.23-gentoo-r3, genkernel). Most things are working well but sasl is making some trouble. After performing a ldapwhoami using sasl interactively (digest-md5) two times which work as expected, it hangs for all the next times I try it. Then, when it hangs, sometimes it prompts for credentials after several minutes.

Trying to shutdown the slapd daemon while it hangs is also getting blocked and at the logs I can find that it is waiting for one thread at least.

Any help would be great, thanks.

Greez,

dialsc

----------

## VinzC

Have you tried a ~masked or downgraded version of the package which the locking thread belongs to?

----------

## dialsc

no, i did not. i installed all the software the regular (emerge) way.

----------

## dialsc

it seams to me that the problem must be more deep inside the libraries used. trying to authenticate against another, a second ldap server via sasl also works only two times than starts hanging.

authenticating against this second ldap server from another machine via sasl works as expected. therefore the problem must to be found at the specific machine.

anyone an idea?Last edited by dialsc on Tue Jan 01, 2008 10:04 pm; edited 1 time in total

----------

## VinzC

Since I have not used ldap yet you might try to inspect what libraries are loaded and compare their versions between both machines (the one that works and the one that doesn't). Use ldd to see what libraries are loaded and equery b <library> to check what package version they belong to. Maybe there's a version of one package that differs, who knows?...

----------

## dialsc

thanks for the tip. i did it and figured out the following for e.g. ldapsearch.

```

ldd /usr/bin/ldapsearch

        linux-gate.so.1 =>  (0xffffe000)

        libldap-2.3.so.0 => /usr/lib/libldap-2.3.so.0 (0xb7f43000)

        liblber-2.3.so.0 => /usr/lib/liblber-2.3.so.0 (0xb7f36000)

        libsasl2.so.2 => /usr/lib/libsasl2.so.2 (0xb7f1e000)

        libdl.so.2 => /lib/libdl.so.2 (0xb7f1a000)

        libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 (0xb7ed8000)

        libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 (0xb7d8e000)

        libcrypt.so.1 => /lib/libcrypt.so.1 (0xb7d5f000)

        libresolv.so.2 => /lib/libresolv.so.2 (0xb7d4d000)

        libc.so.6 => /lib/libc.so.6 (0xb7c1d000)

        /lib/ld-linux.so.2 (0xb7f80000)

```

inspecting the libraries showed me that glibc and openssl are newer at the problem server. the working one uses glibc 2.5-r4 whereas the problem server uses glibc 2.6.1. the working server has openssl 0.9.8e-r2 installed and the problem server uses openssl 0.9.8g.

therefore downgrading either glibc oder openssl could fix the problem but how can i tell portage to install (and not update) a specific version of a package (sorry, i'm a gentoo noob). which one to start with and what will be the impact when downgrading the glibc package?

greez,

dialscLast edited by dialsc on Tue Jan 01, 2008 10:05 pm; edited 1 time in total

----------

## VinzC

 *dialsc wrote:*   

> ...
> 
> therefore downgrading either glibc oder openssl could fix the problem but how can i tell portage to install (and not update) a specific version of a package (sorry, i'm a gentoo noob). which one to start with and what will be the impact when downgrading the glibc package?

 

I'd first try downgrading glibc - this is a bit risky in fact since it can break your system. If this is a live system things can be quite dangerous. If you can clone your system and test it in a safely fashion then go for it. If you cannot do that then downgrade openssl, that's much safer. So don't even think of downgrading glibc on a production server!

Note: downgrading glibc usually goes with emerge -e system... If nothing breaks, then emerge -e world - will make 100% sure everything is ok.

There is another way to check glibc: build the same system on another (testing) machine - that takes time for sure  :Smile:  but much less than downgrading -  except for glibc of which you'd mask the exact version you *don't* want using /etc/portage/package.mask. See man portage for syntax details.

You can tell emerge to install a given package version with emerge =package-[version number]. For example:

```
emerge =openssl-0.9.8g
```

You can also tell emerge to install (compile) the highest version number of a package (i.e. the latest version, normally) that comes before a given value:

```
emerge <openssl-0.9.8g
```

The disadvantage is that the package will be upgraded regardless of what you installed when you perform a world upgrade, for instance. To be sure a given package won't be upgraded, I usually mask the versions I don't want using /etc/portage/package.mask:

```
# Don't upgrade openssl to versions above and including 0.9.8g

>=openssl-0.9.8g
```

There are other techniques that many Gentooers will share, it's all up to you to use the one that you find best for you.

EDIT: In both cases it's a good idea to file a bug with what you observed.

----------

## dialsc

hi,

i tried to downgrade glibc. unfortunately glibc fails to build with the following message.

```

 * Sanity check to keep you from breaking your system:

 *  Downgrading glibc is not supported and a sure way to destruction

 *

 * ERROR: sys-libs/glibc-2.5-r4 failed.

 * Call stack:

 *             ebuild.sh, line 1717:  Called dyn_setup

 *             ebuild.sh, line  768:  Called qa_call 'pkg_setup'

 *             ebuild.sh, line   44:  Called pkg_setup

 *   glibc-2.5-r4.ebuild, line 1077:  Called die

 * The specific snippet of code:

 *                      die "aborting to save your system"

 *  The die message:

 *   aborting to save your system

 *

 * If you need support, post the topmost build error, and the call stack if relevant.

 * A complete build log is located at '/var/tmp/portage/sys-libs/glibc-2.5-r4/temp/build.log'.

```

i think i will resetup the server from scratch. than i will mark the two packages and look what will happen. than i will update openssl and look again. certainly this will take some time. afterwards i will post my experiences.

thanks a lot, greez,

dialsc

ps: i've created a bug report: https://bugs.gentoo.org/show_bug.cgi?id=203991

----------

## VinzC

Why don't you downgrade OpenSSL alone before rebuliding your system? At least it has no specific version requirement on glibc and you can downgrade it on a live system without breaking it (be sure to run revdep-rebuild though)...

----------

## dialsc

yes, you're right. i'm gonna try it right now. do i have to remerge all the packages that depend on openssl?

----------

## dialsc

hi,

i've downgraded openssl now. the interessting thing is, that it now needs about 8 authentications until a further try is hanging.

even more interessting, at the machine which i thought of that it works i can see the same behaviour when i do a bit more authentications in a sequence. after about 10 authentications it also begins to hang. but this machine is quite faster comming back so i can perform a further authentication.

seams that there is a general problem.

or is it a feature that i do not know yet...   :Wink: 

----------

## Jokey_

Can you try to run the openldap server in foreground and full debug mode to track that "hang" down? (Note it is really noisy but should give an idea what it does until it hangs)

suggested:

/usr/lib/openldap/slapd -u ldap -g ldap <your options from /etc/conf.d/slapd here> -d <debuglevel>

debug levels are shown here:

http://www.bind9.net/manual/openldap/2.3/runningslapd.html

note that when you turn on packet logging, your passwords may be printed clear-text to screen

----------

## dialsc

hi,

i run slapd in foreground with debug mode -1. i used ldapsearch from the same machine with debug mode 9. two times it worked, the third time it started hanging.

output slapd (third run):

```

daemon: activity on 1 descriptor

daemon: activity on:

>>> slap_listener(ldap://)

daemon: listen=8, new connection on 16

daemon: added 16r (active) listener=(nil)

conn=2 fd=16 ACCEPT from IP=127.0.0.1:58296 (IP=0.0.0.0:389)

daemon: epoll: listen=7 active_threads=0 tvp=zero

daemon: epoll: listen=8 active_threads=0 tvp=zero

daemon: epoll: listen=9 active_threads=0 tvp=zero

daemon: activity on 1 descriptor

daemon: activity on: 16r

daemon: read active on 16

connection_get(16)

connection_get(16): got connid=2

connection_read(16): checking for input on id=2

ber_get_next

ldap_read: want=8, got=8

  0000:  30 18 02 01 01 60 13 02                            0....`..

ldap_read: want=18, got=18

  0000:  01 03 04 00 a3 0c 04 0a  44 49 47 45 53 54 2d 4d   ........DIGEST-M

  0010:  44 35                                              D5

ber_get_next: tag 0x30 len 24 contents:

ber_dump: buf=0x8232488 ptr=0x8232488 end=0x82324a0 len=24

  0000:  02 01 01 60 13 02 01 03  04 00 a3 0c 04 0a 44 49   ...`..........DI

  0010:  47 45 53 54 2d 4d 44 35                            GEST-MD5

ber_get_next

ldap_read: want=8 error=Resource temporarily unavailable

daemon: epoll: listen=7 active_threads=0 tvp=zero

daemon: epoll: listen=8 active_threads=0 tvp=zero

daemon: epoll: listen=9 active_threads=0 tvp=zero

do_bind

ber_scanf fmt ({imt) ber:

ber_dump: buf=0x8232488 ptr=0x823248b end=0x82324a0 len=21

  0000:  60 13 02 01 03 04 00 a3  0c 04 0a 44 49 47 45 53   `..........DIGES

  0010:  54 2d 4d 44 35                                     T-MD5

ber_scanf fmt ({m) ber:

ber_dump: buf=0x8232488 ptr=0x8232492 end=0x82324a0 len=14

  0000:  00 0c 04 0a 44 49 47 45  53 54 2d 4d 44 35         ....DIGEST-MD5

ber_scanf fmt (}}) ber:

ber_dump: buf=0x8232488 ptr=0x82324a0 end=0x82324a0 len=0

>>> dnPrettyNormal: <>

<<< dnPrettyNormal: <>, <>

do_sasl_bind: dn () mech DIGEST-MD5

conn=2 op=0 BIND dn="" method=163

==> sasl_bind: dn="" mech=DIGEST-MD5 datalen=0

SASL [conn=2] Debug: DIGEST-MD5 server step 1

```

output ldapsearch (third run):

```

ldapsearch -U LoginUser -Y DIGEST-MD5 -b "dc=mydomain,dc=local" -d 9 -w VERY_SECRETE

ldap_create

ldap_sasl_interactive_bind_s: user selected: DIGEST-MD5

ldap_int_sasl_bind: DIGEST-MD5

ldap_new_connection 1 1 0

ldap_int_open_connection

ldap_connect_to_host: TCP 127.0.0.1:389

ldap_new_socket: 3

ldap_prepare_socket: 3

ldap_connect_to_host: Trying 127.0.0.1:389

ldap_connect_timeout: fd: 3 tm: -1 async: 0

ldap_int_sasl_open: host=ldap.mydomain.local

SASL/DIGEST-MD5 authentication started

ldap_sasl_bind_s

ldap_sasl_bind

ldap_send_initial_request

ldap_send_server_request

ber_scanf fmt ({it) ber:

ber_scanf fmt ({i) ber:

ber_flush: 26 bytes to sd 3

ldap_result ld 0x8058de0 msgid 1

ldap_chkResponseList ld 0x8058de0 msgid 1 all 1

ldap_chkResponseList returns ld 0x8058de0 NULL

wait4msg ld 0x8058de0 msgid 1 (infinite timeout)

wait4msg continue ld 0x8058de0 msgid 1 all 1

** ld 0x8058de0 Connections:

* host: 127.0.0.1  port: 389  (default)

  refcnt: 2  status: Connected

  last used: Wed Jan  2 18:29:05 2008

** ld 0x8058de0 Outstanding Requests:

 * msgid 1,  origid 1, status InProgress

   outstanding referrals 0, parent count 0

** ld 0x8058de0 Response Queue:

   Empty

ldap_chkResponseList ld 0x8058de0 msgid 1 all 1

ldap_chkResponseList returns ld 0x8058de0 NULL

ldap_int_select

```

than it hangs and i killed ldapsearch by Ctrl-C.

at the point of time i killed ldapsearch, slapd printed out the following

```

daemon: activity on 1 descriptor

daemon: activity on: 16r

daemon: read active on 16

connection_get(16)

connection_get(16): got connid=3

connection_read(16): checking for input on id=3

ber_get_next

ldap_read: want=8, got=0

ber_get_next on fd 16 failed errno=0 (Success)

connection_read(16): input error=-2 id=3, closing.

connection_closing: readying conn=3 sd=16 for close

connection_close: deferring conn=3 sd=16

daemon: epoll: listen=7 active_threads=0 tvp=zero

daemon: epoll: listen=8 active_threads=0 tvp=zero

daemon: epoll: listen=9 active_threads=0 tvp=zero

daemon: activity on 1 descriptor

daemon: activity on:

daemon: epoll: listen=7 active_threads=0 tvp=zero

daemon: epoll: listen=8 active_threads=0 tvp=zero

daemon: epoll: listen=9 active_threads=0 tvp=zero

```

i've repeated the ldapsearch several times and the outputs shown are always the same. trying to stop slapd via Ctrl-C showed the following output from slapd.

```

daemon: shutdown requested and initiated.

daemon: closing 7

daemon: closing 8

daemon: closing 9

connection_close: deferring conn=3 sd=16

connection_close: deferring conn=4 sd=20

connection_close: deferring conn=5 sd=22

connection_close: deferring conn=6 sd=24

connection_close: deferring conn=7 sd=26

connection_closing: readying conn=8 sd=28 for close

connection_close: deferring conn=8 sd=28

slapd shutdown: waiting for 6 threads to terminate

```

and than the shutdown hangs too.

does this show anything interessting to you?

greez,

dialsc

----------

## idler

Howdy,

Could you please post you slapd.conf ?

Maybe run slapd with -d 256 or 512 or such, and see the if the logs look any clearer.

----------

## dialsc

hi,

currently i'm doing a totaly new gentoo installation as describted at the gentoo installation handbook. after it has finished i will reinstall the openldap like i did it before. than i will test it again and tell you what happend afterwards.

so long...

dialsc

----------

