# What benefits there are to NPTL?

## Zarhan

Not sure whether this would belong here or the portage forum, but here goes...

What is this NPTL USE flag that I occasionally see? Native Posix Thread Library, yes. A replacement for linuxthreads I gather, but the question is: What good will it do from the user point of view? Improved performance? More efficient memory usage? Anything else?

Is it yet stable to take into production use? If I want to start using NPTL, are there any other steps besides setting the USE flag and re-emerging packages that have the nptl option? Do I need to compile any options into kernel (I'm using 2.6 series). 

However, doing a quick check, emerge -epv world | grep nptl yields only

[ebuild  N    ] sys-libs/glibc-2.3.3.20040420-r1  -build -debug -debug -erandom +nls -nptl -pic  18 kB

In effect, the only package that actually needs to be recompiled is Glibc  should I want to take advantage of NPTL?

----------

## GatoVolador

Zarhan - 

If I understand correctly, the main advantage of NPTL is that it uses lightweight threads. With linux threads every thread gets its own process, which, if you have a lot of threads, needlessly eats up a lot of resources in both CPU time and memory. NPTL does away with this, now multiple threads can exist inside one process, decreasing resource and cpu usage.

I (and I believe a good number of others I've read about here) have been using NPTL with 2.6.5-gentoo-r1 for a few months and have seen no problems whatsoever, so it feels stable. Thread management is part of glibc, so this is the only package you have to rebuild, but then other apps that use threads will benefit (these apps won't have the nptl use flag, that's why you didn't find anything).

For me the big winner was Java. I do a lot of heavy multithread stuff, and before I'd have one Java program where the JVM would have 100 thread-processes, now I have one JVM process that handles the 100 lightweight threads. Much less resources are used and Java apps now seem much more stable.

Regards,

Jay

----------

## Zarhan

 *jayb wrote:*   

> Zarhan - 
> 
> If I understand correctly, the main advantage of NPTL is that it uses lightweight threads. With linux threads every thread gets its own process, which, if you have a lot of threads, needlessly eats up a lot of resources in both CPU time and memory. NPTL does away with this, now multiple threads can exist inside one process, decreasing resource and cpu usage.
> 
> 

 

  So I guess there is not too much benefit if you're not running a server that spawns huge number of threads. 

  Does it require a patched kernel, ie. either gentoo-sources or some other patch set? Vanilla (development-sources) is not enough?

----------

## butters

NPTL requires a 2.6.x kernel such as gentoo-dev-sources or development-sources or mm-sources, and also requires the 2.6.x kernel headers.  Get those by emerging linux26-headers (this should be detected as a dependency for glibc if nptl is in your USE flags).  If your emerge of glibc still fails with linux26-headers installed, edit the file

/etc/make.profile/virtuals

and change two instances of "linux-headers" to "linux26-headers" in that file.  Whether or not you need to do this depends on how recent your portage version is (you might want to do emerge portage before emerge glibc to avoid this).

You should consider NPTL a Good Thing.  UNIX variants have had real POSIX threads for years, while linux has emulated threads as processes (adding significant overhead as mentioned previously).  You don't need to be running servers in order to reap the benefits.  Most web browsers and email clients (as well as p2p clients and many other network apps) make extensive use of multithreading.  Firefox may run with 10 or more threads concurrently.  NPTL can make a big difference is such cases, especially if your internet connection is on the slow side.

Similarly to XFree86-->Xorg, I moved from linux-threads to NPTL primarily on the basis that it is the next step in GNU/Linux evolution, so why not give the future a shot?

----------

## Zarhan

 *butters wrote:*   

> NPTL requires a 2.6.x kernel such as gentoo-dev-sources or development-sources or mm-sources, and also requires the 2.6.x kernel headers. 

 

  Great, this finally gives me a reason to try out the 2.6-headers. I've been running with 2.4-headers because that is the recommended Gentoo way, especially because there has been some dependency problems reported on these boards and the benefits of running with 2.6 have been more or less vague. I'll test this with one computer first and see how it goes, and if it works then go for it with all the others I have.

----------

## Zarhan

 *butters wrote:*   

> NPTL requires a 2.6.x kernel such as gentoo-dev-sources or development-sources or mm-sources, and also requires the 2.6.x kernel headers.  Get those by emerging linux26-headers (this should be detected as a dependency for glibc if nptl is in your USE flags).  

 

  Are you sure that the linux26-headers package are required? It is not a depedency even when NPTL is set. Looking into the ebuild (glibc-2.3.3.20040420-r1.ebuild) it checks only the kernel version. Apparently it uses headers directly from the installed Linux sources. 

  2.3.3.20040420-r1 is the latest glibc that is not masked. Later, ~x86 versions (such as 2.3.4.20040808) do check for linux26-headers and there are comments about the unreliability of using headers from actually installed sources. I'm not sure about running masked glibc, though...

  I'll try without installing the 26headers for now, since the ebuild does not require it. Let's see what breaks..

----------

## codergeek42

 *Zarhan wrote:*   

>  *butters wrote:*   NPTL requires a 2.6.x kernel such as gentoo-dev-sources or development-sources or mm-sources, and also requires the 2.6.x kernel headers.  Get those by emerging linux26-headers (this should be detected as a dependency for glibc if nptl is in your USE flags).   
> 
>   Are you sure that the linux26-headers package are required? It is not a depedency even when NPTL is set. Looking into the ebuild (glibc-2.3.3.20040420-r1.ebuild) it checks only the kernel version. Apparently it uses headers directly from the installed Linux sources. 
> 
>   2.3.3.20040420-r1 is the latest glibc that is not masked. Later, ~x86 versions (such as 2.3.4.20040808) do check for linux26-headers and there are comments about the unreliability of using headers from actually installed sources. I'm not sure about running masked glibc, though...
> ...

 I use NPTL with the 2.4.24 headers. No problems yet. jEdit seems a lot smoother ^_^

----------

## John5788

so all i need to do to enable NPTL is to add nptl to my use flags and re-emerge glibc along with packages i want to have nptl?

----------

## Voltago

 *John5788 wrote:*   

> so all i need to do to enable NPTL is to add nptl to my use flags and re-emerge glibc along with packages i want to have nptl?

 

Rather recompile all packages that use POSIX threads.

----------

## Zarhan

 *John5788 wrote:*   

> so all i need to do to enable NPTL is to add nptl to my use flags and re-emerge glibc along with packages i want to have nptl?

 

  Well, since you probably haven't used 'static' (ie everything in your system is dynamically linked), then just re-emerge your glibc, no need to reinstall anything else. Might require a reboot. That's my theory, anyway, I'll see after glibc finishes compiling  :Smile: 

----------

## codergeek42

 *John5788 wrote:*   

> so all i need to do to enable NPTL is to add nptl to my use flags and re-emerge glibc along with packages i want to have nptl?

 

```
# USE="nptl" emerge glibc

# revdep-rebuild
```

?

----------

## GatoVolador

Actually if you're running a server that fires a lot of threads it would be useful. While of course nothing can make each thread run faster (except optimizing the  code, if possible), with NPTL resources usage goes way down since you don't have to spawn a new process (costly) for each new thread.

I am using linux 2.4 headers with NPTL and haven't seen any problems. In the next couple of weeks I'll take a look at moving to linux26-headers. 

And no, you don't have to rebuild anything except glibc unless you have statically linked apps, like codergeek said. Glibc abstracts away the details of thread management, so the apps don't care how it gets done, they just want it done. By the way whether you're using NPTL or linuxthreads,  you're still using POSIX threads, since they are both just implementations of the POSIX thread specification.

Regards,

Jay

----------

## codergeek42

 *jayb wrote:*   

> By the way whether you're using NPTL or linuxthreads,  you're still using POSIX threads, since they are both just implementations of the POSIX thread specification.

 Except that NPTL is a better implementation   :Wink: 

----------

## Zarhan

Glibc finished compiling. Restarted firefox, it started running with NPTL threads immediately (not creating n+10 processes when browsing). No need to reboot or anything else for that matter. Great!

----------

## finr

I just updated my system to an NPTL glibc but to my surprise gkrellm2 is showing more proc's than before (i always had like 95 runnig before, but since a reboot this number is usually > 110 but also greatly varying).. Is this normal? As I understood NPTL should decrease the number of processes since more threads can be run in a single process?

BTW, for firefox it seems NPTL has had it's effect, it's down from 7 processes to 2 directly after being started.. But still, my over-all process count seems to hae increased?

----------

## WaVeX

ok I got a question Whats all this statically linked and dynamically lnked stuff.

How do I check what my gentoo is?

Which is better?

----------

## thechris

dynamic linking uses less disk space and is more secure.  most apps in linux will be dynamically linked.

----------

## butters

Glad to see you've been successful.  Sorry about the whole 2.6-headers thing, when I tried to emerge glibc with nptl, the configure script failed saying I needed 2.6.5-headers or newer.  It could be that newer glibc builds no longer need 2.6-headers for NPTL.

IMHO gkrellm is and always was a resource hog.  It's a shame that a system monitor should have so much affect on system performance.  Try re-emerging gkrellm now that NPTL is in glibc, since gkrellm might be statically linked.

----------

## jdgill0

How about NPTL and OpenMosix ... the pthreads will not migrate due to reasons discussed at openmosix.org ... Now I know OpenMosix is currently stable only for 2.4 kernels, however I read a post earlier today that said OpenMosix for 2.6 would be released in 5 days ... When OpenMosix gets released for the 2.6 kernels (whether that is 5 days or not), will the NPTL be able to migrate?  If they will migrate between cluster nodes, then I would see NPTL having a huge advantage over pthreads for clusters.

----------

## charlieg

 *jayb wrote:*   

> I (and I believe a good number of others I've read about here) have been using NPTL with 2.6.5-gentoo-r1 for a few months and have seen no problems whatsoever, so it feels stable.

 

I've been using it since before 2.6.0, well before.  It's as stable as non-NPTL imho.

----------

## asiobob

after enabling nptl xmms and mplayer wouldn't start (segfault), the only common thing here is perhaps the reliance of the nvidia drivers (glx) on both. A remerge of that fixed the problem (emerge nvidia-glx).

----------

## charlieg

 *ASIO_BOB wrote:*   

> after enabling nptl xmms and mplayer wouldn't start (segfault), the only common thing here is perhaps the reliance of the nvidia drivers (glx) on both. A remerge of that fixed the problem (emerge nvidia-glx).

 

Yeah, it does require a few things to be recompiled.  IIRC don't you also have to recompile glibc?

----------

## asiobob

yes of course, however I was under the impression that by compiling glibc with NPTL was the only thing that had to be done "enable" NPTL (assuming one had a 2.6 kernel). Are you hinting there's a kernel config option for this?

----------

## John5788

i added nptl to my use flags and recompiled glibc. it seems like im using NPTL and firefox seems to use 3 less processes than before, never needed to emerge linux26-headers

```
~

john5788@john5788: pts/3: 10 files 291Kb -> /lib/libc.so.6 

GNU C Library stable release version 2.3.3, by Roland McGrath et al.

Copyright (C) 2004 Free Software Foundation, Inc.

This is free software; see the source for copying conditions.

There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A

PARTICULAR PURPOSE.

Compiled by GNU CC version 3.3.3 20040412 (Gentoo Linux 3.3.3-r6, ssp-3.3.2-2, pie-8.7.6).

Compiled on a Linux 2.6.7-gentoo-r14 system on 2004-08-15.

Available extensions:

        GNU libio by Per Bothner

        crypt add-on version 2.1 by Michael Glad and others

        NPTL 0.61 by Ulrich Drepper

        BIND-8.2.3-T5B

        NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk

Thread-local storage support included.

Report bugs using the `glibcbug' script to <bugs@gnu.org>.
```

```
~

john5788@john5788: pts/3: 10 files 291Kb -> ps aux | grep firefox | wc -l

3
```

----------

