# Apache gets slower... and slower...

## Chewi

Hello. I've been running an Apache server for a while so I feel like I've got it figured but I just moved my fairly busy forum to it and it's not performing as well as I'd hoped. The thing is, it seems to start out okay but after a while, it gets slower and slower. And then if I simply do /etc/init.d/apache restart, it's really fast again. What's going on? I've tried various things including turning off the access log and turning off MMAP mode. They seem to have small improvements but they haven't fixed the overall problem. I'm running PHP5. Could be that. Not sure. I'm also using the pre-release version of InvisionBoard. That may well be the problem but the InvisionBoard guys are using the same version themselves and it seems okay. Any ideas?

http://forums.ffaura.com

----------

## jkt

what apache version? which MPM? prefork||worker||...?

----------

## Chewi

Sorry should have said.

apache-2.0.49-r3

php-5.0.0_rc1

Invision Board 2.0 PDR4

I'm not sure what you mean by the last two things. I'm just starting it with the /etc/init.d/apache script that gets installed by Portage.

----------

## UberLord

MPM types worker and prefork are Apache 2 specific and have to be compiled with that option

Apache 2 by default (in Gentoo) assumes the prefork worker unless you have the "threads" use flag set which forces Apache 2 to be compiled with the worker MPM

Apache 2 with the worker MPM + nptl enabled glibc kicks ass  :Very Happy: 

----------

## jkt

is InvisionBoard a PHP apllication?

in apache2, there're so called MPMs, eg. prefork, worker, perchild,... there's a lot of differences between them, some of them uses threads, others don't.

anyway, your problem might be in low memory, try to login into your box when it seems to be "slow" and execute `free`.

----------

## jkt

ok, UberLord was faster  :Smile: . It seems that you defined to start a lot of apache childs, more than you can afford.

----------

## Chewi

Well InvisionBoard is using PHP and MySQL. The machine is an AthlonXP 1100 with 512MB RAM. That seems adequate to me and like I said, it works great after I've just restarted the server. I'll give these things a go a little later though. Kinda tied up right now. Thanks.

----------

## jkt

 *Chewi wrote:*   

> The machine is an AthlonXP 1100 with 512MB RAM. That seems adequate to me and like I said, [b]it works great after I've just restarted the server[/]b.

 

sure, after restart there are no running apache instances <flame>with all the memory leaks in php/5</flame>  :Smile: 

----------

## Chewi

Yeah I had a feeling it could be related to PHP5. I'm writing PHP5 specific code on my other site (same server) though so I kinda need it. I came from an object-oriented background so I like what it has to offer. Hopefully those problems will get ironed out in due course.

This NPTL thing sounds nice. If I enable the USE flag, what must I recompile? Is there anything I need to consider before using it? I'm running 2.6.5 at the moment.

----------

## jkt

if you're talking about threads in Apache, recompile apache2 & also PHP as it needs to now about thread support.

----------

## Chewi

Actually I just checked, I already had threads turned on. I didn't think I did.

Actually no wait. I enabled threads in ufed but when I type emerge -pv apache, it doesn't show as enabled. Strange.

Actually scratch all that, I loaded ufed again it was switched off again. I think I'm going crazy. Not enough sleep.  :Sad: 

----------

## Chewi

Ah I've just noticed that PHP5 is up to RC3 now. I'll have to hunt down an ebuild for it or failing that, modify the one I've got.

----------

## UberLord

 *Chewi wrote:*   

> 
> 
> This NPTL thing sounds nice. If I enable the USE flag, what must I recompile? Is there anything I need to consider before using it? I'm running 2.6.5 at the moment.

 

You need to emerge glibc

Recompiling the kernel may help as well

----------

## kashani

apache2 + php + threads = unstable

All versions, even 5, of PHP will have issues if you run a threaded Apache server. You may manage to avoid them if you are using an incredibly small subset of the PHP command set, but most of us are going to start having problems immediately.

http://lists.userlinux.com/pipermail/discuss/2003-December/001119.html

kashani

----------

## Chewi

Oh. Well umm... I've JUST recompiled everything with threads support and it's flying at the moment. Maybe I'll still see the slowdowns, I'll have to wait and see. Am I right in thinking that NPTL is an alternative to thread support? If so, I guess that would be better.

----------

## UberLord

 *kashani wrote:*   

> apache2 + php + threads = unstable
> 
> All versions, even 5, of PHP will have issues if you run a threaded Apache server. You may manage to avoid them if you are using an incredibly small subset of the PHP command set, but most of us are going to start having problems immediately.
> 
> http://lists.userlinux.com/pipermail/discuss/2003-December/001119.html
> ...

 

I hate misinformation.

Apache2 + PHP + threads = stable

However, PHP can be linked against non-threadsafe libraries which does cause problems with threads. Here's a list of libraries PHP can use which shows their threadsafe status

http://uk.php.net/manual/en/faq.obtaining.php#faq.obtaining.optional

http://uk.php.net/manual/en/install.apache2.php

The above page has a small note about Apache2 and the worker mpm.

Basically, just use threadsafe libraries and you'll be just fine  :Smile: 

----------

## UberLord

 *Chewi wrote:*   

> Am I right in thinking that NPTL is an alternative to thread support? If so, I guess that would be better.

 

Yeah

NPTL = Native POSIX Threading Library

It's designed to be a much better than the current linuxthreads library in glibc.

Infact, a lot of the apache mpm worker problems can be attributed to linuxthreads - nptl fixes most of these right now.

----------

## kashani

 *UberLord wrote:*   

> 
> 
> I hate misinformation.
> 
> Apache2 + PHP + threads = stable
> ...

 

I'd argure that in VERY specific cases PHP might be stable with threaded Apache, but for the general use server it will be unstable.

Let's talk about that list of libraries. I don't see mysql, png, ssl, xml, or gdbm on there and those are just the major ones. Suppose Invision needs a few less common libs to work. Are you willing to say that those libraries are thread safe? 

Since Chewi is already having problems moving to what most of the PHP programmers themselves consider a highly beta enviroment doesn't seem to be a great idea. 

kashani

----------

## UberLord

 *kashani wrote:*   

> 
> 
> I'd argure that in VERY specific cases PHP might be stable with threaded Apache, but for the general use server it will be unstable.
> 
> Let's talk about that list of libraries. I don't see mysql, png, ssl, xml, or gdbm on there and those are just the major ones. Suppose Invision needs a few less common libs to work. Are you willing to say that those libraries are thread safe? 
> ...

 

I'm an admin for http://www.driverheaven.net

It runs RHEL which ships (by default I think) with Apache2 running the worker MPM + php + nptl threading.

We run a few sites off that server - vBulliten forum for the main site and Deluxe Portal for the others. Both use mysql as a backend. We also make extensive use of ssl and xml.

I also run an Apache2 server on my Gentoo server at home (worker mpm, nptl) which makes extensive use of the gd library for image processing - it generates thumnails of my pictures (jpeg and a few pngs). My server uses postgresql for the backend

Both servers have had zero problems regarding thread safety in PHP.

Well, that's a small lie - I can't get turck-mmcache to work under a threadsafe PHP on my Gentoo server. No great loss though - it works fine on the driverheaven server.

Make of that what you will.

----------

## kashani

I'm fine with that, but a bit surprised that it hasn't exploded on you. I've seen a number of people have issues that reverting back to prefork Apache fixed. 

In Red Hat 9.0 the Apache2 packageinstalls a httpd and httpd-worker. By default httpd is used. I'm not sure how the Enterprise versions work. I'd be amazed if worker in the default as Red Hat tends to move pretty slowly in there areas.

kashani

----------

## kashani

 *UberLord wrote:*   

> 
> 
> I'm an admin for http://www.driverheaven.net
> 
> It runs RHEL which ships (by default I think) with Apache2 running the worker MPM + php + nptl threading.
> ...

 

I hope I don't come off like I beating this to death. I really really do want a fully threaded Apache2/PHP webserver. I could cut at least 5 machines out of the www farms and get a lot more out of my db servers. Problem is I looked into it and couldn't find anyone who'd say it was safe for production environments.

http://www.driverheaven.net/phpinfo.php

I'd delete that asap or put it in a much harder to find place. I see thread safety is off so I'm assuming that means you're using the prefork Apache. 

kashani

----------

## UberLord

 *kashani wrote:*   

> 
> 
> I'd delete that asap or put it in a much harder to find place. I see thread safety is off so I'm assuming that means you're using the prefork Apache. 
> 
> 

 

Ahhh shite - wtf put that there???? Thats coming off! Thanks

Well, it was running worker at least 2 weeks back and there's no thread in the admin forum to say why it was changed.....bleh. Maybe I just imagined it was worker when we first got the server...... I dunno. Time to change it back I guess  :Smile: 

----------

## splooge

One thing I've noticed browsing these forums: kashani = 100% signal to noise ratio (or should that be 0%?  =/).  

I search for posts from him just to learn things.

----------

## splooge

That and the lego avatar is the ownage.

----------

## Chewi

Well with normal threading, it was fast and I was perfectly happy with that but I decided to give NPTL a go anyway. Daaaaamn. Now that's fast. I can't believe how much faster it is to before I made this thread. Okay it might be a bit risky (and yes, I do know exactly what thread-safe means) but I think it's a risk well worth taking. I have a database and webroot backup made every other night anyway. Thanks so much, guys!

----------

## UberLord

 *kashani wrote:*   

> I hope I don't come off like I beating this to death. I really really do want a fully threaded Apache2/PHP webserver. I could cut at least 5 machines out of the www farms and get a lot more out of my db servers. Problem is I looked into it and couldn't find anyone who'd say it was safe for production environments.

 

Well, why not try it out in a test environment and run testbed applications that use 100% of the libraries your customer users. If it proves stable after a given amount of time - say a week to a month - then move the testing setup over to production.

If no-one takes the plunge, then no-one can tell you if it works in a production environment or not.

On another note, Apache and PHP for windows only come in threadsafe versions - so if a library works on both then it's pretty safe to be threadsafe. IIRC PHP 4.3.0 was the first stable version for threads as the developers had help from some Microsoft IIS developers.

----------

## Chewi

Hmmm it slowed down a tad so I checked the memory usage. Two apache processes hogging 45% of the RAM. I think there's some serious memory leakage going on here! Dammit. I suspect the same thing was happening in normal threaded mode.

Clearly the memory leaks aren't uncommon and I can see that work is done to resolve the issue so hopefully it won't be too long before things get straightened out. I guess what I can do in the meantime is get Apache to restart every few hours using cron.

----------

## kashani

 *UberLord wrote:*   

> 
> 
> Well, why not try it out in a test environment and run testbed applications that use 100% of the libraries your customer users. If it proves stable after a given amount of time - say a week to a month - then move the testing setup over to production.
> 
> If no-one takes the plunge, then no-one can tell you if it works in a production environment or not.
> ...

 

It's a bit more complicated in our environment, which I think affects a number of people as well so I'll try explain in depth.

I'm currently working at a managed hosting company. Generally our customers have had serious problems due to growing pains, incompetence, or just bad design. We usually move them from a single server from some guy's garage to multiple web servers with a dedicated db server. Load balance the whole thing and now they're seeing three or four 9's of uptime. We'd probably see five 9's but their design is usually so bad it's neigh impossible.

For one of our larger clients I'd need mod_php, mod_mysqlauth, mod_perl, mod_ssl, and  mod_mmapstatic. I'm not even sure what libs those require. I don't think mod_authmysql has been ported to Apache2, mod_perl is getting there, and mmap_static is a touchy son of a bitch.

Now say I spend 2 months going through this mess with two test servers and actually get it working. How much speed am I really going to see?

For the sake of arguement let's assume I see a phenomenal 50% increase. I'd guess 10% at most in real life. Let's also assume that PHP and MySQL can mutiplex properly. So instead of a few hundred connections to the db server I have ten. This would be pretty damn sweet. Until the customer comes to me the next week wanting some new functionality which isn't thread safe or supported in Apache2.

Until the threadedness has been worked out a bit more I don't really see any motivation to do this, unless you have a very specific application/site or don't care about stability. 

kashani

----------

