# [SOLVED] Apache hangs with MPM_worker and PHP 5.3.5

## superbrain666

I made a full emerge -e world system upgrade after 1,5 Years runtime.

Now i have the problem, that apache is hanging after little more than 12h runtime.

I think the problem is that the apache child processes are not effectively getting killed.

if i do

```
/etc/init.d/apache restart

```

I get 

```
* Stopping apache2 ...                                                                                               [ ok ]

* Starting apache2 ...

httpd (pid 26049) already running                                                                                  [ ok ]

```

So i have to restart again

```
/etc/init.d/apache2 restart

 * Stopping apache2 ...

httpd (no pid file) not running                                                                                     [ ok ]

 * Starting apache2 ... 

```

in /var/log/apache/error_log i found:

```
[Mon Mar 07 09:28:58 2011] [warn] child process 26052 still did not exit, sending a SIGTERM

[Mon Mar 07 09:28:58 2011] [warn] child process 26054 still did not exit, sending a SIGTERM

[Mon Mar 07 09:28:58 2011] [warn] child process 10011 still did not exit, sending a SIGTERM

[Mon Mar 07 09:29:00 2011] [warn] child process 26052 still did not exit, sending a SIGTERM

[Mon Mar 07 09:29:00 2011] [warn] child process 26054 still did not exit, sending a SIGTERM

[Mon Mar 07 09:29:00 2011] [warn] child process 10011 still did not exit, sending a SIGTERM

[Mon Mar 07 09:29:02 2011] [warn] child process 26052 still did not exit, sending a SIGTERM

[Mon Mar 07 09:29:02 2011] [warn] child process 26054 still did not exit, sending a SIGTERM

[Mon Mar 07 09:29:02 2011] [warn] child process 10011 still did not exit, sending a SIGTERM

[Mon Mar 07 09:29:04 2011] [error] child process 26052 still did not exit, sending a SIGKILL

[Mon Mar 07 09:29:04 2011] [error] child process 26054 still did not exit, sending a SIGKILL

[Mon Mar 07 09:29:04 2011] [error] child process 10011 still did not exit, sending a SIGKILL

[Mon Mar 07 09:29:05 2011] [notice] caught SIGTERM, shutting down

[Mon Mar 07 09:29:08 2011] [notice] Apache/2.2.16 (Unix) mod_ssl/2.2.16 OpenSSL/1.0.0d PHP/5.3.5-pl0-gentoo configured -- resuming normal operations

```

Any Ideas whats happening here?

I was not touching any apache related configfiles nor changing any use flags.

Just the basesystem was upgraded(gcc, glibc...)Last edited by superbrain666 on Mon Mar 07, 2011 5:50 pm; edited 4 times in total

----------

## hydrapolic

Try to kill the apache2 processes (find them with ps).

----------

## superbrain666

If i do that during normal operation, the apache is shutting down normaly:

```
killall apache2
```

brings:

```
[Mon Mar 07 12:34:15 2011] [notice] caught SIGTERM, shutting down

```

But this is normal as the server does not hang. The same thing happens with the init script. 

I have to wait some time until it hangs again.

----------

## hydrapolic

In what order did you upgrade the machine ?  Was it something like the following: kernel, linux-headers, glibc, gcc, libtool ?

----------

## superbrain666

the following:

 *Quote:*   

> 
> 
> emerge -ave system
> 
> 

 

```

gcc-config x86_64-pc-linux-gnu-4.4.5

```

 *Quote:*   

> 
> 
> emerge -ave world
> 
> 

 

 *Quote:*   

> 
> 
> emerge --depclean
> 
> 

 

should i run emerge -e world once again? Just to be sure?

----------

## hydrapolic

Have you also upgraded your kernel and/or have you rebuilded/upgraded apache ? Maybe the output of emerge --info can also be helpful.

----------

## superbrain666

I didn't upgraded the kernel. It is running still an old one 2.6.34-gentoo-r1

output of emerge --info

```

Portage 2.1.9.25 (hardened/linux/amd64, gcc-4.4.5, glibc-2.11.3-r0, 2.6.34-gentoo-r1 x86_64)

=================================================================

System uname: Linux-2.6.34-gentoo-r1-x86_64-Intel-R-_Core-TM-_i7_CPU_920_@_2.67GHz-with-gentoo-1.12.14

Timestamp of tree: Sun, 06 Mar 2011 23:30:01 +0000

app-shells/bash:     4.1_p9

dev-lang/python:     2.6.6-r2, 3.1.3-r1

dev-util/cmake:      2.8.1-r2

sys-apps/baselayout: 1.12.14-r1

sys-apps/sandbox:    2.4

sys-devel/autoconf:  2.65-r1

sys-devel/automake:  1.5-r1, 1.11.1

sys-devel/binutils:  2.20.1-r1

sys-devel/gcc:       4.4.5

sys-devel/gcc-config: 1.4.1

sys-devel/libtool:   2.2.10

sys-devel/make:      3.81-r2

virtual/os-headers:  2.6.36.1 (sys-kernel/linux-headers)

ACCEPT_KEYWORDS="amd64"

ACCEPT_LICENSE="* -@EULA"

CBUILD="x86_64-pc-linux-gnu"

CFLAGS="-O2 -pipe"

CHOST="x86_64-pc-linux-gnu"

CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa /var/bind"

CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"

CXXFLAGS="-O2 -pipe"

DISTDIR="/usr/portage/distfiles"

FEATURES="assume-digests binpkg-logs distlocks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"

GENTOO_MIRRORS="http://de-mirror.org/distro/gentoo/ http://gentoo.mneisen.org/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ ftp://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/ http://ftp.snt.utwente.nl/pub/os/linux/gentoo http://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/"

LANG="de_DE.UTF-8"

LDFLAGS="-Wl,-O1 -Wl,--as-needed"

MAKEOPTS="-j7"

PKGDIR="/usr/portage/packages"

PORTAGE_CONFIGROOT="/"

PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"

PORTAGE_TMPDIR="/ebuild/working"

PORTDIR="/usr/portage"

SYNC="rsync://rsync.gentoo.org/gentoo-portage"

USE="acl amd64 apache2 bash-completion berkdb bzip2 cgi cli cracklib crypt ctype cups cxx dovecot dri exif gd gdbm gnutls gpm gs hardened hash iconv imap innodb jpeg justify lcms ldap managesieve mmx modules mudflap multilib mysql mysqli ncurses nls nptl nptlonly openmp pam pcntl pcre pdo perl php png posix pppd python readline session sieve simplexml sqlite sse sse2 sse3 ssl ssse3 suhosin sysfs tcpd threads tidy tiff tokenizer truetype unicode urandom vpopmail xml xmlreader xmlwriter xorg zip zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" APACHE2_MPMS="worker" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga neomagic nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" 

Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY

```

----------

## Anon-E-moose

why are you not running "march=native" in your CFLAGS?

I don't think that has anything to do with the apache problem, 

but you'll probably get better system performance with it.

Edit to add: I'm assuming that you doing restarts, etc as "root"

----------

## superbrain666

I was not concerning about CFLAGS as the system is incredibly fast, also with no optimization.

Now i was inserting CFLAGS regarding http://en.gentoo-wiki.com/wiki/Safe_Cflags/Intel#Core_i7_and_Core_i5.2C_Xeon_55xx.

now im doing emerge -e system and a emerge -e world . Maybe something changes.

I'm doing the restarts a root. It is a root-server  :Smile: 

----------

## hydrapolic

Maybe mtune is a safer choice, because it allows you to restore your system on other hardware. But that is your decision.

Regarding the apache problem, try to re-emerge apache, see if it helps. By the way, you are running a hardened profile without a hardened kernel ?

----------

## superbrain666

You are right it was a relic from a time where the "normal" server profile was not supported.

I switch to 10.0/server and remerge everything

----------

## superbrain666

I probably solved it by moving from MPM_worker to MPM_prefork.

I do not know which of my apache / PHP modules is not threadsafe.. but yes it seems to work now.

I will file a bug.

Many thanks to the help with the CFLAGS and the profile.. I will move back to hardened if i have more time.

----------

## mimosinnet

 *superbrain666 wrote:*   

> Now i have the problem, that apache is hanging after little more than 12h runtime.
> 
> I think the problem is that the apache child processes are not effectively getting killed.

 

 *superbrain666 wrote:*   

> I probably solved it by moving from MPM_worker to MPM_prefork. 

 

I have come across the same issue and a MPM_worker seems a nice option for a multi-core system. I have followed this thread, and set the use-flags +threads and +ithreads. At the moment, it seems to be working.

Thanks!

----------

