# MySQL keeps crashing with the following error ....

## Jiffy

This post didn't help.

https://forums.gentoo.org/viewtopic.php?t=104434&highlight=mysqld+signal+11

Besides the log file, is there anything else I can provide to help resolve this?

```

040108 11:12:41  mysqld started

Warning: Ignoring user change to 'mysql' because the user was set to 'mysql' earlier on the command line

/usr/sbin/mysqld: ready for connections.

Version: '4.0.16'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306

mysqld got signal 11;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

We will try our best to scrape up some info that will hopefully help diagnose

the problem, but since we have already crashed, something is definitely wrong

and this may fail.

key_buffer_size=16777216

read_buffer_size=131072

max_used_connections=24

max_connections=200

threads_connected=11

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 451582 K

bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

thd=(nil)

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

frame pointer (ebp) is NULL, did you compile with

-fomit-frame-pointer? Aborting backtrace!

The manual page at http://www.mysql.com/doc/en/Crashing.html contains

information that should help you find out what is causing the crash.

Number of processes running now: 0

040108 11:19:14  mysqld restarted

Warning: Ignoring user change to 'mysql' because the user was set to 'mysql' earlier on the command line

/usr/sbin/mysqld: ready for connections.

Version: '4.0.16'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306

mysqld got signal 11;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

We will try our best to scrape up some info that will hopefully help diagnose

the problem, but since we have already crashed, something is definitely wrong

and this may fail.

key_buffer_size=16777216

read_buffer_size=131072

max_used_connections=18

max_connections=200

threads_connected=2

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 451582 K

bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

thd=(nil)

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

frame pointer (ebp) is NULL, did you compile with

-fomit-frame-pointer? Aborting backtrace!

The manual page at http://www.mysql.com/doc/en/Crashing.html contains

information that should help you find out what is causing the crash.

Number of processes running now: 0

040108 11:20:56  mysqld restarted

Warning: Ignoring user change to 'mysql' because the user was set to 'mysql' earlier on the command line

/usr/sbin/mysqld: ready for connections.

Version: '4.0.16'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306

mysqld got signal 11;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

We will try our best to scrape up some info that will hopefully help diagnose

the problem, but since we have already crashed, something is definitely wrong

and this may fail.

key_buffer_size=16777216

read_buffer_size=131072

max_used_connections=5

max_connections=200

threads_connected=5

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 451582 K

bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

thd=(nil)

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

frame pointer (ebp) is NULL, did you compile with

-fomit-frame-pointer? Aborting backtrace!

The manual page at http://www.mysql.com/doc/en/Crashing.html contains

information that should help you find out what is causing the crash.

Number of processes running now: 0

040108 11:20:59  mysqld restarted

Warning: Ignoring user change to 'mysql' because the user was set to 'mysql' earlier on the command line

/usr/sbin/mysqld: ready for connections.

Version: '4.0.16'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306

mysqld got signal 11;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

We will try our best to scrape up some info that will hopefully help diagnose

the problem, but since we have already crashed, something is definitely wrong

and this may fail.

key_buffer_size=16777216

read_buffer_size=131072

max_used_connections=13

max_connections=200

threads_connected=13

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 451582 K

bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

thd=(nil)

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

frame pointer (ebp) is NULL, did you compile with

-fomit-frame-pointer? Aborting backtrace!

The manual page at http://www.mysql.com/doc/en/Crashing.html contains

information that should help you find out what is causing the crash.

Number of processes running now: 0

040108 11:21:14  mysqld restarted

Warning: Ignoring user change to 'mysql' because the user was set to 'mysql' earlier on the command line

/usr/sbin/mysqld: ready for connections.

Version: '4.0.16'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306

```

Thank you.

----------

## Janne Pikkarainen

Signal 11 really is often a sign of some kind of hardware problem - a bad RAM, CPU running too hot, bad motherboard... and it can get triggered in all kind of weird situations. Does memtest86 run ok? Your mysqld seems to want to allocate pretty much memory and if there's something wrong with the RAM, strange things are about to happen.

----------

## adaptr

PLEASE don't post the exact same crash three times...

 *Quote:*   

> It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured.

 

So - have you remerged it ? With --deep ?

And this time without -fomit-frame-pointer in CFLAGS, as suggested ?

A stack trace would definitely come in useful when the error is obscure.

 *Quote:*   

> It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 451582 K bytes of memory.
> 
> Hope that's ok; if not, decrease some variables in the equation. 

 

This ring any bells ?

450 MEGAbytes seems excessive to me...

----------

## Jiffy

memtest86 passed.

I should have added that this happens on three different servers.

Two DELL PowerEdge 6350's and a Compaq Proliant.

All servers are Quad 550Mhz w/ 1 Gig of RAM running the same kernel

uname -a

Linux entdb 2.4.22-gentoo-r3 #1 SMP Wed Jan 7 14:18:04 EST 2004 i686 Pentium III (Katmai) GenuineIntel GNU/Linux

Yes, I have re-emerged it. Then I built another PowerEdge with the same results. I though it might be something with the Dell boxes, so I built the Compaq server and it crashed there too.

Original CFLAGS="-march=pentium3 -mfpmath=sse,387 -pipe -fomit-frame-pointer -funroll-loops -O2"

Currect CFLAGS="-march=pentium3 -mfpmath=387 -pipe -funroll-loops -O2"

USE="apache2 curl flash freetds gd mbox mcal mmx mysql oci8 odbc php pic ppds samba snmp sse usb -arts -kde -gnome -qt -X"

----------

## Jiffy

I should have posted this log as this is without -fomit-frame-pointer:

```

Version: '4.0.17'  socket: '/tmp/mysql.sock'  port: 3306

mysqld got signal 11;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

We will try our best to scrape up some info that will hopefully help diagnose

the problem, but since we have already crashed, something is definitely wrong

and this may fail.

key_buffer_size=268435456

read_buffer_size=1044480

max_used_connections=28

max_connections=200

threads_connected=17

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 670942 K

bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

thd=0x42c41948

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

Cannot determine thread, fp=0xbbdfea78, backtrace may not be correct.

Stack range sanity check OK, backtrace follows:

0x80fdf9f

0x49154635

0x80b8737

0x811e8f2

0x80f4eda

0x812d0e0

0x812712d

0x8125cb8

0x810b458

0x810f899

0x810a4cc

0x810a015

0x8109969

0x4914ebd0

0x48ebc997

New value of fp=(nil) failed sanity check, terminating stack trace!

Please read http://www.mysql.com/doc/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved

stack trace is much more helpful in diagnosing the problem, so please do

resolve it

Trying to get some variables.

Some pointers may be invalid and cause the dump to abort...

thd->query at 0x8847030 = SELECT results FROM ping

                             WHERE node='servername.deleted' AND

                                   results>='0'

thd->thread_id=152762

The manual page at http://www.mysql.com/doc/en/Crashing.html contains

information that should help you find out what is causing the crash.

Number of processes running now: 0

```

----------

## Jiffy

Rebuilt Server from scratch with less aggressive CFLAGS and 2.6.1 kernel and it crashed again.

USE="mmx mysql nptl pic ppds sse -arts -kde -gnome -gtk -nls -qt -X -xmms -xv -ssl"

CHOST="i686-pc-linux-gnu"

CFLAGS="-march=pentium3 -pipe -funroll-loops -O2"

CXXFLAGS="${CFLAGS}"

Linux tempbuild 2.6.1-gentoo #1 SMP Mon Jan 12 07:35:18 EST 2004 i686 Pentium III (Katmai) GenuineIntel GNU/Linux

```

040112 10:02:15  mysqld started

Warning: Ignoring user change to 'mysql' because the user was set to 'mysql' earlier on the command line

/usr/sbin/mysqld: ready for connections.

Version: '4.0.16'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306

mysqld got signal 11;

This could be because you hit a bug. It is also possible that this binary

or one of the libraries it was linked against is corrupt, improperly built,

or misconfigured. This error can also be caused by malfunctioning hardware.

We will try our best to scrape up some info that will hopefully help diagnose

the problem, but since we have already crashed, something is definitely wrong

and this may fail.

key_buffer_size=33554432

read_buffer_size=131072

max_used_connections=3

max_connections=200

threads_connected=1

It is possible that mysqld could use up to

key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 467966 K

bytes of memory

Hope that's ok; if not, decrease some variables in the equation.

thd=(nil)

Attempting backtrace. You can use the following information to find out

where mysqld died. If you see no messages after this, something went

terribly wrong...

Cannot determine thread, fp=0xbfff8018, backtrace may not be correct.

Stack range sanity check OK, backtrace follows:

0x80fa5e8

0x60b03c

0x52999e

0x528cf2

0x5283cb

0x528187

0x80fc5a1

0x80fbac9

0x3d890c

0x80ab9d1

New value of fp=(nil) failed sanity check, terminating stack trace!

Please read http://www.mysql.com/doc/en/Using_stack_trace.html and follow instructions on how to resolve the stack trace. Resolved

stack trace is much more helpful in diagnosing the problem, so please do

resolve it

The manual page at http://www.mysql.com/doc/en/Crashing.html contains

information that should help you find out what is causing the crash.

Number of processes running now: 0

040112 10:23:13  mysqld restarted

Warning: Ignoring user change to 'mysql' because the user was set to 'mysql' earlier on the command line

/usr/sbin/mysqld: ready for connections.

Version: '4.0.16'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306

```

----------

## Jiffy

I installed Slackware 9.1 with MySQL 4.0.17 on one of the constantly crashing servers.

I used the same settings in my.conf (except for directory structure) and same CFLAGS to build MySQL, not one crash over night as opposed to crashing every 10 to 15 minutes.

Any ideas?

----------

## Jiffy

Did I offend somebody?

I could really use some help please.

----------

## STEALER.net

I've had the same problem. I'm running sys-libs/glibc-2.3.2-r3 and dev-db/mysql-4.0.16. MySQL crashes with segfault from time to time (especially when doing OPTIMIZE TABLE) when both packages are emerged with nptl enabled. Re-emerging sys-libs/glibc and dev-db/mysql without nptl stopped the problem. My kernel version is 2.6.0.

Seems like the nptl implemtation in kernel and sys-libs/glibc-2.3.2-r3 in cooperation with MySQL seem to be faulty. All other programs run without problems. MySQL-4.0.17 also segfaults. sys-libs/glibc-2.3.3_pre20031222 is stable for me with MySQL and Kernel 2.6.1

----------

## Jiffy

Hi, STEALER,

Thanks for the tip, I'll give it a try.

I just would like to know if 

 *Quote:*   

> sys-libs/glibc-2.3.3_pre20031222 is stable for me with MySQL and Kernel 2.6.1

 

is with or without nptl and is MySQL .16 or .17?

----------

## STEALER.net

 *Jiffy wrote:*   

> 
> 
> I just would like to know if 
> 
>  *Quote:*   sys-libs/glibc-2.3.3_pre20031222 is stable for me with MySQL and Kernel 2.6.1 
> ...

 

I've got 2 setups:

1. "developer" system

- ACCEPT_KEYWORDS="~x86"

- gentoo-dev-sources-2.6.1-r1

- dev-db/mysql-4.0.17

- sys-libs/glibc-2.3.3_pre20031222

- works fine w/ and w/o nptl in USE

2. "productive" system

- ACCEPT_KEYWORDS="x86"

- gentoo-dev-sources-2.6.0

- dev-db/mysql-4.0.16 also tested with dev-db/mysql-4.0.17

- sys-libs/glibc-2.3.2-r3

- mysql segfaults w/ nptl in USE

- mysql runs fine w/o nptl in USE

emerge --info on developer system

```

Portage 2.0.50_pre16 (default-x86-1.4, gcc-3.3.2, glibc-2.3.3_pre20031222-r0, 2.6.1-G-R1)

=================================================================

System uname: 2.6.1-G-R1 i686 AMD Athlon(tm) Processor

Gentoo Base System version 1.4.3.12

distcc 2.12.1 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]

ccache version 2.3 [enabled]

Autoconf: sys-devel/autoconf-2.59

Automake: sys-devel/automake-1.7.8

ACCEPT_KEYWORDS="x86 ~x86"

AUTOCLEAN="yes"

CFLAGS="-O3 -pipe -fomit-frame-pointer -fprefetch-loop-arrays

-funroll-loops -fforce-addr -fmove-all-movables -march=athlon-tbird

-mmmx -m3dnow"

CHOST="i686-pc-linux-gnu"

COMPILER="gcc3"

CONFIG_PROTECT="/etc /opt/tomcat/conf /usr/X11R6/lib/X11/xkb

/usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config

/usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/

/usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/

/usr/share/texmf/xdvi/ /var/bind /var/qmail/control"

CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d"

CXXFLAGS="-O3 -pipe -fomit-frame-pointer -fprefetch-loop-arrays

-funroll-loops -fforce-addr -fmove-all-movables -march=athlon-tbird

-mmmx -m3dnow -Wno-deprecated"

DISTDIR="/usr/portage/distfiles"

FEATURES="autoaddcvs ccache cvs digest sandbox usersandbox"

GENTOO_MIRRORS="http://ftp.uni-erlangen.de/pub/mirrors/gentoo

ftp://trumpetti.atm.tut.fi/gentoo/ http://gentoo.linux.no/

ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ http://ftp.easynet.nl/mirror/gentoo/

ftp://ftp.uninett.no/pub/linux/Gentoo"

MAKEOPTS="-j3"

PKGDIR="/usr/portage/packages"

PORTAGE_TMPDIR="/var/tmp"

PORTDIR="/usr/portage"

PORTDIR_OVERLAY="/usr/local/portage"

SYNC="rsync://gentoo.mirror.at.stealer.net/gentoo-portage"

USE="3dnow 3ds X X509 aalib acl alsa apache2 avi berkdb cdr cjk crypt

cups curl directfb doc encode esd evms2 exiscan exiscan-acl fbcon

ffmpeg firebird flash foomaticdb freetds gd gd-external gdbm gif gnome

gnomedb gpm gstreamer gtk gtk2 guile imagemagick imap imlib informix

innodb java jpeg ldap libg++ libwww lmtp mad maildir mbox mozaccess

mozcalendar mozilla mozinterfaceinfo mozp3p mozsvg mozxmlterm

mysql nas ncurses nls nptl odbc oggvorbis oss pam pdflib perl php png

postgres python qt quicktime readline ruby samba sasl slang slp socks5

spell sqlite sse ssl tcltk tcpd tetex tiff truetype usb wmf x86 xchattext xfs

xml xml2 xmms zlib"

```

emerge --info on productive system

```

Portage 2.0.49-r20 (default-x86-1.4, gcc-3.2.3, glibc-2.3.2-r3, 2.6.0-G)

=================================================================

System uname: 2.6.0-G i686 AMD Athlon(tm) XP 2200+

Gentoo Base System version 1.4.3.10

ACCEPT_KEYWORDS="x86"

AUTOCLEAN="yes"

CFLAGS="-O3 -pipe -fomit-frame-pointer -fprefetch-loop-arrays

-funroll-loops -fforce-addr -fmove-all-movables -march=athlon-xp

-mmmx -msse -m3dnow"

CHOST="i686-pc-linux-gnu"

COMPILER="gcc3"

CONFIG_PROTECT="/etc /usr/kde/2/share/config

/usr/kde/3/share/config /usr/share/config /var/bind /var/qmail/control"

CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d"

CXXFLAGS="-O3 -pipe -fomit-frame-pointer -fprefetch-loop-arrays

-funroll-loops -fforce-addr -fmove-all-movables -march=athlon-xp

-mmmx -msse -m3dnow -Wno-deprecated"

DISTDIR="/usr/portage/distfiles"

FEATURES="autoaddcvs ccache sandbox"

GENTOO_MIRRORS="http://ftp.uni-erlangen.de/pub/mirrors/gentoo

ftp://trumpetti.atm.tut.fi/gentoo/ http://gentoo.linux.no/

http://ftp.easynet.nl/mirror/gentoo/"

MAKEOPTS="-j3"

PKGDIR="/usr/portage/packages"

PORTAGE_TMPDIR="/var/tmp"

PORTDIR="/usr/portage"

PORTDIR_OVERLAY="/usr/local/portage"

SYNC="rsync://gentoo.mirror.at.stealer.net/gentoo-portage"

USE="3dnow X509 acl apache2 berkdb crypt cscope curl evms2 exiscan

exiscan-acl fam flash gd gd-external gdbm gif gpm imap imlib innodb

java jpeg kerberos lcms libg++ libwww lmtp maildir mbox mmx mysql

ncurses nls nocd pam passfile pdflib perl pg-hier php png postgres python

readline ruby sasl skey slang snmp sse ssl tcpd tiff truetype wmf x86 xfs

xml xml2 xv zlib"

```

Hope this helps.

<edit>

Corrected wrong Kernel version

</edit>

----------

## Jiffy

Thank you very much STEALER.

I'm going to rebuild the server following your productive system configuration where possible.

I have to ask, since my original build was glibc-2.3.2-r3 without nptl but the 2.4.22-gentoo-r3 kernel, as opposed to yours with the gentoo-dev-sources-2.6.0 kernel, could that have been the issue?

----------

## STEALER.net

 *Jiffy wrote:*   

> I have to ask, since my original build was glibc-2.3.2-r3 without nptl but the 2.4.22-gentoo-r3 kernel, as opposed to yours with the gentoo-dev-sources-2.6.0 kernel, could that have been the issue?

 

Well, I've never tried gentoo-sources-2.4.22. I switched directly from gentoo-sources-2.4.20-r9 to 2.6.0.

I've another setup running gentoo-sources-2.4.20-r9, glibc-2.3.3_pre20031222-r0 and mysql-4.0.17. With ACCEPT_KEYWORDS=~x86 and w/o nptl. That setup works without a problem. But since kernels in the 2.4 series don't support nptl, the USE flag nptl should have no effect when emerging sys-libs/glibc.

----------

