# Kernel panic after moving from kernel 3.5.7 to 3.6.11

## Olis

Hello and Merry X-Mas to everybody!

While others are preparing for Christmas I try to upgrade my system from kernel 3.5.7 to the new stable kernel 3.6.11.

While kernel updates worked without problems for me during the last years this one is causing some trouble. For unknown reason the kernel panics during boot phase.

I have no idea what's going wrong as I can only see the trace on the screen, I don't know where it is going wrong and I don't have any boot.msg to see what went wrong.

All I could do was to take a screen shot. Here's the trace, maybe it helps to analyze the problem:

http://a.yfrog.com/img29/6920/ygar.jpg

Is there a way to get the boot.msg or a boot.log again? I'm on baselayout 2. All I get is the /var/log/rc.log which doesn't really help when the kernel panics.

Before building the new kernel I copied the .config file from kernel 3.5.7, ran 'make oldconfig' and selected all default answers for the new options. Then I build the kernel using 'make && make modules_install' and after that 'make install' to move the kernel to /boot.Last edited by Olis on Wed Dec 26, 2012 9:07 am; edited 1 time in total

----------

## eccerr0r

It looks like it couldn't mount your root partition during a pivot (?).

Double check your drivers - disk drivers, filesystem drivers, and make sure your root= is correct (as well as your initrd).

Not sure how far it really got, did it get to running your init?  (meaning did it print out "Gentoo" in color?)

FWIW, the kernels from 3.3 to 3.7 the .config file changed significantly somehow for my system, but not sure (I totally skipped 3.4...).  I normally could copy .config without changing much but somewhere in these kernels it required another careful scrutiny of the options else it would result in a broken kernel...

----------

## NeddySeagoon

Olis,

Please boot your old kernel, so your system works, then use wgetpaste to post both your old and new kernel .config files.

Your lspci output will be useful too.

The panic message itself has scrolled off the screen, so it you only changed the kernel, the solution can be found by looking at the before and after .config files.

----------

## Olis

 *eccerr0r wrote:*   

> It looks like it couldn't mount your root partition during a pivot (?).
> 
> Double check your drivers - disk drivers, filesystem drivers, and make sure your root= is correct (as well as your initrd).
> 
> Not sure how far it really got, did it get to running your init?  (meaning did it print out "Gentoo" in color?)
> ...

 

As far as I can tell (information is scrolling too fast over the screen) it doesn't get far enough  to run init. I didn't change anything related to disk or file systems drivers.

As the system comes up with no problems using kernel 3.5.7 I don't think it is related to drivers. Not sure why you mentioned kernel 3.3 as I switched from 3.5.7 to 3.6. 11. But I didn't have any problems with kernels 3.1, 3.2, 3.3 or 3.4.

----------

## Olis

 *NeddySeagoon wrote:*   

> Olis,
> 
> Please boot your old kernel, so your system works, then use wgetpaste to post both your old and new kernel .config files.
> 
> Your lspci output will be useful too.
> ...

 

Here are the config files:

Kernel 3.5.7: http://bpaste.net/show/66368/

Kernel 3.6.11: http://bpaste.net/show/66369/

Here's the diff: http://bpaste.net/show/66370/

And here's the output of lspci:

```
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)

00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)

00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)

00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)

00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)

00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)

00:1c.1 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 2 (rev b5)

00:1c.2 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 3 (rev b5)

00:1c.3 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 4 (rev b5)

00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 5 (rev b5)

00:1c.5 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 6 (rev b5)

00:1c.6 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 7 (rev b5)

00:1c.7 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 8 (rev b5)

00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)

00:1f.0 ISA bridge: Intel Corporation H67 Express Chipset Family LPC Controller (rev 05)

00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller (rev 05)

00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)

01:00.0 VGA compatible controller: NVIDIA Corporation GF108 [GeForce GT 530] (rev a1)

01:00.1 Audio device: NVIDIA Corporation GF108 High Definition Audio Controller (rev a1)

06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)

07:00.0 USB controller: Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller (rev 02)
```

Yes, I only upgraded the kernel. The system comes up with no problems running kernel 3.5.7.

Here's a list of all kernels I used since setting up this system (I completely reinstalled Gentoo and switched from 32 to 64 bits when I got this system in January).

```
     Sun Jan 29 15:13:48 2012 >>> sys-kernel/gentoo-sources-3.2.1-r2

     Sat Mar 24 13:06:55 2012 >>> sys-kernel/gentoo-sources-3.2.12

     Fri Jun 22 06:57:11 2012 >>> sys-kernel/gentoo-sources-3.3.8

     Sat Aug 25 11:27:28 2012 >>> sys-kernel/gentoo-sources-3.4.9

     Sun Oct 21 07:59:23 2012 >>> sys-kernel/gentoo-sources-3.5.7

     Tue Oct 30 06:29:54 2012 >>> sys-kernel/gentoo-sources-3.5.7

     Sat Dec 22 10:58:34 2012 >>> sys-kernel/gentoo-sources-3.6.11
```

So there's really no way to get a boot.msg file from the kernel?

Oliver

----------

## eccerr0r

You have to use a serial console and a helper machine to get the pre-boot information.  However I still think you have a driver missing.  The reason why I said 3.3 is because I was using 3.3.  The jump somewhere between 3.3 and 3.7 caused me some grief.

Are you using an initrd?  That's probably one way to single-machine do this.  Else you can try to use fbcon or KMS and with high resolution during boot so more of the debug logs stays on the screen.

----------

## Olis

 *eccerr0r wrote:*   

> You have to use a serial console and a helper machine to get the pre-boot information.  However I still think you have a driver missing.  The reason why I said 3.3 is because I was using 3.3.  The jump somewhere between 3.3 and 3.7 caused me some grief.
> 
> Are you using an initrd?  That's probably one way to single-machine do this.  Else you can try to use fbcon or KMS and with high resolution during boot so more of the debug logs stays on the screen.

 

I don't use initrd. / and /boot are normal partitions (/ is ext4, /boot ext2), for all other partitions I use LVM. I still have a separate /usr, but this should work with udev-171.

I tried to get a higher resolution during boot but I never got it working on this machine, although it worked on my previous system. Graphics controller is an NVIDIA GT530. 

If I'm really missing a driver than it must be a driver that wasn't required for all other stable 3.x kernels as they worked without problems. 

Oliver

----------

## necron

Edit:  compiling ext2 support did not change anything.  the actual message I am getting is telling me that the error asociated with smp.c has to do with (what I guess is a function) 

```
native_smp_send_reschedule+0x4f/0x60()
```

I've been having the same issue moving from 3.5.7 to 3.6.11.  The two kernels have the same configuration except for the fact that i compiled 3.6.11 without support for "Tickless System" because I couldn't find the damned option.  I might be wrong, but I'm pretty sure that's why it isn't working for me

I would share the actual error message if I could, but failing that I can tell you that there's a complaint coming out of my machine about smp.c, so I'm guessing that it does have something to with processor(s) not having what they need?

Here's lspci, for whatever that's worth.  I think we're similar enough where we're having the same issue

```
00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM             

 Controller (rev 09)

00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Process             

or Family Integrated Graphics Controller (rev 09)

00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset             

 Family MEI Controller #1 (rev 04)

00:16.3 Serial controller: Intel Corporation 6 Series/C200 Series Chipset Family             

 KT Controller (rev 04)

00:19.0 Ethernet controller: Intel Corporation 82579LM Gigabit Network Connectio             

n (rev 04)

00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family US             

B Enhanced Host Controller #2 (rev 04)

00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High             

 Definition Audio Controller (rev 04)

00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Ex             

press Root Port 1 (rev b4)

00:1c.1 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Ex             

press Root Port 2 (rev b4)

00:1c.3 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Ex             

press Root Port 4 (rev b4)

00:1c.4 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Ex             

press Root Port 5 (rev b4)

00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family US             

B Enhanced Host Controller #1 (rev 04)

00:1f.0 ISA bridge: Intel Corporation QM67 Express Chipset Family LPC Controller             

 (rev 04)

00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family 6             

 port SATA AHCI Controller (rev 04)

00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Contr             

oller (rev 04)

03:00.0 Network controller: Intel Corporation Centrino Ultimate-N 6300 (rev 35)

0d:00.0 System peripheral: Ricoh Co Ltd PCIe SDXC/MMC Host Controller (rev 05)

0d:00.3 FireWire (IEEE 1394): Ricoh Co Ltd R5C832 PCIe IEEE 1394 Controller (rev             

 04)
```

----------

## eccerr0r

On my 3.7 build, "tickless kernel" got relocated to general setup/timers subsystem... This was one option that moved significantly since the last time I built a kernel.

In that screen shot I see "mount_root" function call - this call should not be run very often, and indicates to me that it's trying to mount root -- very early on in the boot process.  This is why I suspect missing root filesystem drivers (filesystem, disk subsystem, disk driver, PCI, etc., etc.)

----------

## Hu

 *necron wrote:*   

> Edit:  compiling ext2 support did not change anything.  the actual message I am getting is telling me that the error asociated with smp.c has to do with (what I guess is a function) 
> 
> ```
> native_smp_send_reschedule+0x4f/0x60()
> ```
> ...

 Please provide the full error output, without paraphrasing.

----------

## Olis

Hi,

for me the problem is solved. I installed genkernel and used it compile kernel 3.6.11. I had to make some changes to the generated .config (eg. enable RTC support) and I now have to compile a lot more modules than before, but at least the kernel now boots without problems.

Because of the many differences between my "hand-made" .config and the one generated by genkernel I can't tell what the reason for my kernel panic with kernel 3.6.11 was. But for the moment I can live with genkernel.

Oliver

----------

