# New kernel hangs after "INIT: version 2.88 booting" (AMD64)

## maguire

I've been running gentoo-sources-3.16.5 since Oct. 31st, 2014.  I tried upgrading to a version 4 kernel, but it would hang.  I decided to try to gentoo-sources-3.18.25, and I used "make oldconfig" using the 3.16.5 config, and I didn't see anything concerning, so I just accepted all the default Y/N responses.  I compiled the kernel and copied it to /boot.  However, booting that kernel also results in a hang immediately after "INIT: version 2.88 booting", and there is no response to any keyboard input; I have to hard power-cycle the box.  I've been looking through the forum and web-searching for days, but nothing seems applicable to my case.

I've looked back through my emerge logs since Oct. 31st, and the following one concerns me, because I don't understand it, and it may be the culprit:

```

 * Package:    sys-boot/grub-0.97-r16

...

 * *** IMPORTANT NOTE: you must run grub and install

 * the new version's stage1 to your MBR.  Until you do,

 * stage1 and stage2 will still be the old version, but

 * later stages will be the new version, which could

 * cause problems such as an unbootable system.

 * 

 * This means you must use either grub-install or perform

 * root/setup manually.

 * 

 * For more help, see the handbook:

 * https://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?part=1&chap=10#grub-install-auto

```

The link does not work (it just goes to the AMD64 handbook table-of-contents).  Immediately after the above warning message, the log shows:

```

 * Copying files from /lib/grub and /usr/share/grub to //boot/grub

 * Grub has been installed to //boot successfully.

```

Can someone help me with the meaning of the GRUB warning-message?

Would this be consistent with the location of my boot hang?

Thanks!

Bruce.

----------

## eccerr0r

Well definitely you should update your grub image to make sure it's consistent and using your new version, but since you see INIT: version 2.88 booting it means that the kernel got through all of its initialization and actually started /sbin/init - so this means initial boot/grub worked just fine.

Most likely you have some option in your kernel set wrong, when using gentoo-sources you should make sure the options for Gentoo Linux are set to 'Y' (COFNIG_GENTOO_* except possibly INIT_SYSTEMD if you're not using systemd).  Also make sure you have console enabled which is another option in your config.

----------

## maguire

Thanks for the reply, "eccerr0r"!

What you said about the grub update makes sense.

Regarding the kernel config causing the problem, I'm at a complete loss, because I don't see any problem settings!

Here are the CONFIG_GENTOO settings for the hanging 3.18.25:

```

$ egrep CONFIG_GENTOO .config

CONFIG_GENTOO_LINUX=y

CONFIG_GENTOO_LINUX_UDEV=y

CONFIG_GENTOO_LINUX_INIT_SCRIPT=y

# CONFIG_GENTOO_LINUX_INIT_SYSTEMD is not set

```

Here are my CONSOLE settings:

```

$ egrep CONSOLE .config

CONFIG_CONSOLE_TRANSLATIONS=y

CONFIG_VT_CONSOLE=y

CONFIG_HW_CONSOLE=y

CONFIG_VT_HW_CONSOLE_BINDING=y

CONFIG_SERIAL_8250_CONSOLE=y

CONFIG_SERIAL_CORE_CONSOLE=y

CONFIG_VGA_CONSOLE=y

CONFIG_DUMMY_CONSOLE=y

CONFIG_FRAMEBUFFER_CONSOLE=y

CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y

# CONFIG_FRAMEBUFFER_CONSOLE_ROTATION is not set

```

Here is my WORKING 3.16.5 config:  https://bpaste.net/show/c910e92ecefb

Here is my HANGING 3.18.25 config:  https://bpaste.net/show/73814360e2f6

Here are the DIFFERENCES between the two:  https://bpaste.net/show/d3edf906b990

In case it helps, here is my 'emerge --info':  https://bpaste.net/show/765490ccb8f8

Any other suggestions?  I appreciate the help!

Thanks,

Bruce.

----------

## eccerr0r

Do you have more than one root partition on your system and somehow device enumeration chooses another disk?  Using initramfs anywhere?

----------

## maguire

eccerr0r:

No, no other root partition, and no initramfs.  The setup is really basic, including a monolithic kernel.

This is my work computer, so I need to make sure I don't lose a workable setup, but I did have an idea that I have not yet tried.  I'm suspicious that the difference is not the kernel or kernel-options, but possibly the tool-chain that compiled the kernel.  I think that the way to test this would be to make sure that the working 3.16.5 kernel is not deleted (saved/renamed), and then rebuild the 3.16.5 kernel--with the same old config--using my current tool-chain, and see if that kernel still boots fine, or if it hangs.  Do you think that this is a reasonable exercise?

Thanks,

Bruce.

----------

## eccerr0r

Well, once again, the kernel got to init, so the kernel is mostly working.  You can try it if you want though, no idea...

You could try init=/bin/bash on the command line (what do you have on you command line as it is?) and see if you can at least get a command line.  Then you could exec /sbin/init and see the hang...?

----------

## maguire

eccerr0r:

 *Quote:*   

> 
> 
> Well, once again, the kernel got to init, so the kernel is mostly working.
> 
> 

 

Right...  I seem to have a hard time getting that through my head!

My boot command-line is simply:

```

root (hd0,0)

kernel /boot/kernel-3.18.25-gentoo root=/dev/sda3 net.ifnames=0

```

Thanks for the suggestion about booting to Bash and manually running /sbin/init.  I will try that and see what I can see.

Thanks again!

Bruce.

----------

## maguire

Unfortunately, using /bin/bash as init did not work.  I append "init=/bin/bash" to the kernel command-line, and this is what I see on the console (which is then hung):

```

...

[    6.XXXXXX] REISERFS (device sda3): Using r5 hash to sort names

[    6.XXXXXX] VFS: mounted root (reiserfs filesystem) readonly on device 8:3.

[    6.XXXXXX] Freeing unused kernel memory: 876k (ffff... - ffff...)

bash: cannot set terminal process group (-1): Inappropriate ioctl for device

bash: no job control in this shell

```

And again, at this point, the system is hung so that no keyboard input is recognized.

When I look at the 'dmesg' output on my older, working kernel, it appears that the new kernel completes just before the point where the USB devices are "found" for the working kernel:

```

...

[    3.385264] REISERFS (device sda3): Using r5 hash to sort names

[    3.387156] VFS: Mounted root (reiserfs filesystem) readonly on device 8:3.

[    3.389516] Freeing unused kernel memory: 856K (ffffffff818c7000 - ffffffff8199d000)

[    3.428308] usb 6-1: new low-speed USB device number 2 using uhci_hcd

[    3.538391] Switched to clocksource tsc

[    3.585934] usb 6-1: New USB device found, idVendor=0461, idProduct=4d15

[    3.587839] usb 6-1: New USB device strings: Mfr=0, Product=2, SerialNumber=0

[    3.589691] usb 6-1: Product: USB Optical Mouse

[    3.607148] input: USB Optical Mouse as /devices/pci0000:00/0000:00:1d.0/usb6/6-1/6-1:1.0/0003:0461:4D15.0001/input/input3

[    3.609305] hid-generic 0003:0461:4D15.0001: input,hidraw0: USB HID v1.11 Mouse [USB Optical Mouse] on usb-0000:00:1d.0-1/input0

[    3.633518] random: nonblocking pool is initialized

[    3.684229] usb 2-2.1: new full-speed USB device number 5 using ehci-pci

[    3.778711] usb 2-2.1: New USB device found, idVendor=0853, idProduct=0100

[    3.780647] usb 2-2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0

[    3.782621] usb 2-2.1: Product: HHKB Professional

[    3.784504] usb 2-2.1: Manufacturer: Topre Corporation

[    3.790290] input: Topre Corporation HHKB Professional as /devices/pci0000:00/0000:00:1d.7/usb2/2-2/2-2.1/2-2.1:1.0/0003:0853:0100.0002/input/input4

[    3.792358] hid-generic 0003:0853:0100.0002: input,hidraw1: USB HID v1.11 Keyboard [Topre Corporation HHKB Professional] on usb-0000:00:1d.7-2.1/input0

[    9.025268]  md0: unknown partition table

[    9.785329] reiserfs: enabling write barrier flush mode

[   10.102258] Adding 24575996k swap on /dev/sda2.  Priority:0 extents:1 across:24575996k 

[   10.104348] Adding 24442892k swap on /dev/sdb2.  Priority:0 extents:1 across:24442892k 

[   10.106397] Adding 24442892k swap on /dev/sdc2.  Priority:0 extents:1 across:24442892k 

[   10.108345] Adding 24442892k swap on /dev/sdd2.  Priority:0 extents:1 across:24442892k 

[   10.130349] REISERFS (device md0): found reiserfs format "3.6" with standard journal

[   10.130355] REISERFS (device md0): using ordered data mode

[   10.130355] reiserfs: using flush barriers

[   10.130586] REISERFS (device md0): journal params: device md0, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30

[   10.130940] REISERFS (device md0): checking transaction log (md0)

[   10.246810] REISERFS (device md0): Using r5 hash to sort names

[   13.521176] r8169 0000:09:00.0 eth0: link down

[   13.575070] r8169 0000:09:00.0 eth0: link down

[   13.676111] r8169 0000:08:00.0 eth1: link down

[   13.676122] r8169 0000:08:00.0 eth1: link down

[   16.212863] r8169 0000:08:00.0 eth1: link up

```

The only difference between the two kernel configs w.r.t. USB (see config-difference: https://bpaste.net/show/d3edf906b990) that I see is the addition of the following 4 new config values:

```

+USBIP_CORE n

+USB_LINK_LAYER_TEST n

+USB_NET_DRIVERS y

+USB_OTG_WHITELIST n

```

These settings seem fine to me, so what would cause the kernel difference?

Thanks,

Bruce.

----------

## Aquous

I once had the problem you describe and it was due to not having CONFIG_DEVTMPFS{,_MOUNT}=y. Maybe check that?

Also, are you using an initrd?

----------

## eccerr0r

I'd say start over with a fresh .config and rebuild your kernel.  Or maybe a depclean before building.  Something wrong with your kernel if it's configured for USB but no USB devices are detected?

Supposedly setting CONFIG_GENTOO_LINUX on a gentoo-sources kernel will automatically set CONFIG_DEVTMPFS but it's worth doublechecking.

----------

## maguire

eccerr0r and Aquous:

Sorry, the USB thing was a red herring!  The USB devices were detected earlier in the kernel messages.  (Because the messages only appear on the console and the system hangs, I can't save them, and I didn't notice that the order changed.)

Even though CONFIG_DEVTMPFS_MOUNT is disabled on my working 3.16.5 kernel, I enabled it on the 3.18.25 kernel, and recompiled.  I also added every command-line debug flag in the book to try to make sure I don't miss any messages; I added the following to the kernel command line:

```

... debug  ignore_loglevel  log_buf_len=16M  print_fatal_signals=1  loglevel=8  earlyprintk=vga,keep  initcall_debug  udev.log_priority=8  bootmem_debug

```

The boot gets a bit further, but still hangs.  I now see the following:

```

...

[    6.XXXXXX] REISERFS (device sda3): Using r5 hash to sort names

[    6.XXXXXX] VFS: mounted root (reiserfs filesystem) readonly on device 8:3.

[    6.XXXXXX] devtmpfs: mounted

[    6.XXXXXX] Freeing unused kernel memory: 876k (ffff... - ffff...)

INIT: version 2.88 booting

OpenRC 0.18.4 is starting up gentoo linux (x86_64)

* mounting /proc...

* mounting /run...

* /run/openrc: creating directory

* /run/lock: creating directory

* /run/lock: correcting owner

```

At which point it hangs.

When I look at the messages from my working 3.16.5 kernel boot, the next thing that appears is:

```

* mounting devtmpfs on /dev

```

but I don't really know if this is where it is hanging for the 3.18.25 kernel.

I tried the same 3.18.25 boot (with all the debug flags), and appended "init=/bin/bash", but I get the same error messages as before:

```

...

[    6.XXXXXX] Freeing unused kernel memory: 876k (ffff... - ffff...)

bash: cannot set terminal process group (-1): Inappropriate ioctl for device

bash: no job control in this shell

```

Any suggestions?

Thanks!

Bruce.

----------

## eccerr0r

I still don't understand why you can't use the keyboard after init=/bin/bash.  You should at least get the bash prompt.  The warning about job control is normal... though I don't recall the process group issue as normal, hmm...

----------

