# Letting udev process events

## dlublink

Hello,

After reading about 10 posts about people who had issues with their network cards with the latest udev update, I decided to start a new thread.

I updated my gentoo this morning. I ran emerge --unmerge coldplug, etc-update remove coldplug, rm /etc/init.d/coldplug,  and emerge -vuDN world.

When I rebooted, linux hangs on 'Letting udev process events...'. CTRL+ALT+DEL comes on and I can't do anything else.

I booted using a gentoo minimal installer CD. I tried to remove and add it again (emerge --unmerge udev (answered yes to the 'this might break your system' message), rm -rf /etc/udev, emerge -uvDN (which reinstalled udev)).

Even after I did this, it still hangs on the same spot.

The messages are as such:

Press I to enter interactive boot mode

* Mounting proc at /proc ... [ ok]

* Mounting sysfs at /sys ... [ ok ]

* Mounting /dev for udev ... [ ok ]

* Seeding /dev with needed nodes ... [ok ]

* Setting up proper hotplug agent ... [ ok ]

*   Using netlink for hotplug events...

* Starting udevd ...

* Populating /dev with existing devices through events ... [ ok ]

* Letting udev process events ...      (at this point it hangs....)

Any ideas? I am recompiling the kernel right now and will post any new information shortly.

David

----------

## jeanfrancis

Are you sure it hangs?

If you wait a bit, does it continue or put any error message?

You may switch to the F12 console to view if you can see any error message.

----------

## dlublink

 *jeanfrancis wrote:*   

> Are you sure it hangs?
> 
> If you wait a bit, does it continue or put any error message?
> 
> You may switch to the F12 console to view if you can see any error message.

 

I let it run for about 90 seconds before I gave up. I will check the console and post anything I find there.

David

----------

## dlublink

 *dlublink wrote:*   

>  *jeanfrancis wrote:*   Are you sure it hangs?
> 
> If you wait a bit, does it continue or put any error message?
> 
> You may switch to the F12 console to view if you can see any error message. 
> ...

 

It turns out that my kernel was not recompiled automatically when I ran my emerge -uvDN world. I updated my kernel to 2.6.18-r3 and it boots. The 'Letting udev' page is slow. But coldplug always did the same thing.

David

----------

## Headrush

 *dlublink wrote:*   

> It turns out that my kernel was not recompiled automatically when I ran my emerge -uvDN world. I updated my kernel to 2.6.18-r3 and it boots.

 

As you've found out, emerging any kernel doesn't compile the kernel, it just installs the sources.

You still have to compile it yourself whether manually or with genkernel.

----------

## max2k5

I am exactly having the same problem here, after having installed a new 2.6.18 kernel (emerged, compiled and copied...  :Wink:  ), my new kernel just comes until 

 *Quote:*   

> 
> 
> * Letting udev process events ...
> 
> 

 

and then blanks the screen, no further hard drive activity, no soft reboot or screen switch possible.

Unfortunately, at this point no log is written to the ahrd drive, and you cannot see anything. I just got to this little information by filming the boot log  :Wink: 

so is there any debug switch, where i can see which module is loaded or what is done by udev exactly?

if I delete the modules directory, the boot process finishes, but of course complains about missing modules.

Thanks in advance,

max

----------

## hpeters

I have this same problem and others.

On my first computer it will boot on about every sixth attempt.

On the second computer it boots every time but on both it and the first computer i end up with the folder /dev/.udev/failed with these devices in it.

-rw-r--r-- 1 root root   0 Nov 27 13:49 block@hda

-rw-r--r-- 1 root root   0 Nov 27 13:49 block@hda@hda1

-rw-r--r-- 1 root root   0 Nov 27 13:49 block@hda@hda2

-rw-r--r-- 1 root root   0 Nov 27 13:49 block@hda@hda3

-rw-r--r-- 1 root root   0 Nov 27 13:49 block@hda@hda5

-rw-r--r-- 1 root root   0 Nov 27 13:49 block@hdc

-rw-r--r-- 1 root root   0 Nov 27 13:49 block@hdd

-rw-r--r-- 1 root root   0 Nov 27 13:49 class@graphics@fb0

-rw-r--r-- 1 root root   0 Nov 27 13:49 class@i2c-dev@i2c-0

-rw-r--r-- 1 root root   0 Nov 27 13:49 class@input@input1@mouse0

-rw-r--r-- 1 root root   0 Nov 27 13:49 class@net@eth0

-rw-r--r-- 1 root root   0 Nov 27 13:49 class@tty@ttyS1

-rw-r--r-- 1 root root   0 Nov 27 13:49 class@usb_device@usbdev1.1

-rw-r--r-- 1 root root   0 Nov 27 13:49 class@usb_device@usbdev2.1

-rw-r--r-- 1 root root   0 Nov 27 13:49 class@usb_device@usbdev2.2

-rw-r--r-- 1 root root   0 Nov 27 13:49 class@usb_device@usbdev3.1

-rw-r--r-- 1 root root   0 Nov 27 13:49 class@usb_device@usbdev4.1

-rw-r--r-- 1 root root   0 Nov 27 13:49 devices@pci0000:00@0000:00:1d.0@usb2@2-0:1.0@usbdev2.1_ep81

-rw-r--r-- 1 root root   0 Nov 27 13:49 devices@pci0000:00@0000:00:1d.0@usb2@2-1@2-1:1.0@usbdev2.2_ep81

-rw-r--r-- 1 root root   0 Nov 27 13:49 devices@pci0000:00@0000:00:1d.0@usb2@2-1@usbdev2.2_ep00

-rw-r--r-- 1 root root   0 Nov 27 13:49 devices@pci0000:00@0000:00:1d.0@usb2@usbdev2.1_ep00

-rw-r--r-- 1 root root   0 Nov 27 13:49 devices@pci0000:00@0000:00:1d.1@usb3@3-0:1.0@usbdev3.1_ep81

-rw-r--r-- 1 root root   0 Nov 27 13:49 devices@pci0000:00@0000:00:1d.1@usb3@usbdev3.1_ep00

-rw-r--r-- 1 root root   0 Nov 27 13:49 devices@pci0000:00@0000:00:1d.2@usb4@4-0:1.0@usbdev4.1_ep81

-rw-r--r-- 1 root root   0 Nov 27 13:49 devices@pci0000:00@0000:00:1d.2@usb4@usbdev4.1_ep00

-rw-r--r-- 1 root root   0 Nov 27 13:49 devices@pci0000:00@0000:00:1d.7@usb1@1-0:1.0@usbdev1.1_ep81

-rw-r--r-- 1 root root   0 Nov 27 13:49 devices@pci0000:00@0000:00:1d.7@usb1@usbdev1.1_ep00

What are these all about ? They were never there before.

Both computers seem to work fine otherwise except the computer that has problems booting runs Mythtv on it and when mythbackend starts it complains about not being able to change the permissions of the failed devices.

Harley

----------

## Lloeki

udev now implements coldplug functionality.

previously 'letting udev process events' was taking nil time, but now you should add the whole coldplug time to that.

a side effect is that udev seems to work more efficiently than coldplug, and for me, more modules were getting loaded, esp the ethernet over firewire (ieee1394) module. thus, it was creating an eth0 for that, displacing all later created ethernet devices. removed eth/ieee1394 support from kernel, and voila, all fixed.

the reason of the hang may be that a wrong module is loaded. try to build your kernel with less modules.

----------

## hpeters

I never used coldplug before and have disabled it now as well. Doesn't help with not booting enabled or not.

With coldplug enabled there are even more "failed" devices.

Harley

----------

## Lloeki

well, I have 112 devices in /dev/.udev/failed/, but everything is fine. so much fine I didn't notice they were there. I don't thing their presence is indicative of anything going wrong.

----------

## hpeters

It doesn't seem to be causing any problems other than for mythbackend complaining about them.

Just wondering what there all about.

Harley

----------

## dlublink

After about the third boot it started freezing again. I had the latest kernel and latest stable portage (without ~x86) and it still didn't work. Since I use this machine for work, I masked the latest udev package and went back to the old system. Maybe over the weekend I will experiment a bit more.

Thanks for your guys' help.

David

----------

## msalerno

Anyone have an update on this?  I am having the same problem.

FYI - https://bugs.gentoo.org/show_bug.cgi?id=150591

----------

## hpeters

Well are systems do have one thing in common mine also has two ivtv based cards in it.

I haven't had any time to mess with it right now and have just hard masked udev-103 for now but that might be a starting point.

The only other thing are systems have in common is the kernel version 2.6.18

Harley

----------

## msalerno

I am also getting some alsa errors.  When I get home, I will remove the ivtv and alsa-dirver packages and modules and try again.  Right now I have RC_COLDPLUG="no" in my /etc/conf.d/rc as a temporary fix.

----------

## guyr

Ugh, I just hit this same problem.  I do not have coldplug installed.  I have hotplug installed but I'm not loading it.  I was running kernel 2.6.18-r2 without issue before this upgrade.  I have the ivtv module loaded for the Hauppage PVR-350.  I am able to successfully boot into kernel 2.6.17-r7, and it loads ivtv okay and uses the new udev.  So I don't understand why the same setup won't work under 2.6.18-r2.   Thank goodness I always keep at least one previous working kernel installed.

I'm using the ATI proprietary driver, but I don't think that has anything to do with it, since the lockup occurs long before the graphics driver gets loaded.

----------

## msalerno

That's 3 of us with the same problem and with the IVTV module...  I'll try tomorrow to see if removing ivtv will resolve the boot issue.

----------

## damato

Make 4 out of it.  :Smile: 

I am having the very same issue since about 1 or 2 kernel versions with gentoo. I have a PVR350 in my Gentoo machine and as soon as I try to load the ivtv module manually via "modprobe ivtv" or if udev is doing the job for me like it pointed out in here, the system simply freezes without ANY kernel panic or "Oops".

The strange thing is, that according to a german forum entry (http://www.vdr-portal.de/board/thread.php?threadid=56364&threadview=0&hilight=&hilightuser=0&page=1) other users have the very same problem. The interesting thing is that they are also just having it with gentoo. People tried to install e.g. Kubuntu and reported that it works. They even tried a vanilla kernel in gentoo but still the system hangs as soon as the ivtv module is loaded.

However, one guy did a deeper debugging by setting up a serial terminal to his linux box and increased the debugging level of the kernel. And he was able to catch this debug output:

 *Quote:*   

> 
> 
> * Letting udev process events ...[42949404.880000] Oops: 0000 [#1]
> 
> [42949404.880000] Modules linked in: i2c_viapro ivtv v4l1_compat tveeprom videodev 8139too
> ...

 

So might this help in any way? Do we have some problems with our module tools? modprobe issue?

I will also do some deeper tests on it, but also hope that the ivtv maintainer here might show up and give us a hint or idea on where to search. I really want to get my barebone running again with the PVR350 under some of the newer kernels (>= 2.6.1 :Cool: 

Another point is also that it seems to be a ivtv<>PVR350 issue only as I haven"t seen a single report on people using e.g. PVR250 that they are having issues with recent kernel versions and recent ivtv versions.

BTW: for those only interested in getting the bootup problems fixed and get the system booted again, it simply helps to disable the coldplugging in /etc/conf.d/rc by setting

```

RC_COLDPLUG="no"

```

Then the system should boot again, however if the ivtv module is then loaded manually via modprobe the system will still hang.

cheers,

jens

----------

## hpeters

After some debugging i have found that for me anyway the problem is with hotplug trying to load the firmware for ivtv failing and hanging.

Deleting /etc/hotplug/firmware.agent allows the computer to boot everytime.

I don't know of anyway to tell hotplug not to load the firmware for a specific device. Deleting this file will prevent it from loading firmware for any device.

For me that is ok since ivtv loads in the firmware itself and i don't have anything else that needs firmware loaded.

I also haven't tried any newer versions of hotplug to see if they work.

Harley

----------

## guyr

Here's another odd datapoint.  I reported above that 2.6.17-r7 boots with the new udev, but 2.6.18-r2 hangs.  Well, upon trying again today, I learned that the former is not consistently true.  I only succeeded in booting 2.6.17-r7 successfully 2 times out of about 8; the other times it also hung on "Letting udev process events".  This seems even more bizarre.  Why would this not be deterministic?  I don't know if this is relevant or not, but I remember when I first built 2.6.18-r2, I was forced to turn on the option CONFIG_VIDEO_CX2341X=m.  I can't remember if the kernel itself failed to build without that option, or if subsequently trying to load ivtv complained that the option was required.   At any rate, that option is not present in 2.6.17-r7.

----------

## hpeters

 *guyr wrote:*   

> I don't know if this is relevant or not, but I remember when I first built 2.6.18-r2, I was forced to turn on the option CONFIG_VIDEO_CX2341X=m.  I can't remember if the kernel itself failed to build without that option, or if subsequently trying to load ivtv complained that the option was required.   At any rate, that option is not present in 2.6.17-r7.

 

Ivtv is in the process of being merged into the kernel. And the above module was merged in to the 2.6.18 kernel and is necessary for ivtv to work.

I was running the 2.6.18 kernel with no problems before i emerged udev-103.

Harley

----------

## guyr

I just installed 2.6.18-r3 to see if building a kernel after installing latest udev would help.   It booted fine.  I then emerged ivtv, and upon rebooting it hung again.  Using Live-CD, I renamed /lib/modules/2.6.18-gentoo-r3/extra/ivtv.ko to ivtv.bak.  Now Gentoo will boot again.  Of course, I've lost use of the PVR-350, but at least I have use of the computer.

What's odd is that on one of the boots of 2.6.17-r7 that worked, I watched the messages going by.  Loading of the ivtv module occurs *after* the "Letting udev process events" message.  I suppose some sort of background process is already doing something with the ivtv module before it's load message appears.  Commenting out ivtv from modules.autoload/kernel-2.6 did not prevent this problem.

----------

## Lloeki

 *Quote:*   

> I suppose some sort of background process is already doing something with the ivtv module before it's load message appears

 

FWIW, udev replaces coldplug, and (on my computers at least) the time coldplug (coldplugging pci/usb/pnp devices) was taking at boot before is exactly the same that now takes place right after 'letting udev process events'.

since module.autoload.d is to load modules regardless of hotplug/coldplug, changing things there won't change coldplug (new or old) behaviour.

----------

## guyr

 *Lloeki wrote:*   

> changing things there won't change coldplug (new or old) behaviour.

 

I'm not understanding where coldplug is coming into play.  In order for ivtv.ko to get loaded, I *thought* it had to be in autoload.  To be honest, it's been awhile since I set that up and I don't remember if I just followed instructions or if I had to put it in there in order for it to work.  The fact that renaming the module allows the OS to boot up would indicate that something related to that module is to blame.  The fact that simply removing that module from autoload does *not* fix the OS hang during boot would indicate that something other than loading the module is causing it to produce the hang.

----------

## Lloeki

 *Quote:*   

> I'm not understanding where coldplug is coming into play. In order for ivtv.ko to get loaded, I *thought* it had to be in autoload. 

 

udev's coldplug seems to load a lot more modules the standalone coldplug was. it is at least the case on both of my computers. maybe it now has the funky habit of loading the ivtv module.

 *Quote:*   

> The fact that renaming the module allows the OS to boot up would indicate that something related to that module is to blame. 

 

sure. that is indeed the key point.

 *Quote:*   

> 
> 
> The fact that simply removing that module from autoload does *not* fix the OS hang during boot would indicate that something other than loading the module is causing it to produce the hang.

 

yes, it is not only loaded by autoload (which roughly just does 'for i in `cat kernel2.6`; do modprobe $i; done'. no magic here), but also by the new udev's coldplug. what's more, it may trigger the starting of init scripts (this is known as rc_hotplug) which may or may not be delayed to a later time (this is rc_coldplug).

therefore, there may be two 'culprits' here: the ivtv module, or the script that is triggered by its insertion. you could discard the init script by setting 'rc_hotplug' to 'no', or deal with 'rc_plug_services' in /etc/conf.d/rc (you would put '!ivtv' if the script name was /etc/init.d/ivtv).

----------

## mociofiletto

 *dlublink wrote:*   

> Hello,
> 
> After reading about 10 posts about people who had issues with their network cards with the latest udev update, I decided to start a new thread.
> 
> I updated my gentoo this morning. I ran emerge --unmerge coldplug, etc-update remove coldplug, rm /etc/init.d/coldplug,  and emerge -vuDN world.
> ...

 

Well I've got this problem:

emerged the new udev-103 and then I rebooted.

The system went on but the network didn't work in any way.

After a couple of attempts I went back to udev-0.87-r1 and coldpulg.

Now the network works good, but I still have in /dev/.udev/failed the following links:

root@peppebook failed # ls -la

total 0

drwxr-xr-x 2 root root 160 Dec  4 15:23 .

drwxr-xr-x 4 root root  80 Dec  4 15:23 ..

lrwxrwxrwx 1 root root  19 Dec  4 15:22 class@net@eth0 -> /sys/class/net/eth0

lrwxrwxrwx 1 root root  17 Dec  4 15:22 class@net@lo -> /sys/class/net/lo

lrwxrwxrwx 1 root root  20 Dec  4 15:22 class@net@tunl0 -> /sys/class/net/tunl0

lrwxrwxrwx 1 root root  21 Dec  4 15:23 class@net@vmnet1 -> /sys/class/net/vmnet1

lrwxrwxrwx 1 root root  21 Dec  4 15:23 class@net@vmnet8 -> /sys/class/net/vmnet8

lrwxrwxrwx 1 root root  20 Dec  4 15:22 class@net@wlan0 -> /sys/class/net/wlan0

Now the only problem I have is that when I boot, I some messages on the console; looking in my /var/log/syslog I've found this entries:

Dec  4 15:23:01 peppebook kernel: eth0: link up, 100Mbps, full-duplex, lpa 0x45E1

Dec  4 15:23:01 peppebook kernel: wlan0 (WE) : Driver using old /proc/net/wireless support, please fix driver !

Dec  4 15:23:10 peppebook kernel: vmmon: no version magic, tainting kernel.

Dec  4 15:23:10 peppebook kernel: /dev/vmmon[11167]: Module vmmon: registered with major=10 minor=165

Dec  4 15:23:10 peppebook kernel: /dev/vmmon[11167]: Module vmmon: initialized

Dec  4 15:23:10 peppebook kernel: vmnet: no version magic, tainting kernel.

Dec  4 15:23:10 peppebook kernel: /dev/vmnet: open called by PID 11205 (vmnet-bridge)

Dec  4 15:23:10 peppebook kernel: /dev/vmnet: hub 0 does not exist, allocating memory.

Dec  4 15:23:10 peppebook kernel: /dev/vmnet: port on hub 0 successfully opened

Dec  4 15:23:10 peppebook kernel: bridge-eth0: enabling the bridge

Dec  4 15:23:10 peppebook kernel: bridge-eth0: up

Dec  4 15:23:10 peppebook kernel: bridge-eth0: already up

Dec  4 15:23:10 peppebook kernel: bridge-eth0: attached

Dec  4 15:23:10 peppebook kernel: /dev/vmnet: open called by PID 11215 (vmnet-bridge)

Dec  4 15:23:10 peppebook kernel: /dev/vmnet: hub 2 does not exist, allocating memory.

Dec  4 15:23:10 peppebook kernel: /dev/vmnet: port on hub 2 successfully opened

Dec  4 15:23:10 peppebook kernel: bridge-eth1: peer interface eth1 not found, will wait for it to come up

Dec  4 15:23:10 peppebook kernel: bridge-eth1: attached

Dec  4 15:23:10 peppebook kernel: /dev/vmnet: open called by PID 11239 (vmnet-natd)

Dec  4 15:23:10 peppebook kernel: /dev/vmnet: hub 8 does not exist, allocating memory.

Dec  4 15:23:10 peppebook kernel: /dev/vmnet: port on hub 8 successfully opened

Dec  4 15:23:20 peppebook kernel: /dev/vmnet: open called by PID 11849 (vmnet-netifup)

Dec  4 15:23:20 peppebook kernel: /dev/vmnet: hub 1 does not exist, allocating memory.

Dec  4 15:23:20 peppebook kernel: /dev/vmnet: port on hub 1 successfully opened

Dec  4 15:23:20 peppebook kernel: /dev/vmnet: open called by PID 11850 (vmnet-netifup)

Dec  4 15:23:20 peppebook kernel: /dev/vmnet: port on hub 8 successfully opened

Dec  4 15:23:20 peppebook udevd-event[11851]: rename_net_if: error changing net interface name: Device or resource busy

Dec  4 15:23:20 peppebook udevd-event[11861]: rename_net_if: error changing net interface name: Device or resource busy

other messages I visualize during boot are related to USB, but I think it would have been better to rm /dev/udev/rules.d/* before to reemerge old udev

My kernel is 2.6.16-suspend2-r8

----------

## robin_elvin

Well I've done an emerge -uD world and my system is fairly broken  :Sad: 

Firstly, udev-103 causes a Kernel oops on boot when it loads. If I downgrade to udev-087-r1 then I can boot again but I get lots of messages about missing nodes and then when Automounter starts I also get lots of Dbus errors.

Finally, eth0 has gone so I can't even get packages without manually copying them to distfiles  :Sad: 

Any ideas?

** UPDATE **

If I set

```
RC_COLDPLUG="no"
```

in /etc/conf.d/rc then it cures most of it apart from the Dbus errors.

----------

## msalerno

Well, I think you are the first one here to get a kernel oops, but my guess is that it is because of a module that udev is loading.  You could setup udev debugging and see what is the last module loaded.

----------

## mociofiletto

 *robin_elvin wrote:*   

> Well I've done an emerge -uD world and my system is fairly broken 
> 
> Firstly, udev-103 causes a Kernel oops on boot when it loads. If I downgrade to udev-087-r1 then I can boot again but I get lots of messages about missing nodes and then when Automounter starts I also get lots of Dbus errors.
> 
> Finally, eth0 has gone so I can't even get packages without manually copying them to distfiles 
> ...

 

I think you can solve your problem doing this:

1) avoid udev-103 to be installed in your system:

```
~# echo "=sys-fs/udev-103" >> /etc/portage/package.mask
```

2) unmerge udev

```
~# emerge -C sys-fs/udev
```

3) delete the udev conf directory or move it to another location

```
~# mv /etc/udev /etc/udev_103
```

4) emerge the "old" udev

```
~# emerge -v sys-fs/udev
```

5) emerge coldplug

```
~# emerge -v sys-apps/coldplug
```

6) add coldplug to boot runlevel

```
~# rc-update add coldplug boot
```

7) reboot

I think in this way you should go back to the old sys-fs/udev-087-r1. The packages you need in distfiles are udev-087.tar.bz2 and hotplug-2004_09_20.tar.gz.

You shold download them from another machine and find a way (floppy cdrom usb-pendrive) to put them in /usr/portage/distfiles

The packages can be found here:

http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev-087.tar.bz2

http://www.kernel.org/pub/linux/utils/kernel/hotplug/hotplug-2004_09_20.tar.gz

----------

## dmpogo

 *mociofiletto wrote:*   

>  *robin_elvin wrote:*   Well I've done an emerge -uD world and my system is fairly broken 
> 
> Firstly, udev-103 causes a Kernel oops on boot when it loads. If I downgrade to udev-087-r1 then I can boot again but I get lots of messages about missing nodes and then when Automounter starts I also get lots of Dbus errors.
> 
> Finally, eth0 has gone so I can't even get packages without manually copying them to distfiles 
> ...

 

I went even further and got rid of udev altogether  :Smile: 

----------

## damato

Let me state that I finally solved my "Letting udev process events..." lockup problems with my Hauppauge PVR-350 card and recent >2.6.17 by having found an interesting FAQ entry of the ivtv driver here:

http://ivtvdriver.org/index.php/Troubleshooting#Lock_up_when_loading_ivtv_module_.28udev_related.29

According to this FAQ entry the udev related lockups during boot or by manually modprobing "ivtv" are due to a SUBSYSTEM=="firmware" entry in the udev rules. After having searched in the udev rules configuration of gentoo I found the following line in the /etc/udev/rules/50-udev.rules file:

```

# Load firmware

SUBSYSTEM=="firmware", ACTION=="add", RUN+="firmware.sh"

```

After having disabling that line (commented out) the ivtv driver properly loads up during bootup or by manually loading the ivtv driver via "modprobe".

So I can only suggest to everyone having similar "Letting udev process events." lockup problems during bootup or if you also have issues loading the ivtv driver via modprobe, to also try to disable the above 'SUBSYSTEM=="firmware"' by commenting it out. At least here it worked and I am now able to use my PVR-350 hauppauge card again with recent kernels.

cheers,

jens

----------

## sr66

That was perfect!! Commenting that line out worked for me.

Thank you.

----------

