# udev unable to receive kernel netlink message

## humbletech99

I have the following error spewing all over my logs and causing big problems from the sheer number of logs (dozens if not hundreds per second!)

```
udevd[1019]: get_netlink_msg: unable to receive kernel netlink message: Socket operation on non-socket
```

I have tried recompiling my kernel with these compiled direcly into the kernel

```

General Settings --> 

  Export task/process statistics through netlink (EXPERIMENTAL)
```

and 

```
Networking --> 

  Network Options --> 

    Network packet filtering (replaces ipchains) --> 

      Core Netfilter Configuration --> 

        Netfilter netlink interface

          Netfilter NFQUEUE over NFNETLINK interface

          Netfilter LOG over NFNETLINK interface
```

in order to try to make it stop, rebooted into new kernel but it still hasn't helped. I have only found one reference to this on the web which is where I got the idea to recompile the kernel with these settings (http://www.mail-archive.com/lfs-dev@linuxfromscratch.org/msg08651.html). I'm using udev-103.

This page http://www.ussg.iu.edu/hypermail/linux/kernel/0511.3/0304.html says that I should have it enable with config_net by default but then why the heck is it complaining about this a million times a minute?

Any ideas on how to fix this?

----------

## Dan

I assume you already have 

udev_log="debug"  in /etc/udev/udev.conf. to try to get more info ?

----------

## humbletech99

The only extra thing I get when I do that is the following interspersed with the previous line:

```
...

udevd[1019]: get_ctrl_msg: unable to receive user udevd message: Socket operation on non-socket

udevd[1019]: get_netlink_msg: unable to receive kernel netlink message: Socket operation on non-socket

...
```

----------

## Contraband

I am having the exact same issue.  My situation is the result of copying (cp -ax) a working gentoo system to another hard drive, and booting from that.

----------

## humbletech99

well, what a coincidence....

I've done this before successfully though. I also did an emerge update world to recompile everything thinking this would make it work better and make sure all installed files are present. ha. should have left it as it was...

----------

## humbletech99

man this is killing me, surely there must be something I have missed...

I am trying an emerge -eD world to see if I can make it replace any missing socket or file or something.

I find it hard to believe that practically nobody has had this problem?

----------

## sanmartin

If it makes you feel better, I'm having this problem! Did your emerge -e world fix it? This post sounds like the same problem.  I've tried re-emerging udev a couple times, I've checked my kernel config, I don't know what's going wrong.  Anyone else have an idea?

----------

## wynn

 *humbletech99 wrote:*   

> I have the following error spewing all over my logs and causing big problems from the sheer number of logs (dozens if not hundreds per second!)
> 
> ```
> udevd[1019]: get_netlink_msg: unable to receive kernel netlink message: Socket operation on non-socket
> ```
> ...

 The thread you pointed at said (as you found later) that netlink is supported if CONFIG_NET is in the config and it is likely that you already have that.

A consistent voice in that thread (Brian Kadzban), which appears to have been ignored, said *Quote:*   

> This is the errno that's coming back from the recv() syscall.  You need to figure out why the file descriptor that's being passed to recv() is not a socket -- my guess, based on the rest of the udevd.c source code, is that the socket was not created or bound successfully.
> 
> These operations (creating the socket and bind()ing it) will generate their own log messages, but they'll be put into the log when udevd starts up, not later.  So you may have to change the logging level in the config file (...if that's even possible anymore) or change the default logging level in the source code, and recompile.
> 
> Then you should be able to start udevd, wait a couple seconds, and kill it again -- this should log some error from socket() or from bind(). (The prefix will be either "error getting socket: " or "bind failed: ".)

 and again *Quote:*   

> And I have to think that the OP has CONFIG_NET turned on.  So it's still back to needing the debug info from udevd.
> 
> Although actually, the problem messages from both socket() and bind() are logged as errors -- so the important bits may be logged without having to mess with the log level.  We'd just need to see what errno is getting set to (which corresponds to the string that gets put into the log message).

 So you might find the answer, as he says, by looking at /var/log/messages at the point where udevd starts.

You might also like to run "equery check sys-fs/udev" to check the MD5 sums and timestamps on the files to make sure they are OK. An old file may not have been removed by the unmerge (its timestamp didn't match the others) and this may be causing the error. Less likely as there are many people with this problem.

----------

## humbletech99

there isn't anything helpful in the /var/log/messages. It starts with the error I gave above and just repeats that a billion times (26575384118 lines of /var/log/messages, nearly all that same line).

In fact I can't even boot the damn machine to do anything with it. It just spews this to the console before the logger is started and then it doesn't every get past it (I have left it for ages).

AHA: fixed it!

This is more obvious than it first seemed and I'm still kicking myself for not fixing it when I first saw it. 

SOLUTION

The reason this doesn't work is that the /dev tree is not properly populated on the root filesystem. This explains why Contraband also had the same issue. You are copying stuff but not the dev tree so when the system first boots it is missing some bits before udev can populate it. Udev itself seems to rely on it so it just pukes all over the place complaining about it.

To fix this you should do

```
mount --bind / /mnt/temp

mkdir /tmp/stage3

tar jxvf stage3-latestblah.tar.bz2 -C /tmp/stage3

cp -a /tmp/stage3/dev/* /mnt/temp/dev/

rm -fr /tmp/stage3

reboot
```

voila.

----------

## sanmartin

Ok, that makes sense then.  The problem is that there's supposed to be a device node to netlink in the kernel, and udev uses this to receive new hardware events.  My mistake was that I accidentally deleted everything in my /dev/ by binding it to the one from the livecd.  Oops    :Mad:  So netlink couldn't open that file, which was entirely my fault.

I fixed it just by copying the /dev/ contents from the livecd on to my permanent /dev/, but does anyone happen to know exactly which file udev was looking for? In the future it'd be better to fix it by just copying the node you need instead of all of them...

Thanks for figuring that out!   :Laughing: 

----------

## humbletech99

I don't know which node it needs exactly, but I'd guess that there may be other nodes that may possibly cause problems so it's the best idea to just copy them all back. It takes about the same time and they are the devices are standardized so it's ok to do that.

----------

## wynn

 *sanmartin wrote:*   

> Ok, that makes sense then.  The problem is that there's supposed to be a device node to netlink in the kernel, and udev uses this to receive new hardware events.  My mistake was that I accidentally deleted everything in my /dev/ by binding it to the one from the livecd. So netlink couldn't open that file, which was entirely my fault.

 netlink is simply a socket protocol, see netlink(7), so it will have a socket not a device in /dev.

```
netstat -x

Active UNIX domain sockets (w/o servers)

Proto RefCnt Flags       Type       State         I-Node Path

unix  2      [ ]         DGRAM                    1385   @/org/kernel/udev/udevd
```

also

```
# lsof|grep udev

udevd      1107    root  cwd       DIR        8,1     4096          2 /

udevd      1107    root  rtd       DIR        8,1     4096          2 /

udevd      1107    root  txt       REG        8,1    50264    1061398 /sbin/udevd

udevd      1107    root  mem       REG        0,0                   0 [heap] (stat: No such file or directory)

udevd      1107    root  mem       REG        8,1    34804     882336 /lib/libnss_files-2.4.so

udevd      1107    root  mem       REG        8,1    34920     882444 /lib/libnss_nis-2.4.so

udevd      1107    root  mem       REG        8,1    68684     882398 /lib/libnsl-2.4.so

udevd      1107    root  mem       REG        8,1  1167888     882581 /lib/libc-2.4.so

udevd      1107    root  mem       REG        8,1    26596     882445 /lib/libnss_compat-2.4.so

udevd      1107    root  mem       REG        8,1   126094     882303 /lib/ld-2.4.so

udevd      1107    root    0u      CHR        1,3                1363 /dev/null

udevd      1107    root    1u      CHR        1,3                1363 /dev/null

udevd      1107    root    2u      CHR        1,3                1363 /dev/null

udevd      1107    root    3u     unix 0xf797cd00                1385 socket

udevd      1107    root    4u     sock        0,4                1386 can't identify protocol

udevd      1107    root    5r     FIFO        0,5                1400 pipe

udevd      1107    root    6w     FIFO        0,5                1400 pipe

udevd      1107    root    7r      DIR        0,8        0        261 inotify
```

udev also has a very large collection of devices in /lib/udev/state/devices.tar.bz2

----------

