# upower keeps segfaulting

## jhon987

since I've upgraded to the latest stable kernel 4.14.8 (now r1) I've encountered all sorts of issues most of which I've been able to solve, but here's one I could use some support:

dmesg gives the following:

```
[   27.876763] upowerd[4639]: segfault at 8 ip 000055dde7ae76b0 sp 00007ffdc10e9a30 error 4 in upowerd[55dde7ac7000+36000]

[   27.887427] upowerd[4651]: segfault at 8 ip 00005636844326b0 sp 00007ffd1e325600 error 4 in upowerd[563684412000+36000]

[   27.934996] upowerd[4664]: segfault at 8 ip 000056144f63c6b0 sp 00007ffc5745eef0 error 4 in upowerd[56144f61c000+36000]

[   36.192489] upowerd[4880]: segfault at 8 ip 00005559339a76b0 sp 00007ffe45e7c8c0 error 4 in upowerd[555933987000+36000]

....
```

in order to sort other issues I had (foolishly) 'make distclean' the kernel and reconfigured it all over again, so I'm not sure whether it's a bug or mis-configuration.

According to this bug report: https://bugs.freedesktop.org/show_bug.cgi?id=102903, it might have something to do with openrc's cgroupv2 feature, though, in my trial and error it seems not - I tried setting rc.conf's rc_cgroup_mode="legacy" but the issue persists.

This thread right here: https://forums.gentoo.org/viewtopic-t-1049614-start-0.html, suggest it might be an issue with upower version 0.99.4 but should be resolved in later versions. I've tried upgrading upower to the latest (unstable) version 0.99.7 but the segfaults are still here.

So what do you think is it: openrc bug? / upower bug? / kernel mis-configuration?

and how can I solve this?

----------

## Hu

By definition, no well-formed program will die with a segfault, so there is a defect somewhere.  It's possible that this is a kernel bug, but it's more likely that it's an application bug provoked by a change elsewhere (whether kernel, openrc, or some library).  What is the backtrace for that crash?

See https://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces if you need instructions for collecting a backtrace.

----------

## jhon987

Hi Hu,

Thanks for replaying, I actually use: https://wiki.gentoo.org/wiki/Debugging for debugging as I only want to debug certain programs.

Anyways, 

```
# gdb /usr/bin/upower

GNU gdb (Gentoo 7.12.1 vanilla) 7.12.1

Copyright (C) 2017 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-pc-linux-gnu".

Type "show configuration" for configuration details.

For bug reporting instructions, please see:

<https://bugs.gentoo.org/>.

Find the GDB manual and other documentation resources online at:

<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".

Type "apropos word" to search for commands related to "word"...

Reading symbols from /usr/bin/upower...Reading symbols from /usr/lib64/debug//usr/bin/upower.debug...done.

done.

(gdb) run

Starting program: /usr/bin/upower 

[Thread debugging using libthread_db enabled]

Using host libthread_db library "/lib64/libthread_db.so.1".

[New Thread 0x7fffeeb6d700 (LWP 11184)]

[New Thread 0x7fffee36c700 (LWP 11185)]

(upower:11180): UPower-WARNING **: Cannot connect to upowerd: Error calling StartServiceByName for org.freedesktop.UPower: GDBus.Error:org.freedesktop.DBus.Error.Spawn.ChildSignaled: Process org.freedesktop.UPower received signal 11

[Thread 0x7fffee36c700 (LWP 11185) exited]

[Thread 0x7fffeeb6d700 (LWP 11184) exited]

[Inferior 1 (process 11180) exited with code 01]

(gdb) backtrace

No stack.
```

Now the thing is - 

```
# rc-service -l

NetworkManager

agetty

alsasound

avahi-daemon

avahi-dnsconfd

binfmt

bootmisc

busybox-ntpd

busybox-watchdog

cgmanager

cgproxy

consolefont

consolekit

cronie

cups-browsed

cupsd

dbus

devfs

device-mapper

dhcpcd

dhcpd

dhcrelay

dhcrelay6

dmcrypt

dmesg

dmeventd

elogind

exim

fsck

fuse

git-daemon

gpm

hostname

hwclock

ip6tables

ipset

iptables

keymaps

killprocs

kmod-static-nodes

lighttpd

local

localmount

loopback

lvm

lvm-monitoring

lvmetad

modules

modules-load

mount-ro

mtab

mysql

mysql-s6

mysql-supervise

net-online

net.enp7s0

net.eth0

net.lo

netmount

nginx

ntp-client

ntpd

numlock

opentmpfiles-dev

opentmpfiles-setup

osclock

pciparm

php-fpm

pktcdvd

procfs

pwcheck

pydoc-2.7

pydoc-3.5

pydoc-3.6

root

rsyncd

runsvdir

s6-svscan

saned

saslauthd

savecache

slapd

sntp

sshd

svnserve

swap

swclock

sysctl

sysfs

sysklogd

termencoding

tor

udev

udev-settle

udev-trigger

urandom

wpa_supplicant

xdm

xdm-setup
```

there is no upowerd service, so I'm really confused. Shouldn't I have a upowerd service, or maybe because of openrc there is no need for one?

As I mentioned previously, I already re-emerged upower when I installed different versions of it and yet the upowerd daemon hasn't become available on my system.

Any ideas?

----------

## Hu

According to your dmesg output, upowerd crashed.  Why did you then debug upower, a completely different program?  Interestingly, it looks like your attempt to debug upower caused DBus to spawn a upowerd, which promptly crashed.  You need to debug the crashing upowerd, not the user interface frontend upower.

I do not know why there is no rcscript for upowerd.  Presumably, it is unnecessary since DBus launches it on demand.

----------

## jhon987

upowerd seem to me like it should be the daemon of upower, since:

```
~ $ whereis upowerd

upowerd: /usr/share/man/man8/upowerd.8.bz2
```

is the only result I got, I figured maybe I should check out upower and then maybe have a clue regarding upowerd, however your reply made me reconsider, so now:

```
$ locate upowerd

/usr/lib64/debug/usr/lib/upower/upowerd.debug

/usr/lib64/upower/upowerd

/usr/share/man/man8/upowerd.8.bz2
```

but then backtrace is meaningless:

```
# gdb /usr/lib64/upower/upowerd

GNU gdb (Gentoo 7.12.1 vanilla) 7.12.1

Copyright (C) 2017 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-pc-linux-gnu".

Type "show configuration" for configuration details.

For bug reporting instructions, please see:

<https://bugs.gentoo.org/>.

Find the GDB manual and other documentation resources online at:

<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".

Type "apropos word" to search for commands related to "word"...

Reading symbols from /usr/lib64/upower/upowerd...(no debugging symbols found)...done.

(gdb) run

Starting program: /usr/lib64/upower/upowerd 

[Thread debugging using libthread_db enabled]

Using host libthread_db library "/lib64/libthread_db.so.1".

[New Thread 0x7fffee20d700 (LWP 23240)]

[New Thread 0x7fffeda0c700 (LWP 23241)]

(upowerd:23236): GLib-GIO-CRITICAL **: g_dbus_proxy_get_connection: assertion 'G_IS_DBUS_PROXY (proxy)' failed

(upowerd:23236): GLib-GIO-CRITICAL **: g_dbus_connection_signal_subscribe: assertion 'G_IS_DBUS_CONNECTION (connection)' failed

(upowerd:23236): GLib-GIO-CRITICAL **: g_dbus_proxy_call_sync_internal: assertion 'G_IS_DBUS_PROXY (proxy)' failed

Thread 1 "upowerd" received signal SIGSEGV, Segmentation fault.

0x00005555555746b0 in ?? ()

(gdb) backtrace

#0  0x00005555555746b0 in ?? ()

#1  0x00007ffff6ec0569 in g_type_create_instance ()

   from /usr/lib64/libgobject-2.0.so.0

#2  0x00007ffff6ea3437 in ?? () from /usr/lib64/libgobject-2.0.so.0

#3  0x00007ffff6ea4e2d in g_object_newv () from /usr/lib64/libgobject-2.0.so.0

#4  0x00007ffff6ea557c in g_object_new () from /usr/lib64/libgobject-2.0.so.0

#5  0x00005555555617d2 in ?? ()

#6  0x00007ffff6ec0569 in g_type_create_instance ()

   from /usr/lib64/libgobject-2.0.so.0

#7  0x00007ffff6ea3437 in ?? () from /usr/lib64/libgobject-2.0.so.0

#8  0x00007ffff6ea4e2d in g_object_newv () from /usr/lib64/libgobject-2.0.so.0

#9  0x00007ffff6ea557c in g_object_new () from /usr/lib64/libgobject-2.0.so.0

#10 0x0000555555562fda in ?? ()

#11 0x0000555555560c34 in ?? ()

#12 0x00007ffff67eb541 in __libc_start_main (main=0x555555560a80, argc=1, 

    argv=0x7fffffffdb08, init=<optimized out>, fini=<optimized out>, 

    rtld_fini=<optimized out>, stack_end=0x7fffffffdaf8)

    at ../csu/libc-start.c:295

#13 0x0000555555560e5a in ?? ()

(gdb)
```

even though:

/etc/portage/env/debugsyms

```

CFLAGS="${CFLAGS} -ggdb"

CXXFLAGS="${CXXFLAGS} -ggdb"

FEATURES="${FEATURES} splitdebug compressdebug -nostrip"

USE="debug"
```

/etc/portage/env/installsources

```

FEATURES="${FEATURES} installsources"
```

and lastly, /etc/portage/package.env

```
sys-libs/glibc debugsyms installsources 

sys-power/upower debugsyms installsources
```

all that even though, as you can see above, debug symbols are installed: 

```
/usr/lib64/debug/usr/lib/upower/upowerd.debug
```

?

----------

## Hu

 *jhon987 wrote:*   

> upowerd seem to me like it should be the daemon of upower

 Agreed. *jhon987 wrote:*   

> 
> 
> ```
> $ locate upowerd
> 
> ...

 The debug symbols are installed, but not in the correct location.  Interestingly, another thread about a different manifestation of this problem came up very recently.  You are seeing another manifestation of kill SYMLINK_LIB=yes support.  Symbols were split to /usr/lib64/debug/usr/lib/upower, but gdb searches for them in /usr/lib64/debug/path-to-upowerd, meaning /usr/lib64/debug/usr/lib64/upower/upowerd.  Note the difference for the inner lib.  You can move the debug symbols to the path gdb expects, use FEATURES=nostrip instead of FEATURES=splitdebug, add a symlink to force gdb to find the symbols anyway, or patch upower to install the files in the correct place.  I think changing FEATURES is the simplest solution for your case.

----------

## jhon987

Thanks Hu for your help!

so, here's the backtrace:

```
# gdb /usr/lib64/upower/upowerd

GNU gdb (Gentoo 7.12.1 vanilla) 7.12.1

Copyright (C) 2017 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-pc-linux-gnu".

Type "show configuration" for configuration details.

For bug reporting instructions, please see:

<https://bugs.gentoo.org/>.

Find the GDB manual and other documentation resources online at:

<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".

Type "apropos word" to search for commands related to "word"...

Reading symbols from /usr/lib64/upower/upowerd...done.

(gdb) run

Starting program: /usr/lib64/upower/upowerd 

[Thread debugging using libthread_db enabled]

Using host libthread_db library "/lib64/libthread_db.so.1".

[New Thread 0x7fffee20d700 (LWP 24531)]

[New Thread 0x7fffeda0c700 (LWP 24532)]

(upowerd:24527): GLib-GIO-CRITICAL **: g_dbus_proxy_get_connection: assertion 'G_IS_DBUS_PROXY (proxy)' failed

(upowerd:24527): GLib-GIO-CRITICAL **: g_dbus_connection_signal_subscribe: assertion 'G_IS_DBUS_CONNECTION (connection)' failed

(upowerd:24527): GLib-GIO-CRITICAL **: g_dbus_proxy_call_sync_internal: assertion 'G_IS_DBUS_PROXY (proxy)' failed

Thread 1 "upowerd" received signal SIGSEGV, Segmentation fault.

0x00005555555746b0 in up_backend_inhibitor_lock_take (backend=0x55555578e4a0)

    at up-backend.c:501

501                     g_warning ("Could not acquire inhibitor lock: %s", error->message);

(gdb) backtrace

#0  0x00005555555746b0 in up_backend_inhibitor_lock_take (

    backend=0x55555578e4a0) at up-backend.c:501

#1  0x00007ffff6ec0569 in g_type_create_instance ()

   from /usr/lib64/libgobject-2.0.so.0

#2  0x00007ffff6ea3437 in ?? () from /usr/lib64/libgobject-2.0.so.0

#3  0x00007ffff6ea4e2d in g_object_newv () from /usr/lib64/libgobject-2.0.so.0

#4  0x00007ffff6ea557c in g_object_new () from /usr/lib64/libgobject-2.0.so.0

#5  0x0000555555575619 in up_backend_new () at up-backend.c:697

#6  0x00005555555617d2 in up_daemon_init (daemon=0x5555557a2150)

    at up-daemon.c:1087

#7  0x00007ffff6ec0569 in g_type_create_instance ()

   from /usr/lib64/libgobject-2.0.so.0

#8  0x00007ffff6ea3437 in ?? () from /usr/lib64/libgobject-2.0.so.0

#9  0x00007ffff6ea4e2d in g_object_newv () from /usr/lib64/libgobject-2.0.so.0

#10 0x00007ffff6ea557c in g_object_new () from /usr/lib64/libgobject-2.0.so.0

#11 0x0000555555562fda in up_daemon_new () at up-daemon.c:1169

#12 0x0000555555560c34 in up_state_new () at up-main.c:72

#13 main (argc=<optimized out>, argv=<optimized out>) at up-main.c:238

(gdb) 
```

Here's what valgrind had to say:

```

valgrind /usr/lib64/upower/upowerd

==24605== Memcheck, a memory error detector

==24605== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.

==24605== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info

==24605== Command: /usr/lib64/upower/upowerd

==24605== 

(upowerd:24605): GLib-GIO-CRITICAL **: g_dbus_proxy_get_connection: assertion 'G_IS_DBUS_PROXY (proxy)' failed

(upowerd:24605): GLib-GIO-CRITICAL **: g_dbus_connection_signal_subscribe: assertion 'G_IS_DBUS_CONNECTION (connection)' failed

(upowerd:24605): GLib-GIO-CRITICAL **: g_dbus_proxy_call_sync_internal: assertion 'G_IS_DBUS_PROXY (proxy)' failed

==24605== Invalid read of size 8

==24605==    at 0x1286B0: up_backend_inhibitor_lock_take (up-backend.c:501)

==24605==    by 0x5B66568: g_type_create_instance (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B49436: ??? (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B4AE2C: g_object_newv (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B4B57B: g_object_new (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x1157D1: up_daemon_init (up-daemon.c:1087)

==24605==    by 0x5B66568: g_type_create_instance (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B49436: ??? (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B4AE2C: g_object_newv (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B4B57B: g_object_new (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x116FD9: up_daemon_new (up-daemon.c:1169)

==24605==    by 0x114C33: up_state_new (up-main.c:72)

==24605==    by 0x114C33: main (up-main.c:238)

==24605==  Address 0x8 is not stack'd, malloc'd or (recently) free'd

==24605== 

==24605== 

==24605== Process terminating with default action of signal 11 (SIGSEGV)

==24605==  Access not within mapped region at address 0x8

==24605==    at 0x1286B0: up_backend_inhibitor_lock_take (up-backend.c:501)

==24605==    by 0x5B66568: g_type_create_instance (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B49436: ??? (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B4AE2C: g_object_newv (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B4B57B: g_object_new (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x1157D1: up_daemon_init (up-daemon.c:1087)

==24605==    by 0x5B66568: g_type_create_instance (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B49436: ??? (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B4AE2C: g_object_newv (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x5B4B57B: g_object_new (in /usr/lib64/libgobject-2.0.so.0.5000.3)

==24605==    by 0x116FD9: up_daemon_new (up-daemon.c:1169)

==24605==    by 0x114C33: up_state_new (up-main.c:72)

==24605==    by 0x114C33: main (up-main.c:238)

```

According to the above, if I understand the code correctly, the pointer *state (on line 238) is never being mallocd which means it should be a bug on other people's systems as well, yet it seems I'm the only one experiencing it.

I tried upgrading to upower 0.99.7 and it's the same error there as well.

Well, I guess I'll report it on Gentoo's bugzilla, but I really am baffled how come I'm the only one experiencing it, the only explanation I can think of is that I'm wrong regarding *state needs to be mallocd... either that, ot I'm not the only one, but no one else reported it on Gentoo's bugzilla

----------

## Hu

I have not looked long at the code, but I think you misunderstood the message.  I read that output to be that the C expression error->message faults because error is NULL.  If I am correct, then either g_dbus_proxy_call_with_unix_fd_list_sync is buggy and failed to set the error pointer, or upowerd is buggy for assuming that g_dbus_proxy_call_with_unix_fd_list_sync would set the error pointer.  To determine blame, we would need to know whether g_dbus_proxy_call_with_unix_fd_list_sync is documented to always set the error pointer on failure.  You could also take the cynical view and say that upowerd is buggy for trusting that g_dbus_proxy_call_with_unix_fd_list_sync would set the pointer, even if the documentation promises that it will always be set.  :Wink: 

Either way, this crash will only affect people for whom g_dbus_proxy_call_with_unix_fd_list_sync returns NULL.  When out is not NULL, that error log path is not executed.  I don't know why that function returns NULL for you, but if other people do not get NULL, they will not get a crash here.

----------

## jhon987

I always get completely lost when I look at other peoples' code. I filed a bug: https://bugs.gentoo.org/642128

I think that if upowerd is relying on g_dbus_proxy... but can't handle exceptions then it may be a bug in g_dbus but it's therefore also a bug in upowerd as well.

I guess I'm cynical...   :Wink: 

----------

## Hu

Hypothetically, if g_dbus_proxy_call_with_unix_fd_list_sync were documented to always either: (a) Return non-NULL or (b) Return NULL and set the error pointer, but never (c) return NULL without setting the error pointer, then it would not be wrong for upowerd to be written as it is.  Though, like you, I would take the cynical view and add a check for NULL error anyway, since it's cheap and can provide a much better user experience if it ever happens.

----------

## jhon987

Agreed.

By the way, if g_dbus_proxy_call_with_unix_fd_list_sync is in fact the culprit then:

 *Quote:*   

> Returns
> 
> NULL if error is set. Otherwise a GVariant tuple with return values. Free with g_variant_unref().
> 
> Since: 2.30

 

https://developer.gnome.org/gio/stable/GDBusProxy.html#g-dbus-proxy-call-with-unix-fd-list-sync

which means that g_dbus might not even be the end of the line, because g_dbus returns a value based on the error's state (whether it's set or not), meaning, not setting the error by itself.

So either you follow the chain of failures until you reach the bug's hive, or you simply make upowerd immune to failures in the external function/library.

Personally, I would start with making upowerd immune and only then I would follow the bug...

----------

