# wpa_supplicant has timeouts after a while

## pun_guin

Currently running a new kernel (with the ath10k module because my laptop's WiFi needs that) ...

```
> uname -a

Linux gentoo 4.14.65-gentoo #1 SMP Thu Aug 30 02:21:53 CEST 2018 x86_64 AMD A8-7410 APU with AMD Radeon R5 Graphics AuthenticAMD GNU/Linux
```

... I have experienced some kind of a long-term problem: After a few hours (during installing Gentoo updates, not "idle"), wpa_supplicant just times out. My WeeChat is disconnected and I can't ping anything anymore.

I can fix it with

```
# rc-service wpa_supplicant restart
```

There's nothing in dmesg or rc.log that looks like a crash. It just... stops. Could that be a firmware issue?

----------

## Goverp

wpa_supplicant has its own log.  I use the GUI, and it's under File->Event history.  Presumably there's a file configured somewhere.   I vaugely recall something about a verbose mode, but that might be something else.   Whatever, it tends to contain info about dropping things.  Whether they're useful is another question...

----------

## toralf

Why do you have a wpa_supplicant service at all ? I do just have

```
# rc-update  | grep -i wpa

# rc-status  | grep wpa

# rc-update  | fgrep net\.

              net.br0 |      default                           

          net.enp0s25 |      default                           

               net.lo |      default                           

           net.wlp3s0 |      default          

```

which seems to be enough, or? And the WPA is then started by a network script:

```
# ps -elfa | grep wpa

5 S root      3184     1  0  80   0 -  6378 -      Sep15 ?        00:00:01 /usr/sbin/wpa_supplicant -c /etc/wpa_supplicant/wpa_supplicant.conf -W -B -i wlp3s0 -P /run/wpa_supplicant-wlp3s0.pid

1 S root      3191     1  0  80   0 -  3403 -      Sep15 ?        00:00:00 /usr/bin/wpa_cli -a /etc/wpa_supplicant/wpa_cli.sh -p /var/run/wpa_supplicant -i wlp3s0 -P /run/wpa_cli-wlp3s0.pid -B

```

----------

## pun_guin

I never thought about that, it's what I have always used whenever I was on a Linux or BSD. Given that wpa_supplicant is currently dropping my connection anyway, the advantage of the service is probably that I can restart it easier...   :Confused: 

----------

## khayyam

 *pun_guin wrote:*   

> I never thought about that, it's what I have always used whenever I was on a Linux or BSD. Given that wpa_supplicant is currently dropping my connection anyway, the advantage of the service is probably that I can restart it easier...  :?

 

pun_guin ... you won't need to, if modules="wpa_supplicant" (which as I remember is enabled by default for wireless interfaces) then a wpa_cli monitor script is run which restarts net.{iface} when DISCONNECT is detected ... see: /etc/wpa_supplicant/wpa_cli.sh

best ... khay

----------

## pun_guin

The relevant question is if a DISCONNECT is detected, because my WiFi monitor script still displays the name of my network...

```
#!/bin/sh

wpa_cli status | awk -F= '/^ssid/{print $2}'
```

(I use it in dwm, updated once a second.)

----------

## khayyam

 *pun_guin wrote:*   

> The relevant question is if a DISCONNECT is detected, because my WiFi monitor script still displays the name of my network...

 

pun_guin ... different question. Difficult to say anything about what's going on without logs but I'm fairly certain that the ath10k driver will be at issue. If you have another pci-e card I'd suggest you swap it out.

best ... khay

----------

## pun_guin

This is a laptop. I'd never buy an ath10k card... in fact, this chip was the reason why I had to use Linux again because other Unix-like systems don't have a driver for it just yet.  :Sad: 

I had not used the 4.9 kernel so much, so I can't say if the situation has changed - but 4.14 had some differences in ath10k support. Hmm.

----------

## khayyam

 *pun_guin wrote:*   

> This is a laptop. I'd never buy an ath10k card... in fact, this chip was the reason why I had to use Linux again because other Unix-like systems don't have a driver for it just yet. :(

 

pun_guin ... well, laptops often have mini-pci for the wireless card, and often they can be removed, and replaced ... if you don't mind cracking open the case.

best ... khay

----------

## pun_guin

... or I'll just file a bug against the kernel.

But I probably need logs for that.

----------

## deagol

Can you explain why you think wpa_supplicant is timing out? (Basically what do you see/observe to say wpa_supplicant times out. Are there any log messages from the kernel/wpa_supplicant/networkmanager/whatever?)

You can get useful logs out of wpa_supplicant by issuing this command, assuming the normal setup with Networkmanager:

```
sudo dbus-send --system --print-reply --dest=fi.w1.wpa_supplicant1 /fi/w1/wpa_supplicant1 org.freedesktop.DBus.Properties.Set string:fi.w1.wpa_supplicant1 string:DebugLevel variant:string:"msgdump"

```

After that command wpa_supplicant will write really verbose debug messages in your syslog, hopefully indicating what goes wrong till you reboot or restart wpa_supplicant.

Ignoring the timeout the rest of your problem descriptions sounds like you may have PTK rekey enabled on your router, which is kind of broken in Linux and problematic with others from what I have seen so far. Disabling that on the router would then be the simplest workaround. But in that case it should be stable with most routers AND a normal disconnect/reconnect should also restore connectivity.

----------

## pun_guin

 *deagol wrote:*   

> Can you explain why you think wpa_supplicant is timing out?

 

Because it works for a while and then, not reliably at the same time (because that would be easier to debug, I guess), it just drops all connections.

I'll check the logs when I'm back on my Linux computer!  :Smile: 

What is a PTK rekey?

----------

## deagol

A timeout is normally always generating some notification. Without any log message telling "x has timed out" I really doubt you have a timeout issue.

More likely something happens after x minutes and the PTK rekey issue I brought up would be one candidate for that. *pun_guin wrote:*   

> 
> 
> What is a PTK rekey?

 A complete answer to that is kind of complicated and you should check out e.g. https://en.wikipedia.org/wiki/IEEE_802.11i-2004 to learn more about that.

The short version is, that this PTK (Pairwise Transient Key) is used to encrypt nearly all of the traffic between the router (access point) and any client connecting to the network. Replacing this key while transmitting packets at the same time has a very high risk to "freeze" the wlan on linux systems.

Maybe here the more obvious symptoms for the PTK rekey problem:

1.) Every indicator claims the connection the the Wlan is still active and working normally. There are no error logs, wpa_supplicant/Networkmanager claims to be still connected.

2.) ping to any IP needing the wlan connection - even the gateway - fails.

3.) When making sure the wlan is never idle - e.g. by downloading something - it reproducible freezes after a fixed time after the connect.

4.) Disconnecting from the wlan and connecting again works normally. (Even when you check the logs.)

5.) NOT taking any action when the connection freezes but waiting the for as long as it was working previously again will resume communication

6.) The problem only occurs with some wlans. Connecting to different router (different vendor) will likely not show the issue

7.) Windows (seems to) work normally on the same device and only Linux has issues. It's probably not, but chances are it looks like it is. (Can't say how BSD or other systems would react on it.) 

But if you want to be verify if you have the issue:

Enable the wpa_debugging as outlined in my previous post. You will get a real flood of messages which should be helpful regardless what your problem is.

But if you see something like 

```
WPA: Installing PTK to the driver
```

 at the time you lose connection instead only when you connect to the wlan your router is reking the PTK and either the router and/or your client do not handle that correctly.

Disabling it on the router is the simple fix for that. If that's not possible I can provide you some experimental patches which should also help.

----------

## pun_guin

I executed the above command. Currently, nothing is in it...

(I don't have NetworkManager, I usually use wpa_gui or wpa_cli - does that make a difference?)

----------

## deagol

Without NetworkManager you probably are not using dbus to control wpa_supplicant. In that case the dbus command to enable debugging will of course also not be able to enable debugging...

But wpa_cli also has a command to enable debug:

```
log_level debug
```

You still will have to figure out where wpa_supplicant is writing the debug messages to.

So you may want to follow the man page for wpa_supplicant and try to start it manually:

 *Quote:*   

> The easiest way to debug problems, and to get debug log for bug reports, is to start wpa_supplicant on foreground with debugging enabled:
> 
>               wpa_supplicant -c/etc/wpa_supplicant.conf -iwlan0 -d

 

----------

## khayyam

deagol, pun_guin, et al ...

as netifrc is in use you can do the following to enable debugging:

```
modules_wlp3s0="wpa_supplicant"

wpa_supplicant_wlp3s0="-Dnl80211 -dd -f /var/log/wpa_supplicant.log"
```

'-ddd' will increase the debug level, but it's probably unnecessary. Otherwise you could have net-wireless/iw follow 'events':

```
# iw event -f
```

HTH & best ... khay

----------

## pun_guin

(Sorry - I've been on tour this week...)

I enabled the log. Probably I'll be able to reproduce the error soon. Thank you (for now)!

----------

## pun_guin

It doesn't create the /var/log/wpa_supplicant.log file even though it still breaks...

----------

