# gentoo-sources kernel panic

## Vieri

Hi,

After a week of running without issues, one of my recently installed Gentoo systems froze with a kernel panic.

```
Kernel panic - not syncing: hung_task: blocked tasks

CPU: 0 PID: 211 Comm: khungtaskd Tainted: G    W  0      4.19.86-gentoo-x86_64 #1

Hardware name: etc...
```

A few lines above the kernel panic message I can see references to "softirq", "ksoftirqd" and "net_rx_action".

This system is a router so maybe the cause of the kernel panic was a network IRQ issue.

However, I've used the exact same hardware (including the same NICs) for years without issues but with an older gentoo-sources or hardened-sources kernel.

I'm worried of course as I don't know if I can use this system for production use.

According to portage, the latest stable kernel is now 4.19.97. 

What can I try other than upgrading the kernel?

[EDIT] Has anyone ever used the "reboot on kernel panic" feature? Does it really work? Does it try logging something on disk before the reboot for future investigation?

----------

## elsandosgrande

1. You will need to provide more that just those few lines, especially what is above them in the log.

2. Was this a one-time occurrence, or is it a regular occurrence now?

3. Why would you not at least try a kernel that is not below 5.0 ("unstable" versions, even though I have always found the testing packages in both Arch Linux and Gentoo to be perfectly stable, including the kernel)?

4. While I have never tried that feature, I have found something that could help you: https://www.cyberciti.biz/tips/reboot-linux-box-after-a-kernel-panic.html

Take care!

----------

## NeddySeagoon

Vieri,

The 5.4.x kernel will be the next Long Term Support (LTS) kernel. That's what qualifies it for being stable in Gentoo.

When the kernel panics, it halts. It can't write any logs. A hung task can be for any number of reasons. Potentially hardware.

Do you have ECC RAM?

Have you seen any hardware errors, recovered or not, in dmesg?

Is this a random thing or can you provoke it?

Does the CPU Heat Sink need the grot cleaned out? 

Boot into memtest86 and see what it has to say after a few cycles.

Don't change kernels yet but kernel 5.4.x is nothing to be afraid of when the time comes.

----------

## Vieri

Here's the complete log:

```
Jan 24 11:33:07 inf-gw2 kernel: ------------[ cut here ]------------

Jan 24 11:33:07 inf-gw2 kernel: WARNING: CPU: 0 PID: 22756 at net/ipv4/tcp_output.c:911 tcp_wfree+0x29/0xe2

Jan 24 11:33:07 inf-gw2 kernel: Modules linked in: arc4 ecb md4 sha512_ssse3 sha512_generic cmac cifs ccm fscache nfnetlink_queue autofs4 xt_mac xt_REDIRECT xt_limit xt_nat xt_recent xt_statistic xt_connmark xt_TARPIT(O) xt_comment xt_ip

range xt_geoip(O) xt_set xt_NFQUEUE ipt_REJECT nf_reject_ipv4 xt_addrtype bridge stp llc xt_mark xt_TCPMSS xt_hashlimit xt_tcpudp xt_CT xt_multiport nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf

_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack

_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp pppoe pppox

Jan 24 11:33:07 inf-gw2 kernel:  ppp_generic slhc ip_set_hash_mac ip_set_bitmap_port ip_set_hash_net ip_set_hash_ip ip_set nfnetlink l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel ip6table_filter ip6_tables sha256_ssse3 sha256_generic 

mcryptd sha1_ssse3 sha1_generic ipv6 arptable_filter arp_tables xt_iface(O) xt_conntrack iptable_mangle iptable_nat nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw iptable_filter ip_tables x_tables sch_fq_codel 

bpfilter sch_fq snd_hda_codec_analog snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm snd_timer k8temp snd parport_pc floppy ohci_pci parport ohci_hcd asus_atk0110 ehci_pci thermal soundcore ehci_hcd fan ata_generic

 button i2c_nforce2 pata_amd pata_acpi msdos configfs fuse f2fs jfs btrfs zstd_decompress zstd_compress xxhash lzo_compress

Jan 24 11:33:07 inf-gw2 kernel:  zlib_deflate sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise ata_piix ahci libahci libata nvme nvme_core virtio_crypto crypto_engine virtio_pci virtio_balloon virtio_rng virtio_console virtio_b

lk virtio_ring virtio

Jan 24 11:33:07 inf-gw2 kernel: CPU: 0 PID: 22756 Comm: W#01 Tainted: G        W  O      4.19.86-gentoo-x86_64 #1

Jan 24 11:33:07 inf-gw2 kernel: Hardware name: System manufacturer System Product Name/M2N-E, BIOS ASUS M2N-E ACPI BIOS Revision 5001 03/23/2010

Jan 24 11:33:07 inf-gw2 kernel: RIP: 0010:tcp_wfree+0x29/0xe2

Jan 24 11:33:07 inf-gw2 kernel: Code: c3 55 53 8b 87 e0 00 00 00 48 8b 6f 18 ff c8 f0 29 85 44 01 00 00 0f 88 0f 4e 08 00 75 0e 48 c7 c7 83 3b d8 81 e8 b6 30 9c ff <0f> 0b 8b 85 44 01 00 00 3d 40 02 00 00 76 1a 65 48 8b 05 cc a8 95

Jan 24 11:33:07 inf-gw2 kernel: RSP: 0000:ffff88811fc03df0 EFLAGS: 00010246

Jan 24 11:33:07 inf-gw2 kernel: RAX: 0000000000000024 RBX: ffff8880d58eb100 RCX: 0000000000000000

Jan 24 11:33:07 inf-gw2 kernel: RDX: 0000000000000000 RSI: ffff88811fc152d8 RDI: ffff88811fc152d8

Jan 24 11:33:07 inf-gw2 kernel: RBP: ffff88810dd63100 R08: 0000000000000001 R09: 0000000000002b00

Jan 24 11:33:07 inf-gw2 kernel: R10: 0000000000000000 R11: 0000000000000044 R12: 0000000000000000

Jan 24 11:33:07 inf-gw2 kernel: R13: ffff88811ae287c0 R14: 000000000000003e R15: ffff88811ae28000

Jan 24 11:33:07 inf-gw2 kernel: FS:  00007fcc878c5700(0000) GS:ffff88811fc00000(0000) knlGS:0000000000000000

Jan 24 11:33:07 inf-gw2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

Jan 24 11:33:07 inf-gw2 kernel: CR2: 0000561e4889b328 CR3: 0000000114036000 CR4: 00000000000006f0

Jan 24 11:33:07 inf-gw2 kernel: Call Trace:

Jan 24 11:33:07 inf-gw2 kernel:  <IRQ>

Jan 24 11:33:07 inf-gw2 kernel:  skb_release_head_state+0x74/0xa4

Jan 24 11:33:07 inf-gw2 kernel:  skb_release_all+0xa/0x20

Jan 24 11:33:07 inf-gw2 kernel:  __kfree_skb+0xa/0x14

Jan 24 11:33:07 inf-gw2 kernel:  e1000_put_txbuf+0x73/0x86

Jan 24 11:33:07 inf-gw2 kernel:  e1000_clean_tx_irq+0xb4/0x23f

Jan 24 11:33:07 inf-gw2 kernel:  e1000e_poll+0x5a/0x223

Jan 24 11:33:07 inf-gw2 kernel:  net_rx_action+0x12e/0x305

Jan 24 11:33:07 inf-gw2 kernel:  __do_softirq+0x114/0x267

Jan 24 11:33:07 inf-gw2 kernel:  irq_exit+0x58/0x64

Jan 24 11:33:07 inf-gw2 kernel:  do_IRQ+0xaa/0xc8

Jan 24 11:33:07 inf-gw2 kernel:  common_interrupt+0xf/0xf

Jan 24 11:33:07 inf-gw2 kernel:  </IRQ>

Jan 24 11:33:07 inf-gw2 kernel: RIP: 0033:0x557d0f315593

Jan 24 11:33:07 inf-gw2 kernel: Code: 48 8b 45 b8 81 e3 ff 7f 00 00 44 89 75 a8 45 8d 46 01 48 c1 e3 09 48 03 5d a0 42 0f b6 14 30 49 8b 03 48 63 04 90 0f b7 1c 43 <66> 85 db 79 c0 48 8b 8d 78 ff ff ff 48 89 d8 25 ff 7f 00 00 48 c1

Jan 24 11:33:07 inf-gw2 kernel: RSP: 002b:00007fcc878c3d40 EFLAGS: 00000206 ORIG_RAX: ffffffffffffffde

Jan 24 11:33:07 inf-gw2 kernel: RAX: 000000000000007d RBX: 0000000000000000 RCX: 000000000000000c

Jan 24 11:33:07 inf-gw2 kernel: RDX: 000000000000007d RSI: 0000000000000000 RDI: 00007fcc878c3d40

Jan 24 11:33:07 inf-gw2 kernel: RBP: 00007fcc878c3ec0 R08: 0000000000000317 R09: 00007fcc640d3228

Jan 24 11:33:07 inf-gw2 kernel: R10: 0000557d1c82d8ec R11: 00007fcc878c5218 R12: 000000008000000c

Jan 24 11:33:07 inf-gw2 kernel: R13: 0000557d1c82d8ec R14: 0000000000000316 R15: 0000557d1be9f0f0

Jan 24 11:33:07 inf-gw2 kernel: ---[ end trace 2a8d072b1efcd8ea ]---

Jan 24 11:33:07 inf-gw2 kernel: ------------[ cut here ]------------

Jan 24 11:33:07 inf-gw2 kernel: WARNING: CPU: 0 PID: 22756 at net/ipv4/tcp_output.c:911 tcp_wfree+0x29/0xe2

Jan 24 11:33:07 inf-gw2 kernel: Modules linked in: arc4 ecb md4 sha512_ssse3 sha512_generic cmac cifs ccm fscache nfnetlink_queue autofs4 xt_mac xt_REDIRECT xt_limit xt_nat xt_recent xt_statistic xt_connmark xt_TARPIT(O) xt_comment xt_ip

range xt_geoip(O) xt_set xt_NFQUEUE ipt_REJECT nf_reject_ipv4 xt_addrtype bridge stp llc xt_mark xt_TCPMSS xt_hashlimit xt_tcpudp xt_CT xt_multiport nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf

_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack

_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp pppoe pppox

Jan 24 11:33:07 inf-gw2 kernel:  ppp_generic slhc ip_set_hash_mac ip_set_bitmap_port ip_set_hash_net ip_set_hash_ip ip_set nfnetlink l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel ip6table_filter ip6_tables sha256_ssse3 sha256_generic 

mcryptd sha1_ssse3 sha1_generic ipv6 arptable_filter arp_tables xt_iface(O) xt_conntrack iptable_mangle iptable_nat nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw iptable_filter ip_tables x_tables sch_fq_codel 

bpfilter sch_fq snd_hda_codec_analog snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm snd_timer k8temp snd parport_pc floppy ohci_pci parport ohci_hcd asus_atk0110 ehci_pci thermal soundcore ehci_hcd fan ata_generic

 button i2c_nforce2 pata_amd pata_acpi msdos configfs fuse f2fs jfs btrfs zstd_decompress zstd_compress xxhash lzo_compress

Jan 24 11:33:07 inf-gw2 kernel:  zlib_deflate sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise ata_piix ahci libahci libata nvme nvme_core virtio_crypto crypto_engine virtio_pci virtio_balloon virtio_rng virtio_console virtio_b

lk virtio_ring virtio

Jan 24 11:33:07 inf-gw2 kernel: CPU: 0 PID: 22756 Comm: W#01 Tainted: G        W  O      4.19.86-gentoo-x86_64 #1

Jan 24 11:33:07 inf-gw2 kernel: Hardware name: System manufacturer System Product Name/M2N-E, BIOS ASUS M2N-E ACPI BIOS Revision 5001 03/23/2010

Jan 24 11:33:07 inf-gw2 kernel: RIP: 0010:tcp_wfree+0x29/0xe2

Jan 24 11:33:07 inf-gw2 kernel: Code: c3 55 53 8b 87 e0 00 00 00 48 8b 6f 18 ff c8 f0 29 85 44 01 00 00 0f 88 0f 4e 08 00 75 0e 48 c7 c7 83 3b d8 81 e8 b6 30 9c ff <0f> 0b 8b 85 44 01 00 00 3d 40 02 00 00 76 1a 65 48 8b 05 cc a8 95

Jan 24 11:33:07 inf-gw2 kernel: RSP: 0000:ffff88811fc03df0 EFLAGS: 00010246

Jan 24 11:33:07 inf-gw2 kernel: RAX: 0000000000000024 RBX: ffff88811a5bac00 RCX: 0000000000000007

Jan 24 11:33:07 inf-gw2 kernel: RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffff88811fc152d0

Jan 24 11:33:07 inf-gw2 kernel: RBP: ffff888069f260c0 R08: 0000000000000001 R09: 0000000000003d00

Jan 24 11:33:07 inf-gw2 kernel: R10: 0000000000000000 R11: 0000000000000044 R12: 0000000000000000

Jan 24 11:33:07 inf-gw2 kernel: R13: ffff88811ae287c0 R14: 000000000000003e R15: ffff88811ae28000

Jan 24 11:33:07 inf-gw2 kernel: FS:  00007fcc878c5700(0000) GS:ffff88811fc00000(0000) knlGS:0000000000000000

Jan 24 11:33:07 inf-gw2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

Jan 24 11:33:07 inf-gw2 kernel: CR2: 0000561e4889b328 CR3: 0000000114036000 CR4: 00000000000006f0

Jan 24 11:33:07 inf-gw2 kernel: Call Trace:

Jan 24 11:33:07 inf-gw2 kernel:  <IRQ>

Jan 24 11:33:07 inf-gw2 kernel:  skb_release_head_state+0x74/0xa4

Jan 24 11:33:07 inf-gw2 kernel:  skb_release_all+0xa/0x20

Jan 24 11:33:07 inf-gw2 kernel:  __kfree_skb+0xa/0x14

Jan 24 11:33:07 inf-gw2 kernel:  e1000_put_txbuf+0x73/0x86

Jan 24 11:33:07 inf-gw2 kernel:  e1000_clean_tx_irq+0xb4/0x23f

Jan 24 11:33:07 inf-gw2 kernel:  e1000e_poll+0x5a/0x223

Jan 24 11:33:07 inf-gw2 kernel:  net_rx_action+0x12e/0x305

Jan 24 11:33:07 inf-gw2 kernel:  __do_softirq+0x114/0x267

Jan 24 11:33:07 inf-gw2 kernel:  irq_exit+0x58/0x64

Jan 24 11:33:07 inf-gw2 kernel:  do_IRQ+0xaa/0xc8

Jan 24 11:33:07 inf-gw2 kernel:  common_interrupt+0xf/0xf

Jan 24 11:33:07 inf-gw2 kernel:  </IRQ>

Jan 24 11:33:07 inf-gw2 kernel: RIP: 0033:0x557d0f315593

Jan 24 11:33:07 inf-gw2 kernel: Code: 48 8b 45 b8 81 e3 ff 7f 00 00 44 89 75 a8 45 8d 46 01 48 c1 e3 09 48 03 5d a0 42 0f b6 14 30 49 8b 03 48 63 04 90 0f b7 1c 43 <66> 85 db 79 c0 48 8b 8d 78 ff ff ff 48 89 d8 25 ff 7f 00 00 48 c1

Jan 24 11:33:07 inf-gw2 kernel: RSP: 002b:00007fcc878c3d40 EFLAGS: 00000206 ORIG_RAX: ffffffffffffffde

Jan 24 11:33:07 inf-gw2 kernel: RAX: 00000000000000b5 RBX: 0000000000000000 RCX: 0000000000000607

Jan 24 11:33:07 inf-gw2 kernel: RDX: 00000000000000b5 RSI: 0000000000000000 RDI: 00007fcc878c3d40

Jan 24 11:33:07 inf-gw2 kernel: RBP: 00007fcc878c3ec0 R08: 0000000000000289 R09: 00007fcc640d3228

Jan 24 11:33:07 inf-gw2 kernel: R10: 0000557d1c6f7fe8 R11: 00007fcc878c5218 R12: 0000000080000607

Jan 24 11:33:07 inf-gw2 kernel: R13: 0000557d1c6f7fe8 R14: 0000000000000288 R15: 0000557d1beab050

Jan 24 11:33:07 inf-gw2 kernel: ---[ end trace 2a8d072b1efcd8eb ]---

Jan 24 11:33:07 inf-gw2 kernel: ------------[ cut here ]------------

Jan 24 11:33:07 inf-gw2 kernel: WARNING: CPU: 0 PID: 4320 at net/ipv4/tcp_output.c:911 tcp_wfree+0x29/0xe2

Jan 24 11:33:07 inf-gw2 kernel: Modules linked in: arc4 ecb md4 sha512_ssse3 sha512_generic cmac cifs ccm fscache nfnetlink_queue autofs4 xt_mac xt_REDIRECT xt_limit xt_nat xt_recent xt_statistic xt_connmark xt_TARPIT(O) xt_comment xt_ip

range xt_geoip(O) xt_set xt_NFQUEUE ipt_REJECT nf_reject_ipv4 xt_addrtype bridge stp llc xt_mark xt_TCPMSS xt_hashlimit xt_tcpudp xt_CT xt_multiport nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf

_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack

_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp pppoe pppox

Jan 24 11:33:07 inf-gw2 kernel:  ppp_generic slhc ip_set_hash_mac ip_set_bitmap_port ip_set_hash_net ip_set_hash_ip ip_set nfnetlink l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel ip6table_filter ip6_tables sha256_ssse3 sha256_generic 

mcryptd sha1_ssse3 sha1_generic ipv6 arptable_filter arp_tables xt_iface(O) xt_conntrack iptable_mangle iptable_nat nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw iptable_filter ip_tables x_tables sch_fq_codel 

bpfilter sch_fq snd_hda_codec_analog snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm snd_timer k8temp snd parport_pc floppy ohci_pci parport ohci_hcd asus_atk0110 ehci_pci thermal soundcore ehci_hcd fan ata_generic

 button i2c_nforce2 pata_amd pata_acpi msdos configfs fuse f2fs jfs btrfs zstd_decompress zstd_compress xxhash lzo_compress

Jan 24 11:33:07 inf-gw2 kernel:  zlib_deflate sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise ata_piix ahci libahci libata nvme nvme_core virtio_crypto crypto_engine virtio_pci virtio_balloon virtio_rng virtio_console virtio_b

lk virtio_ring virtio

Jan 24 11:33:07 inf-gw2 kernel: CPU: 0 PID: 4320 Comm: syslog-ng Tainted: G        W  O      4.19.86-gentoo-x86_64 #1

Jan 24 11:33:07 inf-gw2 kernel: Hardware name: System manufacturer System Product Name/M2N-E, BIOS ASUS M2N-E ACPI BIOS Revision 5001 03/23/2010

Jan 24 11:33:07 inf-gw2 kernel: RIP: 0010:tcp_wfree+0x29/0xe2

Jan 24 11:33:07 inf-gw2 kernel: Code: c3 55 53 8b 87 e0 00 00 00 48 8b 6f 18 ff c8 f0 29 85 44 01 00 00 0f 88 0f 4e 08 00 75 0e 48 c7 c7 83 3b d8 81 e8 b6 30 9c ff <0f> 0b 8b 85 44 01 00 00 3d 40 02 00 00 76 1a 65 48 8b 05 cc a8 95

Jan 24 11:33:07 inf-gw2 kernel: RSP: 0000:ffff88811fc03df0 EFLAGS: 00010246

Jan 24 11:33:07 inf-gw2 kernel: RAX: 0000000000000024 RBX: ffff88810e3b5700 RCX: 0000000000000007

Jan 24 11:33:07 inf-gw2 kernel: RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffff88811fc152d0

Jan 24 11:33:07 inf-gw2 kernel: RBP: ffff888069fb2d40 R08: 0000000000000001 R09: 0000000000004f00

Jan 24 11:33:07 inf-gw2 kernel: R10: 0000000000000000 R11: 0000000000000044 R12: 0000000000000000

Jan 24 11:33:07 inf-gw2 kernel: R13: ffff88811ae287c0 R14: 000000000000003e R15: ffff88811ae28000

Jan 24 11:33:07 inf-gw2 kernel: FS:  00007f26f179f700(0000) GS:ffff88811fc00000(0000) knlGS:0000000000000000

Jan 24 11:33:07 inf-gw2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

Jan 24 11:33:07 inf-gw2 kernel: CR2: 0000561e4889b328 CR3: 00000000ce4d6000 CR4: 00000000000006f0

Jan 24 11:33:07 inf-gw2 kernel: Call Trace:

Jan 24 11:33:07 inf-gw2 kernel:  <IRQ>

Jan 24 11:33:07 inf-gw2 kernel:  skb_release_head_state+0x74/0xa4

Jan 24 11:33:07 inf-gw2 kernel:  skb_release_all+0xa/0x20

Jan 24 11:33:07 inf-gw2 kernel:  __kfree_skb+0xa/0x14

Jan 24 11:33:07 inf-gw2 kernel:  e1000_put_txbuf+0x73/0x86

Jan 24 11:33:07 inf-gw2 kernel:  e1000_clean_tx_irq+0xb4/0x23f

Jan 24 11:33:07 inf-gw2 kernel:  e1000e_poll+0x5a/0x223

Jan 24 11:33:07 inf-gw2 kernel:  net_rx_action+0x12e/0x305

Jan 24 11:33:07 inf-gw2 kernel:  __do_softirq+0x114/0x267

Jan 24 11:33:07 inf-gw2 kernel:  irq_exit+0x58/0x64

Jan 24 11:33:07 inf-gw2 kernel:  do_IRQ+0xaa/0xc8

Jan 24 11:33:07 inf-gw2 kernel:  common_interrupt+0xf/0xf

Jan 24 11:33:07 inf-gw2 kernel:  </IRQ>

Jan 24 11:33:07 inf-gw2 kernel: RIP: 0033:0x7f26f2081d63

Jan 24 11:33:07 inf-gw2 kernel: Code: 1f 84 00 00 00 00 00 e9 bb fe ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 55 53 48 83 ec 08 48 8b 2f 48 85 ed 74 12 48 89 ef 8b 3f <48> 83 c4 08 5b 5d e9 62 2a f8 ff 66 90 48 89 fb 48 8b 7f 08 e8 74

Jan 24 11:33:07 inf-gw2 kernel: RSP: 002b:00007f26f179cb90 EFLAGS: 00000206 ORIG_RAX: ffffffffffffffde

Jan 24 11:33:07 inf-gw2 kernel: RAX: 00007f26ec001b00 RBX: 000055a7084148b0 RCX: 00007f26f1f1ff2a

Jan 24 11:33:07 inf-gw2 kernel: RDX: 00007f26f21dae80 RSI: 0000000000000001 RDI: 0000000000000000

Jan 24 11:33:07 inf-gw2 kernel: RBP: 000055a7083ec2a0 R08: 00007f26f1d8d760 R09: 0000000000000000

Jan 24 11:33:07 inf-gw2 kernel: R10: 00007f26f179cbe0 R11: 0000000000000206 R12: 000055a708408c00

Jan 24 11:33:07 inf-gw2 kernel: R13: 0000000000000020 R14: 00007f26f179f700 R15: 00007f26f179cd40

Jan 24 11:33:07 inf-gw2 kernel: ---[ end trace 2a8d072b1efcd8ec ]---

Jan 24 11:33:38 inf-gw2 kernel: ------------[ cut here ]------------

Jan 24 11:33:38 inf-gw2 kernel: WARNING: CPU: 0 PID: 4321 at net/ipv4/tcp_output.c:911 tcp_wfree+0x29/0xe2

Jan 24 11:33:38 inf-gw2 kernel: Modules linked in: arc4 ecb md4 sha512_ssse3 sha512_generic cmac cifs ccm fscache nfnetlink_queue autofs4 xt_mac xt_REDIRECT xt_limit xt_nat xt_recent xt_statistic xt_connmark xt_TARPIT(O) xt_comment xt_ip

range xt_geoip(O) xt_set xt_NFQUEUE ipt_REJECT nf_reject_ipv4 xt_addrtype bridge stp llc xt_mark xt_TCPMSS xt_hashlimit xt_tcpudp xt_CT xt_multiport nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf

_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack

_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp pppoe pppox

Jan 24 11:33:38 inf-gw2 kernel:  ppp_generic slhc ip_set_hash_mac ip_set_bitmap_port ip_set_hash_net ip_set_hash_ip ip_set nfnetlink l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel ip6table_filter ip6_tables sha256_ssse3 sha256_generic 

mcryptd sha1_ssse3 sha1_generic ipv6 arptable_filter arp_tables xt_iface(O) xt_conntrack iptable_mangle iptable_nat nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw iptable_filter ip_tables x_tables sch_fq_codel 

bpfilter sch_fq snd_hda_codec_analog snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm snd_timer k8temp snd parport_pc floppy ohci_pci parport ohci_hcd asus_atk0110 ehci_pci thermal soundcore ehci_hcd fan ata_generic

 button i2c_nforce2 pata_amd pata_acpi msdos configfs fuse f2fs jfs btrfs zstd_decompress zstd_compress xxhash lzo_compress

Jan 24 11:33:38 inf-gw2 kernel:  zlib_deflate sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise ata_piix ahci libahci libata nvme nvme_core virtio_crypto crypto_engine virtio_pci virtio_balloon virtio_rng virtio_console virtio_b

lk virtio_ring virtio

Jan 24 11:33:38 inf-gw2 kernel: CPU: 0 PID: 4321 Comm: proftpd Tainted: G        W  O      4.19.86-gentoo-x86_64 #1

Jan 24 11:33:38 inf-gw2 kernel: Hardware name: System manufacturer System Product Name/M2N-E, BIOS ASUS M2N-E ACPI BIOS Revision 5001 03/23/2010

Jan 24 11:33:38 inf-gw2 kernel: RIP: 0010:tcp_wfree+0x29/0xe2

Jan 24 11:33:38 inf-gw2 kernel: Code: c3 55 53 8b 87 e0 00 00 00 48 8b 6f 18 ff c8 f0 29 85 44 01 00 00 0f 88 0f 4e 08 00 75 0e 48 c7 c7 83 3b d8 81 e8 b6 30 9c ff <0f> 0b 8b 85 44 01 00 00 3d 40 02 00 00 76 1a 65 48 8b 05 cc a8 95

Jan 24 11:33:38 inf-gw2 kernel: RSP: 0018:ffff88811fc03df0 EFLAGS: 00010246

Jan 24 11:33:38 inf-gw2 kernel: RAX: 0000000000000024 RBX: ffff8880d581b700 RCX: 0000000000000000

Jan 24 11:33:38 inf-gw2 kernel: RDX: 0000000000000000 RSI: ffff88811fc152d8 RDI: ffff88811fc152d8

Jan 24 11:33:38 inf-gw2 kernel: RBP: ffff888044296a00 R08: 0000000000000001 R09: 000000000000b200

Jan 24 11:33:38 inf-gw2 kernel: R10: 0000000000000000 R11: 0000000000000044 R12: 0000000000000000

Jan 24 11:33:38 inf-gw2 kernel: R13: ffff88811ae287c0 R14: 000000000000003e R15: ffff88811ae28000

Jan 24 11:33:38 inf-gw2 kernel: FS:  00007f7f62962740(0000) GS:ffff88811fc00000(0000) knlGS:0000000000000000

Jan 24 11:33:38 inf-gw2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

Jan 24 11:33:38 inf-gw2 kernel: CR2: 00007f7f629b82b4 CR3: 000000010a3ea000 CR4: 00000000000006f0

Jan 24 11:33:38 inf-gw2 kernel: Call Trace:

Jan 24 11:33:38 inf-gw2 kernel:  <IRQ>

Jan 24 11:33:38 inf-gw2 kernel:  skb_release_head_state+0x74/0xa4

Jan 24 11:33:38 inf-gw2 kernel:  skb_release_all+0xa/0x20

Jan 24 11:33:38 inf-gw2 kernel:  __kfree_skb+0xa/0x14

Jan 24 11:33:38 inf-gw2 kernel:  e1000_put_txbuf+0x73/0x86

Jan 24 11:33:38 inf-gw2 kernel:  e1000_clean_tx_irq+0xb4/0x23f

Jan 24 11:33:38 inf-gw2 kernel:  e1000e_poll+0x5a/0x223

Jan 24 11:33:38 inf-gw2 kernel:  net_rx_action+0x12e/0x305

Jan 24 11:33:38 inf-gw2 kernel:  __do_softirq+0x114/0x267

Jan 24 11:33:38 inf-gw2 kernel:  irq_exit+0x58/0x64

Jan 24 11:33:38 inf-gw2 kernel:  do_IRQ+0xaa/0xc8

Jan 24 11:33:38 inf-gw2 kernel:  common_interrupt+0xf/0xf

Jan 24 11:33:38 inf-gw2 kernel:  </IRQ>

Jan 24 11:33:38 inf-gw2 kernel: RIP: 0010:vma_interval_tree_remove+0xcf/0x231

Jan 24 11:33:38 inf-gw2 kernel: Code: 24 e9 7d 01 00 00 48 8b 4d 10 48 89 ea 48 85 c9 75 14 48 8b 43 70 48 8b 7d 08 48 89 45 18 48 89 e8 e9 8e 00 00 00 48 8b 41 10 <48> 85 c0 74 08 48 89 ca 48 89 c1 eb ef 48 8b 79 08 48 89 7a 10 48

Jan 24 11:33:38 inf-gw2 kernel: RSP: 0018:ffffc90003f87cf0 EFLAGS: 00000282 ORIG_RAX: ffffffffffffffde

Jan 24 11:33:38 inf-gw2 kernel: RAX: 0000000000000000 RBX: ffff8880ce7bbc38 RCX: ffff88810bd46f70

Jan 24 11:33:38 inf-gw2 kernel: RDX: ffff88810bd46e00 RSI: ffff888091c2ac00 RDI: ffff8880ce7bbc38

Jan 24 11:33:38 inf-gw2 kernel: RBP: ffff88810e233560 R08: 00007f7f628fe000 R09: ffff88810bc1cb80

Jan 24 11:33:38 inf-gw2 kernel: R10: ffff88810bc1cba0 R11: 0000000000000000 R12: ffff888091c2ac00

Jan 24 11:33:38 inf-gw2 kernel: R13: ffff8880ce7bbc90 R14: ffff88810bd46eb8 R15: ffff88810bc1cb80

Jan 24 11:33:38 inf-gw2 kernel:  unlink_file_vma+0x37/0x4c

Jan 24 11:33:38 inf-gw2 kernel:  free_pgtables+0x65/0x8d

Jan 24 11:33:38 inf-gw2 kernel:  unmap_region+0xcd/0x105

Jan 24 11:33:38 inf-gw2 kernel:  ? page_add_new_anon_rmap+0x57/0x72

Jan 24 11:33:38 inf-gw2 kernel:  do_munmap+0x24d/0x2c0

Jan 24 11:33:38 inf-gw2 kernel:  vm_munmap+0x63/0x94

Jan 24 11:33:38 inf-gw2 kernel:  __x64_sys_munmap+0xd/0x10

Jan 24 11:33:38 inf-gw2 kernel:  do_syscall_64+0x1c5/0x278

Jan 24 11:33:38 inf-gw2 kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Jan 24 11:33:38 inf-gw2 kernel: RIP: 0033:0x7f7f631e5b07

Jan 24 11:33:38 inf-gw2 kernel: Code: 0f 1f 00 f7 d8 89 05 f8 e5 00 00 48 c7 c0 ff ff ff ff eb 99 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 0b 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8d 0d c9 e5 00 00 f7 d8 89 01 48 83

Jan 24 11:33:38 inf-gw2 kernel: RSP: 002b:00007ffdc0762bd8 EFLAGS: 00000202 ORIG_RAX: 000000000000000b

Jan 24 11:33:38 inf-gw2 kernel: RAX: ffffffffffffffda RBX: 0000561e48840270 RCX: 00007f7f631e5b07

Jan 24 11:33:38 inf-gw2 kernel: RDX: 0000000000000002 RSI: 0000000000004040 RDI: 00007f7f628fb000

Jan 24 11:33:38 inf-gw2 kernel: RBP: 00007ffdc0762e40 R08: 00007f7f631f3930 R09: 0000000000000000

Jan 24 11:33:38 inf-gw2 kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffdc0762d71

Jan 24 11:33:38 inf-gw2 kernel: R13: 00007ffdc0762d08 R14: 00007f7f631f3000 R15: 0000561e48840270

Jan 24 11:33:38 inf-gw2 kernel: ---[ end trace 2a8d072b1efcd8ed ]---

Jan 24 11:33:38 inf-gw2 kernel: ------------[ cut here ]------------

Jan 24 11:33:38 inf-gw2 kernel: WARNING: CPU: 0 PID: 4321 at net/ipv4/tcp_output.c:911 tcp_wfree+0x29/0xe2

Jan 24 11:33:38 inf-gw2 kernel: Modules linked in: arc4 ecb md4 sha512_ssse3 sha512_generic cmac cifs ccm fscache nfnetlink_queue autofs4 xt_mac xt_REDIRECT xt_limit xt_nat xt_recent xt_statistic xt_connmark xt_TARPIT(O) xt_comment xt_ip

range xt_geoip(O) xt_set xt_NFQUEUE ipt_REJECT nf_reject_ipv4 xt_addrtype bridge stp llc xt_mark xt_TCPMSS xt_hashlimit xt_tcpudp xt_CT xt_multiport nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf

_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack

_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp pppoe pppox

Jan 24 11:33:38 inf-gw2 kernel:  ppp_generic slhc ip_set_hash_mac ip_set_bitmap_port ip_set_hash_net ip_set_hash_ip ip_set nfnetlink l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel ip6table_filter ip6_tables sha256_ssse3 sha256_generic 

mcryptd sha1_ssse3 sha1_generic ipv6 arptable_filter arp_tables xt_iface(O) xt_conntrack iptable_mangle iptable_nat nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw iptable_filter ip_tables x_tables sch_fq_codel 

bpfilter sch_fq snd_hda_codec_analog snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm snd_timer k8temp snd parport_pc floppy ohci_pci parport ohci_hcd asus_atk0110 ehci_pci thermal soundcore ehci_hcd fan ata_generic

 button i2c_nforce2 pata_amd pata_acpi msdos configfs fuse f2fs jfs btrfs zstd_decompress zstd_compress xxhash lzo_compress

Jan 24 11:33:38 inf-gw2 kernel:  zlib_deflate sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise ata_piix ahci libahci libata nvme nvme_core virtio_crypto crypto_engine virtio_pci virtio_balloon virtio_rng virtio_console virtio_b

lk virtio_ring virtio

Jan 24 11:33:38 inf-gw2 kernel: CPU: 0 PID: 4321 Comm: proftpd Tainted: G        W  O      4.19.86-gentoo-x86_64 #1

Jan 24 11:33:38 inf-gw2 kernel: Hardware name: System manufacturer System Product Name/M2N-E, BIOS ASUS M2N-E ACPI BIOS Revision 5001 03/23/2010

Jan 24 11:33:38 inf-gw2 kernel: RIP: 0010:tcp_wfree+0x29/0xe2

Jan 24 11:33:38 inf-gw2 kernel: Code: c3 55 53 8b 87 e0 00 00 00 48 8b 6f 18 ff c8 f0 29 85 44 01 00 00 0f 88 0f 4e 08 00 75 0e 48 c7 c7 83 3b d8 81 e8 b6 30 9c ff <0f> 0b 8b 85 44 01 00 00 3d 40 02 00 00 76 1a 65 48 8b 05 cc a8 95

Jan 24 11:33:38 inf-gw2 kernel: RSP: 0018:ffff88811fc03df0 EFLAGS: 00010246

Jan 24 11:33:38 inf-gw2 kernel: RAX: 0000000000000024 RBX: ffff8881173ff300 RCX: 0000000000000007

Jan 24 11:33:38 inf-gw2 kernel: RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffff88811fc152d0

Jan 24 11:33:38 inf-gw2 kernel: RBP: ffff888069dc9380 R08: 0000000000000001 R09: 000000000000c900

Jan 24 11:33:38 inf-gw2 kernel: R10: 0000000000000000 R11: 0000000000000044 R12: 0000000000000002

Jan 24 11:33:38 inf-gw2 kernel: R13: ffff88811ae287c0 R14: 0000000000000114 R15: ffff88811ae28000

Jan 24 11:33:38 inf-gw2 kernel: FS:  00007f7f62962740(0000) GS:ffff88811fc00000(0000) knlGS:0000000000000000

Jan 24 11:33:38 inf-gw2 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

Jan 24 11:33:38 inf-gw2 kernel: CR2: 00007f7f629b82b4 CR3: 000000010a3ea000 CR4: 00000000000006f0

Jan 24 11:33:38 inf-gw2 kernel: Call Trace:

Jan 24 11:33:38 inf-gw2 kernel:  <IRQ>

Jan 24 11:33:38 inf-gw2 kernel:  skb_release_head_state+0x74/0xa4

Jan 24 11:33:38 inf-gw2 kernel:  skb_release_all+0xa/0x20

Jan 24 11:33:38 inf-gw2 kernel:  __kfree_skb+0xa/0x14

Jan 24 11:33:38 inf-gw2 kernel:  e1000_put_txbuf+0x73/0x86

Jan 24 11:33:38 inf-gw2 kernel:  e1000_clean_tx_irq+0xb4/0x23f

Jan 24 11:33:38 inf-gw2 kernel:  e1000e_poll+0x5a/0x223

Jan 24 11:33:38 inf-gw2 kernel:  net_rx_action+0x12e/0x305

Jan 24 11:33:38 inf-gw2 kernel:  __do_softirq+0x114/0x267

Jan 24 11:33:38 inf-gw2 kernel:  irq_exit+0x58/0x64

Jan 24 11:33:38 inf-gw2 kernel:  do_IRQ+0xaa/0xc8

Jan 24 11:33:38 inf-gw2 kernel:  common_interrupt+0xf/0xf

Jan 24 11:33:38 inf-gw2 kernel:  </IRQ>

Jan 24 11:33:38 inf-gw2 kernel: RIP: 0010:vma_interval_tree_remove+0xcf/0x231

Jan 24 11:33:38 inf-gw2 kernel: Code: 24 e9 7d 01 00 00 48 8b 4d 10 48 89 ea 48 85 c9 75 14 48 8b 43 70 48 8b 7d 08 48 89 45 18 48 89 e8 e9 8e 00 00 00 48 8b 41 10 <48> 85 c0 74 08 48 89 ca 48 89 c1 eb ef 48 8b 79 08 48 89 7a 10 48

Jan 24 11:33:38 inf-gw2 kernel: RSP: 0018:ffffc90003f87cf0 EFLAGS: 00000282 ORIG_RAX: ffffffffffffffde

Jan 24 11:33:38 inf-gw2 kernel: RAX: 0000000000000000 RBX: ffff8880ce7bbc38 RCX: ffff88810bd46f70

Jan 24 11:33:38 inf-gw2 kernel: RDX: ffff88810bd46e00 RSI: ffff888091c2ac00 RDI: ffff8880ce7bbc38

Jan 24 11:33:38 inf-gw2 kernel: RBP: ffff88810e233560 R08: 00007f7f628fe000 R09: ffff88810bc1cb80

Jan 24 11:33:38 inf-gw2 kernel: R10: ffff88810bc1cba0 R11: 0000000000000000 R12: ffff888091c2ac00

Jan 24 11:33:38 inf-gw2 kernel: R13: ffff8880ce7bbc90 R14: ffff88810bd46eb8 R15: ffff88810bc1cb80

Jan 24 11:33:38 inf-gw2 kernel:  unlink_file_vma+0x37/0x4c

Jan 24 11:33:38 inf-gw2 kernel:  free_pgtables+0x65/0x8d

Jan 24 11:33:38 inf-gw2 kernel:  unmap_region+0xcd/0x105

Jan 24 11:33:38 inf-gw2 kernel:  ? page_add_new_anon_rmap+0x57/0x72

Jan 24 11:33:38 inf-gw2 kernel:  do_munmap+0x24d/0x2c0

Jan 24 11:33:38 inf-gw2 kernel:  vm_munmap+0x63/0x94

Jan 24 11:33:38 inf-gw2 kernel:  __x64_sys_munmap+0xd/0x10

Jan 24 11:33:38 inf-gw2 kernel:  do_syscall_64+0x1c5/0x278

Jan 24 11:33:38 inf-gw2 kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xa9

Jan 24 11:33:38 inf-gw2 kernel: RIP: 0033:0x7f7f631e5b07

Jan 24 11:33:38 inf-gw2 kernel: Code: 0f 1f 00 f7 d8 89 05 f8 e5 00 00 48 c7 c0 ff ff ff ff eb 99 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 0b 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8d 0d c9 e5 00 00 f7 d8 89 01 48 83

Jan 24 11:33:38 inf-gw2 kernel: RSP: 002b:00007ffdc0762bd8 EFLAGS: 00000202 ORIG_RAX: 000000000000000b

Jan 24 11:33:38 inf-gw2 kernel: RAX: ffffffffffffffda RBX: 0000561e48840270 RCX: 00007f7f631e5b07

Jan 24 11:33:38 inf-gw2 kernel: RDX: 0000000000000002 RSI: 0000000000004040 RDI: 00007f7f628fb000

Jan 24 11:33:38 inf-gw2 kernel: RBP: 00007ffdc0762e40 R08: 00007f7f631f3930 R09: 0000000000000000

Jan 24 11:33:38 inf-gw2 kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffdc0762d71

Jan 24 11:33:38 inf-gw2 kernel: R13: 00007ffdc0762d08 R14: 00007f7f631f3000 R15: 0000561e48840270

Jan 24 11:33:38 inf-gw2 kernel: ---[ end trace 2a8d072b1efcd8ee ]---
```

I installed the new Gentoo system last week, and it happened once today. I cannot reproduce the problem.

The hardware is apparently OK -- no need for cleaning.

It was running an older Gentoo OS for years without a glitch.

Well, maybe it's time to retire it (it's been running since 2006, day and night).

Still, I'd like to know what to make out of this kernel log.

Thanks

----------

## NeddySeagoon

Vieri,

I don't understand to log very well. 

```
Jan 24 11:33:07 inf-gw2 kernel:  e1000_put_txbuf+0x73/0x86

Jan 24 11:33:07 inf-gw2 kernel:  e1000_clean_tx_irq+0xb4/0x23f

Jan 24 11:33:07 inf-gw2 kernel:  e1000e_poll+0x5a/0x223 
```

Its not happy with the Intel e1000 and or e1000e network card drivers.

That can be a kernel issue or a hardware issue.

If you don't have any Intel NICs that use those drivers, they should not even be run.

----------

## Vieri

I require that Intel driver:

```
# ip link list

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

2: dmz: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 1000

    link/ether 00:04:75:9e:17:8f brd ff:ff:ff:ff:ff:ff

3: isp3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 1000

    link/ether 00:60:08:6e:62:ea brd ff:ff:ff:ff:ff:ff

4: isp1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000

    link/ether 00:15:17:d3:5e:f6 brd ff:ff:ff:ff:ff:ff

5: isp2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000

    link/ether 00:15:17:d3:5e:f7 brd ff:ff:ff:ff:ff:ff

6: lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000

    link/ether 00:17:31:83:79:7c brd ff:ff:ff:ff:ff:ff

7: enp1s6: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000

    link/ether 00:80:5a:69:60:5a brd ff:ff:ff:ff:ff:ff

8: ppp1: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 3

    link/ppp 

9: ppp2: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 3

    link/ppp 

10: ppp3: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UNKNOWN mode DEFAULT group default qlen 3

    link/ppp 

 # ethtool --driver dmz

driver: 3c59x

version: 

firmware-version: 

expansion-rom-version: 

bus-info: 0000:01:07.0

supports-statistics: yes

supports-test: no

supports-eeprom-access: no

supports-register-dump: no

supports-priv-flags: no

# ethtool --driver isp3

driver: 3c59x

version: 

firmware-version: 

expansion-rom-version: 

bus-info: 0000:01:08.0

supports-statistics: yes

supports-test: no

supports-eeprom-access: no

supports-register-dump: no

supports-priv-flags: no

# ethtool --driver isp2

driver: e1000e

version: 3.2.6-k

firmware-version: 5.11-2

expansion-rom-version: 

bus-info: 0000:02:00.1

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: no

# ethtool --driver isp1

driver: e1000e

version: 3.2.6-k

firmware-version: 5.11-2

expansion-rom-version: 

bus-info: 0000:02:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: no

# ethtool --driver lan 

driver: forcedeth

version: 0.64

firmware-version: 

expansion-rom-version: 

bus-info: 0000:00:08.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: no

supports-register-dump: yes

supports-priv-flags: no

# ethtool --driver enp1s6

driver: r8169

version: 

firmware-version: 

expansion-rom-version: 

bus-info: 0000:01:06.0

supports-statistics: yes

supports-test: no

supports-eeprom-access: no

supports-register-dump: yes

supports-priv-flags: no

```

Should I report this to bugs.gentoo.org or to the Linux kernel ML?

----------

## NeddySeagoon

Vieri,

Before you file a bug, try to rule out a hardware issue.

Try several kernels. If it works with some and not others, it may be software.

If its really software, lots of people will be affected, not just you, so google will know all about it.

e1000 and e1000e are popular network drivers.

If google knows about it, there may already be a fix.

Of course, someone has to be first but that's an exception 

There may be some module parameters you can fiddle with too.

If e1000e is build as a loadable module then 

```
modinfo e1000e
```

will list them at the end of the output.

----------

## Vieri

Even if the kernel doesn't panic, every once in a while I see this in syslog:

```
Jan 26 17:21:34 kernel: ------------[ cut here ]------------

Jan 26 17:21:34 kernel: WARNING: CPU: 1 PID: 20926 at net/ipv4/tcp_output.c:911 tcp_wfree+0x29/0xe2

Jan 26 17:21:34 kernel: Modules linked in: arc4 ecb md4 sha512_ssse3 sha512_generic cmac cifs ccm fscache nfnetlink_queue autofs4 xt_mac xt_REDIRECT xt_limit xt_nat xt_recent xt_statistic xt_connmark xt_TARPIT(O) xt_comment xt_iprange xt_geoip(O) xt_set xt_NFQUEUE ipt_REJECT nf_reject_ipv4 xt_addrtype bridge stp llc xt_mark xt_TCPMSS xt_hashlimit xt_tcpudp xt_CT xt_multiport nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp pppoe pppox

Jan 26 17:21:34 kernel:  ppp_generic slhc ip_set_hash_mac ip_set_bitmap_port ip_set_hash_net ip_set_hash_ip ip_set nfnetlink l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel ip6table_filter ip6_tables sha256_ssse3 sha256_generic mcryptd sha1_ssse3 sha1_generic ipv6 arptable_filter arp_tables xt_iface(O) xt_conntrack iptable_mangle iptable_nat nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw iptable_filter ip_tables x_tables sch_fq_codel bpfilter sch_fq snd_hda_codec_analog snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm k8temp snd_timer parport_pc floppy ohci_pci parport fan snd asus_atk0110 ohci_hcd soundcore thermal ehci_pci button ehci_hcd ata_generic i2c_nforce2 pata_amd pata_acpi msdos configfs fuse f2fs jfs btrfs zstd_decompress zstd_compress xxhash lzo_compress

Jan 26 17:21:34 kernel:  zlib_deflate sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise ata_piix ahci libahci libata nvme nvme_core virtio_crypto crypto_engine virtio_pci virtio_balloon virtio_rng virtio_console virtio_blk virtio_ring virtio

Jan 26 17:21:34 kernel: CPU: 1 PID: 20926 Comm: W#01 Tainted: G           O      4.19.86-gentoo-x86_64 #1

Jan 26 17:21:34 kernel: Hardware name: System manufacturer System Product Name/M2N-E, BIOS ASUS M2N-E ACPI BIOS Revision 5001 03/23/2010

Jan 26 17:21:34 kernel: RIP: 0010:tcp_wfree+0x29/0xe2

Jan 26 17:21:34 kernel: Code: c3 55 53 8b 87 e0 00 00 00 48 8b 6f 18 ff c8 f0 29 85 44 01 00 00 0f 88 0f 4e 08 00 75 0e 48 c7 c7 83 3b d8 81 e8 b6 30 9c ff <0f> 0b 8b 85 44 01 00 00 3d 40 02 00 00 76 1a 65 48 8b 05 cc a8 95

Jan 26 17:21:34 kernel: RSP: 0000:ffff88811fc83ee8 EFLAGS: 00010246

Jan 26 17:21:34 kernel: RAX: 0000000000000024 RBX: ffff88807ffceee8 RCX: 0000000000000000

Jan 26 17:21:34 kernel: RDX: 0000000000000000 RSI: ffff88811fc952d8 RDI: ffff88811fc952d8

Jan 26 17:21:34 kernel: RBP: ffff8880376e2600 R08: 0000000000000001 R09: 0000000000009c00

Jan 26 17:21:34 kernel: R10: 0000000000000000 R11: 0000000000000044 R12: ffff88807ffceee8

Jan 26 17:21:34 kernel: R13: 0000000000000000 R14: 0000000000000002 R15: 0000000000000002

Jan 26 17:21:34 kernel: FS:  00007f35b6f15700(0000) GS:ffff88811fc80000(0000) knlGS:0000000000000000

Jan 26 17:21:34 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

Jan 26 17:21:34 kernel: CR2: 000055ba3dcdf848 CR3: 000000010f2dc000 CR4: 00000000000006e0

Jan 26 17:21:34 kernel: Call Trace:

Jan 26 17:21:34 kernel:  <IRQ>

Jan 26 17:21:34 kernel:  skb_release_head_state+0x74/0xa4

Jan 26 17:21:34 kernel:  skb_release_all+0xa/0x20

Jan 26 17:21:34 kernel:  __kfree_skb+0xa/0x14

Jan 26 17:21:34 kernel:  net_tx_action+0xff/0x1bc

Jan 26 17:21:34 kernel:  __do_softirq+0x114/0x267

Jan 26 17:21:34 kernel:  irq_exit+0x58/0x64

Jan 26 17:21:34 kernel:  do_IRQ+0xaa/0xc8

Jan 26 17:21:34 kernel:  common_interrupt+0xf/0xf

Jan 26 17:21:34 kernel:  </IRQ>

Jan 26 17:21:34 kernel: RIP: 0033:0x564f07ea47e7

Jan 26 17:21:34 kernel: Code: 49 89 fc 55 48 89 cd 53 48 89 d3 4c 8b b2 a0 00 00 00 eb 12 0f 1f 80 00 00 00 00 41 80 7e 01 00 75 41 49 83 c6 10 41 0f b6 16 <49> 8b 4e 08 48 89 ee 4c 89 e7 48 8d 04 d5 00 00 00 00 48 29 d0 48

Jan 26 17:21:34 kernel: RSP: 002b:00007f35b6f13fb0 EFLAGS: 00000206 ORIG_RAX: ffffffffffffffd7

Jan 26 17:21:34 kernel: RAX: 0000564f093d71f0 RBX: 0000564f093d6880 RCX: 00007f359c009bc0

Jan 26 17:21:34 kernel: RDX: 0000000000000004 RSI: 0000564f0ffc7f90 RDI: 00007f35940d3110

Jan 26 17:21:34 kernel: RBP: 00007f359c009bc0 R08: 00007f35b6f14100 R09: 00007f35b6f14100

Jan 26 17:21:34 kernel: R10: 0000000001080007 R11: 0000000000005885 R12: 00007f35940d3110

Jan 26 17:21:34 kernel: R13: 0000564f08319400 R14: 0000564f0ffc7f50 R15: 00007f3574328aa0

Jan 26 17:21:34 kernel: ---[ end trace f5d35299bace3ecb ]---
```

I don't see anything regarding e1000 in this trace.

----------

## NeddySeagoon

Vieri,

```
Jan 26 17:21:34 kernel: WARNING: CPU: 1 PID: 20926 at net/ipv4/tcp_output.c
```

Its a network thing somewhere.

----------

## Ant P.

 *Vieri wrote:*   

> [EDIT] Has anyone ever used the "reboot on kernel panic" feature? Does it really work? Does it try logging something on disk before the reboot for future investigation?

 

I've used it on my headless servers and it's saved me a few times from kernel versions with hard-to-trigger bugs. It's worth leaving it on there.

If you want it to log something when that happens you'll need either pstore or some kind of external logging system. I'd advise keeping pstore off by default until you need it as it's led to me having to CMOS-reset one of those servers more than once after filling up its nvram.

----------

## Tony0945

Vieri,

I did a quick diff on that file between 4.19.97 and 4.19.98   - no difference

I also checked against 5.2.20 and 5.3.18  there are differences from 4.19.97

5.2.20 and 5.3.18 have differences between themselves but fewer.

I mention this is case you were considering trying a different kernel.

I rather doubt if it's a kernel problem because that driver is very popular.  Perhaps the kernel or driver were damaged?

It might be worthwhile rebuilding the kernel with no changes, but I suspect hardware problems. Realtek and Intel are the two most common NIC's

Realtek often has problems because numerous variations are supported by the same code. Realtek licenses a design and someone else actually designs the chips.

----------

## Vieri

Thank you all for your feedback.

However, I'm still in high seas, and it's getting worse.

I'm reproducing the same problem on new enterprise-grade hardware (with Intel, Broadcom and Mellanox NICs -- no Realtek).

I filed a bug report here below, but I'm not getting much from that: 

https://bugs.gentoo.org/706572

Imagine seeing these messages every couple of seconds on a production routing server...

The kernel didn't panic yet, but I'm not too comfortable with this in the log.

The fact that the kernel is "tainted" means that it would be useless to ask for support on the official Linux kernel mailing lists, right?

Is it tainted because I'm using gentoo-sources, or is it because I've installed xtables-addons as well as *l7* packages?

The problem is that I need xtables-addons and the xt_geoip module until I migrate to using ipsets, in which case I could ditch xtables-addons.

I posted my .config in the bug report, but no one seems to be able to reproduce this issue.

It's really, really weird.

My kernel building process is as follows:

1) emerge new gentoo-sources

2) download a custom config file with all the features I require

3) rename that config file to .config in /usr/src/linux

4) call make oldconfig (actually genkernel should do that for me, but I do it anyway before calling genkernel)

5) run genkernel and build everything

6) emerge xtables-addons, net-misc/l7-protocols, net-misc/l7-filter

What do you suggest I try?

I was thinking of skipping step 2 and running genkernel menuconfig to select my required options.

----------

## Hu

 *Vieri wrote:*   

> The fact that the kernel is "tainted" means that it would be useless to ask for support on the official Linux kernel mailing lists, right?

 Mostly, yes.  The first response you get will almost certainly be one that I use here quite a lot: "Can you reproduce the problem with an untainted kernel?"  I tend to use it because of people who were tainted by the nVidia proprietary drivers, but there are other sources of kernel tainting.

 *Vieri wrote:*   

> Is it tainted because I'm using gentoo-sources, or is it because I've installed xtables-addons as well as *l7* packages?

 According to your output, it is tainted for reasons G and O.  Looking at the kernel source, you will see that O means you loaded an out-of-tree module, and G means your kernel is tainted, but has no taint from proprietary modules.  Therefore, we can focus on the out-of-tree taint.  This is an easy one, since your module list shows you which modules were considered out-of-tree.  A bit of simple Vim regex can split them out:

```
%s/^.*kernel://

1s/Modules.*:/&\r/

2,$j

s/ \zs\w\+\ze //g

s/\s\+/ /g

%j
```

```
Modules linked in: xt_TARPIT(O) xt_geoip(O) xt_iface(O)
```

This tells us that you have three out-of-tree modules causing your kernel to be tainted.  Blacklist those from auto-loading, reboot into a clean kernel, and you will have an untainted kernel with which to test.  I see that you want to keep xt_geoip, but if you want not to be tainted, you cannot keep it.

----------

## NeddySeagoon

Vieri,

It looks like you need to do git bisect and build and test lots of kernels.

The idea is to do a binary search on all the commits between the current kernel and the last one that worked for you.

You will identify the commit that actually introduced the problem.

As you do a binary search, it will take 20 builds and tests to bisect 2^20 (just over 1,000,000) commits.

Its difficult for anyone except you to do this as they need to have your setup to work with.

Once you have identified the actual commit, identifying the problem becomes easier. 

Any 'out of tree' kernel module taints the kernel. Out of tree means not in the kernel code distributed by Linus.

As you say, sympathy from kernel devs will be minimal until you can demonstrate the problem with an untainted kernel.

----------

## Vieri

I have to thank you guys here in the forum, otherwise I'd feel pretty lost.

If I search syslog for anything related to kernel tainting before my dreaded traces, the only line I see is the following:

kernel: xt_iface: loading out-of-tree module taints kernel.

It does not mention xt_geoip or xt_TARPIT.

I don't care for either tarpit or iface for now, but since unloading xt_geoip would require me to change my geoip setup (eg. use ipsets instead, etc.) I'm wondering if it's safe to assume that unloading xt_iface alone would *not* taint the kernel? I mean, if all three modules (xt_TARPIT xt_geoip xt_iface) were tainting the kernel, wouldn't syslog show me the "out-of-tree" message above for all three of them and not just one?

```
# lsmod | grep xt_

xt_mac                 16384  35

xt_REDIRECT            16384  9

xt_limit               16384  16

xt_nat                 16384  28

xt_recent              20480  1

xt_statistic           16384  4

xt_connmark            16384  3

xt_TARPIT              16384  1

xt_comment             16384  149

xt_iprange             16384  15

xt_geoip               16384  51

xt_set                 16384  427

xt_NFQUEUE             16384  4

xt_AUDIT               16384  0

xt_addrtype            16384  83

xt_mark                16384  55

xt_TCPMSS              16384  1

xt_hashlimit           20480  87

xt_CT                  16384  22

xt_multiport           16384  247

xt_NFLOG               16384  0

xt_LOG                 16384  87

ip_set                 45056  5 ip_set_hash_mac,ip_set_bitmap_port,ip_set_hash_ip,xt_set,ip_set_hash_net

xt_iface               16384  2

xt_conntrack           16384  220

nf_nat                 36864  11 nf_nat_irc,nf_nat_ftp,nf_nat_ipv4,xt_nat,nf_nat_tftp,nf_nat_amanda,nf_nat_pptp,nf_nat_h323,nf_nat_proto_gre,nf_nat_sip,xt_RED

IRECT

nf_conntrack          172032  28 xt_conntrack,nf_nat_irc,nf_nat,nf_conntrack_tftp,nf_nat_ftp,nf_conntrack_pptp,nf_conntrack_netbios_ns,nf_conntrack_sane,nf_na

t_ipv4,xt_nat,nf_nat_tftp,nf_nat_amanda,nf_conntrack_sip,nf_conntrack_h323,nf_nat_pptp,nf_conntrack_broadcast,nf_conntrack_irc,nf_conntrack_amanda,nf_conntrac

k_netlink,xt_connmark,nf_conntrack_proto_gre,nf_conntrack_ftp,xt_CT,nf_nat_h323,nf_conntrack_snmp,nf_nat_snmp_basic,nf_nat_sip,xt_REDIRECT

bnxt_en               217088  0

ipv6                  557056  556 rdma_cm,bridge,l2tp_core,xt_TARPIT,ib_core,mlx5_core

```

Anyway, supposing I unload xt-iface or whichever other module taints the kernel, I guess it's good news I can still use gentoo-sources instead of having to switch to vanilla-sources.

The problem with reproducing the issue is that there must be a reasonable amount of traffic being routed through this server.

Booting the system under low traffic won't give me any traces in syslog. That's why I need to put it in production. That's also why I need geoip support.

So my first step will be to unload unnecessary modules, then if that doesn't work, try to set up an ipset-based geoip system and ditch xt_geoip.

It seems strange though that no one else is having this problem. It's not that I have an outstanding setup or anything...

----------

## NeddySeagoon

Vieri,

There are apps that will generate a test load for you. I'm not sure if they require a second system.

Debugging in a production environment is a nono.

You need a test environment, well away from production, where you can recreate the symptoms under controlled conditions.

----------

## Hu

As a minor point, if your goal is to reproduce on an untainted kernel, unloading the bad modules is not enough.  You need to reboot and not load them at all during that boot cycle.  Once a kernel is tainted, you cannot untaint it by unloading the bad module. *Vieri wrote:*   

> If I search syslog for anything related to kernel tainting before my dreaded traces, the only line I see is the following: 

 As I read the relevant kernel code, you only get a message when the kernel first becomes tainted by an out-of-tree module.  Loading additional out-of-tree modules will not elicit further warnings.

----------

## dmpogo

I still did not see what uses    e1000  driver    ( I do see e1000e in your ethtool output)

----------

## Tony0945

 *dmpogo wrote:*   

> I still did not see what uses    e1000  driver    ( I do see e1000e in your ethtool output)

 

Right! Vieri, a quick test would be to blacklist the e1000 driver so it doesn't load.

----------

## Vieri

 *Hu wrote:*   

> This tells us that you have three out-of-tree modules causing your kernel to be tainted.

 

Sorry, but could you explain this again in layman's terms? How did you exactly see which modules were out-of-tree and tainting the kernel? The regex lines are neat, but I must admit I got a bit lost there.

So, I blacklisted those modules.

```
# cat /etc/modprobe.d/blacklist.conf 

blacklist xt_iface

blacklist xt_geoip

blacklist xt_TARPIT
```

I then rebooted the system, and all seemed OK except I mistakenly thought that the kernel would actively deny any userland program to manually load the modules.

The xt_geoip module was being systematically loaded even though it was not being used.

So I made sure it wouldn't be loaded, and rebooted the system again.

Actually, I found a neat trick on an archlinux wiki. Instead of blacklisting it, use the "install" command.

```
# cat /etc/modprobe.d/blacklist.conf 

install xt_iface /bin/true

install xt_geoip /bin/true

install xt_TARPIT /bin/true

```

Now, both /proc/modules and syslog do NOT report geoip loading (as well as the other two), and there is no sign in the logs that the kernel is tainted.

So now I can just sit and wait...

 *Tony0945 wrote:*   

>  *dmpogo wrote:*   I still did not see what uses    e1000  driver    ( I do see e1000e in your ethtool output) 
> 
> Right! Vieri, a quick test would be to blacklist the e1000 driver so it doesn't load.

 

The issue is network-related, but I doubt it has anythind to do with e1000. That's because at first I was seeing this issue on a system with e1000 NICS. However, I am now running into the same problem with a whole new hardware without e1000 NICs.

Ethtool reports the following drivers on this system:

```
driver: bnxt_en

version: 1.9.2

firmware-version: 214.0.191.0

driver: igb

version: 5.4.0-k

firmware-version: 1.63, 0x800009fb, 1.1747.0

driver: ixgbe

version: 5.1.0-k

firmware-version: 0x80000aee, 1.1876.0

driver: mlx5_core

version: 5.0-0

firmware-version: 14.25.1020 (MT_2420110034)

```

----------

## Tony0945

So why load the driver if there is no hardware to use it?

----------

## Vieri

 *Tony0945 wrote:*   

> So why load the driver if there is no hardware to use it?

 

It is not loaded.

```
# lsmod | grep e3000

# modinfo e1000e

filename:       /lib/modules/4.19.97-gentoo-x86_64/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko

version:        3.2.6-k

license:        GPL

description:    Intel(R) PRO/1000 Network Driver

author:         Intel Corporation, <linux.nics@intel.com>

srcversion:     ACE9487E5B832BEE4B5D034

alias:          pci:v00008086d000015E2sv*sd*bc*sc*i*

alias:          pci:v00008086d000015E1sv*sd*bc*sc*i*

alias:          pci:v00008086d000015E0sv*sd*bc*sc*i*

alias:          pci:v00008086d000015DFsv*sd*bc*sc*i*

alias:          pci:v00008086d000015BCsv*sd*bc*sc*i*

alias:          pci:v00008086d000015BBsv*sd*bc*sc*i*

alias:          pci:v00008086d000015BEsv*sd*bc*sc*i*

alias:          pci:v00008086d000015BDsv*sd*bc*sc*i*

alias:          pci:v00008086d000015D6sv*sd*bc*sc*i*

alias:          pci:v00008086d000015E3sv*sd*bc*sc*i*

alias:          pci:v00008086d000015D8sv*sd*bc*sc*i*

alias:          pci:v00008086d000015D7sv*sd*bc*sc*i*

alias:          pci:v00008086d000015B9sv*sd*bc*sc*i*

alias:          pci:v00008086d000015B8sv*sd*bc*sc*i*

alias:          pci:v00008086d000015B7sv*sd*bc*sc*i*

alias:          pci:v00008086d00001570sv*sd*bc*sc*i*

alias:          pci:v00008086d0000156Fsv*sd*bc*sc*i*

alias:          pci:v00008086d000015A3sv*sd*bc*sc*i*

alias:          pci:v00008086d000015A2sv*sd*bc*sc*i*

alias:          pci:v00008086d000015A1sv*sd*bc*sc*i*

alias:          pci:v00008086d000015A0sv*sd*bc*sc*i*

alias:          pci:v00008086d00001559sv*sd*bc*sc*i*

alias:          pci:v00008086d0000155Asv*sd*bc*sc*i*

alias:          pci:v00008086d0000153Bsv*sd*bc*sc*i*

alias:          pci:v00008086d0000153Asv*sd*bc*sc*i*

alias:          pci:v00008086d00001503sv*sd*bc*sc*i*

alias:          pci:v00008086d00001502sv*sd*bc*sc*i*

alias:          pci:v00008086d000010F0sv*sd*bc*sc*i*

alias:          pci:v00008086d000010EFsv*sd*bc*sc*i*

alias:          pci:v00008086d000010EBsv*sd*bc*sc*i*

alias:          pci:v00008086d000010EAsv*sd*bc*sc*i*

alias:          pci:v00008086d00001525sv*sd*bc*sc*i*

alias:          pci:v00008086d000010DFsv*sd*bc*sc*i*

alias:          pci:v00008086d000010DEsv*sd*bc*sc*i*

alias:          pci:v00008086d000010CEsv*sd*bc*sc*i*

alias:          pci:v00008086d000010CDsv*sd*bc*sc*i*

alias:          pci:v00008086d000010CCsv*sd*bc*sc*i*

alias:          pci:v00008086d000010CBsv*sd*bc*sc*i*

alias:          pci:v00008086d000010F5sv*sd*bc*sc*i*

alias:          pci:v00008086d000010BFsv*sd*bc*sc*i*

alias:          pci:v00008086d000010E5sv*sd*bc*sc*i*

alias:          pci:v00008086d0000294Csv*sd*bc*sc*i*

alias:          pci:v00008086d000010BDsv*sd*bc*sc*i*

alias:          pci:v00008086d000010C3sv*sd*bc*sc*i*

alias:          pci:v00008086d000010C2sv*sd*bc*sc*i*

alias:          pci:v00008086d000010C0sv*sd*bc*sc*i*

alias:          pci:v00008086d00001501sv*sd*bc*sc*i*

alias:          pci:v00008086d00001049sv*sd*bc*sc*i*

alias:          pci:v00008086d0000104Dsv*sd*bc*sc*i*

alias:          pci:v00008086d0000104Bsv*sd*bc*sc*i*

alias:          pci:v00008086d0000104Asv*sd*bc*sc*i*

alias:          pci:v00008086d000010C4sv*sd*bc*sc*i*

alias:          pci:v00008086d000010C5sv*sd*bc*sc*i*

alias:          pci:v00008086d0000104Csv*sd*bc*sc*i*

alias:          pci:v00008086d000010BBsv*sd*bc*sc*i*

alias:          pci:v00008086d00001098sv*sd*bc*sc*i*

alias:          pci:v00008086d000010BAsv*sd*bc*sc*i*

alias:          pci:v00008086d00001096sv*sd*bc*sc*i*

alias:          pci:v00008086d0000150Csv*sd*bc*sc*i*

alias:          pci:v00008086d000010F6sv*sd*bc*sc*i*

alias:          pci:v00008086d000010D3sv*sd*bc*sc*i*

alias:          pci:v00008086d0000109Asv*sd*bc*sc*i*

alias:          pci:v00008086d0000108Csv*sd*bc*sc*i*

alias:          pci:v00008086d0000108Bsv*sd*bc*sc*i*

alias:          pci:v00008086d0000107Fsv*sd*bc*sc*i*

alias:          pci:v00008086d0000107Esv*sd*bc*sc*i*

alias:          pci:v00008086d0000107Dsv*sd*bc*sc*i*

alias:          pci:v00008086d000010B9sv*sd*bc*sc*i*

alias:          pci:v00008086d000010D5sv*sd*bc*sc*i*

alias:          pci:v00008086d000010DAsv*sd*bc*sc*i*

alias:          pci:v00008086d000010D9sv*sd*bc*sc*i*

alias:          pci:v00008086d00001060sv*sd*bc*sc*i*

alias:          pci:v00008086d000010A5sv*sd*bc*sc*i*

alias:          pci:v00008086d000010BCsv*sd*bc*sc*i*

alias:          pci:v00008086d000010A4sv*sd*bc*sc*i*

alias:          pci:v00008086d0000105Fsv*sd*bc*sc*i*

alias:          pci:v00008086d0000105Esv*sd*bc*sc*i*

depends:        

retpoline:      Y

intree:         Y

name:           e1000e

vermagic:       4.19.97-gentoo-x86_64 SMP mod_unload modversions 

sig_id:         PKCS#7

signer:         

sig_key:        

sig_hashalgo:   unknown

signature:      

parm:           debug:Debug level (0=none,...,16=all) (int)

parm:           copybreak:Maximum size of packet that is copied to a new buffer on receive (uint)

parm:           TxIntDelay:Transmit Interrupt Delay (array of int)

parm:           TxAbsIntDelay:Transmit Absolute Interrupt Delay (array of int)

parm:           RxIntDelay:Receive Interrupt Delay (array of int)

parm:           RxAbsIntDelay:Receive Absolute Interrupt Delay (array of int)

parm:           InterruptThrottleRate:Interrupt Throttling Rate (array of int)

parm:           IntMode:Interrupt Mode (array of int)

parm:           SmartPowerDownEnable:Enable PHY smart power down (array of int)

parm:           KumeranLockLoss:Enable Kumeran lock loss workaround (array of int)

parm:           WriteProtectNVM:Write-protect NVM [WARNING: disabling this can lead to corrupted NVM] (array of int)

parm:           CrcStripping:Enable CRC Stripping, disable if your BMC needs the CRC (array of int)

# modinfo e1000 

filename:       /lib/modules/4.19.97-gentoo-x86_64/kernel/drivers/net/ethernet/intel/e1000/e1000.ko

version:        7.3.21-k8-NAPI

license:        GPL

description:    Intel(R) PRO/1000 Network Driver

author:         Intel Corporation, <linux.nics@intel.com>

srcversion:     6FD2D67A7F4ECC185727145

alias:          pci:v00008086d00002E6Esv*sd*bc*sc*i*

alias:          pci:v00008086d000010B5sv*sd*bc*sc*i*

alias:          pci:v00008086d00001099sv*sd*bc*sc*i*

alias:          pci:v00008086d0000108Asv*sd*bc*sc*i*

alias:          pci:v00008086d0000107Csv*sd*bc*sc*i*

alias:          pci:v00008086d0000107Bsv*sd*bc*sc*i*

alias:          pci:v00008086d0000107Asv*sd*bc*sc*i*

alias:          pci:v00008086d00001079sv*sd*bc*sc*i*

alias:          pci:v00008086d00001078sv*sd*bc*sc*i*

alias:          pci:v00008086d00001077sv*sd*bc*sc*i*

alias:          pci:v00008086d00001076sv*sd*bc*sc*i*

alias:          pci:v00008086d00001075sv*sd*bc*sc*i*

alias:          pci:v00008086d00001028sv*sd*bc*sc*i*

alias:          pci:v00008086d00001027sv*sd*bc*sc*i*

alias:          pci:v00008086d00001026sv*sd*bc*sc*i*

alias:          pci:v00008086d0000101Esv*sd*bc*sc*i*

alias:          pci:v00008086d0000101Dsv*sd*bc*sc*i*

alias:          pci:v00008086d0000101Asv*sd*bc*sc*i*

alias:          pci:v00008086d00001019sv*sd*bc*sc*i*

alias:          pci:v00008086d00001018sv*sd*bc*sc*i*

alias:          pci:v00008086d00001017sv*sd*bc*sc*i*

alias:          pci:v00008086d00001016sv*sd*bc*sc*i*

alias:          pci:v00008086d00001015sv*sd*bc*sc*i*

alias:          pci:v00008086d00001014sv*sd*bc*sc*i*

alias:          pci:v00008086d00001013sv*sd*bc*sc*i*

alias:          pci:v00008086d00001012sv*sd*bc*sc*i*

alias:          pci:v00008086d00001011sv*sd*bc*sc*i*

alias:          pci:v00008086d00001010sv*sd*bc*sc*i*

alias:          pci:v00008086d0000100Fsv*sd*bc*sc*i*

alias:          pci:v00008086d0000100Esv*sd*bc*sc*i*

alias:          pci:v00008086d0000100Dsv*sd*bc*sc*i*

alias:          pci:v00008086d0000100Csv*sd*bc*sc*i*

alias:          pci:v00008086d00001009sv*sd*bc*sc*i*

alias:          pci:v00008086d00001008sv*sd*bc*sc*i*

alias:          pci:v00008086d00001004sv*sd*bc*sc*i*

alias:          pci:v00008086d00001001sv*sd*bc*sc*i*

alias:          pci:v00008086d00001000sv*sd*bc*sc*i*

depends:        

retpoline:      Y

intree:         Y

name:           e1000

vermagic:       4.19.97-gentoo-x86_64 SMP mod_unload modversions 

sig_id:         PKCS#7

signer:         

sig_key:        

sig_hashalgo:   unknown

signature:      

parm:           TxDescriptors:Number of transmit descriptors (array of int)

parm:           RxDescriptors:Number of receive descriptors (array of int)

parm:           Speed:Speed setting (array of int)

parm:           Duplex:Duplex setting (array of int)

parm:           AutoNeg:Advertised auto-negotiation setting (array of int)

parm:           FlowControl:Flow Control setting (array of int)

parm:           XsumRX:Disable or enable Receive Checksum offload (array of int)

parm:           TxIntDelay:Transmit Interrupt Delay (array of int)

parm:           TxAbsIntDelay:Transmit Absolute Interrupt Delay (array of int)

parm:           RxIntDelay:Receive Interrupt Delay (array of int)

parm:           RxAbsIntDelay:Receive Absolute Interrupt Delay (array of int)

parm:           InterruptThrottleRate:Interrupt Throttling Rate (array of int)

parm:           SmartPowerDownEnable:Enable PHY smart power down (array of int)

parm:           copybreak:Maximum size of packet that is copied to a new buffer on receive (uint)

parm:           debug:Debug level (0=none,...,16=all) (int)

```

I didn't have to wait long...

Here's how it all starts:

```
Feb 11 16:53:40 kernel: ------------[ cut here ]------------

Feb 11 16:53:40 kernel: WARNING: CPU: 6 PID: 0 at net/ipv4/tcp_output.c:915 tcp_wfree.cold+0xc/0x13

Feb 11 16:53:40 kernel: Modules linked in: autofs4 nfnetlink_queue xt_mac xt_REDIRECT xt_limit xt_nat xt_recent xt_statistic xt_connmark xt_comment xt_iprange l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel xt_set xt_NFQUEUE xt_AUDIT ipt_REJECT nf_reject_ipv4 xt_addrtype bridge stp llc xt_mark xt_TCPMSS xt_hashlimit xt_CT xt_multiport nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp pppoe pppox ppp_generic slhc ip_set_hash_mac ip_set_bitmap_port

Feb 11 16:53:40 kernel:  ip_set_hash_net ip_set_hash_ip ip_set nfnetlink ip6table_filter ip6_tables arptable_filter arp_tables xt_conntrack iptable_mangle iptable_nat nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw sch_fq tcp_cdg tcp_bbr iptable_filter ip_tables bpfilter mlx5_ib ipmi_ssif ib_uverbs edac_mce_amd kvm_amd kvm ast ttm irqbypass crct10dif_pclmul efi_pstore ghash_clmulni_intel drm_kms_helper pcspkr efivars ixgbe igb sp5100_tco mlx5_core drm joydev bnxt_en i2c_algo_bit mdio i2c_piix4 mlxfw ccp dca i2c_core ipmi_si ipmi_devintf ipmi_msghandler pinctrl_amd pcc_cpufreq acpi_cpufreq mac_hid efivarfs aesni_intel crypto_simd cryptd glue_helper aes_x86_64 algif_rng algif_aead algif_hash algif_skcipher af_alg crc32c_intel crc32_pclmul crc32_generic msdos fat cramfs overlay squashfs

Feb 11 16:53:40 kernel:  loop fuse f2fs xfs nfs lockd grace sunrpc fscache jfs reiserfs btrfs ext4 mbcache jbd2 multipath linear raid10 raid1 raid0 dm_zero dm_verity reed_solomon dm_thin_pool dm_switch dm_snapshot dm_raid raid456 md_mod async_raid6_recov async_memcpy async_pq raid6_pq dm_mirror dm_region_hash dm_log_writes dm_log_userspace dm_log dm_integrity async_xor async_tx xor dm_flakey dm_delay dm_crypt dm_cache_smq dm_cache dm_persistent_data libcrc32c dm_bufio dm_bio_prison dm_mod firewire_core crc_itu_t hid_sunplus hid_sony hid_samsung hid_pl hid_petalynx hid_monterey hid_microsoft hid_logitech_dj hid_logitech ff_memless hid_gyration hid_ezkey hid_cypress hid_chicony hid_cherry hid_belkin hid_apple hid_a4tech sl811_hcd ohci_hcd uhci_hcd uas usb_storage xhci_plat_hcd pata_sl82c105 pata_via pata_jmicron

Feb 11 16:53:40 kernel:  pata_marvell pata_netcell pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_oldpiix pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_sil680 pata_pdc2027x pata_mpiix lpfc nvmet_fc qla2xxx megaraid_mbox megaraid_mm aacraid sx8 hpsa 3w_9xxx 3w_xxxx 3w_sas mptsas mptfc scsi_transport_fc atp870u dc395x qla1280 dmx3191d sym53c8xx gdth initio BusLogic arcmsr aic7xxx aic79xx sr_mod cdrom sg sd_mod mpt3sas raid_class scsi_transport_sas megaraid megaraid_sas mptspi mptscsih mptbase scsi_transport_spi pdc_adma sata_inic162x sata_mv sata_qstor sata_vsc sata_uli sata_sis pata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24

Feb 11 16:53:40 kernel:  sata_sil sata_promise ata_piix ahci libahci nvme_fc nvme_loop nvmet nvme_rdma rdma_cm iw_cm ib_cm ib_core configfs ipv6 crc_ccitt nvme_fabrics nvme nvme_core

Feb 11 16:53:40 kernel: CPU: 6 PID: 0 Comm: swapper/6 Not tainted 4.19.97-gentoo-x86_64 #1

Feb 11 16:53:40 kernel: Hardware name: Supermicro AS -1114S-WTRT/H12SSW-NT, BIOS 1.0b 11/15/2019

Feb 11 16:53:40 kernel: RIP: 0010:tcp_wfree.cold+0xc/0x13

Feb 11 16:53:40 kernel: Code: 9d 04 00 00 00 5b c6 85 9b 04 00 00 00 5d c3 48 c7 c7 70 93 06 b0 e8 f7 f7 94 ff 0f 0b c3 48 c7 c7 70 93 06 b0 e8 e8 f7 94 ff <0f> 0b e9 46 a5 ff ff 48 c7 c7 70 93 06 b0 e8 d5 f7 94 ff 0f 0b b8

Feb 11 16:53:40 kernel: RSP: 0018:ffff9e9c2b183d90 EFLAGS: 00010246

Feb 11 16:53:40 kernel: RAX: 0000000000000024 RBX: ffff9e9bc099cee8 RCX: 0000000000000000

Feb 11 16:53:40 kernel: RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 0000000000000300

Feb 11 16:53:40 kernel: RBP: ffff9e9bbef09980 R08: ffff9e9c2b1968b8 R09: 0000000000000001

Feb 11 16:53:40 kernel: R10: 0000000000000000 R11: 0000000000000001 R12: ffff9e9bc099cee8

Feb 11 16:53:40 kernel: R13: ffff9e9a900100a8 R14: ffff9e9c0155a8c0 R15: 0000000000000026

Feb 11 16:53:40 kernel: FS:  0000000000000000(0000) GS:ffff9e9c2b180000(0000) knlGS:0000000000000000

Feb 11 16:53:40 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

Feb 11 16:53:40 kernel: CR2: 00007f90cb702820 CR3: 00000007d41b2000 CR4: 0000000000340ee0

Feb 11 16:53:40 kernel: Call Trace:

Feb 11 16:53:40 kernel:  <IRQ>

Feb 11 16:53:40 kernel:  skb_release_head_state+0x64/0xb0

Feb 11 16:53:40 kernel:  skb_release_all+0xe/0x30

Feb 11 16:53:40 kernel:  consume_skb+0x27/0x80

Feb 11 16:53:40 kernel:  bnxt_tx_int+0xd0/0x360 [bnxt_en]

Feb 11 16:53:40 kernel:  bnxt_poll+0x20f/0x870 [bnxt_en]

Feb 11 16:53:40 kernel:  net_rx_action+0x148/0x3b0

Feb 11 16:53:40 kernel:  __do_softirq+0xe8/0x2f1

Feb 11 16:53:40 kernel:  irq_exit+0x100/0x110

Feb 11 16:53:40 kernel:  do_IRQ+0x81/0xe0

Feb 11 16:53:40 kernel:  common_interrupt+0xf/0xf

Feb 11 16:53:40 kernel:  </IRQ>

Feb 11 16:53:40 kernel: RIP: 0010:cpuidle_enter_state+0xc3/0x320

Feb 11 16:53:40 kernel: Code: e8 82 68 a0 ff 80 7c 24 0b 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 30 02 00 00 31 ff e8 84 55 a6 ff fb 66 0f 1f 44 00 00 <48> ba cf f7 53 e3 a5 9b c4 20 4c 29 f5 48 89 e8 48 c1 fd 3f 48 f7

Feb 11 16:53:40 kernel: RSP: 0018:ffffb4440021fe80 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6

Feb 11 16:53:40 kernel: RAX: ffff9e9c2b1a2200 RBX: ffff9e9c02dfe800 RCX: 000000000000001f

Feb 11 16:53:40 kernel: RDX: 0000000000000000 RSI: 000000002c234d74 RDI: 0000000000000000

Feb 11 16:53:40 kernel: RBP: 000002de50bf72ea R08: 000002de50bf72ea R09: 0000000000000035

Feb 11 16:53:40 kernel: R10: 00000000ffffffff R11: ffff9e9c2b1a12e8 R12: 0000000000000002

Feb 11 16:53:40 kernel: R13: ffffffffb03954a0 R14: 000002de4de20c75 R15: ffff9e95044bcc80

Feb 11 16:53:40 kernel:  do_idle+0x1dc/0x270

Feb 11 16:53:40 kernel:  cpu_startup_entry+0x6f/0x80

Feb 11 16:53:40 kernel:  start_secondary+0x1a7/0x200

Feb 11 16:53:40 kernel:  secondary_startup_64+0xb6/0xc0

Feb 11 16:53:40 kernel: ---[ end trace 828aa59c66af655f ]---

[...]

Feb 11 16:57:21 kernel: ------------[ cut here ]------------

Feb 11 16:57:21 kernel: WARNING: CPU: 1 PID: 0 at net/ipv4/tcp_output.c:915 tcp_wfree.cold+0xc/0x13

Feb 11 16:57:21 kernel: Modules linked in: autofs4 nfnetlink_queue xt_mac xt_REDIRECT xt_limit xt_nat xt_recent xt_statistic xt_connmark xt_comment xt_iprange l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel xt_set xt_NFQUEUE xt_AUDIT ipt_REJECT nf_reject_ipv4 xt_addrtype bridge stp llc xt_mark xt_TCPMSS xt_hashlimit xt_CT xt_multiport nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp pppoe pppox ppp_generic slhc ip_set_hash_mac ip_set_bitmap_port

Feb 11 16:57:21 kernel:  ip_set_hash_net ip_set_hash_ip ip_set nfnetlink ip6table_filter ip6_tables arptable_filter arp_tables xt_conntrack iptable_mangle iptable_nat nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw sch_fq tcp_cdg tcp_bbr iptable_filter ip_tables bpfilter mlx5_ib ipmi_ssif ib_uverbs edac_mce_amd kvm_amd kvm ast ttm irqbypass crct10dif_pclmul efi_pstore ghash_clmulni_intel drm_kms_helper pcspkr efivars ixgbe igb sp5100_tco mlx5_core drm joydev bnxt_en i2c_algo_bit mdio i2c_piix4 mlxfw ccp dca i2c_core ipmi_si ipmi_devintf ipmi_msghandler pinctrl_amd pcc_cpufreq acpi_cpufreq mac_hid efivarfs aesni_intel crypto_simd cryptd glue_helper aes_x86_64 algif_rng algif_aead algif_hash algif_skcipher af_alg crc32c_intel crc32_pclmul crc32_generic msdos fat cramfs overlay squashfs

Feb 11 16:57:21 kernel:  loop fuse f2fs xfs nfs lockd grace sunrpc fscache jfs reiserfs btrfs ext4 mbcache jbd2 multipath linear raid10 raid1 raid0 dm_zero dm_verity reed_solomon dm_thin_pool dm_switch dm_snapshot dm_raid raid456 md_mod async_raid6_recov async_memcpy async_pq raid6_pq dm_mirror dm_region_hash dm_log_writes dm_log_userspace dm_log dm_integrity async_xor async_tx xor dm_flakey dm_delay dm_crypt dm_cache_smq dm_cache dm_persistent_data libcrc32c dm_bufio dm_bio_prison dm_mod firewire_core crc_itu_t hid_sunplus hid_sony hid_samsung hid_pl hid_petalynx hid_monterey hid_microsoft hid_logitech_dj hid_logitech ff_memless hid_gyration hid_ezkey hid_cypress hid_chicony hid_cherry hid_belkin hid_apple hid_a4tech sl811_hcd ohci_hcd uhci_hcd uas usb_storage xhci_plat_hcd pata_sl82c105 pata_via pata_jmicron

Feb 11 16:57:21 kernel:  pata_marvell pata_netcell pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_oldpiix pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_sil680 pata_pdc2027x pata_mpiix lpfc nvmet_fc qla2xxx megaraid_mbox megaraid_mm aacraid sx8 hpsa 3w_9xxx 3w_xxxx 3w_sas mptsas mptfc scsi_transport_fc atp870u dc395x qla1280 dmx3191d sym53c8xx gdth initio BusLogic arcmsr aic7xxx aic79xx sr_mod cdrom sg sd_mod mpt3sas raid_class scsi_transport_sas megaraid megaraid_sas mptspi mptscsih mptbase scsi_transport_spi pdc_adma sata_inic162x sata_mv sata_qstor sata_vsc sata_uli sata_sis pata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24

Feb 11 16:57:21 kernel:  sata_sil sata_promise ata_piix ahci libahci nvme_fc nvme_loop nvmet nvme_rdma rdma_cm iw_cm ib_cm ib_core configfs ipv6 crc_ccitt nvme_fabrics nvme nvme_core

Feb 11 16:57:21 kernel: CPU: 1 PID: 0 Comm: swapper/1 Tainted: G        W         4.19.97-gentoo-x86_64 #1

Feb 11 16:57:21 kernel: Hardware name: Supermicro AS -1114S-WTRT/H12SSW-NT, BIOS 1.0b 11/15/2019

Feb 11 16:57:21 kernel: RIP: 0010:tcp_wfree.cold+0xc/0x13

Feb 11 16:57:21 kernel: Code: 9d 04 00 00 00 5b c6 85 9b 04 00 00 00 5d c3 48 c7 c7 70 93 06 b0 e8 f7 f7 94 ff 0f 0b c3 48 c7 c7 70 93 06 b0 e8 e8 f7 94 ff <0f> 0b e9 46 a5 ff ff 48 c7 c7 70 93 06 b0 e8 d5 f7 94 ff 0f 0b b8

Feb 11 16:57:21 kernel: RSP: 0018:ffff9e9c2b043d90 EFLAGS: 00010246

Feb 11 16:57:21 kernel: RAX: 0000000000000024 RBX: ffff9e9b3ac956e8 RCX: 0000000000000000

Feb 11 16:57:21 kernel: RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 0000000000000300

Feb 11 16:57:21 kernel: RBP: ffff9e9b2028e600 R08: ffff9e9c2b0568b8 R09: 0000000000000001

Feb 11 16:57:21 kernel: R10: 0000000000000000 R11: 0000000000000001 R12: ffff9e9b3ac956e8

Feb 11 16:57:21 kernel: R13: ffff9e9a900100a8 R14: ffff9e9c0155a8c0 R15: 00000000000001ec

Feb 11 16:57:21 kernel: FS:  0000000000000000(0000) GS:ffff9e9c2b040000(0000) knlGS:0000000000000000

Feb 11 16:57:21 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

Feb 11 16:57:21 kernel: CR2: 00007f8e19ca3566 CR3: 00000007cf314000 CR4: 0000000000340ee0

Feb 11 16:57:21 kernel: Call Trace:

Feb 11 16:57:21 kernel:  <IRQ>

Feb 11 16:57:21 kernel:  skb_release_head_state+0x64/0xb0

Feb 11 16:57:21 kernel:  skb_release_all+0xe/0x30

Feb 11 16:57:21 kernel:  consume_skb+0x27/0x80

Feb 11 16:57:21 kernel:  bnxt_tx_int+0xd0/0x360 [bnxt_en]

Feb 11 16:57:21 kernel:  bnxt_poll+0x20f/0x870 [bnxt_en]

Feb 11 16:57:21 kernel:  net_rx_action+0x148/0x3b0

Feb 11 16:57:21 kernel:  __do_softirq+0xe8/0x2f1

Feb 11 16:57:21 kernel:  irq_exit+0x100/0x110

Feb 11 16:57:21 kernel:  do_IRQ+0x81/0xe0

Feb 11 16:57:21 kernel:  common_interrupt+0xf/0xf

Feb 11 16:57:21 kernel:  </IRQ>

Feb 11 16:57:21 kernel: RIP: 0010:cpuidle_enter_state+0xc3/0x320

Feb 11 16:57:21 kernel: Code: e8 82 68 a0 ff 80 7c 24 0b 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 30 02 00 00 31 ff e8 84 55 a6 ff fb 66 0f 1f 44 00 00 <48> ba cf f7 53 e3 a5 9b c4 20 4c 29 f5 48 89 e8 48 c1 fd 3f 48 f7

Feb 11 16:57:21 kernel: RSP: 0018:ffffb4440016fe80 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6

Feb 11 16:57:21 kernel: RAX: ffff9e9c2b062200 RBX: ffff9e9c02df8800 RCX: 000000000000001f

Feb 11 16:57:21 kernel: RDX: 0000000000000000 RSI: 000000002c234d74 RDI: 0000000000000000

Feb 11 16:57:21 kernel: RBP: 00000311ace513f9 R08: 00000311ace513f9 R09: 000000000000018e

Feb 11 16:57:21 kernel: R10: 0000000000002159 R11: ffff9e9c2b0612e8 R12: 0000000000000002

Feb 11 16:57:21 kernel: R13: ffffffffb03954a0 R14: 00000311aaa90c77 R15: ffff9e950448cc80

Feb 11 16:57:21 kernel:  do_idle+0x1dc/0x270

Feb 11 16:57:21 kernel:  cpu_startup_entry+0x6f/0x80

Feb 11 16:57:21 kernel:  start_secondary+0x1a7/0x200

Feb 11 16:57:21 kernel:  secondary_startup_64+0xb6/0xc0

Feb 11 16:57:21 kernel: ---[ end trace 828aa59c66af6560 ]---

Feb 11 16:58:11 kernel: ------------[ cut here ]------------

Feb 11 16:58:11 kernel: WARNING: CPU: 3 PID: 0 at net/ipv4/tcp_output.c:915 tcp_wfree.cold+0xc/0x13

Feb 11 16:58:11 kernel: Modules linked in: autofs4 nfnetlink_queue xt_mac xt_REDIRECT xt_limit xt_nat xt_recent xt_statistic xt_connmark xt_comment xt_iprange l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel xt_set xt_NFQUEUE xt_AUDIT ipt_REJECT nf_reject_ipv4 xt_addrtype bridge stp llc xt_mark xt_TCPMSS xt_hashlimit xt_CT xt_multiport nfnetlink_log xt_NFLOG nf_log_ipv4 nf_log_common xt_LOG nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp pppoe pppox ppp_generic slhc ip_set_hash_mac ip_set_bitmap_port

Feb 11 16:58:11 kernel:  ip_set_hash_net ip_set_hash_ip ip_set nfnetlink ip6table_filter ip6_tables arptable_filter arp_tables xt_conntrack iptable_mangle iptable_nat nf_nat_ipv4 nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 iptable_raw sch_fq tcp_cdg tcp_bbr iptable_filter ip_tables bpfilter mlx5_ib ipmi_ssif ib_uverbs edac_mce_amd kvm_amd kvm ast ttm irqbypass crct10dif_pclmul efi_pstore ghash_clmulni_intel drm_kms_helper pcspkr efivars ixgbe igb sp5100_tco mlx5_core drm joydev bnxt_en i2c_algo_bit mdio i2c_piix4 mlxfw ccp dca i2c_core ipmi_si ipmi_devintf ipmi_msghandler pinctrl_amd pcc_cpufreq acpi_cpufreq mac_hid efivarfs aesni_intel crypto_simd cryptd glue_helper aes_x86_64 algif_rng algif_aead algif_hash algif_skcipher af_alg crc32c_intel crc32_pclmul crc32_generic msdos fat cramfs overlay squashfs

Feb 11 16:58:11 kernel:  loop fuse f2fs xfs nfs lockd grace sunrpc fscache jfs reiserfs btrfs ext4 mbcache jbd2 multipath linear raid10 raid1 raid0 dm_zero dm_verity reed_solomon dm_thin_pool dm_switch dm_snapshot dm_raid raid456 md_mod async_raid6_recov async_memcpy async_pq raid6_pq dm_mirror dm_region_hash dm_log_writes dm_log_userspace dm_log dm_integrity async_xor async_tx xor dm_flakey dm_delay dm_crypt dm_cache_smq dm_cache dm_persistent_data libcrc32c dm_bufio dm_bio_prison dm_mod firewire_core crc_itu_t hid_sunplus hid_sony hid_samsung hid_pl hid_petalynx hid_monterey hid_microsoft hid_logitech_dj hid_logitech ff_memless hid_gyration hid_ezkey hid_cypress hid_chicony hid_cherry hid_belkin hid_apple hid_a4tech sl811_hcd ohci_hcd uhci_hcd uas usb_storage xhci_plat_hcd pata_sl82c105 pata_via pata_jmicron

Feb 11 16:58:11 kernel:  pata_marvell pata_netcell pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_oldpiix pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_sil680 pata_pdc2027x pata_mpiix lpfc nvmet_fc qla2xxx megaraid_mbox megaraid_mm aacraid sx8 hpsa 3w_9xxx 3w_xxxx 3w_sas mptsas mptfc scsi_transport_fc atp870u dc395x qla1280 dmx3191d sym53c8xx gdth initio BusLogic arcmsr aic7xxx aic79xx sr_mod cdrom sg sd_mod mpt3sas raid_class scsi_transport_sas megaraid megaraid_sas mptspi mptscsih mptbase scsi_transport_spi pdc_adma sata_inic162x sata_mv sata_qstor sata_vsc sata_uli sata_sis pata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24

Feb 11 16:58:11 kernel:  sata_sil sata_promise ata_piix ahci libahci nvme_fc nvme_loop nvmet nvme_rdma rdma_cm iw_cm ib_cm ib_core configfs ipv6 crc_ccitt nvme_fabrics nvme nvme_core

Feb 11 16:58:11 kernel: CPU: 3 PID: 0 Comm: swapper/3 Tainted: G        W         4.19.97-gentoo-x86_64 #1

Feb 11 16:58:11 kernel: Hardware name: Supermicro AS -1114S-WTRT/H12SSW-NT, BIOS 1.0b 11/15/2019

Feb 11 16:58:11 kernel: RIP: 0010:tcp_wfree.cold+0xc/0x13

Feb 11 16:58:11 kernel: Code: 9d 04 00 00 00 5b c6 85 9b 04 00 00 00 5d c3 48 c7 c7 70 93 06 b0 e8 f7 f7 94 ff 0f 0b c3 48 c7 c7 70 93 06 b0 e8 e8 f7 94 ff <0f> 0b e9 46 a5 ff ff 48 c7 c7 70 93 06 b0 e8 d5 f7 94 ff 0f 0b b8

Feb 11 16:58:11 kernel: RSP: 0018:ffff9e9c2b0c3d90 EFLAGS: 00010246

Feb 11 16:58:11 kernel: RAX: 0000000000000024 RBX: ffff9e9c041278e8 RCX: 0000000000000000

Feb 11 16:58:11 kernel: RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 0000000000000300

Feb 11 16:58:11 kernel: RBP: ffff9e9bbefa7700 R08: ffff9e9c2b0d68b8 R09: 0000000000000001

Feb 11 16:58:11 kernel: R10: 0000000000000000 R11: 0000000000000001 R12: ffff9e9c041278e8

Feb 11 16:58:11 kernel: R13: ffff9e9a900100a8 R14: ffff9e9c0155a8c0 R15: 00000000000001aa

Feb 11 16:58:11 kernel: FS:  0000000000000000(0000) GS:ffff9e9c2b0c0000(0000) knlGS:0000000000000000

Feb 11 16:58:11 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

Feb 11 16:58:11 kernel: CR2: 00007ff0d0235c28 CR3: 0000000708fea000 CR4: 0000000000340ee0

Feb 11 16:58:11 kernel: Call Trace:

Feb 11 16:58:11 kernel:  <IRQ>

Feb 11 16:58:11 kernel:  skb_release_head_state+0x64/0xb0

Feb 11 16:58:11 kernel:  skb_release_all+0xe/0x30

Feb 11 16:58:11 kernel:  consume_skb+0x27/0x80

Feb 11 16:58:11 kernel:  bnxt_tx_int+0xd0/0x360 [bnxt_en]

Feb 11 16:58:11 kernel:  bnxt_poll+0x20f/0x870 [bnxt_en]

Feb 11 16:58:11 kernel:  net_rx_action+0x148/0x3b0

Feb 11 16:58:11 kernel:  __do_softirq+0xe8/0x2f1

Feb 11 16:58:11 kernel:  irq_exit+0x100/0x110

Feb 11 16:58:11 kernel:  do_IRQ+0x81/0xe0

Feb 11 16:58:11 kernel:  common_interrupt+0xf/0xf

Feb 11 16:58:11 kernel:  </IRQ>

Feb 11 16:58:11 kernel: RIP: 0010:cpuidle_enter_state+0xc3/0x320

Feb 11 16:58:11 kernel: Code: e8 82 68 a0 ff 80 7c 24 0b 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 30 02 00 00 31 ff e8 84 55 a6 ff fb 66 0f 1f 44 00 00 <48> ba cf f7 53 e3 a5 9b c4 20 4c 29 f5 48 89 e8 48 c1 fd 3f 48 f7

Feb 11 16:58:11 kernel: RSP: 0018:ffffb44400207e80 EFLAGS: 00000246 ORIG_RAX: ffffffffffffffd6

Feb 11 16:58:11 kernel: RAX: ffff9e9c2b0e2200 RBX: ffff9e9c02df9c00 RCX: 000000000000001f

Feb 11 16:58:11 kernel: RDX: 0000000000000000 RSI: 000000002c234d74 RDI: 0000000000000000

Feb 11 16:58:11 kernel: RBP: 0000031d6a89bd5e R08: 0000031d6a89bd5e R09: ffff9e9c2b0dd7a0

Feb 11 16:58:11 kernel: R10: 000000000000a282 R11: ffff9e9c2b0e12e8 R12: 0000000000000002

Feb 11 16:58:11 kernel: R13: ffffffffb03954a0 R14: 0000031d69082e46 R15: ffff9e95044a2640

Feb 11 16:58:11 kernel:  do_idle+0x1dc/0x270

Feb 11 16:58:11 kernel:  cpu_startup_entry+0x6f/0x80

Feb 11 16:58:11 kernel:  start_secondary+0x1a7/0x200

Feb 11 16:58:11 kernel:  secondary_startup_64+0xb6/0xc0

Feb 11 16:58:11 kernel: ---[ end trace 828aa59c66af6561 ]---
```

Lots more after that...

Why is the kernel tainted starting from the second trace?

"W": if a warning has previously been issued by the kernel. 

"G": GPL modules

OK, so that's all is needed to taint a kernel?

Should I send the first trace to a Linux Kernel mailing list?

----------

## Hu

 *Vieri wrote:*   

> Sorry, but could you explain this again in layman's terms? How did you exactly see which modules were out-of-tree and tainting the kernel?

 The kernel marked them with a trailing (O) when it listed your modules.  The regex deleted all the unmarked modules, leaving the offending ones with nowhere to hide. *Vieri wrote:*   

> I then rebooted the system, and all seemed OK except I mistakenly thought that the kernel would actively deny any userland program to manually load the modules.

 Sorry.  I've never needed to blacklist modules, so I wasn't aware of that quirk.  I would have warned you if I had known that blacklisting was insufficient. *Vieri wrote:*   

> Now, both /proc/modules and syslog do NOT report geoip loading (as well as the other two), and there is no sign in the logs that the kernel is tainted.

 Good. *Vieri wrote:*   

>  *Tony0945 wrote:*   So why load the driver if there is no hardware to use it? It is not loaded.
> 
> ```
> # lsmod | grep e3000
> ```
> ...

 e3000?  Shouldn't you be using e1000 here? *Vieri wrote:*   

> I didn't have to wait long...
> 
> Here's how it all starts:
> 
> Why is the kernel tainted starting from the second trace?
> ...

 Correct.  The theory is that warnings are not issued idly, so if you got a warning before, something is already weird, and the second diagnostic could be merely fallout from the first problem leaving a mess.

 *Vieri wrote:*   

> Should I send the first trace to a Linux Kernel mailing list?

 Yes, although you might get better results if you can reproduce the problem on a v5.5.x kernel, since that is much more recent.  You could start with what you have shown here.  Just be prepared for a request to test something recent.

----------

## Vieri

 *Hu wrote:*   

> Shouldn't you be using e1000 here?

 

Doesn't really matter. I don't need any of these on my current hardware.

# lsmod | egrep "e1000|e3000"

Shows nothing.

I posted the non-tainted message to the LKML.

As Neddy said earlier, I need to pinpoint which commit caused this. 

So now I 'm trying to find one that "works".

I'm supposing that kernel 4.12.12 worked for me, but I need to confirm it.

Portage has 4.14.156 and 4.9.210 (gentoo-sources), so I'm starting off with 4.9.210. If that doesn't give me any issues, I'll try 4.14.156.

Then I'll also try the latest 5.x (but I already did that on another hardware and had the same results, but I'll do it anyway just to confirm).

If I wanted to emerge + use genkernel version 4.14.14, what's the easiest way? Custom overlay/ebuild?

----------

## Vieri

When installing multiple kernels on a machine, I understand that version-specific kernel modules are placed in a separate /lib/modules directory.

However, what about packages such as iptables that depend on the kernel that's being booted?

If I re-emerge iptables for a newly-built kernel then I may end up with a non-working iptables for the previous kernel, right?

```
# equery files iptables

 * Searching for iptables ...

 * Contents of net-firewall/iptables-1.6.1-r3:

/etc

/etc/conf.d

/etc/conf.d/ip6tables

/etc/conf.d/iptables

/etc/init.d

/etc/init.d/ip6tables

/etc/init.d/iptables

/lib

/lib/systemd

/lib/systemd/system

/lib/systemd/system/ip6tables-restore.service

/lib/systemd/system/ip6tables-store.service

/lib/systemd/system/iptables-restore.service

/lib/systemd/system/iptables-store.service

/lib64

/lib64/libip4tc.so.0 -> libip4tc.so.0.1.0

/lib64/libip4tc.so.0.1.0

/lib64/libip6tc.so.0 -> libip6tc.so.0.1.0

/lib64/libip6tc.so.0.1.0

/lib64/libiptc.so.0 -> libiptc.so.0.0.0

/lib64/libiptc.so.0.0.0

/lib64/libxtables.so.12 -> libxtables.so.12.0.0

/lib64/libxtables.so.12.0.0

/sbin

/sbin/ip6tables -> xtables-multi

/sbin/ip6tables-apply -> iptables-apply

/sbin/ip6tables-restore -> xtables-multi

/sbin/ip6tables-save -> xtables-multi

/sbin/iptables -> xtables-multi

/sbin/iptables-apply

/sbin/iptables-restore -> xtables-multi

/sbin/iptables-save -> xtables-multi

/sbin/xtables-multi

/usr

/usr/bin

/usr/bin/iptables-xml -> /sbin/xtables-multi

/usr/include

/usr/include/ip6tables.h

/usr/include/iptables

/usr/include/iptables.h

/usr/include/iptables/internal.h

/usr/include/libiptc

/usr/include/libiptc/ipt_kernel_headers.h

/usr/include/libiptc/libip6tc.h

/usr/include/libiptc/libiptc.h

/usr/include/libiptc/libxtc.h

/usr/include/libiptc/xtcshared.h

/usr/include/xtables-version.h

/usr/include/xtables.h

/usr/lib64

/usr/lib64/libip4tc.so

/usr/lib64/libip6tc.so

/usr/lib64/libiptc.so

/usr/lib64/libxtables.so

/usr/lib64/pkgconfig

/usr/lib64/pkgconfig/libip4tc.pc

/usr/lib64/pkgconfig/libip6tc.pc

/usr/lib64/pkgconfig/libiptc.pc

/usr/lib64/pkgconfig/xtables.pc

/usr/lib64/xtables

/usr/lib64/xtables/libebt_802_3.so

/usr/lib64/xtables/libebt_ip.so

/usr/lib64/xtables/libebt_log.so

/usr/lib64/xtables/libebt_mark_m.so

/usr/lib64/xtables/libip6t_DNAT.so

/usr/lib64/xtables/libip6t_DNPT.so

/usr/lib64/xtables/libip6t_HL.so

/usr/lib64/xtables/libip6t_LOG.so

/usr/lib64/xtables/libip6t_MASQUERADE.so

/usr/lib64/xtables/libip6t_NETMAP.so

/usr/lib64/xtables/libip6t_REDIRECT.so

/usr/lib64/xtables/libip6t_REJECT.so

/usr/lib64/xtables/libip6t_SNAT.so

/usr/lib64/xtables/libip6t_SNPT.so

/usr/lib64/xtables/libip6t_ah.so

/usr/lib64/xtables/libip6t_dst.so

/usr/lib64/xtables/libip6t_eui64.so

/usr/lib64/xtables/libip6t_frag.so

/usr/lib64/xtables/libip6t_hbh.so

/usr/lib64/xtables/libip6t_hl.so

/usr/lib64/xtables/libip6t_icmp6.so

/usr/lib64/xtables/libip6t_ipv6header.so

/usr/lib64/xtables/libip6t_mh.so

/usr/lib64/xtables/libip6t_rt.so

/usr/lib64/xtables/libipt_CLUSTERIP.so

/usr/lib64/xtables/libipt_DNAT.so

/usr/lib64/xtables/libipt_ECN.so

/usr/lib64/xtables/libipt_LOG.so

/usr/lib64/xtables/libipt_MASQUERADE.so

/usr/lib64/xtables/libipt_NETMAP.so

/usr/lib64/xtables/libipt_REDIRECT.so

/usr/lib64/xtables/libipt_REJECT.so

/usr/lib64/xtables/libipt_SNAT.so

/usr/lib64/xtables/libipt_TTL.so

/usr/lib64/xtables/libipt_ULOG.so

/usr/lib64/xtables/libipt_ah.so

/usr/lib64/xtables/libipt_icmp.so

/usr/lib64/xtables/libipt_realm.so

/usr/lib64/xtables/libipt_ttl.so

/usr/lib64/xtables/libxt_AUDIT.so

/usr/lib64/xtables/libxt_CHECKSUM.so

/usr/lib64/xtables/libxt_CLASSIFY.so

/usr/lib64/xtables/libxt_CONNMARK.so

/usr/lib64/xtables/libxt_CONNSECMARK.so

/usr/lib64/xtables/libxt_CT.so

/usr/lib64/xtables/libxt_DSCP.so

/usr/lib64/xtables/libxt_HMARK.so

/usr/lib64/xtables/libxt_IDLETIMER.so

/usr/lib64/xtables/libxt_LED.so

/usr/lib64/xtables/libxt_MARK.so

/usr/lib64/xtables/libxt_NFLOG.so

/usr/lib64/xtables/libxt_NFQUEUE.so

/usr/lib64/xtables/libxt_NOTRACK.so

/usr/lib64/xtables/libxt_RATEEST.so

/usr/lib64/xtables/libxt_SECMARK.so

/usr/lib64/xtables/libxt_SET.so

/usr/lib64/xtables/libxt_SYNPROXY.so

/usr/lib64/xtables/libxt_TCPMSS.so

/usr/lib64/xtables/libxt_TCPOPTSTRIP.so

/usr/lib64/xtables/libxt_TEE.so

/usr/lib64/xtables/libxt_TOS.so

/usr/lib64/xtables/libxt_TPROXY.so

/usr/lib64/xtables/libxt_TRACE.so

/usr/lib64/xtables/libxt_addrtype.so

/usr/lib64/xtables/libxt_bpf.so

/usr/lib64/xtables/libxt_cgroup.so

/usr/lib64/xtables/libxt_cluster.so

/usr/lib64/xtables/libxt_comment.so

/usr/lib64/xtables/libxt_connbytes.so

/usr/lib64/xtables/libxt_connlabel.so

/usr/lib64/xtables/libxt_connlimit.so

/usr/lib64/xtables/libxt_connmark.so

/usr/lib64/xtables/libxt_conntrack.so

/usr/lib64/xtables/libxt_cpu.so

/usr/lib64/xtables/libxt_dccp.so

/usr/lib64/xtables/libxt_devgroup.so

/usr/lib64/xtables/libxt_dscp.so

/usr/lib64/xtables/libxt_ecn.so

/usr/lib64/xtables/libxt_esp.so

/usr/lib64/xtables/libxt_hashlimit.so

/usr/lib64/xtables/libxt_helper.so

/usr/lib64/xtables/libxt_ipcomp.so

/usr/lib64/xtables/libxt_iprange.so

/usr/lib64/xtables/libxt_ipvs.so

/usr/lib64/xtables/libxt_length.so

/usr/lib64/xtables/libxt_limit.so

/usr/lib64/xtables/libxt_mac.so

/usr/lib64/xtables/libxt_mangle.so

/usr/lib64/xtables/libxt_mark.so

/usr/lib64/xtables/libxt_multiport.so

/usr/lib64/xtables/libxt_nfacct.so

/usr/lib64/xtables/libxt_osf.so

/usr/lib64/xtables/libxt_owner.so

/usr/lib64/xtables/libxt_physdev.so

/usr/lib64/xtables/libxt_pkttype.so

/usr/lib64/xtables/libxt_policy.so

/usr/lib64/xtables/libxt_quota.so

/usr/lib64/xtables/libxt_rateest.so

/usr/lib64/xtables/libxt_recent.so

/usr/lib64/xtables/libxt_rpfilter.so

/usr/lib64/xtables/libxt_sctp.so

/usr/lib64/xtables/libxt_set.so

/usr/lib64/xtables/libxt_socket.so

/usr/lib64/xtables/libxt_standard.so

/usr/lib64/xtables/libxt_state.so

/usr/lib64/xtables/libxt_statistic.so

/usr/lib64/xtables/libxt_string.so

/usr/lib64/xtables/libxt_tcp.so

/usr/lib64/xtables/libxt_tcpmss.so

/usr/lib64/xtables/libxt_time.so

/usr/lib64/xtables/libxt_tos.so

/usr/lib64/xtables/libxt_u32.so

/usr/lib64/xtables/libxt_udp.so

/usr/share

/usr/share/doc

/usr/share/doc/iptables-1.6.1-r3

/usr/share/doc/iptables-1.6.1-r3/INCOMPATIBILITIES.bz2

/usr/share/doc/iptables-1.6.1-r3/iptables.xslt.bz2

/usr/share/man

/usr/share/man/man1

/usr/share/man/man1/iptables-xml.1.bz2

/usr/share/man/man8

/usr/share/man/man8/ip6tables-restore.8

/usr/share/man/man8/ip6tables-save.8

/usr/share/man/man8/ip6tables.8

/usr/share/man/man8/iptables-apply.8.bz2

/usr/share/man/man8/iptables-extensions.8.bz2

/usr/share/man/man8/iptables-restore.8.bz2

/usr/share/man/man8/iptables-save.8.bz2

/usr/share/man/man8/iptables.8.bz2

/var

/var/lib

/var/lib/ip6tables

/var/lib/ip6tables/.keep_net-firewall_iptables-0

/var/lib/iptables

/var/lib/iptables/.keep_net-firewall_iptables-0

```

----------

## Tony0945

You can rebuild your out of kernel modules with "emerge @module-rebuild"

Please read this detailed article Gentoo wiki kernel/Upgrade

----------

## Vieri

 *Tony0945 wrote:*   

> You can rebuild your out of kernel modules with "emerge @module-rebuild"

 

Sure, but that's for packages such as xtables-addons. I'm referring to the user-space programs such as iptables.

When compiling/emerging iptables, the current /usr/src/linux directory is scanned. So, my doubt is whether I require to recompile/re-emerge iptables each time I install a new kernel, and whether that implies having issues with the other kernels. The @module-rebuild command does not affect the iptables package, or does it?

----------

## dmpogo

 *Vieri wrote:*   

>  *Tony0945 wrote:*   You can rebuild your out of kernel modules with "emerge @module-rebuild" 
> 
> Sure, but that's for packages such as xtables-addons. I'm referring to the user-space programs such as iptables.
> 
> When compiling/emerging iptables, the current /usr/src/linux directory is scanned. So, my doubt is whether I require to recompile/re-emerge iptables each time I install a new kernel, and whether that implies having issues with the other kernels. The @module-rebuild command does not affect the iptables package, or does it?

 

User space programs do not ( as far as I ever encountered) depend on specific version of kernel.   The depend on particular functionality enabled in the kernel.   So I can see how installing new kernel with required functionality disabled may make iptables fail, but I can't see how it can make it not work with old kernel with which it worked before.

----------

## Vieri

I think I've found the root cause for this issue (not the solution).

The warning messages I reported (which *could* lead to a system hang after a long period running) disappear if I stop using NFQUEUE.

In my specific case I use NFQUEUE balance 0:5 with iptables.

----------

## Vieri

The problem is reproducible when using Suricata (or similar program) in NFQ repeat mode.

It seems to be a netfilter issue.

----------

