# system failing with a "tainted" cpu?

## jyoung

Hi,

My NFS server keeps crashing. When it does, messages scroll over the screen every few seconds, including

CPU ## PID ## comm nfsd Tainted G D W L

Does anyone have any idea what could cause this?

thanks

----------

## mike155

Please post the output of

```
emerge --info
```

It will tell us how your machine is set up.

----------

## Ant P.

 */usr/src/linux/Documentation/admin-guide/tainted-kernels.rst wrote:*   

> 
> 
> ```
> 
>                                          Table for decoding tainted state
> ...

 

So your server is running proprietary drivers, had recent errors, warnings and hangs. The first is usually the cause of the others.

----------

## jyoung

Here's the output of emerge --info

```
Portage 2.3.40 (python 3.5.5-final-0, default/linux/amd64/17.0/desktop, gcc-7.3.0, glibc-2.27-r4, 4.12.4-gentoo x86_64)

=================================================================

System uname: Linux-4.12.4-gentoo-x86_64-Intel-R-_Xeon-R-_CPU_E5607_@_2.27GHz-with-gentoo-2.6

KiB Mem:    55549004 total,  33306412 free

KiB Swap:   16777212 total,  16777212 free

Timestamp of repository gentoo: Mon, 18 Jun 2018 01:30:01 +0000

Head commit of repository gentoo: b51261d5d1bd0325f97c91ec87db4f0ea3fd8846

sh bash 4.4_p23

ld GNU ld (Gentoo 2.28 p1.2) 2.28

app-shells/bash:          4.4_p23::gentoo

dev-lang/perl:            5.26.2::gentoo

dev-lang/python:          2.7.15::gentoo, 3.4.8-r1::gentoo, 3.5.5-r1::gentoo

dev-util/cmake:           3.11.4::gentoo

dev-util/pkgconfig:       0.29.2::gentoo

sys-apps/baselayout:      2.6::gentoo

sys-apps/openrc:          0.37::gentoo

sys-apps/sandbox:         2.13::gentoo

sys-devel/autoconf:       2.13::gentoo, 2.69-r4::gentoo

sys-devel/automake:       1.13.4-r1::gentoo, 1.15.1::gentoo, 1.16.1-r1::gentoo

sys-devel/binutils:       2.28-r2::gentoo, 2.28.1::gentoo, 2.30-r3::gentoo

sys-devel/gcc:            5.4.0-r3::gentoo, 6.4.0-r1::gentoo, 7.3.0-r3::gentoo

sys-devel/gcc-config:     1.9.1::gentoo

sys-devel/libtool:        2.4.6-r5::gentoo

sys-devel/make:           4.2.1-r3::gentoo

sys-kernel/linux-headers: 4.16-r1::gentoo (virtual/os-headers)

sys-libs/glibc:           2.27-r4::gentoo

Repositories:

gentoo

    location: /usr/portage

    sync-type: rsync

    sync-uri: rsync://rsync.gentoo.org/gentoo-portage

    priority: -1000

    sync-rsync-verify-metamanifest: yes

    sync-rsync-verify-max-age: 24

    sync-rsync-extra-opts: 

    sync-rsync-verify-jobs: 1

ACCEPT_KEYWORDS="amd64 ~amd64"

ACCEPT_LICENSE="*"

CBUILD="x86_64-pc-linux-gnu"

CC="gcc"

CFLAGS="-O2 -pipe"

CHOST="x86_64-pc-linux-gnu"

CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"

CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"

CXXFLAGS="-O2 -pipe"

DISTDIR="/usr/portage/distfiles"

ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"

FCFLAGS="-O2 -pipe"

FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"

FFLAGS="-O2 -pipe"

GENTOO_MIRRORS="http://mirror.lug.udel.edu/pub/gentoo/ ftp://cosmos.illinois.edu/pub/gentoo/"

LANG="en_US.utf8"

LDFLAGS="-Wl,-O1 -Wl,--as-needed"

MAKEOPTS="-j14"

PKGDIR="/usr/portage/packages"

PORTAGE_CONFIGROOT="/"

PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"

PORTAGE_TMPDIR="/var/tmp"

USE="X a52 aac acl acpi alsa amd64 apulse berkdb bluetooth branding bzip2 cairo cdda cddb cdr cli consolekit crypt cups cxx dbus dri dts dvd dvdr emboss encode exif f77 fam flac fortran gdbm gif glamor gpm gtk iconv ipv6 jpeg lcms ldap libnotify libtirpc mad mng modules mp3 mp4 mpeg multilib ncurses nls nptl ogg opengl openmp pango pcre pdf png policykit ppds qt3support qt5 readline sdl seccomp spell ssl startup-notification svg tcpd tiff truetype udev udisks unicode upower usb vorbis wxwidgets x264 xattr xcb xml xv xvid zlib ~amd64" ABI_X86="64 32" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon plan sheets stage words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python2_7 python3_5" RUBY_TARGETS="ruby23" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy v4l" XFCE_PLUGINS="clock" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"

Unset:  CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

```

----------

## Hu

G/P is a weird one.  It will be P if the user has loaded proprietary drivers, G if no proprietary drivers, but some taint flag is set, or the string "Not tainted" if there are no bits set at all.

----------

## Ant P.

 *Hu wrote:*   

> G/P is a weird one.  It will be P if the user has loaded proprietary drivers, G if no proprietary drivers, but some taint flag is set, or the string "Not tainted" if there are no bits set at all.

 

Right, probably nothing to do with proprietary drivers then.

The fact that the kernel in use hasn't even been available in the main gentoo tree for over two years might be related to the symptoms, though.

----------

## bunder

 *Quote:*   

> CPU ## PID ## comm nfsd Tainted G D W L 

 

we need to see the complete error(s) from dmesg, that one line isn't enough to tell us what happened

----------

## jyoung

Here's the top of the output from dmesg. It keeps going with similar messages from other CPU #s.

```
[60001.867437] CPU: 0 PID: 4368 Comm: nfsd Tainted: G        W       4.12.4-gentoo #3

[60001.867438] Hardware name: Dell Inc. Precision WorkStation T5500  /0CRH6C, BIOS A10 07/22/2011

[60001.867439] task: ffff900ca4748a00 task.stack: ffffa434c1810000

[60001.867449] RIP: 0010:nfsd4_open+0x248/0x6a0 [nfsd]

[60001.867450] RSP: 0018:ffffa434c1813d90 EFLAGS: 00010286

[60001.867451] RAX: 0000000000000043 RBX: ffff900c9a15b068 RCX: 0000000000000000

[60001.867452] RDX: ffff900cb06149a8 RSI: ffff900cb060cc28 RDI: ffff900cb060cc28

[60001.867453] RBP: ffffa434c1813dc0 R08: 0000000000000001 R09: 0000000000001e33

[60001.867454] R10: 0000000000000852 R11: 0000000000001e33 R12: 0000000018270000

[60001.867455] R13: ffff900c9a15a3e0 R14: ffff900ca20f1400 R15: ffff900ac71d4600

[60001.867457] FS:  0000000000000000(0000) GS:ffff900cb0600000(0000) knlGS:0000000000000000

[60001.867458] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033

[60001.867459] CR2: 000000000170c440 CR3: 000000067d409000 CR4: 00000000000006f0

[60001.867460] Call Trace:

[60001.867470]  nfsd4_proc_compound+0x33e/0x590 [nfsd]

[60001.867477]  nfsd_dispatch+0x9e/0x1b0 [nfsd]

[60001.867503]  svc_process_common+0x425/0x5b0 [sunrpc]

[60001.867514]  svc_process+0x12e/0x1a0 [sunrpc]

[60001.867519]  nfsd+0xe9/0x150 [nfsd]

[60001.867524]  kthread+0x109/0x140

[60001.867529]  ? nfsd_destroy+0x60/0x60 [nfsd]

[60001.867531]  ? kthread_create_on_node+0x40/0x40

[60001.867534]  ret_from_fork+0x27/0x40

[60001.867535] Code: de 4c 89 f7 e8 ea e0 00 00 85 c0 41 89 c4 74 1d 41 80 bd 15 01 00 00 00 74 13 44 89 e6 48 c7 c7 20 7e 12 c1 0f ce e8 2e e9 24 e2 <0f> ff 4d 85 ff

 74 55 49 39 df 74 50 48 89 df e8 94 22 ff ff 49 

[60001.867562] ---[ end trace 40ae1fd4e3a25c94 ]---

[60002.058695] nfsd4_process_open2 failed to open newly-created file! status=10008

[60002.058705] ------------[ cut here ]------------

[60002.058716] WARNING: CPU: 2 PID: 4370 at fs/nfsd/nfs4proc.c:450 nfsd4_open+0x248/0x6a0 [nfsd]

[60002.058717] Modules linked in: rpcsec_gss_krb5 nfsd auth_rpcgss nfs_acl ipv6 crc_ccitt cfg80211 rfkill 8021q garp stp llc snd_hda_codec_hdmi snd_hda_codec_analog snd_hda_codec_generic dell_wmi coretemp snd_hda_intel radeon snd_hda_codec dell_smbios dcdbas kvm ttm snd_hda_core snd_hwdep sparse_keymap video dell_smm_hwmon drm_kms_helper iTCO_wdt iTCO_vendor_support irqbypass crc32c_intel snd_pcm ppdev ghash_clmulni_intel drm snd_timer syscopyarea cryptd sysfillrect wmi parport_pc snd serio_raw firewire_ohci pcspkr sysimgblt fb_sys_fops soundcore i7core_edac i2c_i801 lpc_ich mfd_core xts cbc sha256_generic ixgb ixgbe tulip cxgb3 cxgb mdio cxgb4 vxge bonding vxlan ip6_udp_tunnel udp_tunnel macvlan vmxnet3 virtio_net virtio_ring virtio tg3 sky2 r8169 pcnet32 mii igb ptp pps_core dca i2c_algo_bit i2c_core

[60002.058757]  e1000 bnx2 atl1c msdos fat configfs cramfs squashfs fuse xfs nfs lockd grace sunrpc fscache jfs reiserfs btrfs ext4 jbd2 ext2 mbcache linear raid10 raid1 raid0 dm_raid raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c dm_snapshot dm_bufio dm_crypt dm_mirror dm_region_hash dm_log dm_mod dax firewire_core crc_itu_t sl811_hcd xhci_pci xhci_hcd usb_storage mpt3sas raid_class aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid sx8 hpsa cciss 3w_9xxx 3w_xxxx 3w_sas mptsas scsi_transport_sas mptfc scsi_transport_fc mptspi mptscsih mptbase imm parport sym53c8xx initio arcmsr aic7xxx aic79xx scsi_transport_spi sr_mod cdrom sg sd_mod pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4

[60002.058801]  sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_pdc202xx_old pata_atiixp pata_amd pata_ali pata_it8213 pata_pcmcia pata_serverworks pata_oldpiix pata_artop pata_it821x pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_sil680 pata_pdc2027x nvme nvme_core

```

----------

## Ant P.

This looks like https://bugzilla.kernel.org/show_bug.cgi?id=195725

----------

