# High load, but the machine is not doing anything?

## pgu

```
uptime

 11:21:06 up 58 days,  2:35,  4 users,  load average: 6.62, 6.88, 6.96
```

The system is not swapping:

```
vmstat 5 5

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----

 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa

 1  6      0 309584 4359604 13452248    0    0     6    53    0    0  1  0 98  0

 1  6      0 309752 4359604 13452248    0    0     0     0 21624 60716  0  3 73 24

 1  6      0 309432 4359604 13452248    0    0     0     0 21624 60664  0  3 73 24

 1  6      0 309304 4359604 13452248    0    0     0     0 21651 60790  0  3 73 24

 2  6      0 309152 4359604 13452248    0    0     0     0 21629 60715  0  3 73 24
```

ps shows all the running processes are in the D state.

What should I do to debug the system?

----------

## pgu

I do see a lot of interrupts occurring on eth1, which is where the root file system is located (NFS)

----------

## pgu

Some more info:

```
uname -srvmpio

Linux 2.6.38-gentoo-r6 #30 SMP Mon Jul 25 17:40:03 CEST 2011 x86_64 Intel(R) Core(TM) i7 CPU X 990 @ 3.47GHz GenuineIntel GNU/Linux
```

```
grep 'model name' /proc/cpuinfo 

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz

model name      : Intel(R) Core(TM) i7 CPU       X 990  @ 3.47GHz
```

----------

## Hu

What processes are in the D state?  Why are they blocked?

----------

## wazoo42

I also have the same problem on a phenom II x6. I did notice that there are 12 of my default user account logged in, and I'm not sure why there are so many by default (in the past I only had a few). Here's the output from top, which shows 20-50% wasted cpu and a load of at least 4. I can close firefox, chromium, dolphin, and okular so that no user apps are running, but the high load persists.

```

top - 13:32:39 up 58 min, 12 users,  load average: 2.75, 4.14, 4.57

Tasks: 186 total,   1 running, 185 sleeping,   0 stopped,   0 zombie

Cpu(s):  2.1%us,  0.9%sy,  0.0%ni, 74.4%id, 22.6%wa,  0.0%hi,  0.0%si,  0.0%st

Mem:  16370908k total,  3528204k used, 12842704k free,   216644k buffers

Swap:        0k total,        0k used,        0k free,   775616k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                               

27474 benjfitz  20   0  813m 351m  35m S    5  2.2   1:39.74 firefox                                                                               

27894 benjfitz  20   0  532m  66m  25m S    4  0.4   0:47.74 chrome                                                                                

 6208 root      20   0  943m 572m 479m S    2  3.6   2:24.29 X                                                                                     

 6454 benjfitz  20   0  490m  28m  16m S    2  0.2   1:01.74 knotify4                                                                              

 6512 benjfitz  20   0  406m  30m  14m S    1  0.2   0:18.46 konsole                                                                               

 6462 benjfitz  20   0  581m  43m  24m S    1  0.3   0:14.94 plasma-desktop                                                                        

27710 benjfitz  20   0  954m  92m  20m S    1  0.6   0:14.62 chrome                                                                                

27764 benjfitz  20   0  918m  52m  17m S    1  0.3   0:02.43 chrome                 

...

```

----------

## Hu

wazoo42: do you have any processes in D state?  What are they doing?

Some terminal programs register as a login, in which case you get a "new user" every time you open a new terminal window.

----------

## wazoo42

Hmmmm...it looks like the high load and lagginess are the only similarity. ps aux doesn't show any processes in the D state. That being said, I wish I knew what processes were causing the high load (it jumps to 15 every once in a while).

Thanks for the hint regarding PTS logins. I do keep a bunch of konsole tabs open.

----------

## Hu

Are you using transparent hugepages?  In some configurations, that can cause the kernel to put processes into a wait state for an extended period while it writes out those pages.  I find that behavior annoying, so I do not run that configuration.  Thus, I am unsure whether a process in that state will show as being in D state.

----------

## wazoo42

I do have hugepage support. I will try eliminating it and report back, but I have had that compiled into the kernel for the past year or more and it is only in the past week that the high load problem cropped up.

----------

## pgu

 *Hu wrote:*   

> What processes are in the D state?  Why are they blocked?

 

My simulation programs. I don't know why they are block as they are mostly CPU intensive. I also have several other machines, same hardware, same configuration, which does not show this behavior. I have rebooted the machine now so I can't do any more inspection of this state.

----------

## wazoo42

I installed gentoo on a new drive and this problem is no longer present (the kernel config was identical). Either my original drive was going bad (fsck had started finding errors on boot) or it was something particular to that install (I had run emerge -e @world several times though).

----------

