# Sad state of affairs with user process scheduling with Linux

## devsk

I was following up on https://bugzilla.kernel.org/show_bug.cgi?id=12309 , which has been there for more than 3 years, I came across this message from someone who caused user space starvation by just running network traffic through a Linux VM. That person basically fixed the issue by moving to the out-of-tree BFS scheduler from Con Kolivas.

Someone got so upset with my message that they closed the bug. Although the issue is still there. I wanted to be a little blunt but the goal was to draw attention to the fact that in-kernel CFS is bad in scheduling user space processes when kernel is spending a lot of time doing things in kernel space. BFS does a much better job of scheduling user space even under severe kernel load.

Anyway, I wanted to know from Gentoo community: are you folks using in-kernel CFS or you have moved to BFS? What are your experiences with desktop interactivity under different kinds of load like cpu load (rebuilding world), IO load (large backup jobs), memory load (running gimp and video editors etc.)?

My own experience is that Linux has miles to go in maintaining good desktop interactivity under load. My mouse freezes and switching tasks and workspaces takes ages during loads. I am thinking of moving to BFS.

----------

## devsk

BTW, nobody replied to the gentleman on the mailing list. I was expecting some kernel folks to at least show some interest.

----------

## HeissFuss

I tried BFS a while ago (2.6.38?) and found it to make my system less stable.

If you're having issues with some applications freezing while others are still responsive while doing things like network or file copies, you may be seeing issues with THP defrag similar to this thread.

----------

## PaulBredbury

I've been using BFS for years, on 32-bit - it's stable.

I eventually suspected 1000hz of being unstable on my laptop, so use 432hz.

I also use:

CONFIG_NO_HZ=y

CONFIG_PREEMPT=y

CONFIG_TREE_PREEMPT_RCU=y

CONFIG_PREEMPT_RCU=y

----------

## ppurka

There is a long long thread in the amd64 forums about unresponsive system on disk access. I do see this problem with the gentoo-sources-3.3.x kernel. I will have to check out BFS and see whether that improves the situation.

----------

## devsk

 *ppurka wrote:*   

> There is a long long thread in the amd64 forums about unresponsive system on disk access. I do see this problem with the gentoo-sources-3.3.x kernel. I will have to check out BFS and see whether that improves the situation.

 Yeah. I have participated in that thread. Its amazing that this issue has been there for so many years and nobody has done anything about it.

----------

## cach0rr0

talk of BFS with mainline kernel people goes /dev/null

mere mention of it seems to be grounds for having the entirety of the rest of what you ask discarded 

I've personally had great results with BFS. Some releases are better than others, but keeping a standard vanilla sources around of the same version as my BFS-patched kernel, I attribute the variations in stability not to BFS, but simply to different releases of the kernel having varying degrees of stability. 

In other words: good luck!

----------

## aCOSwt

 *devsk wrote:*   

> I wanted to know from Gentoo community: are you folks using in-kernel CFS or you have moved to BFS? What are your experiences with desktop interactivity under different kinds of load.

 

On my Intel Core II duo, I use gentoo-sources (=> CFS) 2.6.38 whenever I need heavy disk I/O, and ck-sources (=> The full ck patchset (not only BFS)) for any other purposes.

The rationale supporting this is the following experience anybody can reproduce and then conclude accordingly :

1/ Building packages with emerge

- CFS : Achieved in less time with MAKEOPTS=-j3 than with MAKEOPTS=-j2

- CK-PATCHED : Achieved in less time with MAKEOPTS=-j2 than CFS + MAKEOPTS=-j3 and time[CK-PATCHED,MAKEOPTS=-j3]=time[CK-PATCHED, MAKEOPTS=-j2]

What is a scheduler which needs to be overfed in order to get better performances ?

I will not use Con-Kolivas words... I'd get banned !   :Laughing: 

2/ Running Interbench on GS-2.6.38 / CK-2.6.38 with all common CONFIG_* set identically

```
--- GS-2.6.38 - Benchmarking simulated cpu of Audio real time in the presence of simulated ---

Load   Latency +/- SD (us)  Max Latency

None     24.0 +/- 24.4      31.0

Video      9.0 +/- 10.1      20.0

X     16.0 +/- 17.2      27.0

Burn      4.0 +/- 4.6        6.0

Write     23.0 +/- 24.5     106.0

Read     16.0 +/- 16.9      27.0

Compile      7.0 +/- 9.6       66.0

--- GS-2.6.38 - Benchmarking simulated cpu of Video real time in the presence of simulated ---

Load   Latency +/- SD (us)  Max Latency   % Desired CPU  % Deadlines Met

None     24.0 +/- 24.3      48.0

X     17.0 +/- 18.2      36.0

Burn      4.0 +/- 4.6       11.0

Write     23.0 +/- 24.7     115.0
```

```
--- CK-2.6.38 - Benchmarking simulated cpu of Audio real time in the presence of simulated ---

Load   Latency +/- SD (us)  Max Latency

None      4.0 +/- 4.4        8.0

Video      4.0 +/- 4.5       10.0

X      4.0 +/- 4.3        7.0

Burn      4.0 +/- 4.2        5.0

Write      6.0 +/- 7.1       15.0

Read      6.0 +/- 6.7       13.0

Compile      6.0 +/- 6.8       14.0

--- CK-2.6.38 - Benchmarking simulated cpu of Video real time in the presence of simulated ---

Load   Latency +/- SD (us)  Max Latency

None      4.0 +/- 4.3        9.0

X      4.0 +/- 4.3       10.0

Burn      4.0 +/- 4.2       10.0

Write      5.0 +/- 6.9       55.0
```

OK, benchmarks are meaningless bla bla...

Being said that whoever knows what latency means will understand these values, I find them matching the real behavior of my system working as a DAW.

This benchmark which highlights a significantly lower and more constant latency actually means 0 XRUNS under CK-PATCHED together with near to perfect audio tracks and audio / video tracks sync !

And this more or less irrespective of the load !

3/ The disk I/O throughput dilemma

OK, a CK-PATCHED kernel does not fulfill all needs :

It is a fact that disk IO throughput is completely sacrificed.

There is a rationale behind this and CK says that this is indeed wished, that everywhere he can reduce this throughput, he reduces it.

BTW, I never moved from GS-2.6.38, results with 3 series kernels show a +25% increase everywhere.

I believe this to be somewhere linked to some overhead induced on the scheduler by the management of the control groups, even if the associated set of options is unselected.

The conclusion of my personal opinion is that, except in the case when heavy disk IO is needed... CK-PATCHES RULE !   :Cool:   :Cool: 

----------

## Ant P.

 *devsk wrote:*   

> What are your experiences with desktop interactivity under different kinds of load like cpu load (rebuilding world), IO load (large backup jobs), memory load (running gimp and video editors etc.)?

 

I still need to tweak some of the IO settings, but for interactivity and raw throughput BFS performs better than CFS. The difference for media-heavy tasks is "good enough" versus "give up and use a proprietary OS".

I do have a problem where encoding videos in ffmpeg leaks large amounts of RAM until a full reboot... don't know if that's the fault of BFS or THP or something else though.

----------

## devsk

 *Quote:*   

> BTW, I never moved from GS-2.6.38, results with 3 series kernels show a +25% increase everywhere. 

 You mean with BFS?

----------

## aCOSwt

 *devsk wrote:*   

>  *Quote:*   BTW, I never moved from GS-2.6.38, results with 3 series kernels show a +25% increase everywhere.  You mean with BFS?

 

No, the 25% increase only concerns CFS and, I think, because of control groups.

Control groups are disabled under ck-sources kernels

----------

## devsk

The guy who posted about user space starvation actually used 3.2.14. I myself use 3.4 vanilla. The issue is not gone in 3 series kernels. There still are freezes. May be not minutes long but still annoyingly many many seconds long.

----------

## devsk

I am testing out BFS with 3.4.2. Let's see how it goes.

----------

## aCOSwt

 *Ant P. wrote:*   

> I do have a problem where encoding videos in ffmpeg leaks large amounts of RAM until a full reboot... don't know if that's the fault of BFS or THP or something else though.

 

Isn't ffmpeg the principal culprit ? I face similar memory leaks with iFFmpeg too.

----------

