# A call for kernel testing help (desktop lovers/-ck users)

## kerframil

Hello folks,

I'm sure lots of you will be familiar with Con Kolivas' patchset, in particular his autoswap  patch and vm_autoregulate patch. Some of you might also have tested Nick Piggin's work which is along similar lines (I don't use love-sources but I believe that uses or used his scheduler interactivity patches).

Now, quite a few people who deal with desktop workloads seem to be in agreement that the performance of 2.6 could be improved a bit when it comes to such matters. These matters have been discussed here and there in the past. Con recently announced his 2.6.7-ck5 patchset (on the new ck mailing list as well as the Linux Kernel Mailing List. This time around it's generated some very interesting discussion with such luminaries as Andrew Morton and Nick Piggin becoming involved.

The challenge to Con is to prove that his work really does have a beneficial effect on performance, and hard numbers are required to back it up. To that end he's asked if anyone would be willing to help as his personal resources are limited.

As I know a lot of us love to use Linux on the desktop, I'm basically imploring anyone who can spare some time and preferably who has access to a machine with a relatively high spec to chip in and offer to conduct some tests. Help a brother out - it can only be a good thing for us all in the long run!   :Cool: 

Just as I was writing this post, he replied to my post (saying I was willing to help) and has defined the testing framework as follows:

```
Grab one of the faster machines build a vanilla 2.6.7 and one with the 

http://ck.kolivas.org/patches/2.6/2.6.7/from_linux-2.6.7_to_vm_autoregulate2 

patch.

Get a clean tree for benchmarks and boot into vanilla in single user 

mode with mem=128M

Go into the clean tree and do

make clean && make mrproper && make defconfig

time make

then do

make clean && make mrproper && make defconfig

time make -jX

where X is increasing the value of jobs, until the value of X is 

something like 5 times larger than the first time make

then do 3 runs of make -jX and average the results

then boot into autoswap single user mode with mem=128M and do another 3 

runs of make -jX

Ideally we do this with different mem= options and different degrees of 

make -jX but this could take forever. This is a fair starting point.
```

So, anyone else up for it   :Question: 

----------

## Tuti

i'd be up for that

anything specific for the kernel configs? like preempt?

----------

## kerframil

 *Quote:*   

> anything specific for the kernel configs? like preempt?

 

No, I would suggest leaving preempt out. Whatever you do, the main thing is to ensure that the configs are practically identical in all cases. I intend to start my testing as soon as the weekend commences; I could post my .config if you are curious.

----------

## barry

I've done some testing with this. So far there's no stress on the system between j1 and j5 (with mem=128M). j10 seems to favour the vanilla kernel slightly, while a heavier load at j15 seems to favour Con's. I will try j20 later.

Vanilla -j5:

5m28.527

5m0.794

0m26.589

CK -j5:

5m31.251

4m59.911

0m26.258

Vanilla -j10 (3 tests):

7m11.933,  6m57.894,  7m25.938

5m2.583,  5m2.369,  5m3.070

0m29.584,  0m28.762,  0m29.275

CK -j10 (3 tests)

7m15.713,  7m37.472,  7m50.441

5m2.658,  5m2.735,  5m2.854

0m29.701,  0m29.775,  0m29.870

Vanilla -j15 (3 tests)

9m47.895,  9m34.718,  8m56.847

5m5.027,  5m4.717,  5m4.949

0m31.625,  0m31.139,  0m31.333

CK -j15 (3 tests)

8m51.580,  8m44.176,  8m52.932

5m4.505,  5m5.846,  5m4.629

0m31.071,  0m31.046,  0m31.260

[1gb swap, identical .config for each kernel.]

----------

## kerframil

Great stuff Barry! I'll be chipping in with my results later tonight after work - I have a nice Proliant ML370 G3 to test on  :Smile: . I'll also ask Con to keep an eye on this thread, or I might just reproduce the results on the original mailing list thread once we have some more results here. Thanks.

EDIT: Any chance you can amend the post just to include a brief overview of your system specs? Just CPU speed, mainboard chipset, memory type and perhaps disk class should do.

EDIT #2: Note also that Con asked for the tests to be conducted with a vanilla tree and a vanilla tree with just the from_linux-2.6.7_to_vm_autoregulate2 patch referenced in his instructions above - not the full -ck5 patch. Just want to make sure that is the case  :Wink: 

----------

## barry

Sure, the specs are 512mb PC3200 RAM (limited to 128M as per instructions), Athlon XP 2500+ slightly overclocked at 2002MHz, Asus A7N8X v2 board,  and Maxtor 6Y120P0 IDE disk. GCC 3.3.3-r6 compiler.

I've run the tests with just the patch mentioned, and not the whole -ck5 set.

I've now run -j20 with the patch, the results are unexpected. The first run was quite different from the following two, so I did a fourth, which was rather disappointing:

CK -j20 (four tests)

19m57.442

12m18.414

12m13.557

28m32.147

The user and sys times are more or less the same in all tests at ~5m4 and ~0m35 respectively.

Update:

The results for the vanilla kernel show a very poor fourth run also. I don't know why this is:

Vanilla -j20 (four tests)

18m36.929

17m52.209

18m35.022

27m20.255

----------

## RoYzter

my sys is not so hi-spec, but anyway, here's my results:

celeron 2,6 Ghz 512mb pc2700 ram (also limited to 128M), msi hermes 651 barebone, seagate ST3120026A ide-disk, GCC 3.3.3-r6

```

kernelcompile unpatched, make

real    16m42.166s

user    15m40.182s

sys     0m52.281s

kernelcompile unpatched, make -j10

1. run

real    21m29.922s

user    15m52.773s

sys     1m3.027s

2. run

real    24m24.538s

user    16m11.104s

sys     1m7.934s

3. run

real    24m47.037s

user    16m12.932s

sys     1m9.140s

kernelcompile patched, make -j10

1. run

real    24m6.645s

user    16m6.914s

sys     1m7.766s

2. run

real    25m15.155s

user    16m8.157s

sys     1m11.069s

3. run

real    24m15.253s

user    16m27.488s

sys     1m9.661s

```

hope this helps, if you need any additional info, feel free to ask  :Smile: 

----------

## barry

I don't think mem=128m and -j10 puts sufficient stress on the system for the patch to change anything worthwhile. I didn't notice much paging at -j10, while at -j20 there's almost constant paging going on. During my 28min runs there was a load of over 18 at the end, and constant disk thrashing during the build - it didn't seem that Con's patch helped there unfortunately.

----------

## tomthewombat

here is an idea

hdparm -X33 /dev/hda

this will force UDMA 33, then the method that uses less disk thrashing should become more apparent.

even better set it to X0 for PIO only mode.  hehe~

----------

## rinnan

Well, this is odd in my opinion.

The implication here is that the goal of a CPU scheduler for an OS to to allow it to bring multple CPU-bound processes to completion as quickly as possible and to initiate and tear down threads as quickly as possible.  But that doesn't seem to be the goal of a CPU scheduler at all.  I mean, following this logic, the sceduler would be more efficient if it just dropped the priority on every task but those most recently run, arbitrarily.  But that would be a terrible sceduler.

There needs to be benchmarks for fairness, latency, and predictability as well as speed and I would say that all three of these other goals are vastly more important ones, none of which are measured by this kind of benchmark.

rinnan

----------

## kerframil

 *rinnan wrote:*   

> There needs to be benchmarks for fairness, latency, and predictability as well as speed and I would say that all three of these other goals are vastly more important ones, none of which are measured by this kind of benchmark.

 

This benchmark was pretty much devised to measure the effect in terms of swap thrashing. The policy of the CPU scheduler is by no means the only factor in terms of having "fair" performance and the tedency of the kernel to swap has been often cited as a bugbear (particulary with respect to desktop performance).

There are perfectly serviceable methods for determining areas of high latency (one only needs to look at the work done recently by people like Andrew Morton, Takahashi Iwai, Ingo Molnar and William Lee Irwin III). Measuring "fairness" is somewhat more subjective ... what makes something fair or unfair? Having said that, there is quite a bit of evidence to suggest that the staircase scheduler is considerably more strict with CPU distribution - more so if the interactive mode is switched off. It's temporary appearance in -mm recently also led to various reports suggesting improvements in a fairly diverse set of workloads (and, indeed, with several homegrown benchmarks) - and, of course, the occasional regression. See kerneltrap.org, LKML et al.

In any case, with respect to the issue that this thread was about, I should have updated it already. Con revised his whole approach and came up with the "hard swappiness" patch which was subsequently metamorphosized into the "mapped watermark" patch. His description is as follows:

```
Added since 2.6.8.1-ck3:

+mapped_watermark.diff

This readjusts the way memory is evicted by lightly removing cached ram 

once the ram is more than 2/3 full, if less than the "mapped watermark" 

percent of ram is mapped ram (ie applications). The normal system is to 

aggresively start scanning ram once it is completely full. The benefits 

of this are:

1. Allocating memory while ram is being lightly scanned is faster and 

cheaper than when it is being heavily scanned.

2. There is usually some free ram which tends to speed up application 

startup times.

3. Swapping is an unusual event instead of a common one if you have 

enough ram for your workload.

4. It is rare for your applications to be swapped out by file cache 

pressure.

Disadvantage: Less file cache - but can be offset with the tunable

The mapped watermark is configurable so a server for example might be 

happy to have a lower mapped percentage. The default is 66 and a server 

might like 33 (0 is also fine)

echo 33 > /proc/sys/vm/mapped

This patch removes the swappiness knob entirely and deprecates all my 

previous vm hacks (autoregulated swappiness, hard swappiness, kiflush).
```

In fact, there was a bug in his initial implementation - anyone using this patch (or the 2.6.8.1-ck4 patchset) should grab this incremental patch to fix it. There is also a minor staircase revision in the same directory on that server.

----------

## luqas

What are you using to benchmark?  Also,  what are the so called "defaults" in the configs.   I would love to assist in this.

----------

## kerframil

 *Dryre wrote:*   

> What are you using to benchmark?  Also,  what are the so called "defaults" in the configs.   I would love to assist in this.

 

That's great - however the work has been done now so this thread is redundant henceforth  :Wink: 

As mentioned above, Con did away with his "autoregulated swappiness" approach eventually, and developed a better approach. If you want to help out with -ck then the best thing is (a) to use it (b) to join and track the mailing list. Thanks.

----------

## luqas

Yea.  I saw he moved to his watermark instead auto-swappiness.  I didn't know if he needed benchmarks for that.  Thanks and I have been wanting to join the mailing list.

----------

