# -ck patches I/O throughput drop from 2.6.38 to 3.2.1 kernel

## aCOSwt

- I get a gentoo-sources kernel and a ck-sources kernel with all common options set identically.

- I moved both from 2.6.38 version to 3.2.1 keeping all common options identical and setting all new options carefully.

- My system is based on an Intel CoreII duo + 4Gb RAM.

On my system running its typical load, I noticed a drop in performances from ck-sources 2.6.38 to ck-sources 3.2.1, drop that I cannot objectively quantify.

I am currently trying to investigate the problem and started throughputs tests with an Iozone benchmark :

```
/usr/sbin/iozone -r 8 -s 50M -l 1 -u 10 -i 0 -i 1 -b 3.2.1-cs-ext4-default-noop-B.xls
```

I generally use the deadline scheduler and specific mount options but I tried here on an ext4 with default mount options and the noop scheduler.

The results for write and rewrite tests show that :

- Under a 2.6.38 + BFS, the throughput is approximately identical to a 2.6.38 + CFS for a number of processes less or equal to 4. With more than 4 processes, the throughput is divided by 4. Normal, well, at least, this is what I am used to.

- Under a 3.2.1 + BFS, the throughput is a fourth of a 3.2.1 + CFS irrespective of the number of processes.

I know that threaded IO has always been a weakness of BFS but my typical load never creates situations in which more than 4 processes are competing for writing on the same IO ressource, so I generally do not care.

- Did you notice the same kind of performance drop ?

- Did I do something wrong with my configuration ?

- Is this benchmark meaningless ?

- I read about the modifications made on ext4 in 3.2.1 but did not realize that they would have any impact here. Am I wrong ?

EDIT 1 : With the deadline scheduler, results are approximately identical to the results with the noop scheduler.

----------

## PaulBredbury

```
cat /proc/sys/kernel/rr_interval

echo 12 > /proc/sys/kernel/rr_interval
```

Would be interesting to check that.

I set it to 4, in a gaming laptop.

----------

## aCOSwt

As I do agree with the argument that : "Worst case latencies being higher than 7ms are far worse than average latencies not being in the microsecond range.", I have always kept the default value of 6 for the rr_interval.

This being said, Increasing it to 12 does not show any significant difference in the mean and rms of the results. Increasing to 300 does show some slight increase of the average throughput per process but I remain very far from reaching the values I obtained under 2.6.38.

----------

## aCOSwt

Had an answer from CK.

To sum up with : This drop in IO throughput is intentional.

 *CK wrote:*   

> It's in the -ck patches and due to changes to the VM and I/O scheduler between that older kernel and the current one that make -ck more effective at reducing throughput...
> 
> ...
> 
> I'm forever doing things in -ck that make I/O slower rather than faster.

 

On a side note which confirms my tries of different values for the rr_interval :

 *CK wrote:*   

> rr_interval has NOTHING to do with I/O throughput

 

----------

## PaulBredbury

Those are bizarre word choices - CK must have a managerial position these days  :Razz: 

I can only suggest to try with the BFS patches only, and without the other patches that make up the -ck patchset.

----------

## aCOSwt

 *PaulBredbury wrote:*   

> I can only suggest to try with the BFS patches only, and without the other patches that make up the -ck patchset.

 

I will try this for sure. That was also among CK's advices in this particular case anyway.

Unfortunately, the sys-kernel/ck-sources package does not make a selective application of patches very handy.   :Crying or Very sad: 

----------

## PaulBredbury

Here's the BFS patch.

----------

## aCOSwt

 *PaulBredbury wrote:*   

> Here's the BFS patch.

 

 :Shocked:  Hey ! You must be joking !   :Evil or Very Mad: 

1/ That patch applied flawlessly on a Vanilla 3.2.1... make etc... etc... lilo etc etc...

2/ Reboot under patched kernel !   :Shocked:   :Shocked:  -> Red Star in front of ERROR IPTABLES...   :Shocked:   :Shocked:  => DO PANIC !

3/ Did not wait for reading carefully or investigating whatever or even shutting down properly... SWITCHED OFF !

4/ Reboot under normal gentoo-sources 3.2.1 : ERROR IPTABLES... SWITCHED OFF ! 

5/ Emergency root & boot filesystem... rm -r * on all filesystems then restored good old backups !

6/ Had a strong night cap !   :Twisted Evil: 

BTW, I did not found anywhere any checksum info about the patch.

COOL !   :Mad: 

My 2.6.38 is going to last longer than what I had planned !   :Cool: 

EDIT : I corrected the title of the thread.

----------

## PaulBredbury

Apply it to your working gentoo-sources, not vanilla.

I've never heard of BFS interfering with iptables.

Congrats on your Sherlock-esque problem-solving skills  :Razz: 

----------

## aCOSwt

 *PaulBredbury wrote:*   

> Congrats on your Sherlock-esque problem-solving skills 

 

I understand your sarcasm commanded by the night cap in phase 6 !   :Wink: 

Well... it was about 1 am... could not decently play violin, could I ?   :Razz: 

OK, more seriously, I will retry this patch tonight on a networkless gentoo-sources 3.2.1

----------

## aCOSwt

All right then, patched a networkless (and netfilter free) 3.2.1 gentoo-sources.

- Latency almost identical to a 2.6.38-ck : Good

- IO throughput when writing came back to 2.6.38-ck's levels... : Good ! (even better when nb processes > 4 but I do not care)

However :

 :Shocked:   - IO throughput when reading divided by 3 (whatever the number of processes reading concurrently) ! compared to a 2.6.38-ck or a 3.2.1-ck !

----------

