# Question about IO schedulers

## Etal

Linux 4.12 includes two new IO schedulers. Curiously, "make oldconfig" asked me if I wanted to enable them but didn't ask me what I wanted as the default, so I decided to double-check with menuconfig:

```
 .config - Linux/x86 4.12.0 Kernel Configuration

 → Enable the block layer → IO Schedulers ─────────────────────────────────────

  ┌───────────────────────────── IO Schedulers ─────────────────────────────┐

  │  Arrow keys navigate the menu.  <Enter> selects submenus ---> (or empty │  

  │  submenus ----).  Highlighted letters are hotkeys.  Pressing <Y>        │  

  │  includes, <N> excludes, <M> modularizes features.  Press <Esc><Esc> to │  

  │  exit, <?> for Help, </> for Search.  Legend: [*] built-in  [ ]         │  

  │ ┌─────────────────────────────────────────────────────────────────────┐ │  

  │ │    [*] Deadline I/O scheduler                                       │ │  

  │ │    [*] CFQ I/O scheduler                                            │ │  

  │ │        Default I/O scheduler (CFQ)  --->                            │ │  

  │ │    [*] MQ deadline I/O scheduler                                    │ │  

  │ │    [*] Kyber I/O scheduler                                          │ │  

  │ │    [ ] BFQ I/O scheduler                                            │ │  

  │ │                                                                     │ │  

  │ │                                                                     │ │  

  │ └─────────────────────────────────────────────────────────────────────┘ │  

  ├─────────────────────────────────────────────────────────────────────────┤  

  │        <Select>    < Exit >    < Help >    < Save >    < Load >         │  

  └─────────────────────────────────────────────────────────────────────────┘
```

The default scheduler is CFQ, but oddly I don't have an option to change the default to any of the multi-queue ones:

```
 .config - Linux/x86 4.12.0 Kernel Configuration

 → Enable the block layer → IO Schedulers ─────────────────────────────────────

       ┌──────────────────── Default I/O scheduler ────────────────────┐

       │  Use the arrow keys to navigate this window or press the      │  

       │  hotkey of the item you wish to select followed by the <SPACE │  

       │  BAR>. Press <?> for additional information about this        │  

       │ ┌───────────────────────────────────────────────────────────┐ │  

       │ │                       ( ) Deadline                        │ │  

       │ │                       (X) CFQ                             │ │  

       │ │                       ( ) No-op                           │ │  

       │ │                                                           │ │  

       │ │                                                           │ │  

       │ │                                                           │ │  

       │ └───────────────────────────────────────────────────────────┘ │  

       ├───────────────────────────────────────────────────────────────┤  

       │                    <Select>      < Help >                     │  

       └───────────────────────────────────────────────────────────────┘
```

Well then, CFQ it is … right? I boot up:

```
$ dmesg | grep scheduler

[    0.546146] io scheduler noop registered

[    0.546147] io scheduler deadline registered

[    0.546169] io scheduler cfq registered (default)

[    0.546171] io scheduler mq-deadline registered

[    0.546172] io scheduler kyber registered
```

And check what my schedulers are…

```
$ grep -H . /sys/block/*/queue/scheduler 

/sys/block/bcache0/queue/scheduler:none

/sys/block/loop0/queue/scheduler:[mq-deadline] kyber none

/sys/block/sda/queue/scheduler:[mq-deadline] kyber none

/sys/block/sdb/queue/scheduler:[mq-deadline] kyber none
```

That's not what I expected!

How did my scheduler end up being MQ Deadline and why can't I even choose CFQ?

----------

## Ant P.

The menu choice for default is only for legacy single-queue schedulers (the ones above it), the ones at the bottom are multiqueue ones. You can only be using one of the two lists at any time depending on if you have CONFIG_SCSI_MQ_DEFAULT or the corresponding bootparam set. The unused ones are dead code and you may as well remove them from your kernel.

There isn't a menu option for default mq scheduler, it picks a per-device default in block/elevator.c based on whether the hardware reports native multiqueue support or not. If you want to override it, the only way right now is to remove mq-deadline or use an /etc/local.d/ script.

----------

## mbar

Also https://www.phoronix.com/scan.php?page=article&item=linux412-hddssd-io&num=1

----------

## Jaglover

Phoronix test does not mention what filesystem was used. XFS wiki.

----------

## CaptainBlood

 *Jaglover wrote:*   

> Phoronix test does not mention what filesystem was used. XFS wiki.

 Are SSD results relevant for /var/tmpfs [RAM] emerging?

Thks 4 ur attention.

----------

## Jaglover

 *CaptainBlood wrote:*   

> Thks 4 ur attention.

 

What you intend to achieve by insulting people?

----------

## BlueFusion

I just upgraded to kernel 4.12 today and wanted to take advantage of in-kernel BFQ IO scheduler.  Perhaps, I misunderstand it's usefulness, however.

I disabled all IO schedulers except BFQ, which is compiled in-kernel.  I added the following to my kernel command line:

```
scsi_mod.use_blk_mq=y dm_mod.use_blk_mq=y elevator=bfq
```

All 3 of my systems I rolled this change out on come up with the "none" scheduler for all drives:

```
proton ~ # cat /sys/block/sd*/queue/scheduler

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq
```

NOTE: This server is entirely HDD.  My desktop has the same issue with a single SSD installed.

I can manually activate bfq, or write a script to do so, but why is the elevator kernel option not working?

```
proton ~ # echo bfq > /sys/block/sda/queue/scheduler

proton ~ # cat /sys/block/sd*/queue/scheduler

[bfq] none

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq 

[none] bfq
```

----------

## thumper

Blk-mq-based device drivers bypass the previous Linux I/O scheduler.

This should get you started in the proper direction.

https://www.thomas-krenn.com/en/wiki/Linux_Multi-Queue_Block_IO_Queueing_Mechanism_(blk-mq)

George

[Moderator edit: fixed [url] tag; auto-linking does not work with embedded parentheses. -Hu]

----------

## BlueFusion

I was under the impression that there are 3 distinct blk-mq schedulers - Kyber, Deadline-mq, and BFQ (maybe a noop-style one, too?).

If I remove the scsi_mod.use_blk_mq=y dm_mod.use_blk_mq=y from the kernel command line, and just specify elevator=bfq, the system does not boot - it stops just after loading modules.  The fact that I can also manually activate the bfq scheduler via the command referenced in my previous post also gives the impression that you can specify specific blk-mq schedulers but it is not possible to activate non-blk-mq schedulers (i.e. CFQ).  BFQ is what used to be called bfq-mq, no?

Thanks for the link, George.  I stumbled upon that earlier in my Google searching.  It seems to be somewhat dated, however.  I've read elsewhere that the reason it bypassed all other IO schedulers is simply because they needed to be rewritten to work together or something of the sort.  Perhaps that is why we can now switch between them?

----------

## mbar

 *Jaglover wrote:*   

> Phoronix test does not mention what filesystem was used. XFS wiki.

 

EXT4, it's mentioned in the table on the first page. And in the text of the article:

 *Quote:*   

> The I/O schedulers were tested on a Linux 4.12 Git kernel on Ubuntu 17.04 x86_64. An EXT4 file-system was used on each drive with the stock mount options. The only changes made during the benchmarking process was changing the I/O scheduler in use.

 

----------

## Etal

Thanks for the pointers, everyone!

From the source file Ant P. pointed out, it looks like that it completely ignores any parameters with MQ schedulers and just selects "mq-deadline" if available and the the device has 1 hardware queue, or "none" otherwise. So the only way to change it is through an init script.

By the way, is there a way to check how many hardware queues a device has?

----------

## Juippisi

The mainline kernel didnt update their Kconfig when they added new schedulers, theres a patch that allows you to select BFQ as default in menuconfig. 

https://patchwork.kernel.org/patch/9825423/

I think you also have enable few things in kernel config in order for the system to boot with BFQ.

```

CONFIG_SCSI_MQ_DEFAULT=y

CONFIG_DM_MQ_DEFAULT=y

```

 -- https://pf.natalenko.name/news/?p=281

----------

