# [SOVED] tape drive has very low speed

## Ilya.A

Hi there!

Gentoo and HPE StoreEver 1/8 G2 LTO-7 autoloader.

I can't get type drive speed above 16MB/s - on write and read both.

With default block size 32kb speed is 2.1MB/s. I able to rise block size up to 256k, speed grows proportionally up to 16MB/s.

With tapestat it looks like this:

[img]https://i.ibb.co/ng6Kvcs/tapestat-00.png[/img]

speed is fast enough for 5 seconds, then drops.

I don't know, may be tapestat shows normal picture. I've saw on forums people uses block size 2MB. Speed will be good if it will grow proportionally to the block size.

but I can't set it that much - anything above 256kb result in error:

 *Quote:*   

> srvAmanda /etc/amanda/ansv # dd if=/dev/nst0 bs=512k count=10240
> 
> dd: error reading '/dev/nst0': Device or resource busy
> 
> 0+0 records in
> ...

 

While playing with block size I've saw in dmesg:

 *Quote:*   

> [ 2537.149171] st 0:0:13:0: [st0] Block limits 1 - 8388608 bytes.

 

Looks like drive should be able to set block size up to 8Mb. But I can't use it.

Any ideas there?

ps

I tried windows on this hardware - tape drive speed is as expected to be - about 300MB/s.Last edited by Ilya.A on Mon Oct 17, 2022 4:27 am; edited 1 time in total

----------

## Zucca

What happens (logs, dmesg, terminal output) if you give dd exactly 8388608 (bs=8388608) as block size?

----------

## Ilya.A

 *Quote:*   

> srvAmanda /etc/amanda/ansv # dd if=/dev/nst0 bs=8388608 count=1
> 
> dd: error reading '/dev/nst0': Device or resource busy
> 
> 0+0 records in
> ...

 

No new messages in dmesg or anywhere else.

----------

## pietinger

Ilya.A,

 *Ilya.A wrote:*   

> [...] anything above 256kb result in error:

 

I have only old knowledge about tapes ... so, maybe its outdated, but manual page is from 2020 ... and 256 kB reminds me to something ... let me quote:

 *Quote:*   

> The driver uses an internal buffer that has to be large enough to  hold
> 
>        at  least one tape block.  In kernels before 2.1.121, the buffer is al-
> 
>        located as one contiguous block.  This limits the  block  size  to  the
> ...

 

(from: http://www.tin.org/bin/man.cgi?section=0&topic=ST)

So, question would be: Do you use variable block length or fixed ?

Maybe this will help you: 

https://www.backupcentral.com/forum/14/180680/ot__hitting_block_size_limitation_on_lto4_drive

https://bugzilla.kernel.org/show_bug.cgi?id=20072

----------

## Ilya.A

While read or write operation autoloader's screen shows, for example, "Drive RD" (read) just for a second, then it stay "Drive Idle" for a while, then again it shows "Drive RD" for a second and so on. Meanwhile tapestat program shows constant but very low speed and 99% full buffers, as on screenshot in the first message.

----------

## Ilya.A

 *Quote:*   

> So, question would be: Do you use variable block length or fixed ?

 

It doesn't really matter. In any position I'v tried I get speed like this:

 *Quote:*   

> Tape:     r/s     w/s   MB_read/s   MB_wrtn/s  %Rd  %Wr  %Oa    Rs/s    Ot/s
> 
> st0         0     160           0          40    0   20   20       0       3
> 
> Tape:     r/s     w/s   MB_read/s   MB_wrtn/s  %Rd  %Wr  %Oa    Rs/s    Ot/s
> ...

 

It is for running  *Quote:*   

> srvAmanda ~ # mt setblk 256k
> 
> srvAmanda ~ # dd if=/dev/random of=/dev/nst0 bs=8M count=1024

 

You can see, column w/s after a few seconds decrease to 64.

Why good speed on start and why it gets low?

With variable block size I can set dd's block size up to 256kb untill busy error. With fixed block size on drive I can use any bs value of dd but still get same speed.

If I set fixed block size more than 256k I get "busy" error, but I can set it up to 8Mb.

----------

## Ilya.A

And I'v tried two more linux distribution - got exactly same results.

----------

## pietinger

How it is connected ? What happens if you dont use /dev/random ? (only a guess: maybe kernel cannot deliver so fast; same results with /dev/null ?)

----------

## Ilya.A

Drive is SAS, connected through SAS controller and SAS expander.

It doesn't looks like hardware problem - I've tried windows on that machine - tape drive works fine.

----------

## logrusx

 *Ilya.A wrote:*   

> Drive is SAS, connected through SAS controller and SAS expander.
> 
> It doesn't looks like hardware problem - I've tried windows on that machine - tape drive works fine.

 

Do you use customized kernel? Did you try distribution kernel?

Regards,

Georgi

----------

## eccerr0r

my /dev/random chokes out very quickly...can't see anyone really getting that much data from it that fast!

Use /dev/urandom or better yet /dev/zero for a garbage source of data for testing.

I should pull out my Exabyte 8200 and see if it still works... though my initial testing is that /dev/random isn't fast enough to write to it, either.

----------

## NeddySeagoon

Like eccerr0r says, /dev/random is slow and blocks when it runs out of entropy. It a good source of small amounts of secure random numbers.

/dev/urandom  is slow does not block when it runs out of entropy, so its not secure.

/dev/zero is fast and as long as you don't care about the date ...

Why dd?

Tape Archiver was made for tapes. It's installed everywhere today too.

You will know it better as tar.

----------

## logrusx

 *NeddySeagoon wrote:*   

> 
> 
> Why dd?
> 
> Tape Archiver was made for tapes. It's installed everywhere today too.
> ...

 

Isn't it the same in this case?

But the OP says speeds are slow both for reading and writing and /dev/random is not involved in reading data off of the device.

p.s. cant dmesg shed some light on how the driver is initialized?

Regards,

Georgi

----------

## NeddySeagoon

logrusx,

Good catch for the read speed.

tar is more convenient than dd. It can do on the fly compression and pause for tape changes.

For less than one tape dd and tar are almost equivalent.

dd with small block sizes is slow as it calls the kernel at every block. The bigger the block size, up to about bs=1M, the faster dd goes.

----------

## Ilya.A

 *logrusx wrote:*   

>  *Ilya.A wrote:*   Drive is SAS, connected through SAS controller and SAS expander.
> 
> It doesn't looks like hardware problem - I've tried windows on that machine - tape drive works fine. 
> 
> Do you use customized kernel? Did you try distribution kernel?
> ...

 

I've tried two other linux distribution in form of live cd - exactly the same result. So I guess this is not my modification of kernel. Actually trying precompiled kernel is on the todo list so i'll give it a try.

----------

## Ilya.A

For actual backups I use Amanda, dd just for testing.

st startup messages: *Quote:*   

> [    5.111257] st: Version 20160209, fixed bufsize 32768, s/g segs 256
> 
> [    5.111732] st 0:0:13:0: Attached scsi tape st0
> 
> [    5.111737] st 0:0:13:0: st0: try direct i/o: yes (alignment 4 B)
> ...

 

----------

## bbgermany

Hi,

I remember a similiar issue with one of my past customers. I found this article:

https://blog.benjojo.co.uk/post/lto-tape-backups-for-linux-nerds

maybe the block size of 256k is not big enough at all. Tapedrives love streams, so the larger the file/package, the happier the drive  :Wink: 

Greeting Stefan

----------

## eccerr0r

word from the enlightened, though not relevant to the discussion at hand:

Do not run a MDRAID on the same ultra SCSI bus as an async SCSI tape drive...

----------

## Ilya.A

I can't set it more than 256kb. I mean I can set but I can't use it - "device busy" error.

----------

## Ilya.A

BTW, amtapetype finished with this:

 *Quote:*   

>         comment "Created by amtapetype; compression disabled"
> 
>         length 5876329216 kbytes
> 
>         filemark 3105 kbytes
> ...

 

What does it mean - LEOM is not supported?

----------

## bbgermany

 *eccerr0r wrote:*   

> word from the enlightened, though not relevant to the discussion at hand:
> 
> Do not run a MDRAID on the same ultra SCSI bus as an async SCSI tape drive...

 

and do not run tapes on raid controller at all  :Wink: 

greeting Stefan

----------

## grknight

 *Ilya.A wrote:*   

> BTW, amtapetype finished with this:
> 
>  *Quote:*           comment "Created by amtapetype; compression disabled"
> 
>         length 5876329216 kbytes
> ...

 

From amanda-devices manual page:

LEOM - (read-write) If this property is true, then the device can detect an EOM condition before actually running out of space, allowing Amanda to forgo caching parts while writing.  For some devices, it is necessary to override the conservative default value of this property.

----------

## logrusx

 *grknight wrote:*   

>  *Ilya.A wrote:*   BTW, amtapetype finished with this:
> 
>  *Quote:*           comment "Created by amtapetype; compression disabled"
> 
>         length 5876329216 kbytes
> ...

 

And from what I could find, it somehow depends on the block size. I guess 256 kbytes is too small for that feature to work.

I also noticed things discussed about much older kernel versions, but I don't understand from the topic enough to draw some conclusion from it. I'd only suggest trying with 5.10 kernel, this is the oldest in tree. You might get lucky with it, Ilya.

p.s. I noticed there's firmware for this device on HPE's site. It might be needed...

Regards,

Georgi

----------

## Ilya.A

Finally I've tried another SAS controller and all of a sudden tape drive works without that speed degradation you can see on screenshot in first message (https://ibb.co/ng6Kvcs). And looks like no shoe-shining. Drive may be used in production, alleluya!

But anyway, I unable to use block size more than 256kb. I saw on the Internet that people use 512kb and more.

So speed of tape drive remains far form its specs.

I'am going to try FreeBSD. I'll let you know the results.

----------

## energyman76b

a) hard drive seeking is really really destroying the ability to stream. And a lot of ssds love to choke when they get warm. 

b) use something like mbuffer - it really helps.

----------

## Ilya.A

Sorry guys, have no time to continue testing.

With another controller on production server amtapetype finished with this:

```
define tapetype LTO7 {

    global

    comment "Created by amtapetype; compression enabled"

    length 5876449792 kbytes

    filemark 615 kbytes

    speed 223193 kps

    blocksize 256 kbytes

}

# LEOM is not supported for this drive and kernel

```

Speed looks good, i'll stick with it.

Thanx for your help!

----------

