# large file to usb -> kernel panic [solved]

## houtworm

I tried to create a bootable usb stick from a 3,5 Gb iso image.

So at first I tried

```
 dd if=image.iso of=/dev/sdc
```

..and after a while there was a kernel panic. Flashing keyboard leds and nothing I could do except pressing the reset switch.

Then I tried another usb stick with the same result.

That was using gentoo-sources-4.9.0

After that I tried it at another computer, also with kernel 4.9.0 with the same result.

I also tried unetbootin .. got also a kernel panic.

Then I booted with kernel 4.8.13 but it did not matter, again a kernel panic.

Is there a way to solve this problem?

It is almost a shame but now I started windows at my laptop and will try if rufus can make this stick.

That worked  :Smile: 

I did make a backup recently at a external harddrive connected to usb without any problems.

So it seems that writing 1 large file goes wrong but multiple smaller files gives no trouble?

For example my /usr is 13 Gb.

----------

## eccerr0r

That's not good...

How about a stable version of the kernel (4.4.39)? 

Are you using vanilla or gentoo-sources all around?

Do you have a dump of the panic?  Is it the same error each time?

----------

## houtworm

I use only gentoo-sources

I will install gentoo-sources and try it again.

With dd on the commandline I can see the kernel dump but I have to write it down because the computer will not respond anymore.

I don't know if the error is the same every time. Perhaps I will try it twice to see it  :Smile: 

But first I must reinstall grub because that other OS has ruined my bootsector.

----------

## houtworm

Ok, I installed kernel gentoo-sources-3.4.113-r1 and tried again.

And.. got another kernel panic:

```

Kernel panic - not syncing: hungtask: blocked tasks

Pid: 48, comm: khungtaskd Not tainted 3.4.113-gentoo-r1 #1

Call trace:

[<ffffffff8163706e>] ? panic+0xbf/0x1b8

[<ffffffff8108bc68>] ? watchdog+0x250/0x260

[<ffffffff8108ba10>] ? hung_task_panic+0x10/0x10

[<ffffffff81645454>] ? kernel_thread_helper+0x4/0x10

[<ffffffff8104e230>] ? kthread_flush_work_fn+0x10/0x10

[<ffffffff81645450>] ? gs_change+0xb/0xb
```

It looks just like the other panic this morning.

Is it usefull to try it again?

dd wrote about 20 minutes to the stick and then the writing stopped. Then 2 minutes later the kernel panic showed up.

It is an 8 Gb usb2 stick and the iso file is 3,2 Gb

----------

## Tony0945

Pardon me for what may be a silly question, but was /dev/sdc mounted during the dd operation? Are you sure the device is /dev/sdc? Did you try bs=8192k as the wiki said? I can see writing on a mounted filesystem causing a panic.

----------

## Hu

That panic looks to be more about the lack of progress writing to the device than due to any confusion associated with changing the filesystem while it was mounted.

OP: what manufacturer/model USB stick is this?

----------

## houtworm

It is a kingston usb stick.

I tried bs=8192 -> kernel panic

then bs=8192k -> kernel panic, did not take long

no bs -> kernel panic, after 20 minutes writing

of course /dev/sdc was not mounted.

Yes I am sure that /dev/sdc was my usb stick.

----------

## eccerr0r

Do you have any other sticks to test with, is it just this one stick?

My guess is that this stick gave up on a write for whatever reason but reported that it's still trying - and the kernel got confused when it couldn't make forward process writing.  Especially if there was a barrier...  This is bad behavior in any case though I'm not exactly sure what the best way to handle this is when there's data to be written in flight.

It could also be a USB hardware issue...

----------

## yilmi

Panic on a hung task seems to be enabled on your system (see https://www.kernel.org/doc/Documentation/sysctl/kernel.txt)

Try : sysctl -w kernel.hung_task_panic=0

However, you device shouldn't be unresponsive for a 120s (default)....  it can also be caused by a hardware error or some cheap USB sticks/SD cards that have "smart" controllers that might produce some unexpected behavior (https://www.bunniestudios.com/blog/?p=3554)

----------

## houtworm

Ok what I have tried..

downgrade core-utils to 8.23 to see if dd has perhaps changed but I still got a kernel panic.

I noticed in gkrellm at the disk activity that the writing suddenly stopt but the reading continues.

At that moment dd can't be killed and I can't reboot either so... reset switch.

I will try another usb stick and see if that works.

I will also try the hint from yilmi.

I will know the result within 10 minutes.

----------

## houtworm

Ok another stick, from lexar this time.

and.. another panic and now I could see it in the logfile, thanks to yilmi:

```
Jan  9 23:21:14 bergwerf kernel: udevd           D    0   675      1 0x00000000

Jan  9 23:21:14 bergwerf kernel:  ffff88042df453d8 ffff88042d37ab00 ffff88042df44f40 ffff88043fd56440

Jan  9 23:21:14 bergwerf kernel:  ffff88042f352200 ffffc90002e27bb8 ffffffff8175b0a4 ffffc90002e27b98

Jan  9 23:21:14 bergwerf kernel:  0000000000000018 ffff88042df44f40 ffff88042df44f40 ffff8803cd256b5c

Jan  9 23:21:14 bergwerf kernel: Call Trace:

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff8175b0a4>] ? __schedule+0x184/0x580

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff8175b4cd>] ? schedule+0x2d/0x80

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff8175b705>] ? schedule_preempt_disabled+0x5/0x10

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff8175c7ab>] ? __mutex_lock_slowpath+0x8b/0x100

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff8175c836>] ? mutex_lock+0x16/0x30

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff8117f60b>] ? __blkdev_get+0x3b/0x390

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff811802e9>] ? blkdev_get+0x109/0x2f0

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff81180510>] ? blkdev_get_by_dev+0x40/0x40

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff81149789>] ? do_dentry_open.isra.1+0x149/0x2d0

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff81159166>] ? path_openat+0x526/0x11d0

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff8115ae39>] ? do_filp_open+0x79/0xd0

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff8116128a>] ? dput+0x2a/0x230

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff8114a943>] ? do_sys_open+0x123/0x1f0

Jan  9 23:21:14 bergwerf kernel:  [<ffffffff8175ebe0>] ? entry_SYSCALL_64_fastpath+0x13/0x94
```

These panics kept coming..

I was perhaps wrong in my previous message. The reading stopped but the writing continued.

Now the usb stick has a led that kept flashing..

The kingston stick was new.

It has no priority anymore but it would be nice if anybody can help to find where it goes wrong.

--Kees

----------

## eccerr0r

Different coreutils shouldn't affect this issue, this is a kernel or more likely hardware problem.

It should not panic or oops or WARN due to a unresponsive disk.

What USB hardware are you using?

This is weird, blkdev_get is used during opening a device - but the bulk write has already happened... or at least part of it...  Did it write anything to the flash disk?

----------

## houtworm

It happens on both computers.

The other pc has a gygabyte x790ud3 mobo

Ths usb2 port works ok. I have copied my gentoo system to it without problems, from an externel harddrive connected to the usb2 port.

There is written on the stick. Perhaps if I can mount an iso.. I can see more.

it looks like everything is on the stick.

I will try if I can boot the pc from it.

..no does not work.

but it looks like everything is written to the stick

du -c  gives the same result for the iso and for the stick

weird things happen..

----------

## Hu

 *houtworm wrote:*   

> and.. another panic and now I could see it in the logfile, thanks to yilmi:
> 
> ```
> Jan  9 23:21:14 bergwerf kernel: udevd           D    0   675      1 0x00000000
> 
> ...

 I think that is not a panic, but a WARN.  Panic, by definition, halts the machine so abruptly that it cannot write the message to any local log file because it never runs the filesystem code after the panic starts.  (You can still save panic output by writing it to a serial console and using a separate machine to record the text shown on the serial console.)

----------

## houtworm

I think it is a kernel panic because my machine blocked every time with flashing keyboard leds. Until I did what yilmi wrote:

```
 sysctl -w kernel.hung_task_panic=0
```

After that I could see the kernel message in the logfile.

It is however a different message from the message I could see with a kernel panic before.

It is strange that dd did not stop with an error message.

'i/o error' or whatever message.

Tomorrow I will try to write a smaller iso to the stick.

----------

## houtworm

Ok, I wrote another iso to the stick and that worked. No bs=xx used with dd

It was 368 Mb instead of 3,5 gb

So it goes wrong if the filesize is too large.

----------

## houtworm

ok problem solved!

When using direct i/o it works:

```
dd if=image.iso of=/dev/sdc bs=4M iflag=direct oflag=direct
```

..and I can't boot from the stick hahaha

But the problem is solved.

Thanks for thinking with me!

--Kees

----------

## Tony0945

Never heard of the direct i/o flag before. It would seem that writing to a device IS direct.

But I found this: http://www.alexonlinux.com/what-is-direct-io-anyway

Maybe you don't have enough swap to cache 3.5G?

Thanks for the thread. One learns more every day.

----------

## NeddySeagoon

Tony0945,

swap is only used for dynamically allocated RAM.  RAM content that does not have a permanent home on disk.

Thus dirty buffers - waiting to be written, will never be written to swap.  They will just wait their turn to be written.

Meanwhile, as RAM fills up with dirty buffers, everything grinds to a halt.

Dynamically allocated RAM is pushed to swap, read only data/code is flushed, as it can be reread later and RAM is filled up by the read task of the copy operation.

----------

## Tony0945

 *NeddySeagoon wrote:*   

> Tony0945,
> 
> swap is only used for dynamically allocated RAM.  RAM content that does not have a permanent home on disk.
> 
> Thus dirty buffers - waiting to be written, will never be written to swap.  They will just wait their turn to be written.
> ...

 

I see.  I had thought the kernel swapped those write buffers. Then those usb writes failed because of insufficient RAM for those buffers? If the OP had another 4G everything would not grind to a halt?

----------

## NeddySeagoon

Tony0945,

He would still have got the panic due to the hung task but up to that point, everything would look normal.

It appears that 120 sec was not enough to write enough to keep the kernel from detecting a hung task.

----------

## Tony0945

Why did --directio solve that problem? What mechanism was bypassed?

----------

## NeddySeagoon

Tony0945,

The kernels cache was bypassed.

----------

## eccerr0r

EIther way, there is still a problem with the kernel or hardware.  This is merely a workaround.

Being able to panic or hang the kernel in userland should not be possible (other than by deliberate corruption, but that's a different issue.)

----------

## houtworm

Anybody can try it.

I just tried a copy:

```
mount /dev/sdc /mnt/stick

cp image.iso /mnt/stick
```

And I returned at the prompt.

But when unmounting the stick and de cache was written, it took a while and... again a kernel panic.

So anybody can test this without destroying a stick with dd.

The iso is about 3,2 Mb so any large file will do.

If you also get a kernel panic, it is most likely not a hardware failure.

If you get no errors the I am interested in your gentoo-kernel config.

Perhaps I can figure out the difference.

----------

## eccerr0r

I tried a 800MB file to a microsd card in a microsd adaptor, and it worked just fine last night...  (copying a map file for my Garmin...)

----------

## houtworm

Thanks for trying!

Yes smaller files gives no trouble.

Please try something from 3,5 Gb via usb2

----------

## eccerr0r

I sure hope it's not due to the 2GB limit where it could go negative if it just exceeds 31 bits of data...  I've seen some issues with that before, though it's been a long time.

I'm currently downloading larger maps to download to my GPS, I estimate I will have around a 3.5 GiB file that I will have to write on FAT32 which will be nearing that filesystem's limit.  So this won't be a useless test as I will have something I really need to write to the MicroSD card.

Just that my network speed is very slow...

----------

## houtworm

Haha no it is not the 31 bit limitation  :Smile: 

I tried something else..

I copied all the files from that iso to a map end then tried to copy that map with files to my stick. Perhaps separate files will work..

but no, again a kernel panic.

Well the copying was very fast... to the cache, but after that I unmounted the stick so the cache should be written.. and then the panic.

I am done with experimenting.

I will wait at what will hapen at your system. I don't go away so it is no problem that your network speed is slow  :Smile: 

----------

## eccerr0r

here we go, finally was able to generate a 2.7GiB file to copy...

$ ls -hl gmapsupp.img

-rw-r--r-- 1 eccerr0r eccerr0r 2.7G Jan 19 05:36 gmapsupp.img

I copied through Thunar... writing to a 8GB microsd card through USB2 at just a 3 or 4 MB/second....

It began to write...

and write ...

and write ...

The window then disappeared...

and it finished!

I hit the eject/unmount button - worked just fine.

No crash!

----------

## houtworm

eccerr0r thank you very much for doing this experiment!

I  now tried with a file from 2,8 Gb and.. again a crash

Ik will emerge thunar and see if that makes any difference.

----------

## eccerr0r

Well the thing is,... it should not crash, thunar or not...

The file I wrote to the microsd card was a dud, so I will need to do it again sometime, will do it with cp when I regenerate the file properly, or whatever experiment I come up with...

----------

## houtworm

Ok, I tried with thunar and when thunar was finished, I waited until the led from the stick stopped flashing. Then I could unmount the stick, no problem.

Again, now with 

```
cp bigfile /mnt/stick
```

, I was back on the prompt in no time but waited until the led stopped flashing. Unmounting the stick.. no problem.

Then I did again cp bigfile /mnt/stick, and when I was back on the prompt, I unmounted the stick and waited for the prompt...

Led kept flashing... and after a while the keyboard leds joined. Kernel panic!

Something does not work right.

----------

