# STRANGE ISSUE: cannot create loop devices with kernel 3.1!

## tclover

Hi,

I've just runned into a very strange issue with kernel 3.1.1: I cannot simply create more loop back devices! 

Actually, when I can or remember to to do so, I usually set a default number of loop devices to 16-32. I usually use a dozen of theme, however some that are used in a initramfs before switching root remain used even if the files that were atached to them are umounted and deleted et. all (they are mounted with `-o loop,ro,nodev'). No problem for it as long as i could create more loop back devices when need be. 

I use this little script to create a few more when need be:

```
#!/bin/sh

for i in $(seq $2 $3); do

  case $1 in

   -a|--add)

      mknod -m0660 /dev/loop$i b 7 $i

      chown root:disk /dev/loop$i

      ;;

   -d|--del)

      rm -f /dev/loop$i

      ;;

  esac

done
```

I've just created a few devices because I needed more devices and losetup just returns `losetup: could not find any free loop device'! i15 devices are used, with /dev/loop[0-5] unusable because they were used in a initramfs andand are still reported as used.

And after creating more devices I have:

```
% ls -l /dev/loop* 

crw------- 1 root root 10, 237 Nov 15 19:40 /dev/loop-control

brw-rw---- 1 root disk  7,   0 Nov 15 19:40 /dev/loop0

brw-rw---- 1 root disk  7,   1 Nov 15 19:40 /dev/loop1

brw-rw---- 1 root disk  7,  10 Nov 15 19:40 /dev/loop10

brw-rw---- 1 root disk  7,  11 Nov 15 19:40 /dev/loop11

brw-rw---- 1 root disk  7,  12 Nov 15 19:40 /dev/loop12

brw-rw---- 1 root disk  7,  13 Nov 15 19:40 /dev/loop13

brw-rw---- 1 root disk  7,  14 Nov 15 19:40 /dev/loop14

brw-rw---- 1 root disk  7,  15 Nov 15 19:40 /dev/loop15

brw-rw---- 1 root disk  7,  16 Nov 15 19:43 /dev/loop16

brw-rw---- 1 root disk  7,  17 Nov 15 19:43 /dev/loop17

brw-rw---- 1 root disk  7,  18 Nov 15 19:43 /dev/loop18

brw-rw---- 1 root disk  7,  19 Nov 15 19:43 /dev/loop19

brw-rw---- 1 root disk  7,   2 Nov 15 19:40 /dev/loop2

brw-rw---- 1 root disk  7,  20 Nov 15 19:43 /dev/loop20

brw-rw---- 1 root disk  7,   3 Nov 15 19:40 /dev/loop3

brw-rw---- 1 root disk  7,   4 Nov 15 19:40 /dev/loop4

brw-rw---- 1 root disk  7,   5 Nov 15 19:40 /dev/loop5

brw-rw---- 1 root disk  7,   6 Nov 15 19:40 /dev/loop6

brw-rw---- 1 root disk  7,   7 Nov 15 19:40 /dev/loop7

brw-rw---- 1 root disk  7,   8 Nov 15 19:40 /dev/loop8

brw-rw---- 1 root disk  7,   9 Nov 15 19:40 /dev/loop9
```

Any input is welcomed. I was just going to reboot but unmounted another no vital loop for a vital one.Last edited by tclover on Fri Nov 25, 2011 2:32 pm; edited 1 time in total

----------

## Hu

How many loop devices did you configure when you loaded loop.ko?  What are you doing in your initramfs that you end up with stuck loop devices?  That should never happen.

----------

## VoidMage

Just a thought: are you using loop-aes ?

----------

## tclover

Nothing special, just mounting files with something like `mount -t someFS -o nodev,loop,ro /some/file /newroot/mount/point'. After switching to `/newroot' those mounted files will be reported as still being in `/newroot' even if I can umount the underlying FS and even delete them (the files), no probleme here. losetup just keeps reporting them as still being in `/newroot' instead of `/' and still being used even if they're not.

I do not append any argument to loop.ko because... loop is built in the kernel. I need it so I've just decided to built in for some time now and create loop devices when need be untill now. So no arguments whatsoever to loop.ko, well aside from appending some like `max_loop=<n>' to kernel cmdline of course but that's not exactly the same thing!

----------

## tclover

Oh, now after deleting the "stucked loop devises (from initramfs)" and re-created them, losetup finds them again as being free! So I can still use my machine without rebooting. Close one.

----------

## tclover

That was a false joy... actually I still had 4 stucked loop back devices, 2 only were freed with that workaround. It seems a few changes to loop devices went into 3.1.1 and possibly to 3.1.0. I did not use 3.1.0 longer to notice if the issue was there or not. And anyway, there are only fixes to 3.1.1 and mostly drivers, did not notice any loop back device changes when I read greg-kh annoucement. Lets investigate!

----------

## nlsa8z6zoz7lyih3ap

 *Quote:*   

> `mount -t someFS -o nodev,loop,ro /some/file /newroot/mount/point'

 

Just to be sure that I understand the details of the situation:

Are you saying that you  give the commands:

(1) losetup -l  (to see what loop device the above creates)  say it is loopx (or is it loop/x  ?)

(2) umount /some/file

(3) losetup -d loopx

losetup -l still lists loopx.

Finally: I don't understand why you are using the script to specifically create nodes etc.

I use loop-back routinely using nothing more that the losetup and mount commands.

----------

## tclover

 *nlsa8z6zoz7lyih3ap wrote:*   

> (1) losetup -l  (to see what loop device the above creates)  say it is loopx (or is it loop/x  ?)
> 
> (2) umount /some/file
> 
> (3) losetup -d loopx

 

First of, I'm not sure what you mean by `losetup -l', so I will assume it's a `losetup -a' (I don't have any `-l' switch) which list unused devices with the respective associated files.

The first post is clear for the "stucked loop back device": some files are mounted in initrmafs (but not unmounted), then switch root. I did not say this but now: the said mounted files are reported as still being in `/newroot' instead of simply `/'. No problem because the mounted files are still usable without any error. 

Now, after unmounted those files, and even delete a few if need be, those devices are still reported as being used. That's all. 

I had that issue with previous kernel versions, however, with previous kernel versions I could not only create new loop back devices with that simple script when need be but I could free a few of the bogus devices with a `losetup -d /dev/loop[0-<n>]. I could not free them all before kernel 3.0, usually two ot three remained unfreeable. But it was fine as long I could create new devices when need arise. I did not, though, try to directly `rm -f /dev/loop[0-<n>]' before and see if I could re-use re-created devices simply because I could create new devices.

NOTE: And by saying no devices available after creating some, I mean `losetup -f' report no available devices or I cannot mount a new file with `-o loop' because of... unavailability of loop back devices.

----------

## nlsa8z6zoz7lyih3ap

 *Quote:*   

> First of, I'm not sure what you mean by `losetup -l', so I will assume it's a `losetup -a' (I don't have any `-l' switch) which list unused devices with the respective associated files. 

 

You are quite right. Sorry about that.

 I don't think that I will be able to help you, but your question has got me curious.

Perhaps you would be willing to educate me.  

I still don't understand why you are using this script to create loopback devices:

```
Code:

#!/bin/sh

for i in $(seq $2 $3); do

  case $1 in

   -a|--add)

      mknod -m0660 /dev/loop$i b 7 $i

      chown root:disk /dev/loop$i

      ;;

   -d|--del)

      rm -f /dev/loop$i

      ;;

  esac

done
```

I have never  had the need to use mknod in conjuction with loopback devices.

The following shows an actual creation and deletion of a loopback device on my laptop.

sheaf owner # uname -a

Linux sheaf 3.1.0-gentoo-r1-a #1 SMP Sat Nov 12 15:49:59 PST 2011 x86_64 Intel(R) Core(TM) i3 CPU M 330 @ 2.13GHz GenuineIntel GNU/Linux

sheaf owner # mkdir /test

sheaf owner # losetup -f

/dev/loop0

sheaf owner # losetup /dev/loop0 /dev/mapper/sda8

sheaf owner # losetup -a

/dev/loop0: [000f]:2986 (/dev/mapper/sda8)

sheaf owner # mount /dev/loop0 /test 

sheaf owner # ls /test

bin   bootcd                     dev  home    iris  lib32  lokalize-scripts  magic  mnt  proc  root  sda1  sda6  sda8  sdb  spectrum  tmp  var             windows

boot  config-2.6.36-gentoo-r3-a  etc  initrd  lib   lib64  lost+found        media  opt  rc    sbin  sda2  sda7  sda9  sdc  sys       usr  vmlinuz-gentoo

sheaf owner # umount /test

sheaf owner # ls /test

sheaf owner # losetup -d /dev/loop0

sheaf owner # losetup -a

So you can see that all of this has been done using a 3.1.0 kernel without using mknod

I am curious as to why you used mknod?

----------

## tclover

 *nlsa8z6zoz7lyih3ap wrote:*   

> So you can see that all of this has been done using a 3.1.0 kernel without using mknod
> 
> I am curious as to why you used mknod?

 

Well, actually you're just using [pre-]created loop back devices. If you run a `ls /dev/loop*' you'll see that you have a few of them, 8 devices is the default. There's an option in kernel config to set up a default value which will give you that number of devices after each reboot/kernel initialization.

I had previously set that option to 32 but changed my mind with kernel 3.x to 16 because I could create new ones easily. I've lost on that as I'm having trouble to create new ones. The option remaining for me is to add a `max_loop=<int>' to the kernel cmdline because it's not an option to recompile the kernel for such a silly thing.

----------

## nlsa8z6zoz7lyih3ap

 *Quote:*   

> Well, actually you're just using [pre-]created loop back devices. If you run a `ls /dev/loop*' you'll see that you have a few of them, 8 devices is the default. There's an option in kernel config to set up a default value which will give you that number of devices after each reboot/kernel initialization. 

 

Thanks

 *Quote:*   

> The option remaining for me is to add a `max_loop=<int>' to the kernel cmdline because it's not an option to recompile the kernel for such a silly thing.

 

Makes sense to me.

PS   I can't replicate your problem. I have also tried your "mknod" method and it works just fine for me without leaving any "hanging" loop devices.

If you are creating them using your script in an initramfs using some init script, maybe the problem lies there rather than in the kernel.  I try lots of things in my initramfs and init that logically should work but don't. No doubt the reasons for this could be traced down, but doing so is very low on my priority list as I have always found workarounds.

----------

## tclover

Yes, I can create new loop devices just fine with kernel 3.0.x but not with 3.1.x...

----------

## py-ro

This change:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=770fe30a46a12b6fb6b63fbe1737654d28e84844

So no need to create loop-Device or to specify them at boot/module-load.

But i dunno if stable losetup can handle this yet.

----------

## nlsa8z6zoz7lyih3ap

 *Quote:*   

> Yes, I can create new loop devices just fine with kernel 3.0.x but not with 3.1.x...

 

A mystery indeed. I had no trouble creating loop devices with the 3.1.0-gentoo-r1 kernel.

No doubt some subtle difference in our kernel configuration and installed software accounts for this,

but I am not inclined to pursue this line of investigation.

I am curious about this, though: Were you creating loop devices and using your script within

an initramfs, later in the boot process or after the boot process?

----------

