# [SOLVED] Developing a simple busybox initramfs

## MALDATA

I'm trying to learn a little about building and using an initramfs, so I was following along with this page: https://wiki.gentoo.org/wiki/Custom_Initramfs

My thought was to try a very simple test, with just the statically-linked busybox executable. So, as in the wiki, I created an initramfs directory:

```

# INITRDDIR=/root/initrdworking

# mkdir -p ${INITRDDIR}

# mkdir -p ${INITRDDIR}/{bin,dev,proc,sys}

# cp --archive /bin/busybox ${INITRDDIR}/bin/

# cp --archive /dev/{null,console,tty,sda} ${INITRDDIR}/dev/

# cp --archive ./init ${INITRDDIR}/

```

The init script is:

```

#!/bin/busybox sh

mount -t proc none /proc

mount -t sysfs none /sys

exec sh

```

And then finally,

```

# chmod +x ${INITRDDIR}/init

# cd ${INITRDDIR}

# find . -print0 | cpio --null --create --verbose --format=newc | gzip --best > /boot/test-initramfs.cpio.gz

```

My kernel already supports gzipped initramfs files, so to test booting it, I reboot my system and just edit the grub boot lines. I edit the initrd line to point at this test initramfs file, then boot. The way I see it, this should use my existing working kernel, but instead of booting my root filesystem as usual, it should run the init script in the context of the initramfs.

The kernel does seem to boot as usual, but I eventually get 

```

Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00

```

If everything ran ok, I should get a busybox shell and the init process shouldn't end. The script does exist at the specified location, it's executable, and so on. That being said, it seems like this would've been TOO easy if it had worked, so I think I probably missed something (like doing something more than just pointing grub toward a different initramfs).

Let me know if you see where I've gone wrong here. Thanks!Last edited by MALDATA on Tue Jan 21, 2020 8:38 pm; edited 1 time in total

----------

## axl

 *MALDATA wrote:*   

> I'm trying to learn a little about building and using an initramfs, so I was following along with this page: https://wiki.gentoo.org/wiki/Custom_Initramfs
> 
> My thought was to try a very simple test, with just the statically-linked busybox executable. So, as in the wiki, I created an initramfs directory:
> 
> ```
> ...

 

I play with some of the same tools you play. 

some comments. 

first of all, it's not just executables. or executable files. like busybox. u most likely had busybox built as static. 

but other things that you might want to include. 

```
[axl@magdalina:~]$ lddtree -la /bin/bash | sort -u

/bin/bash

/lib64/ld-linux-x86-64.so.2

/lib64/libc.so.6

/lib64/libdl.so.2

/lib64/libhistory.so.8

/lib64/libncursesw.so.6

/lib64/libreadline.so.8

/lib64/libtinfo.so.6

/lib64/libtinfow.so.6
```

so let's assume, you figure out the static shared library thing and you DO include all things in it... 

devtmpfs. automatically built in kernel. have to do it manually... something to look at. 

and then... cpio. it's cpio. u have to crc32. beware 64 bits systems. 

that xz gzip bzip lzo whatever is completely irrelevant. 

to make it work, the initrd.cpio image has to have a /init or a /linuxrc. literally. an executable file, named init or linuxrc. inside the cpio. could be a shell using busybox. 

either you mount dev or it's self mount based on kernel. 

even dirs. like dev, proc, sys, run, tmp. u have to create them in the cpio, which is a sort of temporary root. coz if you dont make them prior to cpio... they wont be there when you start mounting shit and starting daemons and where is tmp!?

i could not stop talking about this. I know I find this stuff incredibly exciting so I'll just stop myself talking and just look for something else to talk about.

----------

## axl

attempting to kill init... that error

usually means u are trying to kill pid 1. 

or pid 1 has nothing to execute. your busybox setup has no inittab file or something. i dont know what u're doing, but what u're doing is 

the system has nothing to execute, therefore "trying to kill pid 1" is like trying to kill self because self has nothing to do.

----------

## 389292

idk, maybe you also need an actual separate shell executable? like /bin/sh and/or /bin/bash with all of the libraries.

I also mount things with -n -t in my initramfs, I don't remember if it's really needed, but since I put it there it must be for some reason..

----------

## axl

https://www.youtube.com/watch?v=hV2Q41o-rwE

|| error.

----------

## GDH-gentoo

 *MALDATA wrote:*   

> The kernel does seem to boot as usual, but I eventually get 
> 
> ```
> 
> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00
> ...

 If the script exited, it should have printed error messages before the kernel panic. Did it not?

----------

## Hu

 *MALDATA wrote:*   

> My thought was to try a very simple test, with just the statically-linked busybox executable.

 In the interest of thoroughness, are you sure your busybox is static?  It can be built as non-static, depending on system configuration. *MALDATA wrote:*   

> So, as in the wiki, I created an initramfs directory:

 Although this may be convenient, particularly for development, you don't need to do this when you are ready to make an initramfs for embedding in the kernel. *MALDATA wrote:*   

> 
> 
> ```
> # cp --archive /dev/{null,console,tty,sda} ${INITRDDIR}/dev/
> ```
> ...

 sda is probably not right here.  It may refer to non-existent hardware if you use an NVMe drive.  Even if you use a drive that sda does reference, the drive is probably partitioned, so having only the base block device will be insufficient.  That is not relevant to your panic though. *MALDATA wrote:*   

> 
> 
> ```
> #!/bin/busybox sh
> 
> ...

 No /dev? *axl wrote:*   

> and then... cpio. it's cpio. u have to crc32. beware 64 bits systems. 

 The shown command is very similar to the one in the kernel documentation:

```
find . | cpio --quiet -H newc -o | gzip -9 -n > /boot/imagefile.img
```

The one OP shows is actually a bit better, since it avoids using whitespace delimited find output. *axl wrote:*   

> to make it work, the initrd.cpio image has to have a /init or a /linuxrc. literally. an executable file, named init or linuxrc. inside the cpio. could be a shell using busybox. 

 OP appears to have already done this.  Do you think his shown commands are incorrect? *axl wrote:*   

> even dirs. like dev, proc, sys, run, tmp. u have to create them in the cpio, which is a sort of temporary root. coz if you dont make them prior to cpio... they wont be there when you start mounting shit and starting daemons and where is tmp!?

 OP showed creation of dev, proc, and sys.  I doubt run or tmp are required for the minimal level of functionality that OP hoped to reach.

----------

## axl

did we just?

ok. put the pin back. one step at a time. easy does it.

----------

## MALDATA

Thanks for all the thoughts, everyone.

 *Quote:*   

> let's assume, you figure out the static shared library thing and you DO include all things in it...

 

I can confirm that my busybox binary definitely is statically linked. That being the case, I shouldn't need any other libraries or executables, other than an init script (or a symlink to busybox as an init).

 *Quote:*   

> devtmpfs. automatically built in kernel. have to do it manually... something to look at.

 

I did not check this, so I'll make sure that this is included.

 *Quote:*   

> and then... cpio. it's cpio. u have to crc32. beware 64 bits systems.

 

This is a 64-bit system. Can you elaborate more on how the procedure would be different on this system? Does the procedure in the wiki skip a step related to the CRC32?

 *Quote:*   

> to make it work, the initrd.cpio image has to have a /init or a /linuxrc. literally. an executable file, named init or linuxrc. inside the cpio. could be a shell using busybox.

 

There is an init script that uses "/bin/busybox sh" as the shell, and it is set to executable.

 *Quote:*   

> dev, proc, sys, run, tmp. u have to create them in the cpio

 

I have bin, dev, proc, and sys, but not tmp. 

 *Quote:*   

> your busybox setup has no inittab file or something.

 

I do not have an inittab file. I can try adding one.

 *Quote:*   

> maybe you also need an actual separate shell executable? like /bin/sh and/or /bin/bash with all of the libraries.
> 
> I also mount things with -n -t in my initramfs, I don't remember if it's really needed, but since I put it there it must be for some reason..

 

Statically-linked busybox should not require anything else. I am not familiar with the -n and -t options for mount, I'll have to read up on those.

 *Quote:*   

> If the script exited, it should have printed error messages before the kernel panic. Did it not?

 

Nope. I recorded the boot log on my phone, and didn't see any error messages. I'd transcribe it here, but that seems like a lot.

 *Quote:*   

> Although this may be convenient, particularly for development, you don't need to do this when you are ready to make an initramfs for embedding in the kernel.

 

Good point, I forgot to mention that the initramfs is not embedded in the kernel, it is separate. The intent was to use the exact same kernel I'm using now, but with an initramfs instead of my normal filesystem.

 *Quote:*   

> No /dev?

 

Nope... I'm just following along with the wiki page, which does copy a few device nodes into the initramfs, but does not mount them. Is that an error?

It's entirely possible that the wiki page has some errors. If I can get this working, I'll gladly make the corrections.

----------

## Hu

I understood it not to be embedded.  The process for embedding it is notably different from what you described doing here.  :Smile: 

Copying device nodes to /dev is fine, provided you copy all the nodes you will need.  Mounting a devtmpfs on /dev will let the kernel handle providing nodes that it can service, which means less work for you.  Could you check the phone's recording for the last 25-30 lines before the panic, to see if there are any other errors or warnings reported?  If you don't see anything there, then try modifying your initramfs to print something before it switches to the interactive shell, and something else after it switches.  This latter message should appear only if switching to the interactive shell fails.  For example:

```
#!/bin/busybox sh

set -x

mount -t proc none /proc

mount -t sysfs none /sys

exec sh

echo exec failed

sleep 10

echo panic
```

We need to determine (1) whether your initramfs is even started and (2) if it does start, why it doesn't produce an interactive shell.

----------

## MALDATA

 *Quote:*   

> Could you check the phone's recording for the last 25-30 lines before the panic, to see if there are any other errors or warnings reported?

 

I checked through it frame-by-frame, and didn't see anything interesting. There was one line that said 

```
Unpacking initramfs...
```

But that's about it. After that, it starts setting up a ton of devices and then panics.

After I posted originally, I added an echo to print something to the screen before hitting the 'exec sh', and I did NOT see it get printed. I will add an echo to the very beginning of the script, and one after the 'exec sh', and see what happens.

----------

## axl

well, if the cpio successfully uncompresses then it's just a matter of what the /init or /linuxrc file is. 

or the inittab. busybox is a distro into itself. 

I DO this:

```

cat << EOF > init

#!/bin/busybox sh

/bin/busybox --install -s

mount -t devtmpfs devtmpfs /dev

if [ ! -x "/dev/pts" ]; then mkdir /dev/pts; fi

if [ ! -x "/dev/shm" ]; then mkdir /dev/shm; fi

mount -t proc proc /proc

mount -t sysfs sysfs /sys

mount -t devpts devpts /dev/pts -o gid=5,mode=620,ptmxmode=000

mount /dev/vda1 /srv

if [ "\$?" -ne 0 ]; then

   xfs_repair /dev/vda1

   mount /dev/vda1 /srv

   if [ "\$?" -ne 0 ]; then

      xfs_repair -L /dev/vda1

      mount /dev/vda1 /srv

      if [ "\$?" -ne 0 ]; then

         echo cannot repair xfs filesystem

         /bin/sh

      fi

   fi

fi

mount -a

exec /sbin/init

EOF

chmod a+x init

chroot . ldconfig

chroot . locale-gen

chroot . depmod -av

chroot . updatedb

```

ofc, this works for me, but its kinda what I do. 

And if you need to mount stuff just like in the gentoo handbook. the dev. the dev/pts. dev/shm. proc, sys. run. tmp. and then start some sort of init. 

i'm not sure what you're doing with it. if you a desktop user, for you the loop ends with switch_root /newroot /sbin/newinit. and that's it. 

I work with machines that never get out of initramfs.

----------

## MALDATA

 *Quote:*   

> or the inittab. busybox is a distro into itself.

 

This might be where I've gone wrong. I don't have an inittab. I figured that if I had created /init as a symlink to busybox, then busybox's implementation of init would look for an inittab. However, since I'm providing my own init as a shell script, I assumed that inittab was not needed. So, even though I'm not using busybox's init, do I still need inittab?

Also, I don't fully understand the script you posted. It looks like your init process is a shell script that sets up a few things and then execs /sbin/init... Is your /sbin/init a symlink to the busybox binary? Then I guess when you exec /sbin/init, then it would need inittab.

Thank you for helping me figure this stuff out, I appreciate it!

----------

## axl

 *MALDATA wrote:*   

>  *Quote:*   or the inittab. busybox is a distro into itself. 
> 
> This might be where I've gone wrong. I don't have an inittab. I figured that if I had created /init as a symlink to busybox, then busybox's implementation of init would look for an inittab. However, since I'm providing my own init as a shell script, I assumed that inittab was not needed. So, even though I'm not using busybox's init, do I still need inittab?
> 
> Also, I don't fully understand the script you posted. It looks like your init process is a shell script that sets up a few things and then execs /sbin/init... Is your /sbin/init a symlink to the busybox binary? Then I guess when you exec /sbin/init, then it would need inittab.
> ...

 

No, that's ok. I'm as invested in this as you are. 

So let's finish this riddle.  :Smile: 

inittab is a remnant of a system with multiple users. 

again. it matters a lot what you trying to accomplish. like start raid. start crypt. start wireless. start pretty much any weird thing... and then... 

launch into the real system. switch_root command. we talked about that. 

so let's put it this way. what are you trying to accomplish in initrd space, that you can't accomplish in root? 

I mean... what is the hurdle to cross from init to switch root ?

----------

## Hu

If your /init is a busybox shell script, you do not need an inittab for that /init to work properly.

----------

## NeddySeagoon

With /dev not mounted, you will be missing /dev/null and /dev/console.

The kernel can mount devtmpfs on /dev itself if that option is selected.

You still need the /dev mount point but none of the content. 

The permissions in /dev will be incorrect and there will not be any symlinks like

```
$ ls -l /dev/std*

lrwxrwxrwx 1 root root 17 May 12  2013 /dev/stderr -> ../proc/self/fd/2

lrwxrwxrwx 1 root root 17 May 12  2013 /dev/stdin -> ../proc/self/fd/0

lrwxrwxrwx 1 root root 17 May 12  2013 /dev/stdout -> ../proc/self/fd/1
```

You don't care about the permissions as the init script runs as root.

The missing symlinks probably don't matter either.

My system with a static /dev and root on lvm on raid5 runs its initrd init script with only

```
 # we have a static /dev so we need all dev entries too

# nod <name> <mode> <uid> <gid> <dev_type> <maj> <min>

# e.g. /dev/console below

nod /dev/tty            0666 0 0 c 5 0

nod /dev/console        0600 0 0 c 5 1

nod /dev/null           0666 0 0 c 1 5

nod /dev/urandom        0644 0 0 c 1 9
```

in dev. Well, it has the sd* and dm-* entries too.

----------

## MALDATA

 *Quote:*   

> it matters a lot what you trying to accomplish. like start raid. start crypt. start wireless. start pretty much any weird thing... and then...
> 
> launch into the real system. switch_root command. we talked about that.
> 
> so let's put it this way. what are you trying to accomplish in initrd space, that you can't accomplish in root?
> ...

 

In short, changes to the root filesystem. I want to be able to make changes to the partitions and filesystems, kind of like a rescue CD. So, I don't want to mount my actual rootfs and switch_root into it. I want to just have a busybox system running in RAM that can make administrative changes.

So, while I will not mount the partitions, I do still need access to the /dev/sdX device nodes.

 *Quote:*   

> With /dev not mounted, you will be missing /dev/null and /dev/console.
> 
> The kernel can mount devtmpfs on /dev itself if that option is selected.
> 
> You still need the /dev mount point but none of the content.
> ...

 

In the boot log, I do see "devtmpfs initialized," and when I create the initramfs, I do

```
# INITRDDIR=/root/initrdworking

# mkdir -p ${INITRDDIR}

# mkdir -p ${INITRDDIR}/{bin,dev,proc,sys}

# cp --archive /bin/busybox ${INITRDDIR}/bin/

# cp --archive /dev/{null,console,tty,sda} ${INITRDDIR}/dev/

# cp --archive ./init ${INITRDDIR}/
```

So, /dev/null and /dev/console should exist in the initramfs. But maybe "devtmpfs initialized" doesn't necessarily mean that devtmpfs is mounted on /dev?

I will try adding the "mount -t devtmpfs devtmpfs /dev" line and see if that helps.

----------

## MALDATA

My init script is now as follows:

```
#!/bin/busybox sh

echo 'Start of init...'

mount -t proc none /proc

mount -t sysfs none /sys

mount -t devtmpfs devtmpfs /dev

echo 'About to run exec...'

exec sh

echo 'This is after exec!!!'
```

I don't see any of the echoed output while booting. I assume it'd show up in the console, right? Still get a kernel panic.

----------

## MALDATA

I tried several more things today, and was able to get it working. My procedure is now:

```
# INITRDDIR=/root/initrdworking

# mkdir -p ${INITRDDIR}

# mkdir -p ${INITRDDIR}/{bin,dev,proc,sys}

# mkdir -p ${INITRDDIR}/lib/modules/$(uname -r)/kernel/drivers/hid/usbhid

# cp --archive /bin/busybox ${INITRDDIR}/bin/

# cp --archive /dev/{null,console,tty,tty0,tty1,sda,zero} ${INITRDDIR}/dev/

# cp --archive /lib/modules/$(uname -r)/modules.* ${INITRDDIR}/lib/modules/$(uname -r)/

# cp --archive /lib/modules/$(uname -r)/kernel/drivers/hid/usbhid/usbhid.ko ${INITRDDIR}/lib/modules/$(uname -r)/kernel/drivers/hid/usbhid/

# cp --archive /lib/modules/$(uname -r)/kernel/drivers/hid/hid.ko ${INITRDDIR}/lib/modules/$(uname -r)/kernel/drivers/hid/

# cp --archive /lib/modules/$(uname -r)/kernel/drivers/hid/hid-generic.ko ${INITRDDIR}/lib/modules/$(uname -r)/kernel/drivers/hid/

# cp --archive ./init ${INITRDDIR}/
```

Comparing this to what I posted initially, this now includes:

 /dev/zero

 /dev/tty0

 /dev/tty1

 Kernel module for hid

 Kernel module for hid-generic

 Kernel module for usbhid

The init script is now:

```
#!/bin/busybox sh

/bin/busybox --install -s

mount -t proc none /proc

mount -t sysfs none /sys

mount -t devtmpfs devtmpfs /dev

modprobe usbhid

modprobe hid-generic

exec setsid sh -c 'exec sh </dev/tty1 >/dev/tty1 2>&1'
```

Compared to my original init script, this runs /bin/busybox --install -s to set up the symlinks, mounts devtmpfs, inserts the kernel modules so the keyboard works, and then avoids the busybox tty problem.

I still create the cpio file the same way:

```
find . -print0 | cpio --null --create --verbose --format=newc | gzip --best > /boot/custom-initramfs.cpio.gz
```

Now in GRUB, I can select my default kernel, hit e to edit the commands, change the initramfs from the regular one to my custom one, and add "console=tty0" to the kernel args. It then boots and gets me to a busybox shell, which is exactly what I was looking for. Thanks for the help, everyone!

----------

