# Kernel Compression / Size Reduction?

## Crimjob

Hi All,

The hardware I'm currently working with requires a kernel in gzip or bzip2 format, and the size must be smaller than 1.8MB.

I tried using the Gentoo Live CD to bzip2 the kernel using

```
bzip2 -c kernel2.6.30-gentoo-r4 > vmlinux.bz2
```

but a du of kernel2.6.30-gentoo-r4 shows 2869 and a du of vmlinux.bz2 shows 2870?

I tried gzip as well

```
gzip -c kernel2.6.30-gentoo-r4 > vmlinux.gz
```

du of vmlinux.gz is lower than the bz2 but still higher than 1.8MB.

And the overall problem, when attempting to boot either, an error is given on the compression medium used. Though I am unsure if this is related to the size restriction or not.

So can anyone tell me how I can either strip useless stuff from the kernel (there's a TON of stuff I don't need, this server's hardware is very restrictive), or compress it so I can boot this bad boy?

P.S. if anyone is interested, I'm trying to get this running on a Sun Cobalt RaQ 550 1U server, however due to the limitations, it has to be installed on a host machine first and then transfer the hard drive to the RaQ and boot. I keep getting kernel panics (the RaQ has a kernel firmware, not bios) when it tries to decompress the Gentoo kernel I made on /dev/hda1 or /boot

----------

## Xarthisius

Kernel (arch/YOUR_ARCH/boot/bzImage) is already compressed, you can choose gzip/bzip2/lzma in General setup/Kernel compression method.

As for the general problem I am complete layman, but quick googling suggested that:

1. you may need some patches for kernel 

2. you can bypass the size restriction with "CoLo" Best regards,

Xarthisius

----------

## geki

build your own kernel.

the only thing you can do is to know about the hardware. disable all useless features.

the other thing would be:

make menuconfig

> General Setup

> > Optimize for size [magic  :Wink:  -Os instead of -O2 optimization]

> > Configure standard kernel features (for small systems) [CONFIG_EMBEDDED to avail more options]

the EMBEDDED config switch allows to set dangerous flags. I recommend to disable only debugging options!

or you may trash your system.  :Smile: 

you have been warned.

know what you got, know what you need and go for it!  :Wink: 

----------

## cach0rr0

you might look at 2 things

-build from one of Pappy's seeds; they already have loads of stuff pruned out that you don't need

-see my sig (which I got from pappy's page) for doing drivers

-build modules as often as you can; your IDE support needs to be built-in, support for your root filesystem needs to be built-in, but beyond that most things can be done as modules. (note this goes against my usual recommendation for servers, but this is a special case)

----------

## Carnildo

Making a small kernel for a server isn't hard (the complete non-modular kernel for my server is 770k): you simply need to know what hardware you've got and what features you need, and only enable those.  Stay away from the "standard kernel features" menu unless you know exactly what you're doing: it's easy to make a non-functional system there, and you don't really save all that much space.

----------

## Crimjob

I just redid everything in there, cutting out a bunch of crap, and enabling bzip2 compression in the kernel. Compiling as I type.

I apologize as I'm a bit new to this, but does this mean I can just cp the arch/kernel/boot/bzImage to /boot/vmlinux.bz2?

----------

## Carnildo

Should work, though it might be expecting the kernel to be called "bzImage" or plain "vmlinux".

----------

## cach0rr0

you can name the kernel image whatever you wish, so long as your boot loader knows the name

in my case i always just name things "kernel-new" and "kernel-old", for most recent and previous kernels respectively

e.g.

```

cp /usr/src/linux/arch/x86/boot/bzImage /boot/kernel-new
```

and in my grub.conf

```

title Gentoo Linux 2.6.30-r5

root (hd0,0)

kernel /boot/kernel-new root=/dev/sda3

```

----------

## Crimjob

No grub / lilo for me, there's a firmware kernel installed that takes args for booting.

From an attached console, I can see this:

```
Second stage kernel: Decompressing -Error bunzip2ing kernel. Trying gunzip

ERROR: unknown compression method

ERROR: internal error, invalid method

Error gunziping kernel. Giving up. 
```

Now I've been tearing apart my kernel, managed to get it to 1.7MB, but it's showing as 1824896Bytes. I'm not sure what else I can rip from the kernel without breaking something or touching something I have no idea about right now.

I'm hoping that boot error is because the kernel is still technically above 1.8MB (in bytes?), but it could also be the way I compiled and copied?

I assumed by changing the compression method in the kernel to bzip2 that it would compile and compress using that method. I then assumed I could just

```
cp arch/linux/x86/boot/bzImage /boot/vmlinux.bz2
```

and it would work, but I'm not 100% sure? All I have to work with are 4 options. The kernel has to follow these options or it won't boot. The kernel must be named /vmlinux.gz, /vmlinux.bz2, /boot/vmlinux.gz or /boot/vmlinux.bz2, hence why the extension must be there.

I'm going to have to go through everything I have left in the kernel, with google, or preferably an expert, to rip whatever else I can to make this kernel smaller, and try again to see what happens. If anyone see's anything wrong with (or knows another way) the way I copied the kernel to boot, please let me know.

On side notes, I'd like to keep this box as Gentoo as possible, primarily due to the community and support. I had Strongbolt running on this box okay, but the support / updates / etc. were just garbage. As far as drivers and the such go, I've gone as light as I can (except for a few things the kernel won't let me remove, by showing a "-*-", even though I don't need them), and only built in things like chipset, network, etc. and I have no use for any input devices or display, etc. I've Y'd the optomize for size and standard kernel features.

I also know there are patches for kernels for my hardware, however I believe that to be somewhat specific to the "special" features the box has, as it is mostly PC hardware under the hood. For the time being, I could care less about the front panel LCD and control buttons  :Smile: . I'll worry about those once I have a running system. As far as "CoLo" goes, I haven't been able to find any documentation on it so I'm not sure how to use it :S

Gotta let my head rest tonight, will stab back at it in the morning. If anyone's available to assist with my kernel shrinking over IRC, let me know  :Smile: 

----------

## geki

 *Carnildo wrote:*   

> Stay away from the "standard kernel features" menu unless you know exactly what you're doing: it's easy to make a non-functional system there, and you don't really save all that much space.

 > General Setup

> > Configure standard kernel features (for small systems)

> > > Load all symbols for debugging/ksymoops [strip symbol names for backtraces -- saved 130K here -- useless for production kernel]

> > Strip assembler-generated symbols during link [another symbol strip]

> Kernel hacking

> > Early printk

I disable these that are only visible for EMBEDDED.  :Surprised: 

- wrt the size, does it require 1.8MB or 1.800.000 bytes? the former would be fine now.

- the copy looks fine, too.  :Wink: 

are you sure the path to the kernel image is okay at boot? maybe it just does not find it instead of bad file. sometimes error messages are fubar.

- wrt the kernel config pastebin(see my sig) your kernel config and pastebin the kernel config generated by cach0rr0 sig. that may help.

----------

## pigeon768

 *Crimjob wrote:*   

> and only built in things like chipset, network, etc.

  You don't need network built into the kernel - build that stuff as modules. Go through again, only try and move as much as you can from <Y> to <M>. About the only thing you'll need built into the kernel is your disk driver, root FS, VESA driver - build everything else as modules.

There's also a lot of size to be saved in debug stuff - but I assume you've already stripped that out.

Out of curiosity - what hardware is this?

----------

## Crimjob

I did poke around in some scary areas, basically said N to anything in there that seemed completely pointless, specific to another arch or hardware type, and some other things, but I didn't touch General Setup, that ksymoops one might be enough for me right then and there.

I've read everywhere it needs to be below 1.8MB, doesn't specify 1,800,000 bytes, but the fact I'm still getting that error makes me think that's the case (unless the compression is fubar'd). As the first stage kernel boots, it basically works as so:

Check for /boot/vmlinux.gz file

Check for /vmlinux.gz file

Check for /boot/vmlinux.bz2 file

Check for /vmlinux.bz2 file

If it can't find any of them, it panics and reboots. If it finds one at any point while checking, it attempts to decompress and boot it. I believe it is a limitation of the first stage kernel not being able to boot a large second stage kernel. The first stage kernel is a customized 2.6.10 kernel that's flashed into a ROM chip on the motherboard. No interest in going there and bricking my system, and the 2.6.10 is the latest (Sun stopped developing updates for this hardware, but there's still a small open source group dedicated to bringing some updates like rom's). I'm looking into "CoLo", which basically looks like I'll be chain loading kernels. CoLo is supposed to be a smaller kernel bootable by the rom, which can then boot the kernel you want to run. The only pain for this is then I'd be chain loading 3 kernels in a row, a little painful for boot time, though I don't plan on booting it too much.

How can I paste the kernel config? (I'm a little rusty), I'm aware of pastebin and all, but the actual config file and how to go about copying it from a cli.

The hardware is Sun Cobalt RaQ 550

   * RaQ 550 (5000 series):

     - Intel Pentium III Tualatin 1.3GHz processor (rare processor  :Smile:  )

     - ServerWorks LE chipset with CSB5 south bridge

     - 2 PC133 DIMM slots (registered ECC DIMMs required, 2GB in there now)

     - 1 32/64 bit PCI slot

     - 2 National Semiconductor DP83815 10/100 ethernet controllers

     - 1 or 2 ATA100 disks, raid capable

http://www.rtk.jp/img/2009040300004.jpg

It's a very interesting piece of hardware, and very low power (PSU is 70 or 80 watts). Loud as hell stock but I have some low noise replacements fans on the way. My intent would be to use this as my router, as my current router is running 2k8 server and eating 500w of power (and malfunctioning like every Microsoft thing does). I had Strongbolt Linux running on it, which supports net booting, though it uses NFS not PXE. It ran okay with a nice interface (google BlueQuartz), but there was definitely something funky going on with it. I had IPtables on it, though I'm a n00b, it still should have ran okay. I was able to get it running for an hour or so then it would crap out. Restarting the script would make everything work again, but doing this every hour was annoying as hell. Tried multiple versions / configs / kernels but no luck. I had a feeling some hourly scheduled cron job was causing it, but nothing was scheduled :S. So I gave up on it since it was Strongbolt V1, and even their Strongbolt V2 forums have maybe 20 posts from 2009 on them? Support was lacking, or costing (they charge for their products), so I decided to try something more mainstream that I wont have iptables problems with (though as you can see I'm having other problems  :Smile:  )

Gonna rip it to below 1,800,000 bytes, and try again. I will also compile 2 kernels to (hopefully) save some time, one bzip2 one gzip. If those fail I might try CoLo. I've got a good feeling it's just the size issue.

----------

## Crimjob

 *geki wrote:*   

>  *Carnildo wrote:*   Stay away from the "standard kernel features" menu unless you know exactly what you're doing: it's easy to make a non-functional system there, and you don't really save all that much space. > General Setup
> 
> > > Configure standard kernel features (for small systems)
> 
> > > > Load all symbols for debugging/ksymoops [strip symbol names for backtraces -- saved 130K here -- useless for production kernel]
> ...

 

Hrmph. I am unable to get rid of most of that stuff. ksymoops and early printk (or any printk) are all shown as -*- not allowing me to remove them. I couldn't find the assembler generated symbols. Scsi support is also -*-, when I have no use for it.

 *pigeon768 wrote:*   

> There's also a lot of size to be saved in debug stuff - but I assume you've already stripped that out.

 

I did manage to find some more stuff I missed and ditched it.

Building the gzipped kernel now, then trying the bzip2 kernel. Here's to hoping.

----------

## cach0rr0

im lost at why you're gzipping an already bzipped kernel

----------

## Crimjob

Sorry I meant then trying the bzip2, not trying to bzip2. All I'm doing is adding the extension according to what I select in the kernel.

So I managed to get the kernel nice and small ripping all the debug stuff. Same problem though, now I'm getting real confused.

I copied all the output of the console from start to end so everyone can see what's going on. It will probably make a decent amount of sense since it is a linux kernel booting in the first stage anyways.

http://pastebin.com/d436aa8f

----------

## xaviermiller

Hello,

Don't forget that the kernel is uncompressed before to run.

The kernel compression is (was) useful only to fit to small devices like diskettes.

----------

## Crimjob

Do you mean the kernel is not compressed even with the option selected in the kernel?

----------

## geki

 *Crimjob wrote:*   

> Hrmph. I am unable to get rid of most of that stuff. ksymoops and early printk (or any printk) are all shown as -*- not allowing me to remove them. I couldn't find the assembler generated symbols. Scsi support is also -*-, when I have no use for it.

 so you have enabled options that depend on them. either you did not remove enough useless features or some feature of yours just needs them..?  :Smile: 

the assembler generated symbols is in: General Setup

----------

## Crimjob

The only thing's I've actually added are drivers, everything else has been a remove. Just used gentoo-sources. I couldn't find anything that depended on it, even the generic enable / disable printk all together is like that.

----------

## Crimjob

Well don't I feel silly 

http://www.gentoo.org/doc/en/handbook/handbook-mips.xml?part=1&chap=10#doc_chap2

Going to try that and see what happens.

----------

## Crimjob

I retract my previous statement. After finally figuring out how to unmask that package, I found out it's only for MIPS arch, I'm on the newer x86. 

Point of interest came up. I was just playing around with everything and I saw a line near the end of kernel compilation said something along the lines of GZip and a path to a gzipped vmlinux.bin.gz. For giggles I pulled a cp arch/x86/boot/compressed/vmlinux.bin.gz /boot/vmlinux.bin.

```
BOOTLOADER: Mapping in physical locations

BOOTLOADER: load_addr=0xc2004000 ret_data=0xc2205cc4

BOOTLOADER: opening "/boot/vmlinux.bz2"

BOOTLOADER: opening "/vmlinux.bz2"

BOOTLOADER: opening "/boot/vmlinux.gz"

BOOTLOADER: reading "/boot/vmlinux.gz"

BOOTLOADER: read 1686097bytes

BOOTLOADER: unmounting /

BOOTLOADER: calling reboot notifiers

md: stopping all md devices.

flushing ide devices: hda

BOOTLOADER: mapping 22M-32M for ride home

BOOTLOADER: disabling interrupts

BOOTLOADER: flushing cache

BOOTLOADER: Leap of faith!

Back in ramcode: done

Second stage kernel: Decompressing -Error bunzip2ing kernel. Trying gunzip.    - done

command line: 'console=ttyS0,115200 debug ip=off '

booting kernel...

```

Unfortunately this is where it loops and reboots. It's an interesting change to what I've been seeing so far though.

I downloaded CoLo from the site and extracted it to the same place Gentoo portage would have. I'm going to try using it that way and see what happens. Here's to hoping again

----------

