# [TRICKS] Speed up system with aufs + squashfs

## stanbogdan

Introduction:

    Hello guys, here's a thing I've been working on lately, trying to speed up my gentoo system and free some hard drive space in the process. It all starts from the observation that these days the hard disk is the choke point of the operating system in most cases; you spend a lot of time waiting for applications to load their data from the disk, while the processor rarely reaches 100% usage (my laptop's Turion 2GH cpu usually sits at 800mhz, since I'm using the ondemand cpu freq governor). So wouldn't it be nice if you could transfer some of the effort that the disk spends reading data to the processor? Actually, there is a way to do this, it's been around for a long time, you can compress the data on the disk, so that you load less data and spend a bit of extra time uncompressing it. Most applications, though, don't keep their data in compressed format, so we need to help them a bit. In this article I will tell you how you can improve data loading speed and free some disk space in the process, by using the aufs and squashfs file systems. 

Warning: if you don't feel comfortable with recompiling your kernel, editing startup scripts, and generally handling by yourself different problems that you bump into, you'd better skip this one, as it requires a lot of manual work to get going, and I am sure that a lot of things _could_ go wrong, depending on your system and the way you do the procedure I am just about to describe. I won't be held responsible for any losses that you might have because you 'just thought to try this', without fully understanding what is happening and how to cope with problems that might appear.

What you need to do:

Step 1: Recompile your kernel with squashfs support. You can get it from http://squashfs.sourceforge.net/ . I recommend you compile the module into the kernel, as it will be needed from the very start of the boot process anyway... 

[EDIT:] As of gentoo-sources-2.6.24 , squashfs is in the kernel already, so nothing to do at this step, if you have an up-to-date kernel

Step 2: Download and install aufs from http://aufs.sourceforge.net/ . You will need to make and install it. Be sure to read how to use it, because you will mount it a bit differently compared to your other file systems. If you ever used unionfs, you know what I mean. After installing it, add it to /etc/modules.autoload.d/kernel-2.6 .

Step 3: Restart your system with the new kernel and play around a bit with aufs and squashfs, to be sure you understand how they work. It's better to do it now, when it is safe, to avoid having difficulties later on

Step 4: Move your portage distfiles folder outside of /usr . We are going to compress the /usr folder into a squashfs image, there's no use in adding a whole bunch of archives to this image, it will just add a lot of time to the compression process... Just move /usr/portage/distfiles to a different location and add the line DISTDIR="<new_folder>" to make.conf

Step 5: Create the following folders (just a suggestion, you can choose different names/locations, after you understand the process):

```
/squashed/usr

/squashed/usr/ro

/squashed/usr/rw
```

Step 6: Compress the /usr folder into a squashed image in /squashed/usr. Just run 

```
mksquashfs /usr /squashed/usr/usr.sfs -b 65536
```

to do this. Note that you can play around with the block size for the squash file system, I didn't run enough tests to be sure that 64k is best for all cases, but I think it is a good enough value to start with.

Step 7: Add the following lines to /etc/fstab:

```
/squashed/usr/usr.sfs   /squashed/usr/ro   squashfs   loop,ro   0 0

usr    /usr    aufs    udba=reval,br:/squashed/usr/rw:/squashed/usr/ro  0 0
```

Step 8: Open /sbin/rc and look for the call to halt.sh. I should be in the part where the reboot and shutdown softlevels are processed. Add the following lines just BEFORE the 'LC_ALL=C exec /etc/init.d/halt.sh "${SOFTLEVEL}"'  line:

```
einfo "Unmounting stubborn squashfs folders"

umount -l /usr

umount -l /squashed/usr/ro
```

[EDIT:] I guess that steps 7&8 could be implemented as a boot script...

Step 9: Reboot and test that your system still works as expected. You should notice a speed improvement (even though at the moment you are using more disk space). 

Step 10: unmount /usr (the aufs mount), rm -rf /usr/* (remove all the initial /usr folder contents!!! Might want to make a backup instead, to be sure nothing bad happens here). Then, re-mount /usr.

Step 11: You are done, enjoy!

Step 12: Once in a while (every couple of weeks, or whenever you think /squashed/usr/rw is already taking too much space), re-make the usr.sfs image. Do something like this:

```
mksquashfs /usr /squashed/usr/usr_tmp.sfs -b 65536

umount -l /usr

umount -l /squashed/usr/ro

rm /squashed/usr/usr.sfs

mv /squashed/usr/usr_temp.sfs /squashed/usr/usr.sfs

rm -rf /squashed/usr/rw/*

mount /squashed/usr/ro

mount /usr
```

    You can/should compress other folders this way. My suggestion is that you at least also compress /opt and /lib, but you could also compress /bin and /sbin. WARNING: DON'T delete the original folder contents of the /lib folder after compressing it. Its contents are needed by the mount.aufs utility itself. If you delete it, you will get a broken system and will need a LIVECD to repair things. Just mount the aufs /lib over the initial /lib folder. You will waste a bit of disk space this way, but you will still get the speed increase, and you are going to be safer. As a matter of fact, any folder that is small enough to begin with (/bin /sbin /lib ) should be treated the same. Don't bother compressing folders that contain non-compressible data, or that change often (/home, /root, /var). Also, for folders that contain a lot of smaller files, you can think about decreasing the squashfs block size to 32k, or 16k, because otherwise you might lose on the speed improvement. Feel free to test different scenarios.

Note: As you have probably already understood, the write speeds are not affected, since all writes are done onto the base file system, in /squashed/usr/rw. Files that get written to once will be read from the rw folder afterwards, therefore losing any speed benefit. That's why it makes no sense to apply this procedure to folders that change frequently.

Benchmark results:

   I have compressed /bin /sbin /lib /opt /usr and then tried the following operations with and without the squashfs images mounted:

1) start the system up to the login screen

2) login and wait until desktop is usable (session is restored, hard disk stops screaming)

3) start openoffice-writer

4) start eclipse

5) generate random file access pattern and record the total time to read all files. 

```
# find /usr -type f -printf "%s %p\n" | sort -R | awk '{ printf $2; printf "\n" }' > /sort

# time cpio -o --quiet -H newc < sort > /dev/null
```

Times without squashfs+aufs:

1) 7.5 seconds to load the kernel, another 10 seconds until file systems are mounted (up to this point no speed increase is possible by this method), another 42 seconds until the login screen pops up (out of which 24 seconds are just for waiting on the network interfaces). A total of 60 seconds, out of which only 18s are relevant to our benchmark

2) 16s

3) 9s

4) 25s

5) 36m31.726s

Times with squashfs+aufs:

1) total time: 58s, out of which 16s are relevant to the benchmark -> 11% decrease in time

2) 10s -> 37.5% decrease

3) 6s -> 33% decrease

4) 22s -> 12% decrease

5) 9m48.873s -> less than 1/3 of the normal time!

Also, the /usr folder only takes 1.1Gb on my disk, compared to 3.1Gb it had before; the /opt folder takes 221mb, instead of 544mb, the others take a bit more right now, since i haven't deleted the original files, just mounted over them (12mb more... barely noticeable).

Note: I'm open to suggestions about other benchmark procedures to use here... Please keep in mind that I need something that will check the read performance for files that already exist on the disk, otherwise it's useless (bonnie++ can't be used, as far as I can tell)

Node: I have 1 GB of RAM, which, as far as I know (I don't pretend to be a linux guru), is being used by the OS to do disk caching as much as possible. And yes, the OS would be caching the compressed sectors (not the uncompressed data) from the squash file every time you try to read something from /usr. Considering that the compress ratio for /usr is almost 3x, you suddenly have 3 times more disk cache than before, without investing in extra RAM chips  :Wink: . If you have something like 2gb ram, you will probably never need to load the same data from /usr twice during a session, as your compressed /usr probably fits entirely into your free ram. You can imagine how fast a simple decompress from memory is, compared to a read from the hard-drive.

NOTE: If you are using squashfs 3.3 (and you probably are at the time I wrote this guide), use the -no-sparse argument when calling mksquashfs. I've had problems with corrupted files without this option, and Phillip Lougher (creator of squashfs) said this is due to a bug in the current release. It would have been a nice option, but not one that will make too much of a difference, anyway... Read the change log for future versions of Squashfs to see when this bug gets fixed.

Credits:

I initially got the idea by reading https://forums.gentoo.org/viewtopic-t-465367-highlight-aufs.html (synss is explaining how to compress the portage folder to save a lot of disk space and to increase the portage syncs speed)

Lots of thanks to Phillip Lougher, the creator of squashfs, for giving me the chance to implement something like this and for providing me with answers to some squashfs problems that I had.

Comments and more test results from your machine are very much welcome.

Note: you can read more about how much better squashfs is compared to other filesystems (including ext3) here: http://elinux.org/Squash_Fs_ComparisonsLast edited by stanbogdan on Wed Feb 06, 2008 9:00 am; edited 8 times in total

----------

## Bones McCracker

Seems like a good idea for a system that's relatively static (like embedded or firewall/router).

----------

## pussi

Thanks for the tip.

I think I'm going to try this :)

----------

## stanbogdan

I suggest that if you just want to give it a try, then create a squash image of the /lib32 (or /lib64 , if you have a 64-bit processor) and mount it using squashfs + aufs over the initial /lib folder. Don't delete anything, it would be unsafe, and expect the system to hang at the end of the shutdown process, because someone is bound to still have files opened from /lib (and therefore, the aufs daemon will refuse to unmount it). You can either just wait at that moment, it will eventually shutdown, after a number of retries, or you can edit the shutdown part from /sbin/rc to call a lazy unmount for the aufs and the squashfs mounts -> that will take care of any problems.

After doing this, you can run benchmarks with and without this setup, see if it helps even a little bit. Please post back any results you get  :Smile: 

----------

## pussi

why /lib? wouldn't /usr be safer and more effective?

----------

## stanbogdan

Well, for /lib I was suggesting that you only make a squashed image and mount it over the original /lib folder, without deleting it first, so as far as safety goes, you don't have to worry about anything, if the mount fails, the original data is still there. Squashing /usr will take a lot longer (it has 8.5 gb on my machine, that takes a while to compress...), and in order to keep it safe for the initial tests, you would also have to keep the original folder, therefore spending a lot of your hard drive space until you finally decide it is safe to delete it entirely (well, almost entirely, as I said, some things from /usr/bin would have to be kept in order to allow the mount.aufs script to run). If you have the hard drive space and time to also compress /usr, feel free to do that also, it should definitely add to the performance.

P.S. if you want to make extensive tests and play a lot with different possibilities, I suggest you keep a LiveCD handy, just in case something goes wrong...

----------

## pussi

ah.. good point. I was going to mount the squashfs over the orginal anyway just in case. My /usr is only 2.6G though so I might just go with it.

----------

## stanbogdan

Just updated the initial post with more detailed information and some other benchmark results.

----------

## stanbogdan

And yet even more benchmark results added to the initial post (the random access pattern part)

----------

## waynesherman

First of all, Stan, thanks for your work on this tip.

I ran across a problem not documented here.  When I tried to reboot after deleting the contents of my /usr directory, I ended up with lots of error messages.  It turns out that my aufs mount failed and /usr was empty.  Looking back at the log revealed why:

1) The aufs mount in /etc/fstab depends on mount.aufs

2) mount.aufs is a shell script which happens to depend on 4 files in /usr/bin:

```
/usr/bin/comm

/usr/bin/diff

/usr/bin/paste

/usr/bin/tee

```

3) Since the /usr directory was empty at that point in the boot process, mount.aufs failed.

The solution to fix the immediate problem is to create a /usr/bin directory and copy those files to it (or when you delete everything in /usr, don't delete the /bin directory and those 4 files).  That way when mount.aufs executes, it can find the 4 files.  aufs will then mount the unioned /usr on top of the existing /usr which in effect hides those files.

A problem remains:  I don't know how those files are going to receive updates.  Any ideas?

Has anyone else seen this behavior?

If yes, how did you fix it?

If no, why does it happen on my system and not others?

Thanks,

Wayne

----------

## stanbogdan

 *waynesherman wrote:*   

> First of all, Stan, thanks for your work on this tip.
> 
> I ran across a problem not documented here.  When I tried to reboot after deleting the contents of my /usr directory, I ended up with lots of error messages.  It turns out that my aufs mount failed and /usr was empty.  Looking back at the log revealed why:
> 
> 1) The aufs mount in /etc/fstab depends on mount.aufs
> ...

 

I have noticed the same problem the first time I tried this trick, and the solution was, as you have tried, to keep all the files needed from /usr/bin. I guess that you could just keep a complete stage3 /usr, to allow not only for mount.aufs to work, but to also have a working system if you run into any problems (vim, gcc, emerge, etc), without needing a LiveCD nearby. I will need to further investigate this matter at some point, right now I am trying to make up some scripts to make things easier for anyone. Until then, if you don't really have hard drive space issues, you can keep your /usr folder untouched and just mount the squashfs+aufs folder on top of it, getting the speed benefit, but losing some space on the drive. 

About your question on how the files you mentioned are going to receive updates... I guess they won't, under these circumstances, not unless you update them manually from time to time (unmount /usr, copy those files from /squashed/usr/ro/bin to usr/bin, remount /usr). But then again, they shouldn't need an update at all, not unless they depend on some library that becomes obsolete and gets deleted from the file system (but, then again, you are keeping your unsquashed /lib folder untouched, right?)

----------

## stanbogdan

UPDATE: If you happen to have (or are able to buy) a fast and big enough usb stick (I got my hands on a Corsair Voyager GT, 20MBps sustained read speed), you can copy the squashed image on it and mount it from there. Not only does the usb stick have a MUCH faster random access speed (2 or 3 times faster, on average, due to the fact that it has no moving parts), but it can be accessed in parallel to your hard drive, therefore taking even more pressure off it. If you want to do this, you should really keep your unsquashed /usr folder untouched on the main drive, you don't want to be unable to use your computer just because you don't have the usb stick nearby...

----------

## Bumbl

 *stanbogdan wrote:*   

> UPDATE: If you happen to have (or are able to buy) a fast and big enough usb stick (I got my hands on a Corsair Voyager GT, 20MBps sustained read speed), you can copy the squashed image on it and mount it from there. Not only does the usb stick have a MUCH faster random access speed (2 or 3 times faster, on average, due to the fact that it has no moving parts), but it can be accessed in parallel to your hard drive, therefore taking even more pressure off it. If you want to do this, you should really keep your unsquashed /usr folder untouched on the main drive, you don't want to be unable to use your computer just because you don't have the usb stick nearby...

 

this would be something like vistas readyboost technologie, wouldn't it?

would you possibly be so kind as to do another benchmark run with the usb-stick because i'm playing with the idea of buying me one of the 16gb crosair sticks, because: you could never have a fast enough system, i like the idea, it would be cool and it would be something to play with (which is why i use gentoo)

----------

## stanbogdan

 *Bumbl wrote:*   

>  *stanbogdan wrote:*   UPDATE: If you happen to have (or are able to buy) a fast and big enough usb stick (I got my hands on a Corsair Voyager GT, 20MBps sustained read speed), you can copy the squashed image on it and mount it from there. Not only does the usb stick have a MUCH faster random access speed (2 or 3 times faster, on average, due to the fact that it has no moving parts), but it can be accessed in parallel to your hard drive, therefore taking even more pressure off it. If you want to do this, you should really keep your unsquashed /usr folder untouched on the main drive, you don't want to be unable to use your computer just because you don't have the usb stick nearby... 
> 
> this would be something like vistas readyboost technologie, wouldn't it?

 

Yep, it would be similar to Vista's readyboost, just that instead of caching only the files that prove to be frequently accessed, it caches the files that YOU want to be cached, preferably all of them (if you have a big enough usb stick). Also, the fact that the data is compressed allows you store more data, and (I suspect, might be different for the usb stick), read it faster.

 *Bumbl wrote:*   

> would you possibly be so kind as to do another benchmark run with the usb-stick because i'm playing with the idea of buying me one of the 16gb crosair sticks, because: you could never have a fast enough system, i like the idea, it would be cool and it would be something to play with (which is why i use gentoo)

 

Preliminary tests show that it is a LOT faster than reading from the hard drive, at least for random reads. I created a random access pattern for 20% of all files from my /usr folder, and tried to read all of them with the squash image on the drive, then with the squashed image on the stick. The stick-mounted image was two times faster... I have changed a lot of things on my drive since the previous benchmarks, so I will just run all the tests mentioned in the main post again, in all 3 cases: with no squashed image, with a hard-drive squashed image, and with a flash stick image. I'm also done with writing a script that will allow for a much better management of squashed images. Expect a new post tomorrow, after I am able to run a couple of more organized tests.

----------

## stanbogdan

New benchmark results, after testing out the Corsair Voyager GT flash stick. The list of tests that were run is:

1) Start the computer and time how long it takes after you enter your password in the login screen, until the desktop is fully usable (hard disk lights stop blinking)

2) Start a lot of programs, save the session, reboot and time how much it takes to go from the login screen to the moment when the entire session is completely restored (same as 1, but with a bigger session to restore)

3) Test sequential read speed for a part of the file system: 

```
#time tar cf /var/tmp/portage/tmp /usr/lib64
```

Note: /var/tmp/portage is mounted in RAM as a tmpfs mount (did it to increase emerge speeds)

4) Test random access speed for a part of the file system: 

```
# find /usr -type f -printf "%s %p\n" | sort -R | awk '{ printf $2; printf "\n" }' > /sort

# delete last 80% lines from /sort 

# time cpio -o --quiet -H newc < /sort > /dev/null
```

Note: I deleted 80% of the sort file because the entire squashed file fits into my RAM, therefore, once a sector from the squashed file system is loaded into memory, it would never be flushed out during the test. Any subsequent reads of files from that sector would be very quick and would not be dependent on the file system or its support (stick / hard-disk)

5) Same test as the one at point 4, but with a reasonably high stress on the cpu (4 consoles doing cat /dev/urandom > /dev/null). The cpio process was unable to get more than 20% of the CPU, as checked with top

Result set 1: no squashed image

1) 6.5s

2) 29s

3) 130s

4) 608s

5) 596s

Result set 2: squashed image on hard-disk

1) 4s

2) 23s

3) 41s

4) 193s

5) 232s

Result set 3: squashed image on usb flash stick

1) 3.5s

2) 20s

3) 32s

4) 74s

5) 228s

Conclusions: 

1) Using a squashed image of your /usr folder visibly increases overall performance of your system, as far as disk loading times are concerned

2) Using a high-speed USB stick as a support for the squashed image further increases access speeds, especially when doing sustained random reads from the disk. 

3) Having a high cpu-load while trying to access the disk has no effect in the normal case, but the squash image performance is seriously affected, because it needs cpu time to uncompress the data loaded from the disk. The performance still does not decrease anywhere near the normal performance (with no squashed image), and you would normally expect an application to first load its data, and only then process it. Still, this means that while reading a lot of data from your squashed /usr folder, you might notice a system-wide slowdown. Not that you wouldn't also see that in the normal case, with the hard-disk screaming like it's about to break down while trying to access hundreds of files per second...

P.S. As I said in my previous post, I have simplified the process of making the squash file, mounting it from different locations, etc. I need to brush up on the script, though, at the moment I still have problems from time to time, having to shut down using the power button... I'll update the main post when I'm done with it, at which point I will also include these last benchmarks in it.

----------

## grayskunk

I have a file server with many compressed, read-only .tar.gz files. Would a squashfs decrease the size of compressed files to where it is worth doing?

----------

## stanbogdan

I doubt that squashfs would be able to compress your files into a smaller size than your current situation... It uses the same compression algorithm (GZIP), but only compresses each sector separately (4k - 1mb at a time, depending on what you choose when creating the squashed file). Also, you would have to keep your data in uncompressed format in your squashfs file system (they would be compressed by squashfs itself, but from the outside they will seem to be uncompressed), which, I guess, is not what you want for a file server. Anyway if you are looking for a way to optimize access to your .tar.gz files, and/or decrease the amount of space they occupy on the drive, using a squashfs image is not going to help, or is not going to help enough to be worth the effort. 

You might have better luck just compressing your /usr folder, that could save some useful space that you can use to store more files on your server, though...

----------

## grayskunk

thanks for the I will try out your tip on my /usr folder. Thannks!

----------

## Phoenix591

would anyone be interested in a bash script that automaticly copies your current kernel configuration (stored in a seperate file) to your new kernel (presymlinked to /usr/src/linux) and patches it for aufs, and builds it? its (pretty) well commented. and Works for Me (tm)

----------

## drescherjm

There is an ebuild for aufs on bgo

https://bugs.gentoo.org/show_bug.cgi?id=160065

----------

## Phoenix591

well, it still was  a good chance to improve my bash   :Smile:  , and it just simply improved my  existing script

----------

## drescherjm

Could you PM me the script though? Please.

Thanks,

----------

## Phoenix591

lost it . I foolishly just copy and pasted the posts instructions to update /usr . there was a typo. lost everything relitivly recent in my /usr.

----------

## drescherjm

Thanks and sorry to hear that.

----------

## SlashBeast

Work for me, THX!   :Wink: .

Imho good idea is mv /usr/src to (for example) /squash/non-squashed/src and link /usr/src to /squash/non-squashed/src.

Without src and distfiles

```
jinchuuriki slashbeast # du -sh /usr

3,1G   /usr

jinchuuriki slashbeast # du -sh /squash/squashed/usr.sfs 

1,4G   /squash/squashed/usr.sfs
```

Faster, smaller, squashy rox.  :Very Happy: 

----------

## kokoko3k

First of all, Thanks!

 *Quote:*   

> And yes, the OS would be caching the compressed sectors (not the uncompressed data) from the squash file every time you try to read something from /usr. 

 

Are you sure about this?

Also, i wonder if it is possible to append rw files to the previously created squashfs file, instead of create a new one from zero (it's a time consuming task!), or to manage multiple squashed files (it should be possible, but i'm not really in aufs)

----------

## stanbogdan

 *kokoko3k wrote:*   

> First of all, Thanks!
> 
>  *Quote:*   And yes, the OS would be caching the compressed sectors (not the uncompressed data) from the squash file every time you try to read something from /usr.  
> 
> Are you sure about this?
> ...

 

Hmmm, I will investigate further, I don't know for sure, but I feel that is what is happening, because of the following symptom: when doing the random access pattern, the squashed file system test was more than 3 times faster than the normal test (a LOT more), even though the compression factor was about 3x. I have no other way to explain this, other than the fact that the entire squashed image of my /usr folder (which is appx 1gb) could be loaded entirely into my RAM (1gb), one sector at a time, and therefore a lot of the random disk reads were actually directly from the OS disk cache, as opposed to the normal case (no squashed image), where the 3gb of information would not fit and therefore a lot of sectors would be loaded into the disk cache, then unloaded, then loaded again, etc.

 *kokoko3k wrote:*   

> 
> 
> Also, i wonder if it is possible to append rw files to the previously created squashfs file, instead of create a new one from zero (it's a time consuming task!), or to manage multiple squashed files (it should be possible, but i'm not really in aufs)

 

You can append data to a squashed image, but you can't delete anything from it, so if you modify a file (that is already contained in the image) and try to re-add it to the squashed image, you would have a filename conflict that squashfs would solve in a non-convenient way (by renaming the second file). You don't have to re-make the squashed image too often, and you can also leave it over night, if it takes too much. You could manage to use multiple squash files with aufs / unionfs, and that would eliminate the file name conflict problem, but you would end up using too much disk space, as some files would be included in multiple squashed images (in different versions, of course) and it will be a bit more complicated to manage them. If you want to go for this last option, I could give you some pointers on how to use aufs for this purpose, just give me a message.

Thanks for trying my trick and for giving me feedback, maybe I will actually make some time to make this easier to use...

----------

## medius

I've been playing with this trick for a few days now.  I've found it pretty effective and wanted to contribute my squashmount init script.  It is more of an outline, and could definitely  be rewritten to take advantage of a conf file in /etc/conf.d but it works for me.  

First some background:  I have a fast 8GB usb flash drive that hosts the squashed images of my /usr and /opt directories.  The flash drive is normally mounted on /mnt/patriot (it is a patriot drive) and is mounted based on the device id.  However if the drive is not available for some reason, then I want the initscript to mount the usr.sfs and opt.sfs that I store on my local disk.  This uses more drive space, but I'm doing this mostly for performance reasons so I prefer to have the security of a local copy of the squashed images.

I've modified my /etc/init.d/localmount so that it does not mount devices of type aufs or squashfs.  Then my squashmount script mounts my opt and usr directories from flash if available, or from disk if not available.

I also modified any other init scripts that need localmount to need squashmount.  This ensures that any init script that needs /usr will not run until after /usr is available.

Since the initscript also has a stop section I let it handle unmounting /usr and /opt and the flash drive.

Here is the snippet from my /etc/fstab that supports my /etc/init.d/squashmount

```

/dev/disk/by-id/usb-_Patriot_Memory_078304AD0A3F-0:0-part1  /mnt/patriot  reiserfs  rw,user,noatime,sync  0 0

/squashed/usr/usr.sfs  /squashed/usr/ro  squashfs  loop,ro,noauto  0 0

/mnt/patriot/squashed/usr.sfs  /squashed/usr/ro  squashfs  loop,ro,noauto  0 0

usr  /usr  aufs  udba=reval,br:/squashed/usr/rw:/squashed/usr/ro  0 0

/squashed/opt/opt.sfs  /squashed/opt/ro  squashfs  loop,ro,noauto  0 0

/mnt/patriot/squashed/opt.sfs  /squashed/opt/ro  squashfs  loop,ro,noauto  0 0

opt  /opt  aufs  udba=reval,br:/squashed/opt/rw:/squashed/opt/ro  0 0

```

Note the noauto options on the squashfs entries.  This lets my script determine if the flashdrive is mounted before attempting to mount the wrong image.

Below is the content of my prototype /etc/init.d/squashmount

```

#!/sbin/runscript

# Copyright 1999-2007 Gentoo Foundation

# Distributed under the terms of the GNU General Public License v2

depend() {

        need localmount

}

start() {

        # Mount squashfs filesystems. Use flash based images if they exist.

        ebegin "Mounting squashfs and aufs filesystems"

        if [[ -e /mnt/patriot/squashed/usr.sfs ]] ; then

                einfo "Mounting usr.sfs from flash"

                mount /mnt/patriot/squashed/usr.sfs > /dev/null

        elif [[ -e /squashed/usr/usr.sfs ]] ; then

                einfo "Mounting usr.sfs from disk"

                mount /squashed/usr/usr.sfs > /dev/null

        else

                eerror "usr.sfs not found"

        fi

        if [[ -e /mnt/patriot/squashed/opt.sfs ]] ; then

                einfo "Mounting opt.sfs from flash"

                mount /mnt/patriot/squashed/opt.sfs > /dev/null

        elif [[ -e /squashed/opt/opt.sfs ]] ; then

                einfo "Mounting opt.sfs from disk"

                mount /squashed/opt/opt.sfs > /dev/null

        else

                eerror "opt.sfs not found"

        fi

        eend $?

        # Mount aufs filesystems in /etc/fstab.

        ebegin "Mounting aufs filesystems"

        mount -at aufs \

                -O no_netdev >/dev/null

        eend $? "Some aufs filesystems failed to mount"

}

stop() {

        ebegin "Unmounting aufs filesystems"

        umount -alt aufs \

                -O no_netdev >/dev/null

        eend $? "Some aufs filesystems failed to unmount"

        ebegin "Unmounting squashfs filesystems"

        umount -alt squashfs \

                -O no_netdev >/dev/null

        eend $? "Some aufs filesystems failed to unmount"

        if [[ -d /mnt/patriot/squashed ]] ; then

                einfo "Unmounting flash"

                sync

                umount -l /mnt/patriot/ > /dev/null

        fi

        sleep 1

}

# vim:ts=4

```

And here is my modifed /etc/init.d/localmount

```

#!/sbin/runscript

# Copyright 1999-2007 Gentoo Foundation

# Distributed under the terms of the GNU General Public License v2

depend() {

        need checkfs

}

start() {

        # Mount local filesystems in /etc/fstab.

        ebegin "Mounting local filesystems"

        mount -at noaufs,nosquashfs,noproc,noshm${NET_FS_LIST:+,no}${NET_FS_LIST// /,no} \

                -O no_netdev >/dev/null

        eend $? "Some local filesystem failed to mount"

        # Make sure we insert usbcore if its a module

        if [[ -f /proc/modules && ! -d /proc/bus/usb ]] ; then

                # >/dev/null to hide errors from non-USB users

                modprobe usbcore &> /dev/null

        fi

        # Check what USB fs the kernel support.  Currently

        # 2.5+ kernels, and later 2.4 kernels have 'usbfs',

        # while older kernels have 'usbdevfs'.

        if [[ -d /proc/bus/usb && ! -e /proc/bus/usb/devices ]] ; then

                local usbfs=$(grep -Fow usbfs /proc/filesystems ||

                        grep -Fow usbdevfs /proc/filesystems)

                if [[ -n ${usbfs} ]] ; then

                        ebegin "Mounting USB device filesystem (${usbfs})"

                        usbgid=$(echo $(getent group usb) | awk -F: '{print $3}')

                        mount -t ${usbfs} usbfs /proc/bus/usb \

                                -o ${usbgid:+devmode=0664,devgid=${usbgid},}noexec,nosuid

                        eend $?

                fi

        fi

        # Setup Kernel Support for miscellaneous Binary Formats

        if [[ -d /proc/sys/fs/binfmt_misc ]] ; then

                local binfmt=$(grep -Fow binfmt_misc /proc/filesystems)

                if [[ -n ${binfmt} ]] ; then

                        ebegin $"Mounting misc binary format filesystem"

                        mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc \

                                -o nodev,noexec,nosuid

                        eend $?

                fi

        fi

        # Setup Kernel Support for the security filesystem

        if [[ -d /sys/kernel/security ]] ; then

                local secfs=$(grep -Fow securityfs /proc/filesystems)

                if [[ -n ${secfs} ]] ; then

                        ebegin $"Mounting security filesystem"

                        mount -t securityfs securityfs /sys/kernel/security \

                                -o nodev,noexec,nosuid

                        eend $?

                fi

        fi

        # We do our swapping here instead of rc so we can get urandom started

        # before us for people that like an encrypted swap.

        ebegin "Activating (possible) swap"

        /sbin/swapon -a

        eend $?

        # Start dm-crypt mappings, if any

        start_addon dm-crypt

}

# vim:ts=4

```

I hope this is useful for someone.

----------

## synss

Nice, I did not think about it myself but it is a great idea! I have done it on my slow but ultra lightweight laptop running ubuntu [also posted my how to on the ubuntu forums] and starting openoffice is a new experience. It will probably save battery life, too.

Thanks!

----------

## DingoDom

Nice Tip..

I've been installing this  tricks for few days... It's works and my gentoo is faster than ever

but it break my gcc 

  gcc-config error: Could not run/locate "x86_64-pc-linux-gnu-gcc"

Do you have any ideas ?

Thanks

----------

## Palatis

a little script to do the updates automatically via cron.

put this script into /etc/cron.weekly OR /etc/cron.monthly.

please modify the three VARIABLES to suite your needs!

i named this script `squashed-rebuild'  :Smile: 

```
#!/bin/bash

BLOCK_SIZE=1048576

MOUNTPOINTS="usr opt"

SQFS_LOCATION="/mnt/sqfs/"

for MP in ${MOUNTPOINTS}

do

   /usr/bin/mksquashfs /${MP} ${SQFS_LOCATION}/${MP}_tmp.sqfs -b ${BLOCK_SIZE}

   /bin/umount -l /${MP}

   /bin/umount -l ${SQFS_LOCATION}/${MP}/ro

   /bin/rm -rf ${SQFS_LOCATION}/${MP}/rw &

   /bin/rm ${SQFS_LOCATION}/${MP}.sqfs

   /bin/mv ${SQFS_LOCATION}/${MP}_tmp.sqfs ${SQFS_LOCATION}

   /bin/mount ${SQFS_LOCATION}/${MP}/ro

   /bin/mount /${MP}

done
```

ohh, my setup is as below (/mnt/sqfs/ on a seperate lvm LV):

 /mnt/sqfs/usr/ro

 /mnt/sqfs/usr/rw

 /mnt/sqfs/usr.sqfs

 /mnt/sqfs/opt/ro

 /mnt/sqfs/opt/rw

 /mnt/sqfs/opt.sqfs

fstab:

```
/mnt/sqfs/usr.sqfs   /mnt/sqfs/usr/ro   squashfs   noatime,nodev,loop,ro   0 0

none         /usr         aufs      udba=reval,br:/mnt/sqfs/usr/rw:/mnt/sqfs/usr/ro  0 0

/home/sqfs/opt.sqfs   /home/sqfs/opt/ro   squashfs   noatime,nodev,loop,ro   0 0

none         /opt         aufs      udba=reval,br:/mnt/sqfs/opt/rw:/mnt/sqfs/opt/ro  0 0
```

----------

## synss

Hi, I'll go with my little script too. I use it on my laptop, which is running Debian Lenny, but it should not matter. Also note that I am using LVM2, that /home/synss/local is mounted on /usr/local (and therefore excluded from the sqfs image) and that /usr/opt is mounted on /opt (and therefore /opt is included in the sqfs image).

The script

compresses /usr

creates a new LV with appropriate size

copies the sqfs filesystem into the LV

Two more files are created,

squash.log, where all the files being squashed are reported, in case mkquashfs gets stuck (it has happened on my system)

dd.log, output of dd, if any

Since I do that on my laptop, I do not want it in a cron nor initscript, so it is fairly simple and I run it by hand when /var/squashed/ gets too big. Using LVM2 makes everything easy.

```
#!/bin/sh

VG="/dev/vg"

DIR_OUT="lenny_usr_$(date +%Y%m%d)"

SQFS_OUT="$DIR_OUT.sqfs"

SQFS_LV="$VG/$SQFS_OUT"

die() 

{ 

   exit 1 

}

cout()

{

   echo " ** $@"

}

#if [ $UID != 0 ]; then

#   cout "Needs elevated privileges"

#   die

#fi

cout "Create $SQFS_OUT"

mksquashfs /usr $SQFS_OUT \

   -info \

   -no-exports -no-progress \

   -e /usr/local/* \

   > squash.log || die

# DEBUG try -lzma compression, now breaks v 3.3

mkdir -p /var/squashed/$DIR_OUT/ro &>/dev/null

mkdir -p /var/squashed/$DIR_OUT/rw &>/dev/null

SIZE=$(ls -l $SQFS_OUT | cut -d" " -f5)

SIZE="$(($SIZE/1024))k"

if [ -e $SQFS_LV ]

then

   cout "Remove $SQFS_LV"

   lvremove $SQFS_LV

fi

cout "Create $SQFS_LV"

lvcreate -n "$SQFS_OUT" -L $SIZE $VG || die

cout "Copy $SQFS_OUT to $SQFS_LV"

dd if=$SQFS_OUT of=$SQFS_LV > dd.log || die

#rm squash.log dd.log $SQFS_OUT

cout "Please edit /etc/fstab now"

```

PS: Don't be scared by the lvremove command, it will actually ask you if you want to do that.

----------

## synss

Has anyone tried to compress the root? excluding, i.e.,

```
/dev/*

/home/*

/media/*

/srv/*

/tmp/*

/var/*

/boot/*

/lost+found/*

/mnt/*

/proc/*

/sys/*

```

(check the -ef option in mksquashfs), I am guessing that, as long as the squashfs and aufs modules are compiled in the kernel or that you have an initramfs, it should work. If somebody means to actually try, she should check live CD how-tos, because some files in /etc will not like that (I am thinking of /etc/mtab). I may give it a try myself later, as that should save some more battery life. EDIT: this might help to get the initramfs script right.

Last question: has anyone tried the user space solutions (i.e., fuse) and how does it perform?

----------

## shgadwa

OK... I have a problem here.

I did everything as described but now my system will not boot. It locks up at 'loading local filesystems'... And I'm unable to mount the filesystem in a Chrooted environment because it says "Unknown filesystem type 'squashfs'. "

Because I was having problems loadng the modules, I just set squashfs and aufs as enabled in the kernel, rather an setting it as a module.

Any ideas?

EDIT: I WAS able to load modules until I did a reboot (after creating the squashfs filesystem and stuff), now I am no longer able to load modules.

I get this:

Microknoppix / # modprobe squashfs

FATAL: Could not load /lib/modules/2.6.28/modules.dep: No such file or directory

Also, I've been thinking... I first followed this guide: http://en.gentoo-wiki.com/wiki/Squashed_Portage_Tree

And squashed the portage tree... then without making any changes, I decided to squash the entire /usr folder... something tells me I did it wrong. That is, I should not of squashed the portage tree if I was planning to squash the /usr folder (?).

----------

## eruvaer_ohta

belikeyeshua, have you deleted the original /usr folder? The kernel needs the files in /usr/src/linux-? to load the module 'aufs' (or any other module), which is then needed to uncompress the squashed filesystem. So if you deleted the uncompressed files, the kernel can't load the files it needs. To circumvent this, you can move the /usr/src/ folder to another location and soft-link it back.

----------

## shgadwa

Well, it does work if I comment out the new entries in fstab, and in the other file, forget what that file was called...

Then the squashfs does not work, but at least it works. I've done everything I can think of.... I conclude that this guide is broken. It does not work for the latest system. I'd recommend that it was updated, or deleted.

Thank you for your help.

----------

## synss

 *belikeyeshua wrote:*   

> Well, it does work if I comment out the new entries in fstab, and in the other file, forget what that file was called...
> 
> Then the squashfs does not work, but at least it works. I've done everything I can think of.... I conclude that this guide is broken. It does not work for the latest system. I'd recommend that it was updated, or deleted.
> 
> Thank you for your help.

 

I do not want to sound rude but the thing is that if you do not understand what you are doing, do not follow how to blindly. Please swallow a little bit at a time and it will work. This how to works now better than ever since squashfs has been accepted in the latest kernels (.29). But it is still highly unsupported and you have to be able to fix it yourself.

This how to needs not being deleted or amended, it is you who need to understand more about your system.

----------

## justinkb

very nice idea, but i compile new stuff too often for this to be useful for me  :Sad:  since write performance gets worse with this.

@belikeyeshua, clearly, you didn't follow the howto properly... :/

----------

## lazy_bum

 *waynesherman wrote:*   

> 1) The aufs mount in /etc/fstab depends on mount.aufs
> 
> 2) mount.aufs is a shell script which happens to depend on 4 files in /usr/bin:
> 
> ```
> ...

 

Why this is a problem? Looking at the example fstab it should first mount sqashfs (with all the /usr content) and then aufs. So it shouldn't be a problem, right? Anyway, you can make some dirty hack. SlashBeast suggested something similar but with /usr/src.

1) copy those files to ie. /usr.needed/bin

2) remove everything from /usr (except the bin dir)

3) make symlinks from /usr/bin/comm -> /usr.needed/bin/comm etc.

4) remember to update those files in /usr.needed when coreutils and/or diffutils are updated.

 *waynesherman wrote:*   

> If no, why does it happen on my system and not others?

 

For me it looks like you tried to mount some aufs *before* squashfs /usr.

----------

## ocbMaurice

I really love this solution for my eeepc installation.

I did adjust the script quite a bit and wanted to share this:

/etc/init.d/squashed.usr.portage :

```
#!/sbin/runscript

# Copyright 1999-2006 Gentoo Foundation

# Distributed under the terms of the GNU General Public License v2

# $Header: $

#

# It requires support for the loop device and squashfs enabled in the kernel,

# module autoloading is also *highly* recommended.

# sys-fs/squashfs and sys-fs/aufs are necessary for read-write support.

#

# Author: Marcel Greter <marcel.greter@crevis.ch>

# Inspired by http://en.gentoo-wiki.com/wiki/Squashed_Portage_Tree

opts="init sync info commit rollback"

# prepend root and normalize

DIRECTORY="/${DIRECTORY}"

DIRECTORY=`echo ${DIRECTORY} | sed 's/\/$//' | sed 's/\/\+/\//g'`

# get mount points and squash fs file

if [ ! ${DEV_SHM} ]; then DEV_SHM="/dev/shm"; fi

if [ ! ${MOUNT_RO} ]; then MOUNT_RO="${SQUASH_ROOT}${DIRECTORY}.ro"; fi

if [ ! ${MOUNT_RW} ]; then MOUNT_RW="${SQUASH_ROOT}${DIRECTORY}.rw"; fi

if [ ! ${MOUNT_SFS} ]; then MOUNT_SFS="${SQUASH_ROOT}${DIRECTORY}.sfs"; fi

if [ ! ${MOUNT_SFS_NEW} ]; then MOUNT_SFS_NEW="${SQUASH_ROOT}${DIRECTORY}.sfs.new"; fi

if [ ! ${MOUNT_SFS_OLD} ]; then MOUNT_SFS_OLD="${SQUASH_ROOT}${DIRECTORY}.sfs.old"; fi

if [ ! ${MV_OPTS} ]; then MV_OPTS="-i"; fi

if [ ! ${RM_OPTS} ]; then RM_OPTS="-rI"; fi

if [ ! ${MKDIR_OPTS} ]; then MKDIR_OPTS="-p"; fi

if [ ! ${RMDIR_OPTS} ]; then RMDIR_OPTS="-p"; fi

if [ ! ${SYNC_THRESHOLD} ]; then SYNC_THRESHOLD="0"; fi

if [ ! ${KEEP_SFS_BACKUP} ]; then KEEP_SFS_BACKUP="0"; fi

# maybe use memory for overlay

if [ ${IN_MEMORY} ]; then MOUNT_RW="${DEV_SHM}${DIRECTORY}"; fi

depend() {

   need localmount

   after localmount

   before hostname

}

start() {

  _start

}

_start() {

  ebegin "Start squashed ${DIRECTORY}"

  mark_service_starting

 

  if [ ! -d ${DIRECTORY} ]; then

    eend 1 "${DIRECTORY} does not exist" || return $?

  fi

  if [ ! -e ${MOUNT_SFS} ]; then

    ewarn "you probably need to init me first?"

    eend 1 "squashed fs file does not exist" || return 1

  fi

  for EXCLUDE in ${EXCLUDE_DIRS}; do

    # prepend root and normalize

    EXCLUDE="/${EXCLUDE}"

    EXCLUDE=`echo ${EXCLUDE} | sed 's/\/$//' | sed 's/\/\+/\//g'`

    # absolute paths

    EXCL_DIR="${DIRECTORY}${EXCLUDE}"

    EXCL_BIND="${SQUASH_ROOT}${DIRECTORY}.bind${EXCLUDE}"

    /bin/mkdir ${MKDIR_OPTS} "${EXCL_DIR}"

    /bin/mkdir ${MKDIR_OPTS} "${EXCL_BIND}"

    einfo "  bind ${EXCL_DIR} to temp location"

    /bin/mount -o bind "${EXCL_DIR}" "${EXCL_BIND}"

    eend $? "  could not bind ${EXCL_DIR} ${EXCL_BIND}" || return $?

 

  done

  einfo "  mount ${DIRECTORY} readonly squasfs ${SQUASHFS_OPTS}"

  /bin/mount -rt squashfs -o loop,ro,nodev \

    "${MOUNT_SFS}" "${MOUNT_RO}" ${SQUASHFS_OPTS}

  eend $? "  could not mount ${MOUNT_RO} ${SQUASHFS_OPTS}" || return $?

  # maybe use memory for overlay

  if [ ${IN_MEMORY} ]; then

    ewarn "  use writeable overlay in memory"

    ewarn "  changes WILL best lost on unexpected shutdown"

    if [ ! ${SYNC_ON_SHUTDOWN} ]; then

      ewarn "  changes WILL be lost on restart"

    else

      if [ 15 -lt ${SYNC_THRESHOLD} ]; then

        ewarn "  changes MAY be lost on restart"

      fi

    fi

  fi

  # create directory for write overlay

  [ -d ${MOUNT_RW} ] || mkdir ${MKDIR_OPTS} ${MOUNT_RW}

  # get info from directory (XXX - replace perl by sed or something like that)

  SFS_MOD=`/usr/bin/stat ${DIRECTORY} -L | /usr/bin/perl -e 'while(<>){ if(m/Access:\s*\(([0-9]{4})\//) { print $1; } }'`

  SFS_USR=`/usr/bin/stat ${DIRECTORY} -L | /usr/bin/perl -e 'while(<>){ if(m/Uid:\s*\(\s*[0-9]+\s*\/\s*([^\)]+)\)\s*Gid:\s*\(\s*[0-9]+\s*\/\s*([^\)]+)\)/) { print join(":", $1, $2); } }'`

  einfo "  chmod ${SFS_MOD} ${MOUNT_RW}"

  /bin/chmod ${SFS_MOD} ${MOUNT_RW}

  eend $? "  could not change file mode"

  einfo "  chown ${SFS_USR} ${MOUNT_RW}"

  /bin/chown ${SFS_USR} ${MOUNT_RW}

  eend $? "  could not change file owner"

  einfo "  mount ${DIRECTORY} writable overlay ${AUFS_OPTS}"

  /bin/mount -t aufs -o nodev,udba=reval,dirs=${MOUNT_RW}=rw:${MOUNT_RO}=ro \

    aufs ${AUFS_OPTS} "${DIRECTORY}"

  eend $? " could not mount ${AUFS_OPTS} ${DIRECTORY}" || return $?

  for EXCLUDE in ${EXCLUDE_DIRS}; do

    # prepend root and normalize

    EXCLUDE="/${EXCLUDE}"

    EXCLUDE=`echo ${EXCLUDE} | sed 's/\/$//' | sed 's/\/\+/\//g'`

    # absolute paths

    EXCL_DIR="${DIRECTORY}${EXCLUDE}"

    EXCL_BIND="${SQUASH_ROOT}${DIRECTORY}.bind${EXCLUDE}"

    einfo "  bind temp location to ${EXCL_DIR}"

    /bin/mount -o bind "${EXCL_BIND}" "${EXCL_DIR}"

    eend $? "  could not bind ${EXCL_BIND} ${EXCL_DIR}" || return $?

  done

 

  mark_service_started

  eend 0

}

stop() {

  _stop

}

_stop() {

  ebegin "Stop squashed ${DIRECTORY}"

  mark_service_stopping

 

  if [ ${SYNC_ON_SHUTDOWN} != 0 ]; then

    if [ $(du -s ${MOUNT_RW} | cut -f 1) -gt ${SYNC_THRESHOLD} ]; then

      einfo "  changes detected, updating image"

      commit; eend $? "  could not commit changes"

      return $?

    else

      einfo "  no changes detected, skipping update"; eend 0

    fi

  fi

  for EXCLUDE in ${EXCLUDE_DIRS}; do

    # prepend root and normalize

    EXCLUDE="/${EXCLUDE}"

    EXCLUDE=`echo ${EXCLUDE} | sed 's/\/$//' | sed 's/\/\+/\//g'`

    # absolute paths

    EXCL_DIR="${DIRECTORY}${EXCLUDE}"

    EXCL_BIND="${SQUASH_ROOT}${DIRECTORY}.bind${EXCLUDE}"

    einfo "  unbind ${EXCL_DIR}"

    /bin/umount "${EXCL_DIR}"

    eend $? "  could not umount ${EXCL_DIR}" || return $?

  done

  einfo "  umount ${DIRECTORY} writable overlay"

  /bin/umount ${DIRECTORY}

  eend $? "  could not umount ${DIRECTORY}" || return $?

  einfo "  umount ${DIRECTORY} readonly overlay"

  /bin/umount "${MOUNT_RO}"

  eend $? "  could not umount ${MOUNT_RO}" || return $?

  for EXCLUDE in ${EXCLUDE_DIRS}; do

    # prepend root and normalize

    EXCLUDE="/${EXCLUDE}"

    EXCLUDE=`echo ${EXCLUDE} | sed 's/\/$//' | sed 's/\/\+/\//g'`

    # absolute paths

    EXCL_DIR="${DIRECTORY}${EXCLUDE}"

    EXCL_BIND="${SQUASH_ROOT}${DIRECTORY}.bind${EXCLUDE}"

    einfo "  unbind ${EXCL_BIND}"

    /bin/umount "${EXCL_BIND}"

    eend $? "  could not umount ${EXCL_BIND}" || return $?

  done

  mark_service_stopped

  eend 0

}

restart() {

  svc_stop

  svc_start

}

init() {

  if service_started ${myservice}; then svc_stop; fi

  if [ ! -d ${DIRECTORY} ]; then

    eend 1 "${DIRECTORY} does not exist" || return $?

  fi

  if [ -e ${MOUNT_SFS} ]; then

    eerror "squashed fs file already exists"; eend 1

    ewarn "remove ${MOUNT_SFS} to force init"; return 1

  fi

  einfo "  initializing directory, please wait"

  /bin/mkdir ${MKDIR_OPTS} ${MOUNT_RO}

  /bin/mkdir ${MKDIR_OPTS} ${MOUNT_RW}

  for EXCLUDE in ${EXCLUDE_DIRS}; do

    # prepend root and normalize

    EXCLUDE="/${EXCLUDE}"

    EXCLUDE=`echo ${EXCLUDE} | sed 's/\/$//' | sed 's/\/\+/\//g'`

    # absolute paths

    EXCL_DIR="${DIRECTORY}${EXCLUDE}"

    EXCL_BIND="${SQUASH_ROOT}${DIRECTORY}.bind${EXCLUDE}"

    EXCL_SKEL="${SQUASH_ROOT}${DIRECTORY}.skel${EXCLUDE}"

    /bin/mkdir ${MKDIR_OPTS} "${EXCL_DIR}"

    /bin/mkdir ${MKDIR_OPTS} "${EXCL_BIND}"

    /bin/mkdir ${MKDIR_OPTS} "${EXCL_SKEL}"

    einfo "  bind empty temp location to ${EXCL_DIR}"

    /bin/mount -o bind "${EXCL_SKEL}" "${EXCL_DIR}"

    eend $? "  could not bind ${EXCL_SKEL} ${EXCL_DIR}" || return $?

 

  done

 

  einfo "  /usr/bin/mksquashfs ${DIRECTORY} ${MOUNT_SFS} ${MKSQUASHFS_OPTS}"

  /usr/bin/mksquashfs ${DIRECTORY} ${MOUNT_SFS} ${MKSQUASHFS_OPTS}

  eend $? "  could not make squashfs ${DIRECTORY} ${MOUNT_SFS}" || return $?

  # einfo mount -rt squashfs -o loop,ro,nodev,noexec "${MOUNT_SFS}" "${MOUNT_RO}"

  # einfo mount -t aufs -o "nodev,noexec,udba=reval,dirs=${MOUNT_RW}=rw:${DIRECTORY}=ro" aufs "${DIRECTORY}"

  einfo "  cleaning ${DIRECTORY}"

  /bin/rm ${RM_OPTS} "${DIRECTORY}"

  eend "  could not remove ${DIRECTORY}"

  for EXCLUDE in ${EXCLUDE_DIRS}; do

    # prepend root and normalize

    EXCLUDE="/${EXCLUDE}"

    EXCLUDE=`echo ${EXCLUDE} | sed 's/\/$//' | sed 's/\/\+/\//g'`

    # absolute paths

    EXCL_DIR="${DIRECTORY}${EXCLUDE}"

    einfo "  unbind empty temp location on ${EXCL_DIR}"

    /bin/umount "${EXCL_DIR}"

    eend $? "  could not umount ${EXCL_DIR}" || return $?

 

  done

  eend 0

  if service_stopped ${myservice}; then  svc_start; fi

}

sync() {

  commit

}

commit() {

  if [ ! -d ${DIRECTORY} ]; then

    eend 1 "${DIRECTORY} does not exist" || return $?

  fi

  if [ ! -e ${MOUNT_SFS} ]; then

    ewarn "you probably need to init me first?"

    eend 1 "squashed fs file does not exist" || return 1

  fi

  if [ -d "${MOUNT_SFS_NEW}" ]; then

    einfo "  /bin/mv ${MOV_OPTS} ${MOUNT_SFS_NEW} ${MOUNT_SFS_OLD}"

    /bin/mv ${MOV_OPTS} "${MOUNT_SFS_NEW}" "${MOUNT_SFS_OLD}"

    eend $? "  could not move ${MOUNT_SFS_NEW} ${MOUNT_SFS_OLD}"

 

  fi

  ebegin "Recreate squashed fs file and commit changes"

  if service_started ${myservice}; then RESTART=1; fi

  # if service_stopping ${myservice}; then RESTART=1; fi

  if service_stopped ${myservice}; then _start; fi

 

  for EXCLUDE in ${EXCLUDE_DIRS}; do

    # prepend root and normalize

    EXCLUDE="/${EXCLUDE}"

    EXCLUDE=`echo ${EXCLUDE} | sed 's/\/$//' | sed 's/\/\+/\//g'`

    # absolute paths

    EXCL_DIR="${DIRECTORY}${EXCLUDE}"

    EXCL_BIND="${SQUASH_ROOT}${DIRECTORY}.bind${EXCLUDE}"

    EXCL_SKEL="${SQUASH_ROOT}${DIRECTORY}.skel${EXCLUDE}"

    /bin/mkdir ${MKDIR_OPTS} "${EXCL_DIR}"

    /bin/mkdir ${MKDIR_OPTS} "${EXCL_BIND}"

    /bin/mkdir ${MKDIR_OPTS} "${EXCL_SKEL}"

    einfo "  bind empty temp location to ${EXCL_DIR}"

    /bin/mount -o bind "${EXCL_SKEL}" "${EXCL_DIR}"

    eend $? "  could not mount ${EXCL_SKEL} ${EXCL_DIR}" || return $?

  done

  einfo /usr/bin/mksquashfs "${DIRECTORY}" "${MOUNT_SFS_NEW}" ${MKSQUASHFS_OPTS}

  /usr/bin/mksquashfs "${DIRECTORY}" "${MOUNT_SFS_NEW}" ${MKSQUASHFS_OPTS}

  eend $? "  could not make squashfs ${DIRECTORY} ${MOUNT_SFS_NEW}" || return $?

  for EXCLUDE in ${EXCLUDE_DIRS}; do

    # prepend root and normalize

    EXCLUDE="/${EXCLUDE}"

    EXCLUDE=`echo ${EXCLUDE} | sed 's/\/$//' | sed 's/\/\+/\//g'`

    # absolute paths

    EXCL_DIR="${DIRECTORY}${EXCLUDE}"

    einfo "  unbind empty temp location on ${EXCL_DIR}"

    /bin/umount "${EXCL_DIR}"

    eend $? "  could not umount ${EXCL_DIR}" || return $?

 

  done

  # disable sync on shutdown

  SYNC_ON_SHUTDOWN="0" _stop

  einfo "  move '${MOUNT_SFS}' '${MOUNT_SFS_OLD}'"

  /bin/mv ${MV_OPTS} "${MOUNT_SFS}" "${MOUNT_SFS_OLD}"

  eend $? "  could not move ${MOUNT_SFS} ${MOUNT_SFS_OLD}" || return $?

  rollback

  einfo "  move '${MOUNT_SFS_NEW}' '${MOUNT_SFS}'"

  /bin/mv ${MV_OPTS} "${MOUNT_SFS_NEW}" "${MOUNT_SFS}"

  eend $? "  could not move ${MOUNT_SFS_NEW} ${MOUNT_SFS}" || return $?

 

  if [ ${KEEP_SFS_BACKUP} == 0 ]; then

    einfo "  delete ${MOUNT_SFS_OLD}"

    /bin/rm ${RM_OPTS} "${MOUNT_SFS_OLD}"

    eend $? "  could not delete ${MOUNT_SFS_OLD}"

  fi

  # restart if service was started before

  if [ ${RESTART} ] ; then _start; fi

  eend $?

}

info() {

  einfo "DIRECTORY        : ${DIRECTORY}";

  if [ -d ${DIRECTORY} ]; then eend 0; else eend 1; fi

  einfo "MOUNT_RO         : ${MOUNT_RO}";

  if [ -d ${MOUNT_RO} ]; then eend 0; else eend 1; fi

  einfo "MOUNT_RW         : ${MOUNT_RW}";

  if [ -d ${MOUNT_RW} ]; then eend 0; else eend 1; fi

  einfo "MOUNT_SFS        : ${MOUNT_SFS}";

  if [ -f ${MOUNT_SFS} ]; then eend 0; else eend 1; fi

  einfo "MOUNT_SFS_NEW    : ${MOUNT_SFS_NEW}";

  if [ ! -e ${MOUNT_SFS_NEW} ]; then eend 0; else eend 1; fi

  einfo "MOUNT_SFS_OLD    : ${MOUNT_SFS_OLD}";

  # this is only a backup file

  # if [ ! -e ${MOUNT_SFS_OLD} ]; then eend 0; else eend 1; fi

  einfo "AUFS_OPTS        : ${AUFS_OPTS}";

  einfo "SQUASHFS_OPTS    : ${SQUASHFS_OPTS}";

  einfo "MKSQUASHFS_OPTS  : ${MKSQUASHFS_OPTS}";

  einfo "MV_OPTS          : ${MV_OPTS}";

  einfo "RM_OPTS          : ${RM_OPTS}";

  einfo "MKDIR_OPTS       : ${MKDIR_OPTS}";

  einfo "RMDIR_OPTS       : ${RMDIR_OPTS}";

  einfo "DEV_SHM          : ${DEV_SHM}";

  einfo "IN_MEMORY        : ${IN_MEMORY}";

  einfo "SYNC_ON_SHUTDOWN : ${SYNC_ON_SHUTDOWN}";

  einfo "SYNC_THRESHOLD   : ${SYNC_THRESHOLD}";

  einfo "  changed so far : $(du -s ${MOUNT_RW} | cut -f 1)";

  for EXCLUDE in ${EXCLUDE_DIRS}; do

    # prepend root and normalize

    EXCLUDE="/${EXCLUDE}"

    EXCLUDE=`echo ${EXCLUDE} | sed 's/\/$//' | sed 's/\/\+/\//g'`

    # absolute paths

    EXCL_DIR="${DIRECTORY}${EXCLUDE}"

    EXCL_BIND="${SQUASH_ROOT}${DIRECTORY}.bind${EXCLUDE}"

    EXCL_SKEL="${SQUASH_ROOT}${DIRECTORY}.skel${EXCLUDE}"

    einfo " exclude dir     : ${EXCL_DIR}";

    einfo " exclude bind    : ${EXCL_BIND}";

    einfo " exclude skel    : ${EXCL_SKEL}";

  done

 

  eend 0

}

rollback() {

  einfo "  ${MOUNT_RW} delete writable overlay"

  /bin/rm ${RM_OPTS} ${MOUNT_RW}

  eend $? "  could not delete ${MOUNT_RW}" || return $?

  einfo "  ${MOUNT_RW} re-create writable overlay"

  /bin/mkdir ${MKDIR_OPTS} ${MOUNT_RW}

  eend $? "  could not create ${MOUNT_RW}" || return $?

}

```

/etc/conf.d/squashed.usr.portage :

```

# the actual directory you want to squash

DIRECTORY="/usr/portage"

# generic data storage directory

# will create sub-directories

SQUASH_ROOT="/mnt/squashed"

# additional options for mksquashfs

MKSQUASHFS_OPTS="-b 65536" # -no-duplicates

# additional squashfs mount options

SQUASHFS_OPTS="-o noexec"

# additional squashfs mount options

AUFS_OPTS="-o noexec"

# redefine auto configuration

# MV_OPTS="-i"

# RM_OPTS="-rI"

# MKDIR_OPTS="-p"

# RMDIR_OPTS="-p"

# directories will be created on demand

# MOUNT_RO="${SQUASH_ROOT}${DIRECTORY}.ro"

# MOUNT_RW="${SQUASH_ROOT}${DIRECTORY}.rw"

# MOUNT_SFS="${SQUASH_ROOT}${DIRECTORY}.sfs"

# MOUNT_SFS_NEW="${SQUASH_ROOT}${DIRECTORY}.sfs.new"

# MOUNT_SFS_OLD="${SQUASH_ROOT}${DIRECTORY}.sfs.old"

# mount writable overlay in memory

# IN_MEMORY=1; # DEV_SHM="/dev/shm";

# do not include in squash image

EXCLUDE_DIRS="distfiles"

# update squash image on shutdown

# only if there have been changes

SYNC_ON_SHUTDOWN=1

# minimum size reported by du to

# consider that something has changed

SYNC_THRESHOLD=14

# keep old sfs image

KEEP_SFS_BACKUP=0

```

Configuration is obviousely done in /etc/conf.d/squashed.usr.portage. If you want to squash other directories, you can make a symlink of the init script and copy the configuration (as with net.* scripts).

Once it's configured, you only have to do :

```
# /etc/init.d/squashed.usr.portage init
```

Other commands :

```
# /etc/init.d/squashed.usr.portage start

# /etc/init.d/squashed.usr.portage stop

# /etc/init.d/squashed.usr.portage restart

  -> should be self explained

# /etc/init.d/squashed.usr.portage sync

# /etc/init.d/squashed.usr.portage commit

  -> save changes to squashed image and clear rw store

# /etc/init.d/squashed.usr.portage rollback

  -> purge rw store without touching old squased image

# /etc/init.d/squashed.usr.portage info

 * DIRECTORY        : /usr/portage

 * MOUNT_RO         : /mnt/squashed/usr/portage.ro

 * MOUNT_RW         : /mnt/squashed/usr/portage.rw

 * MOUNT_SFS        : /mnt/squashed/usr/portage.sfs

 * MOUNT_SFS_NEW    : /mnt/squashed/usr/portage.sfs.new

 * MOUNT_SFS_OLD    : /mnt/squashed/usr/portage.sfs.old

 * AUFS_OPTS        : -o noexec

 * SQUASHFS_OPTS    : -o noexec

 * MKSQUASHFS_OPTS  : -b 65536

 * MV_OPTS          : -i

 * RM_OPTS          : -rI

 * MKDIR_OPTS       : -p

 * RMDIR_OPTS       : -p

 * DEV_SHM          : /dev/shm

 * IN_MEMORY        :

 * SYNC_ON_SHUTDOWN : 1

 * SYNC_THRESHOLD   : 1500

 *   changed so far : 12

 *  exclude dir     : /usr/portage/distfiles

 *  exclude bind    : /mnt/squashed/usr/portage.bind/distfiles

 *  exclude skel    : /mnt/squashed/usr/portage.skel/distfiles

```

Features :

- Define sub directories to be excluded from the squash image

- Configurable threshold for auto syncing the image on shutdown

- So far you will be asked for confirmation before overwriting stuff

I needed to add this to the init script

```
depend() {

   need localmount

   after localmount

   before hostname

}

```

so I could compress stuff like

- /usr/share

- /var/cache

- /var/db

I'm not sure if this is the best way, for me it seems to work.

My current setup is like this (all in boot runlevel) :

```
dir /etc/init.d/squashed.*

/etc/init.d/squashed.usr.portage

/etc/init.d/squashed.usr.share -> squashed.usr.portage

/etc/init.d/squashed.usr.src -> squashed.usr.portage

/etc/init.d/squashed.var.cache -> squashed.usr.portage

/etc/init.d/squashed.var.db -> squashed.usr.portage
```

I tested the script quite well but I'm not responsible if anything goes wrong!

Anyway, feedback is very welcome!

Hope you like it,

Maurice

----------

## iamboredr

really good idea.. but still not very much dynamic

----------

## mv

I didn't look through the whole thread, but it seems to me like a dupe of Compressing portage using squashfs: initscript method thread (at least at the newer parts of that thread). In particular, ocbMaurice's script is practically completely covered with the corresponding (meanwhile rather mature, although there were revisions recently) initscript from this page which was tested with several directories.

The only difference seems to me the threshhold test which however could be easily configured in /etc/conf.d/squash_... by setting there the "MAGIC_FILE" variable depending on some "ds" command. However, I think the original magic file feature of the script on the webpage is more powerful, since you can manually decide (without changing any config files) whether you want to save or not, just by creating a magic ".no-save" file in the corresponding directory.

Or am I missing something and the scripts in this thread here can really do something crucial  which you cannot do easily with the script on the webpage?

----------

## ocbMaurice

 *mv wrote:*   

> I didn't look through the whole thread, but it seems to me like a dupe of Compressing portage using squashfs: initscript method thread (at least at the newer parts of that thread). In particular, ocbMaurice's script is practically completely covered with the corresponding (meanwhile rather mature, although there were revisions recently) initscript from this page which was tested with several directories.

 

Thanks for pointing me to this, I really missed that one. Well I saw the page, but didn't really investigate the script as I did now. And it really looks like the thing I was initially looking for.

I think there are only two functions missing in this script:

- The threshold as in my script (which is IMO quite usefull)

- Set certain Directories to be excluded from the squash image

- [edit] my script sets the permissions for the new mounted dir to the same as the old (chmod/chown)

As you said, the first one might be even possible with the current solution (the other script even seems to have a better function to check if anything has changed).

There also could be some more options for the init script (like my init, sync or info function). I also think a "uninit" method would be nice to undo the init step (not done in my script yet either).

Well, the script from M. Väth seems to be more mature than mine (specially when it comes to housekeeping).

IMO, you find quite a lot of info about this on the web or on the forum. But it's difficult to really find the best solution (that's why I coded my own script). There are even two threads here about the same issue.

It seems that M. Väth still updates his script, so he probably can integrate those features in his script. It then would be wise to put that in a prominent Position here (IMO this could even get into portage tree).

FYI: To exclude some dirs from squashing I bind/mount them to some temp location, then mount squash ro and aufs rw, then bind/mount the temp location back to the original location.

When creating the squash image, the location is bound to an empty temp location (so the directory will be empty in the squash image). IMO this is a must, since it is quite easy to do and works like a charm!

Maurice

p.s. I still don't know if it's possible to query if anything has changed. IMO it's not possible to know that anything has changed if you only delte files ... I also checked out the proc stuff ...

p.p.s. This approach does not only save harddisk space, it also can free up quite some inodes  :Wink: Last edited by ocbMaurice on Mon May 11, 2009 10:56 pm; edited 4 times in total

----------

## ocbMaurice

 *iamboredr wrote:*   

> really good idea.. but still not very much dynamic

 

To be honest, I didn't get the point about dynamic?

How should/could it be more dynamic?

Maybe you can enlighten me!

----------

## mv

 *ocbMaurice wrote:*   

> - Set certain Directories to be excluded from the squash image

 

I think the proper way to exclude mounted subdirectories is to specify it in the mksquashfs-options with -ef (which can be done with the other script).

If you want to mount/umount them automatically, I suggest you do this in another init-script which "need"s the squash_dir-script: If all would be done in the same script, then in case of errors, it would be unreasonable hard to track down the problems; e.g. what to do if mounting to a temporary directory succeeds, but the aufs-mount fails or something similar. Not to speak about the problems which happen if you are in a --make-shared/--make-slave directory or something similar.

 *Quote:*   

> my script sets the permissions for the new mounted dir to the same as the old (chmod/chown)

 

I do not understand this: Why should it be unreasonable to use the permissions stored in the squash-file? The content also has the permissions stored in that file, so why should it be different for the directory itself?

 *Quote:*   

> p.s. I still don't know if it's possible to query if anything has changed. IMO it's not possible to know that anything has changed if you only delte files

 

Not sure what you mean. The other script does detect this: If you delete files, aufs will store a corresponding file in the writable branch. Of course, this would not go well with the threshhold option. Or do you mean deletion of flies which were only created during the last boot? These do not have to be detected, since if all of them are deleted, there is no reason to resquash the image.

----------

## x-demon

Is it possible to recreate squash img based on old img? Because building image from scratch takes about 30-40 mins on my netbook...

----------

## ^marcs

Sorry for digging such an old topic, but it's best I can find. 

Anyone managed to install aufs2 patch in 2.6.38? 

Way those patches are distributed is a mystery to me

----------

## lazy_bum

You can try this one → aufs2-2.6.38.patch.  I didn't test it yet, but it should work.

----------

## tclover

First off, thanks for this tricks and tips. I cannot possibly run my system on now without usr mounted on a squashsf/aufs combo. Once you get used to the responsiveness of the thing, you cannot come back.

Here are two modified scripts grabbed from this thread, I'll just post 'em in case, well if ever, somebody could find some use. No waranty whatsover of course nor for some usefulness:

 *Quote:*   

> #!/bin/sh
> 
> #~/sqfsd-build.sh
> 
> BSIZE=131072
> ...

 

Now fstab is optional, I keep only usr lines related as a safe net, as the contains of usr disapeared, so localmount can mount it if need be:

 *Quote:*   

> /sqfsd/usr.sfs   /sqfs/usr/ro   squashfs   noatime,nodev,loop,ro   0 0
> 
> usr         /usr         aufs      udba=reval,br:/sqfs/usr/rw:/sqfs/usr/ro  0 0

 

And here my /etc/init.d/sqfsmount:

 *Quote:*   

> #!/sbin/runscript
> 
> #!/sbin/runscript
> 
> # Copyright 1999-2011 Gentoo Foundation
> ...

 

 *Quote:*   

> #/etc/conf.d/sqfsdmount
> 
> SQFSDIR="/sqfsd"
> 
> SQFSD="usr sbin opt var/db var/lib/layman"
> ...

 

An yeah, the idea of getting /lib32 and /lib64 squashed seems appealing but I have an issue when booting. I have baselayout-2.0 openrc-0.8 and it seems /sbin/init write in /lib64 in the bootup process. When the bootup process end some services 'die' somewhere. I guess some services infos get masked after the squashed files got mounted. I don't have any guess on what is happening in the bootup process. I didn't have the time to dig up about that.

EDIT: EDITED Mai 09, 2011,  new update version of sqfsd-build.sh. And now I have almost the same script for fcron. I modified the sqfsd="some dirs" line to sqfsd="`echo ${1}|tr ':' ' '`"  and named the script sqfsd-rebuild.sh and put it to /root. So now you can call the rebuild script using a line like this in crontab: 

* * 1,15 * *    root    $HOME/sqfsd-rebuild.sh usr:bin:sbin:lib32 

or/and

 * * 1,10,20 * *    root    $HOME/sqfsd-rebuild.sh var/db:var/cache/edb:var/lib/layman:var/portage:var/local/portage

say you can put different cron job depending on your needs.

EDIT2: WHAT'S SO GOOD ABOUT SQUASHING STUFF YOU ASK!?

I happen to have wiped out the whole [squashed] usr, after having wiped out the real usr, directory. Well, with a squashed usr... you have to recover only your squashed image. A go in debugfs </dev/sdxx> and then a "cd /sqfsd && ls -d" you note the inode number for usr.sfs and everything is back in no time! And, of course a wiped out squashed file is easy to recover... just rm the .wh* related file in */rw and...

No good you say!? Doesn't worth the trouble?

I could never go back to the much slower usr again. Just the pain to wait emerge to... calculate dependencies and...Last edited by tclover on Tue May 10, 2011 11:29 am; edited 13 times in total

----------

## tclover

just adding a side note of my previous post for those who would want to grab sqfsmount init script, you still had to add a 'need dependency' to the following init services, otherwise you will get errors in the bootup process, but your system could just work with 'em, those errors disappear after sqfsmount is runned: acpid, bootmisc, consolefont, setfont.

----------

## tclover

Yes, again.

Lets start with a candy script intended for experiments...

A squashfs build script that could, maybe, avoid doing the same thing several times:

 *Quote:*   

> EDIT: sqfsd-[re]build.sh script was updated Apr 26 2011 to the previous page.

 

I've just added the related excluded line later, not sure if it's fully working.

That said, I've just opened my eyes the other day and noticed this in the output of mount:

 *Quote:*   

> rc-svcdir on /lib64/rc/init.d type tmpfs (rw,nosuid,nodev,noexec,relatime,size=1024k,mode=755)
> 
> cachedir on /lib64/splash/cache type tmpfs (rw,relatime,size=4096k,mode=644)

 

Those two line explain why my blind attempt failed. So I've just went right away to exclude /lib64/{rc,splash} from my lib64.sfs image but... the thing is, it just doesn't work simply as that. And that I can understand why, lib64 is mounted on /lib64 after all, excluding it or not from the image doesn't make any difference in the sense that when attempting to write to /lib64{rc,splash} will result to writing to /sqfs/lib64/rw/{rc,splash}. Anyone know what I could do?  I started reading the aufs[2] documentation but I did not find anything yet. Is it possible to exclude a sub-directory from aufs branch? or something like that? so that when writing to that sub-directory, the real sub-directory is written instead of the sub-branch? Anybody?Last edited by tclover on Wed Apr 27, 2011 11:51 am; edited 6 times in total

----------

## tclover

An, of course, if somebody knows how to change rc-svcdir and cachedir mount points, do not hesitate to post something. I didn't manage to find out where  I could find something to configure those two mount points from openrc or splashutils files.

----------

## tclover

Well, It happen I wanted an encrypted root partition, an encrypted keys and squashing stuff.

So I've just went into updating the init script of DM-Crypt_with_LUKS http://en.gentoo-wiki.com/wiki/DM-Crypt_with_LUKS#Alternate_init_script_with_squashfs.2Baufs.2Bloop_back_encrypted_keyfile.2BTuxOnIce section of the wiki, and it ended up being a major update, hopping to get rid of the lib64 issue. Well, nothing worked. The squashed lib64 get mounted as other squashed dirs and everything looks fine expect nobody can log in but root. I smashed my head over some walls because I haven't any clue of the why it's happening.

I spent some time reading some scripts from /lib64/rc/sh that do the necessary steps for rc, like mounting rc-svcdir, and only found a single, or did I miss something, parameters assignment--`: ${RC_SVCDIR:=/lib64/rc/init.d}'--in init-common-post.sh. So when this script is called in the chain rc-svcdir get mounted on the right location. I changed it to somewhere else only to found that it didn't take any effect. I don't get this either because I did not find anything assigning another value to RC_SVCDIR. 

Nobody knows something about this? [RE-BUMP]

This is what I got when gdm is launched:

 *Quote:*   

> gdm-simple-slave[3812]: WARNING: Could not spawn command: Failed to execute child process "/usr/bin/dbus-launch" (Permission denied)
> 
> gdm-simple-slave[3812]: WARNING: Unable to launch D-Bus daemon: Failed to execute child process "/usr/bin/dbus-launch" (Permission denied)
> 
> gdm-simple-slave[3812]: WARNING: Could not spawn command: Failed to execute child process "/usr/bin/gnome-session" (Permission denied)
> ...

 

Loggin is also broken in any terminal, when trying to log in as a user I have this:

 *Quote:*   

> Apr 26 14:20:48 xxyyzz polkitd(authority=local): Registered Authentication Agent for unix-session:/org/freedesktop/ConsoleKit/Session1 (system bus name :1.16 [/usr/libexec/polkit-gnome-authentication-agent-1], object path /org/gnome/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8)
> 
> Apr 26 14:21:03 xxyyz gdm-session-worker[3693]: pam_unix(gdm:auth): authentication failure; logname= uid=0 euid=0 tty=:0 ruser= rhost=  user=xyz

 

As one can see above, my uid=1000 gid=1001 is just plain wrong so I cannot never login in any terminal or in gdm. And all this goe after mounted a squashed lib64. I exclude /lib64/{rc/init.d/*,splash/cache/*} when building the sfs image to ensure that I don't get other services info in rc. And even if ever I left init.d contains, it get flushed when rc start, that's part of the init process of rc.

----------

## shentino

Sounds like you got nailed by a chicken and egg with a module that was on a filesystem whose type wasn't modprobed yet.

----------

