# HOWTO: Gentoo Diskless with ATA over Ethernet (AoE)

## VoVaN

LAST EDITED: 24-04-06

24-04-06: Added the note about using initramfs instead of initrd. Now the boot time and application execution on the diskless PC even faster then on my local Gentoo installation!

I'm using diskless PC's with Gentoo Linux for quite a while. It works just fine and you even don't notice performance differences for light DE's like XFCE4, but if you'd like to switch to KDE... that's another story. So, I've decided to open the topic here in order to accumulate all performance tricks for diskless desktops. 

The most basic setup is like bellow: 

- PC boots using PXE and downloading kernel + initrd

- the boot process set all parameters using DHCP and mount all necessary shares using NFS

In order to use heavy DE (KDE) I have to do some adjustments.

1) make squashfs separately for /usr, /opt and mount them via loopback

2) mount /tmp and /var using tmpfs

With the two adjustments on diskless PC I already can't notice performance degree even with KDE  :Smile: 

What could be optimized/tuned more? I tried using Network Block Devices (NBD) for mounting /usr and /opt, but got just a lot of troubles. At the moment I'm thinking about mounting /usr and /opt using ATA over Ethernet (AoE) instead loopback and NFS. In my opinion it can make system even faster with less load on the server and network.

If somebody has any ideas/experiences please share them! 

P.S. If somebody needs a detailed "how to optimize" I can post requested details.

EDIT:AoE works like a charm... and with combination with squashfs I'm getting even quicker boot now.

P.S.S. Who wants step-by-step HOWTO!?

HOWTO

!!!Change net.home to you domain and bsd to your server name bellow in config files. 

1) Configure your server (probably Gentoo as well  :Smile:  )

If you need more information how to configure server look at http://www.faqs.org/docs/Linux-mini/Remote-Boot.html#ss4.4. Here I'll explane only "my way".

configure DHCP, I'm using dnsmasq, which can act as DNS server for small/middle network as well.

Example of dnsmasq.conf. 

```
domain-needed

bogus-priv

filterwin2k

local=/net.home/

interface=eth0

expand-hosts

domain=net.home

dhcp-range=192.168.1.50,192.168.1.100,12h

dhcp-option=1,255.255.255.0

dhcp-option=3,192.168.1.1

dhcp-option=6,192.168.1.1

dhcp-option=17,192.168.1.1:/mnt/data/diskless/

dhcp-option=28,192.168.1.255

dhcp-option=44,192.168.1.1

dhcp-option=58,300

dhcp-option=59,300

dhcp-option=135,menu

#dhcp-option=252,"http://proxy/proxy.pac"

dhcp-boot=/pxelinux.0,bsd.net.home
```

 Configure tftp server. Under Gentoo I'm using atftp. 

Example of /etc/conf.d/atftp

```
TFTPD_ROOT="/mnt/local/tftproot"

TFTPD_OPTS="--daemon --user nobody --group nobody --bind-address 192.168.1.1"
```

Download syslinux (http://syslinux.zytor.com/) and copy chain.c32 menu.c32 memdisk pxelinux.0 to tftp root directory. Create pxelinux.cfg dir and create the default file under pxelinux.cfg directory:

```

PROMPT 0

DEFAULT menu.c32

TIMEOUT 50

MENU TITLE Boot menu

label local

    MENU LABEL ^Windows XP

    LOCALBOOT 0

label gnet-nfsroot

    MENU LABEL Gentoo Network NFS root

    kernel kernel/bzImage-2.6.15-ck2-rr4

    append init=/sbin/init ramdisk_size=32000 root=/dev/nfs ip=dhcp nfsroot=bsd:/mnt/data/diskless/net.home/root video=vesafb,mtrr vga=0x31A

label gnet-aoe

    MENU LABEL Gentoo Network AoE 

    MENU DEFAULT

    kernel kernel/bzImage-2.6.15-ck2-rr4-aoe

    append initrd=kernel/initrd.gz ramdisk_size=32000 root=/dev/ram0 rw ip=dhcp video=vesafb,mtrr vga=0x31A

```

In the example above all kernel and initrd images are kept under kernel subdirectory of tftp root directory.

 

```
emerge av sys-block/vblade
```

The server part is finished at the moment, so now the clien (diskless) PC. Let assume you have Gentoo Desktop successfully installed, which we will use as "prototype" of a diskless system. Firstly let have some notes of the kernel configuration. Be sure that you have:

-	Network driver compiled in the kernel (not as module)

-	

```
 [*]   IP: kernel level autoconfiguration            

 [*]     IP: DHCP support                            

 [*]     IP: BOOTP support 
```

- ATA ove Ethernet support compiled as module

```
<M> ATA over Ethernet support
```

-	Initial ramdisk support compiled in the kernel

```
 <*> RAM disk support                  

(16)  Default number of RAM disks     

(32000) Default RAM disk size (kbytes)
```

Now the optimization part begins.

1) I would suggest creating a separate directory for the diskless snapshot somewhere and copy all necessary parts there. Let assume that the snapshot directory is /mnt/diskless

```
mkdir /mnt/tmp

mount o bind / /mnt/tmp

cd /mnt/tmp

find . print | cpio pamd /mnt/diskless

```

At this point we have the snapshot under /mnt/diskless.

2) You can easy remove /mnt/diskless/usr/src, /mnt/diskless/usr/include, (/mnt/diskless/usr/kde/kde3.x/include), /mnt/diskless/usr/doc directories and /mnt/diskless/boot. Optionally /mnt/diskless/usr/man and /mnt/diskless/usr/info could be removed as well.

3) You should adjust come config files in the /etc directory for the diskless workstation as you wish. Bellow is only example of fstab for usling with AoE:

```
 /dev/ram0                               /       ext2            rw,noatime                      0 0

/dev/etherd/e0.0                        /usr    squashfs        ro,defaults                     0 0

/dev/etherd/e0.1                        /opt    squashfs        ro,defaults                     0 0

bsd:/mnt/data/diskless/net.home         /mnt/diskless   nfs     rw,hard,intr,nolock,nfsvers=3   0 0

bsd:/mnt/data/diskless/net.home/home    /home   nfs             rw,hard,intr,nolock,nfsvers=3   0 0

tmpfs                                   /tmp     tmpfs          size=64m,mode=1777              0 0

tmpfs                                   /var/tmp tmpfs          size=64m,mode=1777              0 0

tmpfs                                   /var/log tmpfs          size=16m                        0 0

proc                    /proc                   proc            defaults                        0 0

shm                     /dev/shm                tmpfs           nodev,nosuid,noexec             0 0

```

4) add

```
aoe aoe_iflist="eth0"
```

 to the kernel-2.6 file in the /etc/modules.autoload.d directory.

5) Now we will make squashfs of /usr and /opt, just

```
cd /mnt/diskless && mksquashfs ./usr  /path/to/usr.sqfs

mksquashfs ./opt  /path/to/opt.sqfs 
```

6) Now we can easily remove content of /usr and /opt under diskless snapshot (/mnt/diskless) and make initrd image of the rest:

```
#!/bin/bash

WORKDIR=/path/to/workdir

cd $WORKDIR

dd if=/dev/zero of=initrd count=64000 bs=512

mke2fs initrd

tune2fs -c0 -i0 initrd

mount -o loop initrd /mnt/initrd && \

cd /mnt/diskless && cp -ar . /mnt/initrd

umount /mnt/initrd && cd $WORKDIR && gzip -9 -c initrd > initrd.gz

```

7) copy initrd.gz and corresponding bzImage for diskless workstation to tftpfoot/kernel

At the moment we need one more step in order to complete server setup:

Copy opt.sqfs and usr.sqfs somewhere to the server and edit /etc/conf.d/vblade:

```
 config_vblade0="0 0 eth0 /path/to/usr.sqfs"

config_vblade1="0 1 eth0 /path/to/opt.sqfs"
```

```
cd /etc/init.d

ln s  vblade.vblade0 vblade.vblade1

./ vblade.vblade0 start

./ vblade.vblade0 start

```

So, its time now try bootig your client  :Wink: 

EDIT: For using initramfs (I'm preferring this way  :Wink:  )

Adjust the kernel patch for using tmpfs instead of initramfs (found on http://www.tux.org/pub/people/kent-robotti/looplinux/rip)

There's a problem with hotplug during early boot, so it should be disabled temporarly (chmod 644 /sbin/hotplug), then I modified /etc/ini.d/bootmisc and place the code bellow in the start section:

```
# Workaround for udevsend early boot error messages

        ebegin "Enabling hotplug"

        /bin/chmod 755 /sbin/hotplug

        eend 0

```

3) fstab should be adjusted

```
 /dev/ram0                               /       tmpfs           size=64m                        0 0

/dev/etherd/e0.0                        /usr    squashfs        ro,defaults                     0 0

/dev/etherd/e0.1                        /opt    squashfs        ro,defaults                     0 0

bsd:/mnt/data/diskless/net.home         /mnt/diskless   nfs     rw,hard,intr,nolock,nfsvers=3   0 0

bsd:/mnt/data/diskless/net.home/home    /home   nfs             rw,hard,intr,nolock,nfsvers=3   0 0

tmpfs                                   /tmp     tmpfs          size=64m,mode=1777              0 0

tmpfs                                   /var/tmp tmpfs          size=64m,mode=1777              0 0

tmpfs                                   /var/log tmpfs          size=16m                        0 0

proc                    /proc                   proc            defaults                        0 0

shm                     /dev/shm                tmpfs           nodev,nosuid,noexec             0 0

```

6) instead of creating initrd we should use initramfs:

```
cd /mnt/diskless

ln -s sbin/init init

find . | cpio -v -o -H newc | gzip -9 > ../initramfs.cgz
```

Finally we have to adjust boot loader configuration:

[codfe]label gnet-nfsroot

    MENU LABEL Gentoo Network NFS root

    kernel kernel/bzImage-2.6.15-ck2-rr4

    append quiet initrd=kernel/initramfs.cgz root=/dev/ram0 rw ip=dhcp video=mtrr,vesa vga=0x31A[/code]Last edited by VoVaN on Mon Apr 24, 2006 8:45 am; edited 10 times in total

----------

## VStrider

Hi. I'm also using diskless clients. Are you using ltsp or something else?

 *VoVaN wrote:*   

> In order to use heavy DE (KDE) I have to do some adjustments.
> 
> 1) make squashfs separately for /usr, /opt and mount them via loopback
> 
> 2) mount /tmp and /var using tmpfs 

 

Interesting, could you give some more details on your setup? Why do you need to mount /usr/ and /tmp?

I'm using gnome on the diskless clients and works great as long as there's no mono apps running (like beagle) and as long as I'm not using FF-1.5 (using FF1.5 will hang the clients after a few mins of use. I suppose it has something to do with the heavy memory use of FF1.5 which drains the res on the server. Btw I read this a couple of days ago, so it looks like mem cache can be disabled or reduced.)

----------

## VoVaN

 *VStrider wrote:*   

> 
> 
> Interesting, could you give some more details on your setup? Why do you need to mount /usr/ and /tmp?
> 
> 

 

As I mentioned I'm using initrd, so root at /dev/ram0 (in memory). The initrd is the same for all PC's, so all specific PC's parameters done on at boot time based on DHCP info. I didn't leave a lot of free space in initrd, so I mounted /tmp as tmpfs, which is basically designed for things like that. Secondly, I don't think it's smart to use ram for /usr, /opt..., so you have to mount it somehow later.

----------

## VoVaN

Who wants step-by-step HOWTO!?

----------

## playahater

 *VoVaN wrote:*   

> Who wants step-by-step HOWTO!?

 

i would like one, please ..  :Smile: 

Cheers

----------

## VoVaN

 *playahater wrote:*   

>  *VoVaN wrote:*   Who wants step-by-step HOWTO!? 
> 
> i would like one, please .. 
> 
> Cheers

 

You got it  :Wink: 

----------

## playahater

Gr8 job man ...  :Smile:  .. this is very usefull  :Smile: 

finally the old Pentium I without hdd will have its usage  :Smile: 

one more question .. i need PXE chip on my eth card, right? .. do i need PXE on both, client and server or .. ?

i do not have any eth card for PI but i`m planning to buy it in couple of days .. so can you recommend one ? .. 

i don`t know wich cards do have PXE  :Sad: 

10x in advace

Cheers

----------

## VoVaN

 *playahater wrote:*   

> Gr8 job man ...  .. this is very usefull 
> 
> finally the old Pentium I without hdd will have its usage 
> 
> one more question .. i need PXE chip on my eth card, right? .. do i need PXE on both, client and server or .. ?
> ...

 

You have to have PXE card only for the client PC. Most Intel and 3Com cards have it. I'm personly using Intel cards. Succes with the setup... later you probably would try to boot Windows95/98 diskless as well as me (for gaming  :Smile:  )

----------

## playahater

excellent  :Smile: 

10x man .. 

Cheers ..  :Smile: 

----------

## cazze

Well, i would like to have a howto too  :Smile: 

this sounds very nice for a silent htpc, so i can skip those noisy hdd's

kammicazze

----------

## playahater

 *kammicazze wrote:*   

> Well, i would like to have a howto too 
> 
> this sounds very nice for a silent htpc, so i can skip those noisy hdd's
> 
> kammicazze

 

1st post ..   :Wink: 

----------

## JeliJami

 *VoVaN wrote:*   

> Download syslinux (http://syslinux.zytor.com/) and copy chain.c32 menu.c32 memdisk pxelinux.0 to tftp root directory. Create pxelinux.cfg dir and create the default file under pxelinux.cfg directory:

 

any reason you don't use the syslinux ebuild?

```
# emerge -s syslinux

Searching...   

[ Results for search key : syslinux ]

[ Applications found : 1 ]

 

*  sys-boot/syslinux

      Latest version available: 3.09

      Latest version installed: [ Not Installed ]

      Size of downloaded files: 1,159 kB

      Homepage:    http://syslinux.zytor.com/

      Description: SysLinux, IsoLinux and PXELinux bootloader

      License:     GPL-2

```

 *Quote:*   

> 1) I would suggest creating a separate directory for the diskless snapshot somewhere and copy all necessary parts there. Let assume that the snapshot directory is /mnt/diskless
> 
> ```
> mkdir /mnt/tmp && mount o bind / /mnt/tmp && cd /mnt/tmp && find . print | cpio pamd /mnt/diskless
> ```
> ...

 

I'm probably missing something here.

the series of commands

```
mkdir /mnt/tmp

mount -o bin / /mnt/tmp

cd /mnt/tmp

```

would result in / being accessible in /mnt/tmp

i guess you need a copy of your install into /mnt/tmp first (or a fresh install)

then

```
# chroot /mnt/tmp /bin/bash
```

and remove stuff in this chroot environment

 *Quote:*   

> 2) You can easy remove /usr/src, /usr/include, (/usr/kde/kde3.x/include), /usr/doc directories and /boot. Optionally /usr/man and /usr/info could be removed as well.
> 
> 

 

you're removing /usr/src, ..., /boot from the / system (your server basically)?

----------

## FormerSlacker

I've got a setup like that myself. Old P100 with no hard drive, booting off the network via etherboot and tftp, and it's root is exported via NFS off the server, and then it launches X and grabs the nice little KDM screen via XDMCP.

I've used Debian for the diskless client OS, seems to work well once you strip away most of the startup services. I'm always amazed to see KDE running perfectly on such a krappy old computer.

Such a neat thing.   :Smile: 

----------

## VoVaN

 *davjel wrote:*   

>  *VoVaN wrote:*   Download syslinux (http://syslinux.zytor.com/) and copy chain.c32 menu.c32 memdisk pxelinux.0 to tftp root directory. Create pxelinux.cfg dir and create the default file under pxelinux.cfg directory: 
> 
> any reason you don't use the syslinux ebuild?
> 
> ```
> ...

 

That's possible as well, but I would preffer downloading: 1) the latest version there is 3.11; 2) I need only 4 files and nothing else.

 *davjel wrote:*   

> 
> 
>  *Quote:*   1) I would suggest creating a separate directory for the diskless snapshot somewhere and copy all necessary parts there. Let assume that the snapshot directory is /mnt/diskless
> 
> ```
> ...

 

you aren't right here.... 

```
mount -o bind / /mnt/tmp
```

 makes root accesseble under /mnt/tmp. 

I'm doing thet in order to get rid of all unnecessary files under /proc /sys /dev directories. 

 *davjel wrote:*   

> then
> 
> ```
> # chroot /mnt/tmp /bin/bash
> ```
> ...

 

Sorry, didn't get your point here  :Sad: 

 *davjel wrote:*   

> 
> 
>  *Quote:*   2) You can easy remove /usr/src, /usr/include, (/usr/kde/kde3.x/include), /usr/doc directories and /boot. Optionally /usr/man and /usr/info could be removed as well.
> 
>  
> ...

 

Thanx for this comment, I'll correct it in the original post. I meant /usr/src, /boot, etc. from the snapshot directory (in our case /mnt/diskless) so it should be /mnt/diskless/usr/src, /mnt/diskless/boot...

----------

## VoVaN

2 davjel: Thanx for reviewing  :Smile:  I corrected what I promissed.

----------

## VoVaN

 *FormerSlacker wrote:*   

> I've got a setup like that myself. Old P100 with no hard drive, booting off the network via etherboot and tftp, and it's root is exported via NFS off the server, and then it launches X and grabs the nice little KDM screen via XDMCP.
> 
> I've used Debian for the diskless client OS, seems to work well once you strip away most of the startup services. I'm always amazed to see KDE running perfectly on such a krappy old computer.
> 
> Such a neat thing.  

 

First of all, I'm glad to hear that I'm not along with the "diskless workstation" idea  :Wink: 

Even in your case you'll benefit if you're doing AoE and squashfs. In my case KDE loads twice quicker just with squashfs, then over NFS. 

As for NFS root: If you have only one computer "NFS root" is just fine, but if you have a lot of them.... it' better to have single initrd  with boottime setup. 

Finally I'd like to mention, that NFS under heavy load takes a lot of CPU time in case of AoE it's 5 times as less in my setup.

----------

## JeliJami

 *davjel wrote:*   

> 
> 
> any reason you don't use the syslinux ebuild?
> 
> 

 

 *VoVaN wrote:*   

> That's possible as well, but I would preffer downloading: 1) the latest version there is 3.11; 2) I need only 4 files and nothing else.
> 
> 

 

and i would prefer to have my packages maintainable by my package manager, in this case portage

but that's just me, i guess  :Smile: 

 *VoVaN wrote:*   

> 1) I would suggest creating a separate directory for the diskless snapshot somewhere and copy all necessary parts there. Let assume that the snapshot directory is /mnt/diskless
> 
> ```
> mkdir /mnt/tmp && mount o bind / /mnt/tmp && cd /mnt/tmp && find . print | cpio pamd /mnt/diskless
> ```
> ...

 

what i missed was

```
find . print | cpio pamd /mnt/diskless
```

i didn't see that one as a copy command   :Embarassed: 

 *VoVaN wrote:*   

>  *davjel wrote:*   then
> 
> ```
> # chroot /mnt/tmp /bin/bash
> ```
> ...

 

what i meant was: use the alternative installation guide to install gentoo in a chrooted environment (your new 'snapshot') using an installed linux system (your 'server')

but never mind, your copy operation (see above) is definitely faster  :Smile: 

 *davjel wrote:*   

> 
> 
>  *Quote:*   2) You can easy remove /usr/src, /usr/include, (/usr/kde/kde3.x/include), /usr/doc directories and /boot. Optionally /usr/man and /usr/info could be removed as well.
> 
>  
> ...

 

 *VoVaN wrote:*   

> Thanx for this comment, I'll correct it in the original post. I meant /usr/src, /boot, etc. from the snapshot directory (in our case /mnt/diskless) so it should be /mnt/diskless/usr/src, /mnt/diskless/boot...

 

you're welcome  :Cool: 

----------

## FormerSlacker

 *VoVaN wrote:*   

>  *FormerSlacker wrote:*   I've got a setup like that myself. Old P100 with no hard drive, booting off the network via etherboot and tftp, and it's root is exported via NFS off the server, and then it launches X and grabs the nice little KDM screen via XDMCP.
> 
> I've used Debian for the diskless client OS, seems to work well once you strip away most of the startup services. I'm always amazed to see KDE running perfectly on such a krappy old computer.
> 
> Such a neat thing.   
> ...

 

With the setup the way I have it, each client would require its own NFS mount, with Debian and X its about 300 megs, not too bad. KDE is loaded on the server, the client just loads a X session and connects to the server.

It's really a dumb thin client, that does nothing but boots linux and starts an X server.

NFS load is minimal, after the inital bootup and starting X (~45s-1min), the nfs is rarely accessed by the client.

What I have been thinking about is getting rid of the NFS server completely, and sending the clients a tiny squashfs initrd and kernel via tftp that contains just enough to connect to an XDMCP X server. I've tinkered with pxes, but its kernel just seg faults right on boot.

----------

## VoVaN

 *FormerSlacker wrote:*   

> 
> 
> With the setup the way I have it, each client would require its own NFS mount, with Debian and X its about 300 megs, not too bad. KDE is loaded on the server, the client just loads a X session and connects to the server.
> 
> It's really a dumb thin client, that does nothing but boots linux and starts an X server.
> ...

 

Could you tell more, probably I can help if you'd like. I've a lot of diskless computers and they all are using pxe + kernel + initrd.

----------

## olger901

Shouldn't this be moved over to Documentation, Tips & Tricks...?

----------

## FormerSlacker

 *VoVaN wrote:*   

>  *FormerSlacker wrote:*   
> 
> With the setup the way I have it, each client would require its own NFS mount, with Debian and X its about 300 megs, not too bad. KDE is loaded on the server, the client just loads a X session and connects to the server.
> 
> It's really a dumb thin client, that does nothing but boots linux and starts an X server.
> ...

 

Such friendly folk around here.   :Smile: 

What I was trying to do initially is send out Pxes to the diskless clients to boot. I've managed to do this, but the Pxes Kernel, 2.4 and 2.6, seems to segfault on boot; the keyboard lights just blink on the clients with no error displayed.

When that didn't work out, I just settled with Debian on an NFS root.

What I'd like to do is just send a tiny distro, Pxes or something else, in an initrd and the kernel to all the clients that would initiate an X session with the server.

Worse comes to worse, it may be time to build a tiny distro on my own.

If you have any pointers to any other tiny Linux distros, with just bare X to connect locally with XDMCP and nothing else, or any help it'd be much appreciated.

----------

## VoVaN

 *FormerSlacker wrote:*   

> 
> 
> Such friendly folk around here.  
> 
> 

 

 :Wink: 

 *FormerSlacker wrote:*   

> 
> 
> When that didn't work out, I just settled with Debian on an NFS root.
> 
> 

 

Could you tell more details about your setup? I'm using PXE, not pxes distro you mentioned.

 *FormerSlacker wrote:*   

> 
> 
> What I'd like to do is just send a tiny distro, Pxes or something else, in an initrd and the kernel to all the clients that would initiate an X session with the server.
> 
> Worse comes to worse, it may be time to build a tiny distro on my own.
> ...

 

You can build relly tiny distro based on Gentoo as well. I think it's better if you have some experiences, look at sys-apps/baselayout-lite in portage  :Wink: 

 *FormerSlacker wrote:*   

> 
> 
> If you have any pointers to any other tiny Linux distros, with just bare X to connect locally with XDMCP and nothing else, or any help it'd be much appreciated.

 

I've seen the floppy based distro exactly for things you mentioned, but can't remember anymore  :Sad:  Look at http://www.linux.org/dist/list.html in "embedded" category, probably you'll fing it, but personally I would preffer to do it with Gentoo.

----------

## FormerSlacker

 *VoVaN wrote:*   

> Could you tell more details about your setup? I'm using PXE, not pxes distro you mentioned.
> 
> 

 

Sure can. Here's the setup. Perhaps this may help some others out there.

1) The client boots up and makes a dhcp request for the boot files. Since the computers in question are real old (P100 to 166, 32 to 128m) and do not support network booting, I had to use a bootdisk for the clients using Etherboot  rom-o-matic. Tape the floppy drive shut and you have ghetto network booting.  :Wink: 

2) DHCP server gives the client the necessary information for it to boot up. 

```
 host george {

      hardware ethernet 00:80:C6:EA:RB:53;

      fixed-address 192.168.0.4;       # clients address

      option host-name "george"

      filename "/tftpboot/clients/george/vmlinuz.nbi"; # network boot image, kernel + initrd

      next-server 192.168.0.3; # instruct the client that dhcp server is not the host, this ip is

      option root-path "/diskless/clients/george/"; # root nfs path

   }

```

3) TFTP and NFS are setup on the server using the above paths for booting and root repsectively.

4) The NBI image is a custom kernel for the client with the "ROOT on NFS" option enabled, made with the mknbi package in portage.

5) The clients NFS root is just a bare debian distro, that was installed then copied over to the clients NFS export directory, configured to query the server for an xsession on boot: "X -query 192.168.0.3"

6) Login manager, KDM, is configured to listen for remote login requests by editing /usr/kde/3.5/share/config/kdm/Xaccess and adding the ip of the client and setting "Enable=true" under "[Xdmcp]" in  /usr/kde/3.5/share/config/kdm/kdmrc.

Thats the long and the short of it. Remote, fast, KDE goodness on ancient machines.

 *VoVaN wrote:*   

> You can build relly tiny distro based on Gentoo as well. I think it's better if you have some experiences, look at sys-apps/baselayout-lite in portage 
> 
> I've seen the floppy based distro exactly for things you mentioned, but can't remember anymore  Look at http://www.linux.org/dist/list.html in "embedded" category, probably you'll fing it, but personally I would preffer to do it with Gentoo.

 

I've debated whether sending the client distro as a initrd and having it mounted from ram is a good idea. I'm not sure if the 32mb clients can afford to do that. I think the only thing left is, as you mentioned, roll out a tiny version of gentoo for this purpose.

 *olger901 wrote:*   

> Shouldn't this be moved over to Documentation, Tips & Tricks...?

 

I suppose since the end goal of all this is getting a desktop environment on diskless workstations, this is also relavant here. Then again what do I know, I'm not a mod.   :Cool: 

----------

## VoVaN

 *FormerSlacker wrote:*   

> 
> 
> 1) The client boots up and makes a dhcp request for the boot files. Since the computers in question are real old (P100 to 166, 32 to 128m) and do not support network booting, I had to use a bootdisk for the clients using Etherboot  rom-o-matic. Tape the floppy drive shut and you have ghetto network booting. 
> 
> 2) DHCP server gives the client the necessary information for it to boot up. 
> ...

 

One aspect of your setup looks strange: if ypu're using NFS root why you need  to create initrd or kernel+initrd(nbi)? Just single kernel with right config is enough to mount root over NFS.

 *FormerSlacker wrote:*   

> I've debated whether sending the client distro as a initrd and having it mounted from ram is a good idea. I'm not sure if the 32mb clients can afford to do that. I think the only thing left is, as you mentioned, roll out a tiny version of gentoo for this purpose.
> 
> 

 

Ayway in you case and for the hardware you have the best solution is there -> http://thinstation.sourceforge.net/wiki/index.php/. This is the distro I've been talking about and I've recently found it for you  :Wink: 

----------

## FormerSlacker

VoVaN you are correct, I'm only sending out the kernel at the moment no initrd, sorry if I got it mixed up in one of my previous posts.

I must thank you kindly for the thinstation link, it is exactly what I'm looking for!   :Surprised:   I'm going to give it a spin and see how well it works. 

Cheers mate!   :Very Happy: 

**EDIT**

Just a FYI, it works great. The kernel and initrd come up to 3.5 megs, and it works great. Thanks again VoVan.   :Cool: 

----------

## Abraxa

Why is this thread in Desktop Environments? Someone should move this to the documentation forum, I reckon.

-Soeren

----------

## VoVaN

I'm going to try more efficient initramfs instead of initrd. Does anybody have any expiriences. As a bonus, it's possible to avoid blinking with bootsplash then.  :Wink: 

----------

## VoVaN

Finally, I've successfully switched to initramfs... even shorter boot time and quicker application execution! I had some troubles during early boot. That was /sbin/hotplug, so that's necessary to disable it somehow (chmod 644) for the time of loading the kernel.

----------

## ldc

Just wanted to give some back after the initial idea from this thread  :Smile: 

Here you go, a generic initramfs that should work for most distributions. The only thing is that it's only premade for x86_64 so if you have something else you will have to replace the busybox.static in there. (follow the guidelines)

http://bladerunner.mine.nu/

Some things in there are specific for my system, but it should still work on most systems with a minimal extra delay. (got a network card that takes 5-10 seconds before any traffic can actually pass on it.)

Hope it's useful for someone.

----------

## VoVaN

2ldc: I switched using very small ~400K initramfs with busybox as well, because tftp transfers (in case of relatively big initramfs) are much slower then NFS or FTP for instance. Now I'm using lzma instead of bzip compressed kernel (patch http://www.udpcast.linux.lu/source.html) and lzma patched squashfs tools (http://www.squashfs-lzma.org/). It makes images smaller and improves decompression speed. Since the initial post I've made even some more improvements, but it looks like it's not very popular in our days or it's wrong place for posting, so I stopped any updates of this topic  :Sad: 

----------

## ldc

Hi VoVaN,

Cannot see how you can think that tftp is much slower than NFS or FTP?... Have done some tests in my setup here and with tftp i have been booting initramfs-images of around 40Mb within 3-4 seconds (+ decompression of course) while nfs have been up to around 10-15 seconds, maybe down to around 5-6 seconds with a idle NFS server.. I where using gPXE for that.

Have not tried using http or ftp so far, but i suspect that those also are affected by the amount of time it takes to init the connection before connecting. Only issue when i have seen when tftp has performed badly is when using a non-compatible tftp server that causes a real slowdown.

Have not really looked into improving decompression-speed since it's still so low that i cannot notice it when the image is this small (2Mb), but thanks for the hints, might be something to use for the HTPC ramdisk later on  :Smile: 

Just some observations that i have made on the setup here.. And maybe not allot of people are using this just because the documentation is so scarce, or just to complicated for the average user. I would be glad to see some basic AoE support implemented in the gentoo initramfs, and it should be quite easy to get that into the generic initramfs, maybe even on the minimal install cd to allow for a install to a AoE disk.

All that's really needed here is: check for AoE in the kernel and if we are trying to boot against a AoE target, bring up all interfaces or atleast the one specified on the kernel-cmd-line, touch the discover, wait for AoE to complete the discovery process and continue with the init.

Probably gonna do some more stuff related to this in the near future so maybe i'll be able to post a good generic patch for genkernel to include this too.

But i got to say that i'm loving booting from a AoE target... Got 3Gb ram in the soe-server and it really boosts the disk-performance. (15 second for a find / -xdev with around 400k files when everything is in the targets buffercache). And planning to implement a GFS or OCFS2 shared volume via AoE soon too, but that needs more research, and some fix to access the disk via AoE on the actual target-system. Maybe i'll write a howto for all this later on too..

----------

## VoVaN

 *ldc wrote:*   

> Hi VoVaN,
> 
> Cannot see how you can think that tftp is much slower than NFS or FTP?... Have done some tests in my setup here and with tftp i have been booting initramfs-images of around 40Mb within 3-4 seconds (+ decompression of course) while nfs have been up to around 10-15 seconds, maybe down to around 5-6 seconds with a idle NFS server.. I where using gPXE for that.
> 
> Have not tried using http or ftp so far, but i suspect that those also are affected by the amount of time it takes to init the connection before connecting. Only issue when i have seen when tftp has performed badly is when using a non-compatible tftp server that causes a real slowdown.

 

I think the keyword here is gPXE. I didn't try gPXE (jet), but using intel PXE + PXELinux (tried a lot of PCI intel cards as well as onboard NIC's) makes tftp transfers really slow. I've done some research a while ago and found some proves concerning my point. BTW, could you explain your setup with gPXE?

 *ldc wrote:*   

> 
> 
> Have not really looked into improving decompression-speed since it's still so low that i cannot notice it when the image is this small (2Mb), but thanks for the hints, might be something to use for the HTPC ramdisk later on 

 

Probably I explained incorrectly, but it's not exactly "decompression speed" of initramfs image (it indeed not really noticeable), but reading LZMA compressed squashfs image via AoE is somehow about 30-50% faster then standard squashfs. As I mentioned briefly already, in my setup I'm using lzma compressed kernel (~1.2Mb instead of ~1.8Mb with bzip) with ~400kb initramfs (busybox for setting up root on initramfs and AoE based on kernel command line parameters), the rest is read-only squashfs image via AoE as well. Using root on initramfs (/tmp, /var/tmp and /var/log on initramfs as well) makes setup more flexible and _really_ improves boot times. It's even boots faster then if I would use just local hard drive on the same PC without any initramfs/squashfs)

 *ldc wrote:*   

> 
> 
> Just some observations that i have made on the setup here.. And maybe not allot of people are using this just because the documentation is so scarce, or just to complicated for the average user. I would be glad to see some basic AoE support implemented in the gentoo initramfs, and it should be quite easy to get that into the generic initramfs, maybe even on the minimal install cd to allow for a install to a AoE disk.
> 
> All that's really needed here is: check for AoE in the kernel and if we are trying to boot against a AoE target, bring up all interfaces or atleast the one specified on the kernel-cmd-line, touch the discover, wait for AoE to complete the discovery process and continue with the init.
> ...

 

I've been thinking about it as well, but in my case it's GFS is useless (squashfs is readonly). It probably would be interesting using GFS+AoE instead of NFS (speed improvements?). Keep us informed, i'm interesting in all these kind on researches.

----------

## ldc

 *VoVaN wrote:*   

> 
> 
> I think the keyword here is gPXE. I didn't try gPXE (jet), but using intel PXE + PXELinux (tried a lot of PCI intel cards as well as onboard NIC's) makes tftp transfers really slow. I've done some research a while ago and found some proves concerning my point. BTW, could you explain your setup with gPXE?
> 
> 

 

Yea, i have seen that behaviour too, until i switched the tftp-server.. There are alot of network cards that has a problem with specific features but at atftp seems to be working good for most cards (and have tried it with intel too)

 *VoVaN wrote:*   

> 
> 
> I've been thinking about it as well, but in my case it's GFS is useless (squashfs is readonly). It probably would be interesting using GFS+AoE instead of NFS (speed improvements?). Keep us informed, i'm interesting in all these kind on researches.
> 
> 

 

Will do... but have run into some road bumps  :Razz: 

AoE dont support access to a locally shared AoE blade, and since i'm using lvol's for the disk-shares it's a bit of a problem to get access to 'virtual' disk.

I'm currently looking into switching from qaoed to one of the kernel-based AoE servers to be able to add local block-device support for the devices. Ie, instead of going via network for accessing the AoE disk i'm looking into adding support into the AoE server to display the shared disks locally that will bypass the network completely.

Have also started looking into OCFS2 (Oracle Cluster FS) since it requires a much simpler setup and some people i have talked to have recommended it to me since i mainly have lots of large (>100MB) files and OCFS2 should have much better performance, but have had some problems with fencing here due to the need of accessing the OCFS2 filesystem on the same machine that i have the disks, and since AoE where not able to share disks locally i tried it with iSCSI. Did work great as long as the load was low, but as soon as i tried maxing out the performance the client fenced due to heartbeat timeouts. Might have been better with AoE since it we much nicer to the network stack by not using tcp/ip connections.

And yea, the performance of GFS do seem to be better than NFS. See the following link for some tests.

http://web.bii.a-star.edu.sg/~james/fs_review/

----------

## ldc

Ignore the previous post grammatic errors... was quite tired when i wrote it  :Smile: 

Have been in contact with the aoeserver owner since that was the one that looked the most promising for my intended use and he seems to have dropped all development of it in favor of qaoed, so i have forked the aoeserver and created a project for it on http://code.google.com/p/aoeserver/

Quite a n00b in device-driver development so it will probably take a while before i get up to speed with this, but keep your eyes open, or maybe help out *wink*, and you might find something there soon.  :Smile: 

----------

