# My Gentoo diskless install howto

## Swiss.Mage

Hi all,

The aim of this post is to provide a tested procedure to setup a diskless Gentoo desktop, using PXE booting capabilities of some network cards and the PXE GRUB bootloader. This procedure has been tested on at least one computer and is really working. 

If you want to do this you will need a network card on the diskless client that uses the PXE protocol to boot, like many 3com cards, for example.

I have to apologize for the quality of the language but English isn't my first language.

In the next lines, I will talk about the "client" and the "server". The client is the diskless machine and the server is the one providing DHCP and NFS filesystem. In all my examples, the client is called "eta" and the server "sigma" ... change this to your needs. The "sigma" server has a static IP address that is 192.168.1.20.

I will separate this precedure into several parts :

1. Server base setup

2. Gentoo setup on the server

3. Booting the client

4. Enable XFree XDMCP

5. Enjoy ...

1. Server base setup

Of course, I wont talk here about the complete setup of the server. For this follow the install guide provided in the Documentation area of the Gentoo's website.

1.1 Create directories

The first thing to do is to create the directories where your diskless system will be stored. Personally, I created a directory called /diskless on which I created a directory for each diskless hosts :

```
# mkdir /diskless

# mkdir /diskless/eta
```

Here "eta" is the name of one of my diskless desktop.

A good thing would be to direcly create a "boot" directory for each host :

```
# mkdir /diskless/eta/boot
```

1.2 DHCP and TFTP setup

The client will get boot informations using DHCP and download all the required files using TFTP. Here's how to setup this.

Just emerge DHCP and configure it for your basic needs. (I wont explain this here). Then, add the following on /etc/dhcp/dhcpd.conf :

 *Quote:*   

> option option-150 code 150 = text ;
> 
> host eta {
> 
>     hardware ethernet 00:00:00:00:00:00;
> ...

 

This provide a static IP adress for the client and the path of a PXE boot image, here pxegrub. You have to replace the MAC address of the Ethernet card of the client and the directory where you will put the client files with the one you use.

For TFTP, I emerged "app-admin/tftp-hpa" that is really easy to use. In /etc/conf.d/in.tftpd, I put the following :

 *Quote:*   

> INTFTPD_PATH="/diskless"
> 
> INTFTPD_USER="nobody"
> 
> INTFTPD_OPTS="-u ${INTFTPD_USER} -l -vvvvvv -p -c -s ${INTFTPD_PATH}"

 

1.3 Setup GRUB

To provide PXE booting I use GRUB. I've done this because I'm already using GRUB on several machines and I'm happy with. The bad piece of news is that you have to compile it by yourself to enable the PXE image compilation ... but that's quite easy.

First, get the latest version of the GRUB source code (over Internet or just do "emerge -f grub" and get it into the /usr/portage/distfiles directory) and copy it to a nice place ... for example /root.

Now, do the following :

```
# tar zxvf grub-0.92.tar.gz

# cd grub-0.92

# ./configure --help
```

You will see a list of available options. In this options, you will see several relative to network interfaces drivers. Look for the option that match you client network card. In my case, I'm using a 3com 3c905 card.

Then, do this :

```
# ./configure --enable-diskless --enable-3c90x

# make
```

Don't do "make install", this is not necessary ! This will compile the GRUB source code and generate the nice pxegrub image we need.

The compilation ended, go to the "stage2" subdir and copy "pxegrub" to your diskless client boot directory :

```
# cd stage2

# cp pxegrub /diskless/eta/boot/pxegrub
```

Now edit the GRUB config file for the the diskless client :

```
# nano -w /diskless/eta/boot/grub.lst
```

And put the following inside (adapt the exemple for your needs) :

 *Quote:*   

> default 0
> 
> timeout 30
> 
> title=Diskless Gentoo
> ...

 

For the nfsroot option, the IP address is the one of the server and the directory is the one where your diskless client files are located (on the server).[/code]

1.4 Setup NFS

NFS is quite easy to configure. The only thing you have to do is to add a line on the /etc/exports config file :

 *Quote:*   

> # /etc/exports: NFS file systems being exported.  See exports(5).
> 
> /diskless/eta           eta(rw,sync,no_root_squash)

 

1.5 Update your hosts

One important thing to do now is to modify your /etc/hosts file to fit your needs. In my configuration, I have the following on this file (I just replaced the domaine name) :

 *Quote:*   

> 127.0.0.1       localhost
> 
> 192.168.1.10    eta.example.com          eta
> 
> 192.168.1.20    sigma.example.com        sigma

 

2. Gentoo setup on the server

I have tried many ways to install the base system of  my diskless client. But the only one that really succeded is the following :

2.1 Reboot the server on a Gentoo LiveCD

I don't think I have to explain this ...

2.2 Then follow the standard install procedure as explained in the Gentoo Install Howto BUT with the following differences :

2.2.1 Mounting file system

When you have to mount the file system, do the following (where hda3 is the partition where you created the /diskless directory) :

```
# mount /dev/hda3 /mnt/gentoo
```

Don't do anything else ... 

2.2.2 Stage tarballs and chroot

WARNING ! Be very careful where you extract your stage tarball ...

I used a stage3 tarball ... as I wanted to have it running quick :

```
# [b]cd /mnt/gentoo/diskless/eta/[/b]

# tar -xvjpf  /mnt/cdrom/gentoo/stage3-*.tar.bz2

# mount -t proc /proc /mnt/gentoo/diskless/eta/proc

# cp /etc/resolv.conf /mnt/gentoo/diskless/eta/etc/resolv.conf

# chroot /mnt/gentoo/diskless/eta/ /bin/bash

# env-update

# source /etc/profile
```

Now follow the usual steps ... until the kernel configuration

2.2.3 Kernel configuration

When you do the "make menuconfig" of your kernel configuration, don't forget to enable the following options with the others recommended into the install howto :

- Your network card device support

- Under "Networking options" :

 *Quote:*   

> [*] TCP/IP networking
> 
> [*]     IP: kernel level autoconfiguration
> 
> [*]          IP: DHCP support
> ...

 

- Under "File systems --> Network File Systems" :

 *Quote:*   

> <*> NFS file system support
> 
> [*]     Provide NFSv3 client support
> 
> [*]     Root file system on NFS

 

2.2.4 /etc/fstab

Just let the following on the file :

 *Quote:*   

> /dev/cdroms/cdrom0      /mnt/cdrom      iso9660         noauto,ro               0 0
> 
> proc                    /proc           proc            defaults                0 0
> 
> tmpfs                   /dev/shm        tmpfs           defaults                0 0

 

2.2.5 Bootloader

Dont install any bootloader because we have already install one (pxegrub).

2.2.6 Restart the server

Finish the install and restart the server. It should boot like the others times (if not you have probably made a mistake  :Razz:  ). Start DHCP, TFTPD and NFS. 

3. Booting the client

Now, just boot the client. Configure the bios and the network card to use PXE in first to boot (before CD-ROM or floppy).

The network card should get an IP address using DHCP and download the GRUB PXE image using TFTP.

Then, you should see a nice black and white GRUB bootmenu where you will select the kernel to boot and press Enter.

If everything is ok the kernel should boot, mount the root filesystem using NFS and provide you the so waited login prompt.

4. Enable XFree XDMCP

Ok ... now you can use your diskless client as if it was a normal one. But the client is a very old machine like a PII 350 for me, running graphical applications on it is a bit stupid, at least if you have a nice P4 with a lot of Mhz an memory as server.

So, I recommend you to emerge distcc and then XFree on the client. After that, follow the instructions given at this URL to enable XDMCP on the server :

http://www.tldp.org/HOWTO/XDMCP-HOWTO/procedure.html

Then, on the client, do :

```
# X -broadcast
```

And you should get the server display manager login prompt as display ... or the one of your neighbour ...  :Razz: 

Something you have to note about this is that everything will run on the remote server ... even sound ! If it's not what you want, start the X server as usual from the client so it will run on it and use it own hardware.

5. Enjoy ...

Ok, now you have a nice diskless desktop that runs all his stuff on a big server located somewhere else ... great no ?

If you are happy, like me, with this, post a message here to gratulate me  :Razz: 

If not, maybe because something gone wrong and my howto doesn't help, search on this nice forum before asking me ... There're a lot of other diskless experiments and you will maybe find your answere.

If you want me to add something to this howto just post here too ...

----------

## darookee

is there a way to do this without a bootable network card?

----------

## Swiss.Mage

This howto covers only the use of a PXE capable network card.

I see how to do without ... maybe with a floppy or a cd-rom ... so look over the forum and the net to find more information about this.

----------

## kermitjunior

Ok,

So I wanted to take a minute to say "THANK YOU"   

How does this differ from LTSP? Any advantages or disadvantages?

----------

## hielvc

You might want to submit it or have it moved to "tips and tricks forum"

hielvc

----------

## Swiss.Mage

In fact I have never heard of this   :Embarassed: 

I will have a look but one of the major advantage of "my" way to create a diskless desktop is to understand how it works ... at least a bit ... I hope   :Confused: 

----------

## BradN

If you don't have a boot ROM for your network card, there's a site that you can create floppy images that act like a boot rom (or create "real" boot rom images and burn them to a chip)...  http://rom-o-matic.net - you could burn one of these as a floppy boot image on a CD if you had a CD drive but no floppy.

And, if you're really insane, I think it might be possible on some BIOS'es to include a net booting module by modifying the bios image (award comes to mind, but i'll leave that as an exercise to the experimentor) - heh, that's why it's called a modular bios - there are in fact modules that can be added into it, space permitting.

----------

## GurliGebis

How about swap, who does your /etc/fstab look?

----------

## ctford0

I have seen other diskless howto's also and I have always had one question.  Why use tftp and not something like ssh?  Is tftp secure?

Thanks for the input...

Chris

----------

## pjp

Moved from Installing Gentoo.

----------

## GurliGebis

Well, Isn't there a way to be able to have swap on this system?

----------

## Swiss.Mage

1. About SWAP

I have only 3 entries in my /etc/fstab. One for the CD-Rom, one for the /proc and one for tmpfs.

A SWAP filesystem need a disk ... I think ... (SWAP in RAM looks a bit strange   :Shocked:  ... And really useless  :Laughing:  ). So you're no more on a diskless setup. And if you have a look at the cost of the RAM it's really incredibly cheap now ! Maybe you could mount a SWAP partition via NFS  :Question:   :Question:   :Question: 

2. About TFTP

First of all, I don't think there's a way to use SSH with a network card ...

Then, there are several aspects that make TFTP not so dangerous : TFTP run as user nobody, is (normally) in read-only mode and has only access to the kernel file and to the GRUB config file ... nothing really dangerous.  But a good firewall may help, and fo course you have to be sure your network is secure, I mean the physical access to the network. You may also have a look at the file /etc/hosts.allow.

----------

## GurliGebis

I was thinking of having a swapfile on one of the root "partition", but I don't think swap over nfs i very stable.

----------

## ctford0

Ok, say I decide to setup 6 diskless clients.  What directories can be shared between them all?

/usr 

/bin

/sbin

/etc ?????

/opt

/lib ???

I really dont want to have duplicates of things that I dont need to.  I know that these I will have to have for each terminal.

/dev

/tmp

/var

Any others to add to this list?

Chris

----------

## Swiss.Mage

Hum ... It depends.

Something you can try is, at least, /usr/portage/* (I have done it between 3 computers and it works quite well).

If the diskless clients are exactly the same, you can try sharing /lib /bin /opt /sbin and /usr but doing this means that all the libraries and binaries will be shared between the computers and so, installing something on one client will install it for all the others ...

If the users, groups, ... are the same on each machine, you could even try sharing /etc and /home ...

I have to "confess" I have never try to share all of this system dirs ... So if you do it, post a message here to provide feedback for the other users (and me too).

----------

## ctford0

Well, what I have is basically dumb terminals, just for X purposes connecting through XDMCP to the server.  The only user on any of the machines will be root because right after bootup I plan to execute an rc-script to launch X doing a query to the server.  This is actually in an office, so with every new office mate another dumb terminal will be added.  Just trying to save myself some work (hehe).  I will probably try to implement this within the next 2 weeks, so I will definately post my results here when finished.

Chris

----------

## Swiss.Mage

Something I can add about this setup is that it will need about 1 GB including the X server.

To connect to a remote X server, the only thing you have to do is to run "X -broadcast" on the client. This can be done with this rc-script :

```
#!/sbin/runscript

depend() {

    after *

}

start() {

    X -broadcast

}

stop() {

    killall X

}
```

----------

## ctford0

The method that I prefer to use, since there are so many XDMCP hosts around me is to do 

```

XFree86 -query xxx.xxx.xxx.xxx

```

this way you always get the server you want, not your neighbor's   :Wink: 

Chris

----------

## HAsker

Hello.. I have read your how-to and realy likes it.. but there is a problem i am having the dhcpd on another computer and when i'm telling it to redirect to that one it does like that but when it is connected to the tftpd it only says that it cant find the file..

```

May 20 17:33:23 hasker in.tftpd[1706]: RRQ from 192.168.0.10 filename /htpc/boot/pxegrub

May 20 17:33:23 hasker in.tftpd[1706]: sending NAK (1, File not found) to 192.168.0.10 
```

This is how my /boot at the diskless client disk..

```
root@hasker boot # ls

boot  bzImage  grub  grub.lst  pxegrub

root@hasker boot # 

```

----------

## Swiss.Mage

Is your DHCP and TFTP server different from your NFS server ?

If it's the case, we are in the same situation (I didn't talk about this into the howto in order to "generalize" it). In fact, in my situation, I copied all the pxegrub image, the kernel image and the grub config file to a directory of the DHCP server : /diskless/boot/eta.

Here's my real /ets/dhcp/dhcpd.conf :

 *Quote:*   

> option option-150 code 150 = text ;
> 
> host eta {
> 
> 	hardware ethernet ...;
> ...

 

And for /etc/conf.d/in.tftpd :

 *Quote:*   

> INTFTPD_PATH="/diskless/boot"
> 
> INTFTPD_USER="nobody"
> 
> INTFTPD_OPTS="-u ${INTFTPD_USER} -l -vvvvvv -p -c -s ${INTFTPD_PATH}"

 

Hope this help ...

----------

## HAsker

The in.tftpd server and the nfs server is on the same machine but the in.tftpd cant find the pxegrub image and the grub.lst .. :/

----------

## Swiss.Mage

I think the in.tftpd server must be on the same machine than the DHCP server ... but I'm not sure !

Check that the directory is accessible by the user in.tftpd is running as and look at the log files to see if there is a special reason given.

----------

## ghuug

actually there is nothing wrong with having NFS as a SWAP. I'm using nearly the same setup as described here in some nodes of our openmosix cluster, and SWAP is exported through NFS.

----------

## GurliGebis

How do I have swap over nfs?

Do I just make a 256 mb swapfile on the nfs mounted root, and add the file to /etc/fstab?

----------

## ghuug

Yes, create file on NFS server.

e.g.:

```

dd if=/dev/zero of=/tmp/swap seek=100 count=1 bs=1M

mkswap /tmp/swap

chmod 0600 /tmp/swap

```

then on NFS client:

```

swapon -a /nfs/tmp/swap

```

----------

## Matje

I'm afraid it's not that simple. I also use swap over NFS, but you need to do this through a loopback device to fool your system, because swap over NFS normally isn't allowed...

----------

## HAsker

humm now i got my system to boot into grub but i wont boot because it restarts just when it's going to boot gentoo though the network...

----------

## ctford0

Anyone have a boot floppy working??  I was trying the suggestion that BradN posted, however after the client downloads the pxegrub it says that it is an invalid image....

Any ideas??

chris

----------

## karch

I had the same invalid image problem with pxegrub, so I downloaded a .lzdsk rom image from rom-o-matic.net and used nbgrub instead -- and everything was much happier...  you can find nbgrub in the same stage2 dir as pxegrub in the grub source tree.

-Mike

 *ctford0 wrote:*   

> Anyone have a boot floppy working??  I was trying the suggestion that BradN posted, however after the client downloads the pxegrub it says that it is an invalid image....
> 
> Any ideas??
> 
> chris

 

----------

## BradN

I'd only tested it with the network boot server that knoppix includes, but it seemed to work fine... maybe check how knoppix does it for clues?  (tip:  let knoppix be the only dhcp server it sees)

----------

## ctford0

Alright, I seem to have gotten past that barrier (thanks karch), however I think I just hit a brick wall.  I get the grub menu and I pick the diskless gentoo then it executes the grub commands.  After this I see: 

```

Booting command-list

root (nd)

  Filesystem type is tftp, using whole disk

kernel /node1/boot/bzImage ip=dhcp root=/dev/nfs nfsroot=192.168.1.1:/diskless/node1/

  [Linux-bzImage, setup=0xc00, size=0x13bb66]

```

Thats as far as I ever get.  Do I need to somehow put the bzImage in some other format or anything?

Thanks again for all the help....

Chris

----------

## HAsker

argh... i wounder why my comp just restarts when it's about to start gentoo :/

anyone have a clue?

----------

## Swiss.Mage

Is your kernel image booting or not ?

If not, it could be an image corruption prolem (check md5sum) or a kernel misconfiguration.

If yes, and if the kernel seems to boot but the computer reboot imediatly, check if the NFS root partition is mounted correctly. Check also your NFS server configuration. Your /etc/exports config file should be like this :

 *Quote:*   

> # /etc/exportfs: NFS file systems being exported. See exports(5).
> 
> /diskless/eta    eta(rw,sync,no_root_squash)

 

----------

## karch

It sounds like something is wrong with your tftp server if this is the last thing you see...  have you tested to see if you can do basic tftp file transfers?

-Mike

 *ctford0 wrote:*   

> 
> 
> ```
> 
> Booting command-list
> ...

 

----------

## ctford0

Could it be the tftp server?  I had to download the grub image before this using tftp right?  

Chris

----------

## karch

Yep... the boot floppy gets you the grub image, which then tells it to use tftp to boot.  What happens if you do a test transfer from your tftp server from your workstation (or from the server itself for that matter)?  It should go something like this:

```
[user@host]$ tftp 192.168.1.1

tftp> get /client/boot/bzImage

Received 994286 bytes in 0.9 seconds
```

 *ctford0 wrote:*   

> Could it be the tftp server?  I had to download the grub image before this using tftp right?  
> 
> Chris

 

----------

## GurliGebis

I got mine booting, but I constantly gets kernel panic   :Crying or Very sad: 

Something with:

```
/dev/ide/host0/bus0/target0/lun0:kernel BUG at sched.c:1141!

invalid operand: 0000
```

And then it dumps the kernel, and end with this:

```
<0>Kernel panic: Aiee, killing interrupt handler!
```

----------

## ctford0

karch:  I tried the tftp on the server itself and everything worked fine.

```

root@dummy ctford0 # tftp 192.168.1.1

tftp> get /node1/boot/bzImage

Received 1312042 bytes in 6.4 seconds

tftp> quit

root@dummy ctford0 #

```

any other suggestions?

chris

----------

## HAsker

ctford0:

I think we have the same problem because that's the only text i can see too... but i'am pxebooting directly.. it's very strange...

----------

## ctford0

HAsker how large is your kernel?  I noticed that mine is quite a bit larger than karch's.  Is there some limit maybe on how large the kernel can be?

Or maybe my network card just sucks...

Chris

----------

## HAsker

maybe it's something like that.. mine is  

```

hasker@hasker boot $ du -h  bzImage 

1.4M    bzImage

```

 :Smile: 

----------

## karch

I wouldn't think that the size of the kernel would matter, but then again, you never know.  I built a very basic kernel without too many bells and whistles - only 968K.  Besides the options, that SwissMage has in the HOWTO, I didn't add much more than the driver for my crappy ol' RTL8139 network adapter.

-Mike

----------

## ctford0

I'll try striping my down tonight and see if it helps.  

Chris

----------

## HAsker

ok.. post the answer because i dont have the time to try it :/

----------

## ctford0

Welp, I cut my kernel down by a couple hundred kb but still doesn't help.  Im beginning to thing that I just have a crappy net card.  Oh well, dont really know where to go from here.

Any suggestions??

chris

----------

## Swiss.Mage

After looking on the net, it seems there's a limit in the size of the file that can be transfered by TFTP but I cannot find a value ...

As example, my kernel image is 1.2 MB big.

----------

## ctford0

I think mine was just a little larger than that...

I might give it another try later tonight. 

Thanks

Chris

----------

## sebest

 *Swiss.Mage wrote:*   

> After looking on the net, it seems there's a limit in the size of the file that can be transfered by TFTP but I cannot find a value ...
> 
> As example, my kernel image is 1.2 MB big.

 

I think it's around 50 or 70Mo.

I used to download a debian rootfs to run many diskless webservers from ram (not root nfs), and i hited that limit. But i don't know if it depends on the tftp client or the tftp server. But with grub i manage to download an initrd as big as 50Mo.

----------

## Edweirdo

Mine seemed to work fine the first time, but everytime after that when I try to boot my client it dies at trying to mount my root filesystem R/W.

 *Quote:*   

> * Root filesystem could not be mounted R/W :( [ !! ]

 

My export file looks like:

```
/diskless/nc001 *(rw,sync,no_root_squash)
```

Any help would be welcome.

----------

## zuluh

If I do as the HOWTO says.. If I happen to have a disc on my discless client   :Laughing:  , can I reach it somehow? I wanna use this method to install gentoo on my laptop that hasn't got CD or discdrive, but a PXE enabled card..

----------

## Swiss.Mage

Of course you can !

As the system is booted, just mount your disk like explained in the standard installation doc.

But maybe it would be easier to use a distro like Knoppix to do that ... or also to use the Gentoo LiveCD remotly.

----------

## Zodiak

I have been using about the same setup to boot 16 Redhat machines over the network for six months now.  Figuring out how to do this on my own took me a long time, and I wish I had had this forum at that time.  My compliments to swiss mage.   :Very Happy: 

Anyway to answer a few questions.  PXE is what has the limit, the PXE standard limits the size of disk image that can be downloaded to I think 512k.  (Some cards do support 640k however.)   So if you are using GRUB, make sure that your GRUB image is less then that. (If you follow the instructions in this forum this will not be a problem.)

After GRUB is booted, you can download a boot image of any size.   I liked to use regular floppy boot disks, and just do a dd of the disk to build an image for the tftp server.   (Easy to test, and build)  Then I used syslinux memdisk to build a RAM disk and copy the boot image into the RAM disk and boot from there.  I never had a problem with the size of the image, and did try up to 2.2MB images without any problems, so I am not sure about that.

My grub.lst looked like the following

title Network Boot

  root(nd)

  kernel /vmlinux root=/dev/nfs nfsroot=10.10.1.1:/clients/%s ip=dhcp

title Diskless FreeDos Boot

  root(nd)

  kernel /memdisk

  initrd /dosfloppyimage.img

In getting Linux to work via a NFS server, you need to make sure that you compile the kernel with built in NFS-root support, and also build the network driver into the kernel, and remove all Network settings from other places. (i.e. initializing the network card during startup scripts gave me kernel panics since it was already initialized in the kernel.)  I also liked to build a RAM drive into my kernel for my use, but I don't know if this is strictly required.

After that configuring my NFS server hosts.allow, and iptables scripts was all that I needed to do to get it to boot.

As a side note, in answer to a few posts,  I have my tftp, dhcp, and nfs server on different machines (And even on different subnets), and it works just fine.   It is by no means a requirement to have them all on the same machine.  I also tried swaping via NFS, but it was so slow it was not even worth it. (I bet swapping to floppy would be faster, and from reading on the Internet most people have similar experience.)  Just buy some more RAM, it's cheap and lots easier to setup too.

----------

## zuluh

 *Edweirdo wrote:*   

> Mine seemed to work fine the first time, but everytime after that when I try to boot my client it dies at trying to mount my root filesystem R/W.
> 
>  *Quote:*   * Root filesystem could not be mounted R/W  [ !! ] 
> 
> My export file looks like:
> ...

 

I got that problem too, after adding a line in fstab it was fixed. 

that line should go something like this:

```

/dev/nfs / ext3 noatime 0 0

```

----------

## snafoo

i followed the instructions to the letter, but when my kernel boots, it gets to the 'freeing kernel memory' part and never switches over to the gentoo scripts. just sits there. 

the root mounts perfectly over NFS.

 :Confused: 

thanks.

----------

## vibidoo

Hello Mr Swiss.made

Thanks a lot for this topics     :Very Happy:  .

I followed your instruction and everything was going well .

Until I  reset the node client .

I sucess to boot normaly the node client only one time .

And because I didn't plug a keyboard I reset (hardware reset) the client .

And still now I have the following error on boot (node client) :

```

* Checking root filesystem

 fsck.ext2: Is a directory while trying to open /

/:

the superblock could not be read or does not describe a correct ext2

filesystem. If the device is valid and it really contaiins an ext2 

filesystem (and not swap or ufs or something else) , then the superblock is

corrupt , and you might try running e2fsck with an allternate superblock :

  e2fsck -b 8193 <device>

* Filesystem repaired

*Remounting root filesystem read/write ...

* Root filesystem could not be mounted R/W :(

(Control D for normal setup)

```

With Control D I finish to boot normaly but I have some errors 

Any idea ??

----------

## Swiss.Mage

I had the same problem yesterday   :Rolling Eyes: 

I did the following and now it's working again :

1. Check the /etc/fstab file of the diskless client. Mine looks like this :

```
# <fs>                                  <mountpoint>    <type>          <opts>                  <dump/pass>

 

192.168.1.20:/diskless/eta              /               nfs             defaults                1 1

/dev/cdroms/cdrom0                      /mnt/cdrom      iso9660         noauto,ro               0 0

 

none                                    /proc           proc            defaults                0 0

none                                    /dev/shm        tmpfs           defaults                0 0
```

2. If this is not enough, check your DHCP server config file and add the root-path option :

```
host eta {

        hardware ethernet ...;

        fixed-address 192.168.1.10;

        option option-150 "/eta/grub.lst";

        filename "/eta/pxegrub";

        option root-path "/dev/root";

}
```

3. If it does not work now, post a message again.

++

----------

## vibidoo

thanks for your quick answer .

the line on the fstab resolve the boot problem .

and I just had to emerge portmap and now I can boot completely.

But there are still an error on the boot . It seems to not affect the system but if you have any idea .

```

* Checking root filesystem 

fsck: fsck.nfs: not found

fsck: Error 2 while executing fsck.nfs for 192.68.0.1:diskless/eta

```

----------

## -=Beelzebub=-

 *vibidoo wrote:*   

> thanks for your quick answer .
> 
> the line on the fstab resolve the boot problem .
> 
> and I just had to emerge portmap and now I can boot completely.
> ...

 

You can get around that by putting "touch /fastboot" in your /etc/conf.d/local.start

----------

## Swiss.Mage

It's seems there're numerous problems concerning the boot of the client.

Here's a summary of all the problems I had and their solution :

Various errors during kernel boot (before the "Gentoo Linux" tag) :

Check your kernel configuration (make menuconfig in the kernel source tree)

System freez just after USB detection

Just reboot the client.

Unable to mount the filesystem read/write

This is an issue with the /etc/fstab. Add an entry for your NFS root filesystem and it should work

fsck.nfs not found

This is not vital and can be let as is ...

Various errors during system services start

I have no solution for this and I cannot find what gives this errors.  But I already have one answer : modprobe complaining about strange missing modules. This is a baselayout problem, don't worry about this.

But I have several others errors and some services are started one or three times ... If someone found a solution for this please post !

----------

## vibidoo

Now I would like to use 2 NIC cards on a client 

Could you help me to setup the Dhcpd.conf file's ?

I really have no idea on how to setup this .

Do I have to setup like this ?

```

host eta {

        hardware ethernet1 ...;

        fixed-address 192.168.1.10;

        option option-150 "/eta/grub.lst";

        filename "/eta/pxegrub";

        hardware ethernet2 ...;

        fixed-address 192.168.0.20;

}

```

 :Question: 

----------

## Swiss.Mage

Nop ...

You just have to create two different host {} blocs ...

For example :

host pc1 {

        hardware ethernet ...;

        fixed-address 192.168.1.10;

        option option-150 "/pc1/grub.lst";

        filename "/pc1/pxegrub";

}

host pc2 {

        hardware ethernet ...;

        fixed-address 192.168.1.20;

        option option-150 "/pc2/grub.lst";

        filename "/pc2/pxegrub";

}

++

----------

## mark

I have root over nfs working, I boot the kernel off of a cd.  My bios supports net booting.

Has anyone done a net boot in combination with a hardware router.  My SMC router currently provides ip addresses via DHCP.  I think I may regret it if I try to run two DHCP servers, what do people think?

I may need to figure out how to use static ip's with my reouter?

Mark

----------

## Swiss.Mage

Hi all,

I'm very proud of announcing that my diskless howto has been integrated in the official "Gentoo Linux alternative installation method HOWTO" available at :

http://www.gentoo.org/doc/en/altinstall.xml#doc_chap6

Thanks to the author that has me integrated as contributor   :Very Happy: 

CU soon (and nice holidays for who is in holidays   :Wink:   )

----------

## Galle

Hey all,

Why do I have to reboot using the LiveCD to install the stages for my diskless clients?

What is the difference from that and just chrooting on the running system?

Regards

Galle

----------

## Swiss.Mage

From the howto :

 *Quote:*   

>  I have tried many ways to install the base system of my diskless client. But the only one that really succeded is the following :
> 
> 2.1 Reboot the server on a Gentoo LiveCD 

 

As said I've try many ways to install the base system but I always had problems ...

Try to run bootstrap or to emerge a new version of GCC on a chrooted environment ... It could work for you but not for me !

----------

## lucasjb

Hi All,

In case anyone was wondering, somebody has already thought about all of these issues and created an easy to install and configure suite of software known as LTSP. It makes server setup very easy and client setup trivial.  I take it there's no ebuild for it at this stage, but since LTSP-4, a complete build environment (known as LBE) for the application has been included that would probably make creating an ebuild quite easy for somebody who is skilled and so inclined. I'm not sure how well it would integrate with Gentoo, but it's fantastic software and really worth checking out if you're interested in thin-client linux systems.

Lucas

----------

## heini

Hi,

nice tutorial, but it seems there is a little misunderstanding about X.

 *Swiss.Mage wrote:*   

> Something I can add about this setup is that it will need about 1 GB including the X server.
> 
> To connect to a remote X server, the only thing you have to do is to run "X -broadcast" on the client.

 

"X -broadcast" starts the X server on the client machine. The server machine serves dhcp, tftp and nfs only, but not X. You don't even need to start X on the server machine.

Bye...

	Dirk

----------

## carambola5

 *lbarbuto wrote:*   

> Hi All,
> 
> In case anyone was wondering, somebody has already thought about all of these issues and created an easy to install and configure suite of software known as LTSP. It makes server setup very easy and client setup trivial.  I take it there's no ebuild for it at this stage, but since LTSP-4, a complete build environment (known as LBE) for the application has been included that would probably make creating an ebuild quite easy for somebody who is skilled and so inclined. I'm not sure how well it would integrate with Gentoo, but it's fantastic software and really worth checking out if you're interested in thin-client linux systems.
> 
> Lucas

 

I've been trying to use LTSP recently but it simply isn't working for me.  Also, the goal of LTSP isn't quite the same as what the original howto in this thread was going for:

LTSP is radically slanted towards building X terminals.  Essentially, you use LTSP to get the client machine up and running, then run every single program on the server (including X).

This document explains how to get a client up and running such that, once running, it is a normal Linux workstation except that the filesystem is mounted via NFS.  This is ideal if your client has the necessary processing speed and RAM to get the job done (as in my case).

----------

## albancrequy

Hi all!

I have tested an diskless install and I have written a documentation:

http://alban.apinc.org/index.php?article=documentation/diskless

(80 pages)

but:

1/ the documentation is in french (sorry...)

2/ the distrib is not Gentoo but Mandrake. (sorry again)

The documentation contains the scripts to configure the diskless

and some config files (fstab, grub.lst, dhcpd.conf, etc.).

If you want a translation, do it yourself: the documentation is under

the GPL license.

You can write me (alban.crequy@laposte.net) if you want more infos.

darookee says:

> is there a way to do this without a bootable network card?

Yes. I use either a PXE card or a bootable CDROM. I use isolinux

to boot from a CDROM as a diskless. Ok, the CDROM contains the linux

kernel and isolinux.cfg (isolinux.cfg is to isolinux as grub.lst is to

grub). I use only 2MB of the CDROM. But all the system is on the NFS

server.

kermitjunior says:

> How does this differ from LTSP? Any advantages or disadvantages?

AFAIK, all programs on LTSP run on the server. With these diskless, all

programs run on the diskless.

ctford0 says:

> Why use tftp and not something like ssh?

Because the program inside the chip of the PXE card want tftp. Why not FTP?

Because FTP is too complex (more than TFTP) for the chip.

GurliGebis says:

> Well, Isn't there a way to be able to have swap on this system?

If you have a hard disk on your diskless, you can. I have tested it

and it works. In my tests, the hard disk was used only to swap.

ctford0 says:

> Ok, say I decide to setup 6 diskless clients. What directories can

> be shared between them all?

Well, I guess it depends of the distrib. I have only tested the mandrake.

Hope it is the same for gentoo.

> /usr

> /bin

> /sbin

> /opt

yes; no problem.

> /etc ?????

no, you can't. Because of:

/etc/fstab: the root mount point is not the same on 2 diskless.

/etc/modules: the modules to be loaded are not the same on 2 diskless.

/etc/cups.conf: this file contains the IP of the diskless...

/etc/sysconfig/network: (only on mandrake but I assume there is another

config file for gentoo) the file contains the static IP of the diskless.

No, it is not possible to use DHCP in this step (because of... hum if

someone want to know, ask me, i'm lazy).

etc.

If you want share /etc/passwd and some other, see below.

> /lib ???

yes, I share it.... but not /lib/dev-state! I don't know if this problem

exists in gentoo but /lib/dev-state can't be shared on mandrake.

So, I share /lib (with a NFS mount point in fstab) and I replace the directory

dev-state to a symbolic link to /dev-state. (symbolic link are resolved

by the diskless, not by the NFS server). So all /lib is shared but not

/lib/dev-state.

> I really dont want to have duplicates of things that I dont need

> to. I know that these I will have to have for each terminal.

>

> /dev

In my tests, I have not shared /dev. I wonder if /dev can be shared if

the diskless use devfs since the devfs mount point hide the files inside

the directory /dev...  not tested.

> /tmp

Can't be shared because of some programs which place locks in /tmp.

> /var

Can't be shared because of some programs which place locks in /var.

For example, NFS locks are in /var/lib/nfs in mandrake.

> Any others to add to this list?

On all my diskless, I share /common with a NFS mount point. And

/etc/passwd and some other are in /common.

On the NFS server side, the files passwd of each diskless are shared

with hard links. It's not possible to use symbolic links because of:

1/ symbolic links are resolved by the diskless (hard links are

"resolved" by the NFS server).

2/ The /common mount point is mounted after the system use passwd.

Some init scripts want to read passwd before /etc/fstab is read.

Of course, /home is shared.

Swiss.Mage says:

> If the users, groups, ... are the same on each machine, you could even try sharing /etc and /home ...

> I have to "confess" I have never try to share all of this system dirs ...

> So if you do it, post a message here to provide feedback for the other users (and me too).

In my tests, I don't use NFS: I used normal passwd, group and shadow.

They are shared with hard links on the server and it works !

Swiss.Mage says:

> I think the in.tftpd server must be on the same machine than the

> DHCP server ... but I'm not sure !

If your TFP server is not on the same machine than the DHCP server, the DHCP

server MUST say to the diskless where is the TFTP server. This is done via the

"next-server" option in the file dhcpd.conf. Example:

host eta {

  ...

  next-server 192.168.1.11; # TFTP server's IP

  filename "/eta/boot/pxegrub";

}

If the NFS server is on the same machine than the DHCP machine, you can

simplify the file grub.lst:

Put:

"kernel /eta/bzImage ip=dhcp root=/dev/nfs nfsroot=/diskless/eta"

Instead of:

"kernel /eta/bzImage ip=dhcp root=/dev/nfs nfsroot=192.168.1.20:/diskless/eta"

The linux kernel will assume that the NFS server has the same IP than the DHCP

server.

Moreover: if you put this line:

"kernel /eta/bzImage ip=dhcp root=/dev/nfs nfsroot=/diskless/%s"

the linux kernel will replace "%s" with the IP adress of the diskless given

by the DHCP server.

Alban Crequy

alban.crequy@apinc.org

http://alban.apinc.org/

----------

## Munter

lbarbuto: I got LTSP Running on gentoo with 12 Thin Clients, works great. Dont have PXE on the netcards though, so right now we are using floppys to boot the thin clients untill we get our hand on some real thin clients and not just old junk boxes  :Smile: 

----------

## vibidoo

I am using the dikless system with success since one month already 

But sometimes one of my system use to stop just before the init 

it stop on : 

```

Mounted devfs on /dev

Freeing unused kernel memory: 100k freed

```

And if I reboot I start normaly and boot to the login .

But now it always stop at this point , even I reboot many time the system it doesn't want to boot completely .

----------

## Ninjabucket

```
Mounted devfs on /dev

Freeing unused kernel memory: 100k freed 
```

I am having this exact same problem...anyone know a fix?

----------

## antik

 *Ninjabucket wrote:*   

> 
> 
> ```
> Mounted devfs on /dev
> 
> ...

 

Search in mailing list.

If you want some robust terminal, look here: http://www.thincan.com/

----------

## patsonrt1

Great How-To ...and thread  :Smile: 

Works great with suggestion by karch and I burnt my own prom to my Linksys card for a completely disk free P233  :Shocked: 

Now off to install Distcc  :Wink: 

Thanks!

----------

## syadnom

First, the initial setup of this system is just right.  But things I would change are the following:

Mount / via NFS, but mount all other folders via NBD, export image files from the server as NBD devices and then mount every other directory as NBD, except /usr/portage;/var, , because then you can share /usr/portage and /var, NBD only allows(safely) one machine to mount an image r/w.  

advantages:

1)NBD allows ANY file system to be used, NFS does not.  This gives you local journaling wither ext3 or reiserfs etc.  Much better

2)SWAP can be mounted via NBD and it works quite well.

3)NFS has its own error correction, which is redundant because TCP/IP has very good error correction, this causes overhead, NBD does not do internat corrention and allows TCP/IP's build in error correction handle this, this allows more efficient data transfers.

4)your system believes that the NBD is a local piece of hardware, you can fdisk to your hearts content on these mounted drive images..

(BTW: NBD = network block device)

5)each directory would be its own file on the server, allowing quick and easy backups.  Simply tar up the disk file and your set.

6) NBD is much faster that NFS, and has support for recovering a disconnected disk.  If your NFS / fails, you lock and must reboot, if your NBD fails, you can fix the network error and NBD will remount the disk and allow operation to continue.

also, 3com has a NBD-on-floppy product available that can completely mount a remote volume and pass boot on to a windows bootloader so you can boot windows via the network.  I have this and it works very well, I only need 1 copy of windows that iI can run anywhere on the network(only 1 instance at a time).

----------

## rogeroberholtzer

I have done everything to set up a diskless client from the current 1.4 live cd.  I have exported the directory containing the gentoo client is to use with rw permissions (as in the description at the top of this thread).  However, when the diskless client boots, it claims / is mounted read-only. A local 'mount' lists it as 'rw'. And the server's /proc/fs/nfs/exports file lists it as 'rw' by my client.

So, why the read-only errors on writes?

I have a suspicion of the 'checkroot' file in the rc directory. There was no mention in the diskless info on doing anything with this file. But it does seem to be odd in that it complains about the nfs mounted / not being an ext2 file system. Well, of course it isn't. Why the complaint?

I am so close I can taste it.

Any pointers?

----------

## rogeroberholtzer

I am trying to use swap via NFS. I seem to need it for a big compile. I only plan to use it as needed.

I made the swap file on the NFS server:

     dd if=/dev/zero of=swp  count=1000 bs=1M

Then, on the diskless client, I ran:

     mkswap /full/path/to/swp

No complaint. 'file /full/path/to/swp' reports that it is a modern swap file. It has the proper permissions (0600) and belongs to root. However, when I run:

     swapon /full/path/to/swp

I get the following error:

     swapon: /full/path/to/swp: Invalid argument

The file exists. If I use some name that does not exist, I get an appropriate error.

The NFS mount is like this (listed by mount):

     acme:/full/path on /full/path type nfs (rw,noatime,bg,soft, 

             rsize=8192,wsize=8192,addr=10.1.6.131)

The swap file is:

     -rw-------    1 root     root     1048576000 Nov 18 13:15 swp

Is there some NFS mount option I have overlooked?

(The diskless client is a VME-based PC to which adding additional RAM is not an option).

----------

## patsonrt1

Just a guess Rogeroberholtzer, but did you add "no_root_squash" in the servers exports file?

----------

## darkjedi521

I have the system booting with the kernel on a floppy.  I followed the howto almost exactly in setup (subsituted floppy for PXE/TFTP).  The kernel boots fine and then gentoo starts booting.  It gets to the point where it attempts to mount local filesystems, and then I get 5-6 screens worth of errors regarding /var/lib before being dumped at a login prompt.  None of my other rc scripts run.  /usr doesn't mount, and pam returns fatal errors after a username is entered.  Setting console one in inittab to be /bin/bash instead of agetty let me poke at the running system.  doing the following:

```
source /etc/profile

env-update

source /etc/profile

rc
```

ran the rest of the rc-scripts, but pam was still having fatal errors.  If anyone knows where to poke at, it would be much appreciated.  Thanks in advance.

----------

## hms

I have succesfully managed to set up a diskless system but when I start xine I get stalls in the movie. I'm guessing this is because the xfree server is run over nfs. Is there any way to improve xfree´s performance for a diskless system. 

I don't have a powerfull enough server to run X there and with my experience a network isn't fast enough to export a fullscreen movie in real time over a network either so this isn't an option.

I have the same system running on a local disk and there xine plays movies smothless. During the tests with the diskless system the actual movie-file being played is on a local disk. My network is a 100Mbit so it shouldn't be the bottleneck.

I read the post about NBD and wonder if that could help me get the performance needed to run xfree over nfs.

/Magnus

----------

## GlennM

EDIT: Whoops, fixed it. Turns out you need to remove the local-only line from /etc/xinetd.conf. Doh!

I was wondering if someone could give me some help with TFTP...it seems that my test workstation can't get the kernel image. When the workstation starts, I get:

```
Searching for server (DHCP)...

..Me: 192.168.1.110, Server 192.168.1.2, Gateway 192.168.1.1

Loading 192.168.1.2:/lts/vmlinuz-2.4.24-ltsp-1......
```

That looks right, except that it sits there forever, with no network traffic.

I double-checked to make sure that the kernel is in /tftpboot/lts, and it is, and named correctly. The server and gateway are correct. I think I've got dhcp, xinet, and tftp configured correctly. The relevant files are as follows:

/etc/dhcp/dhcpd.conf:

```

# Some general options

default-lease-time            21600;

max-lease-time                21600;

use-host-decl-names           on;

ddns-update-style             ad-hoc;

# Bootp options

allow booting;

allow bootp;

# Network Options

option subnet-mask            255.255.255.0;

option broadcast-address      192.168.1.255;

option routers                192.168.1.1;

option domain-name-servers    192.168.1.2;

option log-servers            192.168.1.2;

option domain-name            "domain-removed.net";

# LTSP Path Options

option root-path              "192.168.1.2:/opt/ltsp/i386";

filename                      "/lts/vmlinuz-2.4.24-ltsp-1";

shared-network WORKSTATIONS {

  subnet 192.168.1.0 netmask 255.255.255.0 {

    # Distribute dynamic IPs to the workstations

    range dynamic-bootp 192.168.1.110 192.168.1.111;

    # Workstation specific configuration for PXE booting

    #host ws001 {

    #  hardware ethernet     00:E0:06:E8:00:84;

    #  fixed-address         192.168.0.1;

    #}

  }

}
```

/etc/xinetd.d/tftp:

```

service tftp 

{ 

   disable = no 

   socket_type = dgram 

   protocol = udp 

   wait = yes 

   user = root 

   server = /usr/sbin/in.tftpd 

}
```

/etc/conf.d/in.tftpd:

```

# Config file for /etc/init.d/in.tftpd

# Remove the -l if you use [x]inetd

INTFTPD_PATH="/tftpboot"

INTFTPD_OPTS="${INTFTPD_PATH}"

```

I really don't have any more ideas. It just looks like tftpd isn't sending the kernel to the workstation. I've tried changing the "filename" line in dhcpd.conf to "/tftpboot/lts/vmlinuz-2.4.24-ltsp-1", but that doesn't fix the problem. Are there any other files I can adjust?

----------

## akmolloy

I've currently got a mythfrontend machine that I've spent quite a bit of time configuring, and it works well. I want to convert the exisiting installation (settings, programs, etc.) to a diskless install.

I've got a server setup and running, and sucessfully booted into a small kernel.

Any tips on how to do this? Could it be as simple as copying over the directories on the exisiting machine to the master server, and then modifying a few files like fstab?

----------

## Ben2040

Hi

When compiling the grub image, both 0.93 and 0.92 fail with the following. I have tried without CFLAGS too....

```

gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../stage1  -Wall -Wmissing-prototypes -Wunused -Wshadow -Wpointer-arith -malign-jumps=1 -malign-loops=1 -malign-functions=1 -Wundef -Os -fno-builtin -nostdinc -I../netboot -DSUPPORT_NETBOOT=1 -DSUPPORT_SERIAL=1 -DSUPPORT_HERCULES=1 -DFSYS_EXT2FS=1 -DFSYS_FAT=1 -DFSYS_FFS=1 -DFSYS_MINIX=1 -DFSYS_REISERFS=1 -DFSYS_VSTAFS=1 -DFSYS_JFS=1 -DFSYS_XFS=1 -DUSE_MD5_PASSWORDS=1 -DFSYS_TFTP=1 -g -c -o pre_stage2_exec-fsys_reiserfs.o `test -f fsys_reiserfs.c || echo './'`fsys_reiserfs.c

cc1: warning: -malign-loops is obsolete, use -falign-loops

cc1: warning: -malign-jumps is obsolete, use -falign-jumps

cc1: warning: -malign-functions is obsolete, use -falign-functions

fsys_reiserfs.c:115: error: long, short, signed or unsigned used invalidly for `j_mount_id'

fsys_reiserfs.c: In function `next_key':

fsys_reiserfs.c:761: warning: operation on `depth' may be undefined

make[2]: *** [pre_stage2_exec-fsys_reiserfs.o] Error 1

make[2]: Leaving directory `/usr/portage/distfiles/grub-0.92/stage2'

make[1]: *** [all-recursive] Error 1

make[1]: Leaving directory `/usr/portage/distfiles/grub-0.92'

make: *** [all] Error 2

```

Also note that I have done this before on a 32bit Athlon, and it worked fine, now I'm in [32 bit mode] gentoo on a 64 bit Athlon 64, if that makes a difference.

Ben

[EDIT]

Just adding that I'm configuring with:

```

./configure --enable-diskless --enable-rtl8139 --enable-3c90x

```

----------

## hygge

thx for the great guide!

what do you think of adding options to the client's /etc/fstab?

maybe nfsvers=3,hard,rsize=8192 and wsize=8192?

will this speed something up or is it just uneccesary?

----------

## markedmann

Hey all:

I'm having a lot of trouble with configuring dhcpd.conf... Could someone please post a (working) sample for me to follow?  I've used the one in the howto, but it appears to be simply a snippet.  

Thanks in advance,

-Mark

----------

## tiny

This worked for me.

```

option option-150 code 150 = text ;

ddns-update-style ad-hoc;

subnet 192.168.254.0 netmask 255.255.255.0 { }

group {

        host nova {

        hardware ethernet 00:08:9b:1e:69:7a;

        fixed-address 192.168.254.57;

        option option-150 "/nova/boot/grub.lst";

        filename "/nova/boot/pxegrub";

        }

}

```

----------

## Donpasquale

Hi.

fisrt of all: thanks for the great howto. 

But i still got some problems with that. If i boot up my client it all works fine till the point where the kernel wants to start the init it just stops and stays there doing nothing. If i mount the nfs share in another pc which runs from hd it does it in not even a sec. 

I tried every option in the kernel i think. Please help me fix that problem. Im going crazy bout this  :Smile: 

Here is my config:

dhcpd.conf:

```
host eta {

        hardware ethernet ::::;

        fixed-address 192.168.0.160;

        option option-150 "/eta/grub.lst";

        filename "/eta/boot/pxegrub";

}

```

and my grub.lst:

```
default 0

timeout 30

title=Gentoo Networkboot

root (nd)

kernel /eta/bzImage ip=dhcp root=/dev/nfs nfsroot=192.168.0.211:/raid/diskless/eta
```

Hopefully someone can help me. It will be an installation server to do automated gentoo installations as i finish it.

Thx

Donpasquale

----------

## dmitrio

I have copied this HOWTO, with permission of Alexandre "Swiss.Mage" Georges, to gentoo-wiki.com 

http://gentoo-wiki.com/HOWTO_Gentoo_Diskless_Install

If you see anything that should be added or changed, feel free to do so. 

Thank you for a great HOWTO.

----------

## lschiere

Can anyone give me any pointers on how this diskless howto would need to be modified to be able to pxe boot (for instance) amd64 clients with an x86 server?

----------

## someguy

Aug 18 06:22:06 horus dhcpd: DHCPDISCOVER from 00:e0:c5:68:22:a7 via eth0

Aug 18 06:22:06 horus dhcpd: DHCPOFFER on 192.168.2.15 to 00:e0:c5:68:22:a7 via eth0

Aug 18 06:22:08 horus dhcpd: Dynamic and static leases present for 192.168.2.15.

Aug 18 06:22:08 horus dhcpd: Remove host declaration ibis or remove 192.168.2.15

Aug 18 06:22:08 horus dhcpd: from the dynamic address pool for 192.168.2.0/24

Aug 18 06:22:08 horus dhcpd: DHCPREQUEST for 192.168.2.15 (192.168.2.4) from 00:e0:c5:68:22:a7 via eth0

Aug 18 06:22:08 horus dhcpd: DHCPACK on 192.168.2.15 to 00:e0:c5:68:22:a7 via eth0

Aug 18 06:22:08 horus in.tftpd[16988]: RRQ from 192.168.2.15 filename /mnt/diskless/ibis/boot/pxegrub

Aug 18 06:22:08 horus in.tftpd[16988]: sending NAK (1, File not found) to 192.168.2.15

Aug 18 06:22:08 horus in.tftpd[16989]: RRQ from 192.168.2.15 filename /mnt/diskless/ibis/boot/pxegrub

Aug 18 06:22:08 horus in.tftpd[16989]: sending NAK (1, File not found) to 192.168.2.15

im getting this too any help would be mucho appreciado

----------

## Skywacker

the ftp program is getting "File not found" for  filename /mnt/diskless/ibis/boot/pxegrub 

look in /etc/conf.d/in.tfpd for your initial ftp directory. chances are your pxegrub file and kernel are not located in /diskless/mnt/diskless/ibis....

basically you need to edit your grub.lst until you have the correct location for your kernel, etc

I had to change my kernel to be -/b8/boot/kernel-3.2.6-gentoo-r8 because of my installation scripts.

----------

## corn

It wasnt mentioned here but the init scripts seem to look for a variable called CDBOOT by setting this in /etc/rc.conf the startup scripts will skip stuff like fsck etc... Im not sure what it should be set to but all the scripts just check if it has a length.

I was wondering about shared resources such as a sysklogd. I heve them turned off right now. I was curious because I will be running 10-20 machines off my server.

----------

## Skywacker

I built 1 diskless node thanks to these instructions, but I'm wondering the best way to roll this out to my other 30+ machines. I tried just copying /diskless/b8 to/diskless/b9 and modifying fstab, hostname, and a couple other files. This gives me a booting, mostly working Gentoo system with one catch: I can't lamboot (lam-mpi) my cluster if I include b9. 

b2 is my master

b8 is my first working diskless node

b9 is ia copy of the b8 dir with a few config files changed.

I can lamboot b2 and b8, but not b9. 

Any ideas?

All other applications seem to work: ganglia, nfs, ntp, ldap authentication.

Thanks,

Skywacker

----------

## riksta

just a quick tip to consolidate the howto commands a little

# mkdir /diskless

# mkdir /diskless/eta

# mkdir /diskless/eta/boot

can all be done with 

# mkdir -p /diskless/eta/boot

 :Smile: 

----------

## Skywacker

No offense to this thread as it was very helpful, but once I had a working 8 node diskless cluster I ran a comparison test versus a diskfull 8 node cluster and the diskless cluster was about 400% slower. They were not identical clusters, and the master node on the diskfull cluster was faster, but I ended up reverting back to a diskfull cluster. Just something to think about if anyone out there is in the planning stage still.

-Skywacker

----------

## kakakoka

The dhcpd server i am using (the freebsd dhcpd server on my lan) wont start with this:

option option-150 code 150 = text ; 

In the header of the dhcpd.conf.

Whats could I be doing wrong?

----------

## rkrenzis

option option-150 code 150 = text ;

Remove the space between the word "text" and the ";" symbol.

option option-150 code 150 = text;

----------

## rkrenzis

Everything works with the nfs-boot.  emerge sync and adding packages is a breeze.  *BUT* when I'm a regular user (non-root or uid 0), I can't resolve hostnames.  Any thoughts?

TIA.

----------

## kakakoka

Removing the space seems to make no difference.

```
 /usr/sbin/dhcpd xl0

Internet Software Consortium DHCP Server 2.0pl5-OpenBSD

Copyright 1995, 1996, 1997, 1998, 1999 The Internet Software Consortium.

All rights reserved.

Please contribute if you find this software useful.

For info, please visit http://www.isc.org/dhcp-contrib.html

/etc/dhcpd.conf line 19: expecting string or hexadecimal data.

option option-150 code

       ^

Configuration file errors encountered -- exiting

exiting.

```

----------

## dsf

I've updated the wiki for the HOWTO by simplifying several manual steps because of an improvement in the grub ebuild.

```
As of grub 0.94-r1 (and later), it now builds pxegrub properly. It enables diskless booting, and compiles in all supported GRUB network drivers.

# cp /usr/lib/grub/pxegrub /diskless/eta/boot/pxegrub
```

----------

## cable_guy

I'm just posting thies here in hopes that other people who have the same problem can find it. There isn't currently anything posted about it on the net.

I originally set up my diskless nodes basically the same as this how to, using nfs to serve all the systems, and pxe to boot. I upgraded my cards to dlink dge-530t gigabit cards, and they don't support pxe boot. Then to my nasty surprise, etherboot doesn't support them either. The cure was to dissect a gentoo livecd installation cd.

I copied the iso image into /mnt/tmp, then did:

```

mkdir /mnt/loop0

mount /mnt/tmp/isoname.iso /mnt/loop0 -o loop

mkdir /mnt/iso

cd /mnt/loop0

find | cpio -pdm /mnt/iso

cd /mnt

umount /mnt/loop0 

rm -r /mnt/loop0

```

This gave me a folder with the exact cd contents. Take everything out of the root cd folder, except the isolinux folder and livecdsquash file. In the lisolinux folder delete everything except the isolinux.bin and isolinux.cfg. Then take your kernel for your diskless node and copy it to the isolinux folder.  Edit the isolinux.cfg and take all the extra junk out, mine looks like:

```

default gentoo

timeout 5

label gentoo

   kernel omkernel

   append ip=dhcp root=/dev/nfs nfsroot=192.168.0.2:/diskless/192.168.0.201 looptype=squashfs loop=/livecd.squashfs

```

where omkernel is the name of my kernel file, and nfsroot="server ip":"root folder on server".

Then:

```

cd /mnt/iso

mkisofs -no-emul-boot -boot-load-size 4 -boot-info-table -r -b isolinux/isolinux.bin -c isolinux/boot.cat -o /mnt/newgentoo.iso . 

```

And you now have a bootable cd image that will boot your kernel, and load everything up the way it used to. Just don't forget to change the MAC address in your server config files to the MAC address of the new card first.

----------

## skarecrow

I have a network which uses Windows 2000 DHCP servers. I setup a reservation for the NIC in my diskless linux box and configured DHCP options 66 and 67 for the pxe boot info. When booting, the client grabs the appropriate IP address, and leaves me at a grub prompt. When I use a linux DHCP server, the diskless workstation boots perfectly. What is the difference here? It seems like when the system gets the IP from the windows DHCP server that it isnt loading grub.lst. Any ideas?

----------

## caar

I've been following this trak. I also set up a few diskless client configurations. I set them up more or less like it is described here. It all works. So far so good - till it comes to emerge! 

Does anybody know a way to run emerge separately for the root (/) and the /usr part of a package?

I used to work quite intensive with Solaris about 10 years ago. The package system on Solaris destinguishes between the part which goes to root (normally /bin and /sbin)  and the part that goes to /usr (/usr/bin, /usr/sbin, /usr/share, ...). Well, Solaris keeps architecture dependencies in  root (/)... 

Why I bother about this: 

- diskless clients do not have to be of the same CPU architecture as the master or host;

  different CFLAGS etc in /etc/make.conf could be used for clients.

- the part of a software that goes to /usr needs only be compiled once (the part that's common

  to all the clients

- the part that goes to root (/) might be different on each client (e.g. for /etc)

A possible solution might be, if each client has only the client specific files in its file tree (/diskless/<hostname>) and there is a default file tree (/diskless/default). On the client a merge is needed of the client specific file tree and the default tree. This way most packages have to be applied only once and all clients profit from it. Unfortunately, I don't know of such a file system...

----------

## Camisa

Does anybody else notice that their diskless client is mostly idle?  I run emerge system or emerge world and observe the idle time at mostly 70%...

I also have determined when the kernel mounts the NFS / it mounts it NFSVERS=2... even though I have specified it to use version 3 in both the kernel config and in the fstab.

This is a new problem for me...   I had a diskless jukebox running for about 9 months.. and I never had the compile time slowdown like I'm experiencing now..  Back then it was the 2.4. kernel and the system was built from stage 1.

Well this time, I'm still doing stage 1; but I'm using the 2.6.11.11 kernel.

Is anyone else aware of their NFS root using version 2?  Do an "nfsstat" check on your server to see if the traffic is all from version 2 or version 3.

Final question:  Should I be doing the NBD thing instead of NFS...?

----------

## b0fh

I installed gentoo successfully for a diskless node. But I have a problem while shutting down the machine: The init-skript nfsmount unmounts my root partition, so the rest of shutdown fails, for example shutting down portmap afterwards (I need nfsmount because I mount other filesystems over nfs, too). Any ideas how to solve this?

----------

## blackphiber

cp /usr/lib/grub/pxegrub /diskless/eta/boot/pxegrub

cp: cannot stat `/usr/lib/grub/pxegrub': No such file or directory

I have grub 0.96, where else would pxegrub be? or am I doing something wrong?  thanks.

----------

## b0fh

Mine was at /boot/grub/pxegrub

----------

## blackphiber

ls: /boot/grub/pxegrub: No such file or directory

using locate pxegrub turns up nothing.  I have grub 0.96-r1.  thanks again.

----------

## dapsaille

Hi all 

 First Thanks for this howto ...

 but i've a little problem at boot it give me 

RAMDISK: Compressed image found at block 0

VFS : Mounted root (ext2 filesystem).

/linuxrc has to be run as the init process as the one

with a PID of 1. Try adding init="/linuxrc" to the

kernel command line or running "exec /linuxrc"

VFS: Cannot open root device "nfs" or unknow-block(0,255)

Please append a correct "root=" boot option

Kernel panic - not syncing: VFS: Unable to mount root fs on unknow-block(0,255)

 Do you have any clue ? 

 I can post config files if necessary but i don't want to polluate this thread :p

----------

## Slammer

I succesfully finished a diskless gentoo installation, following this How-To. Now the client runs normally. The only problem is during shutdown I receive multiple error messages while the services are closed. (write not permitted in /var/lib/init.d directory). I am sure that the /var directory is mounted as r/w.

Additionally, I want to build a second diskless client using as more as possible common directories with the first client. I understand that is very easy to create a second client in a different directory, but how is possible to share common parts? NFS can not share symlinks (for example /diskless/client2/usr  cannot symlinked to /diskless/client1/usr).

----------

## wimalopaan

For all who are interested setting up multiple Gentoo diskless clients wie have composed a little howto (currently only in german). The installation procesdure is similar to what is decribed in this thread. Bit it is much more flexible in that you have only to maintain a single chroot for all clients. And you can make the nfs-export ro ! 

The location: 

http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/diskless.pdf

We are working further on this topic and we now have an installtion with no interference to the normal gentoo baselayout (in contrast to the above how-to). Currently there is no actual documentation, but this is coming soon. The system can be used as a thin-client (remote login via XDMCP or RDP) or as a diskless-client (local login) or as a virtual-diskless-client running vmware with (potential) local cached images.

Of special interest in this topic ist also pam_cifs, which can mount and umount windows CIFS shares on the fly. 

http://mozart.informatik.fh-kl.de/download/Software/pam_cifs/pam_cifs.html

pam_cifs is similar to pam_mount, but much more simpler and has a different architecture, which makes it a litlle bit more flexible. Currently there is no ebuild, but on Gentoo it compiles straight forward.

If you are interested don't hesitate to contact me.

I will post a follow up if the documentation is ready.

--

Wilhelm

----------

## krani1

so i've followed this article, but my dhcpc server doesn't wake up, and I've got this in logs :S

```
Aug 28 20:13:54 server dhcpd: /etc/dhcp/dhcpd.conf line 33: unknown option dhcp.option-150

Aug 28 20:13:54 server dhcpd:            option option-150 "/diskless/eta/boot/grub.lst"

Aug 28 20:13:54 server dhcpd:                ^
```

any help?

----------

## krani1

* bump *

----------

## rsala

I'm having the same problem as Slammer.  I assume that the nfsroot is getting umounted as part of the shutdown and then the system no longer has access to /var for some last minute bookkeeping/cleanup.  I'm just not sure why no one has reported this problem before.  Perhaps it has to do with some of the changes to the base layout over the years.

Has anyone come across an elegant solution to this problem?

----------

## wimalopaan

I have made a new version of the diskless gentoo client guide (see below). This installation uses unionfs to leave the normal gentoo-chroot-installation as is. Only minimal modifications are required and the setup is independent of the number of diskless clients or network setup.

http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/diskless2.pdf

Of special interest in this topic ist also pam_cifs, which can mount and umount windows CIFS shares on the fly. 

http://mozart.informatik.fh-kl.de/download/Software/pam_cifs/pam_cifs.html

pam_cifs is similar to pam_mount, but much more simpler and has a different architecture, which makes it a litlle bit more flexible. Currently there is no ebuild, but on Gentoo it compiles straight forward. 

If you need to print to authenticated printers via SMB (Windows or Samba) pam_cups comes in handy:

http://mozart.informatik.fh-kl.de/download/Software/pam_cups/pam_cups.html

 -- 

 Wilhelm

----------

## corn

Thanks for this great HOWTO. I used it at work to set up a number cruncher farm. Currently the farm has 88 nodes up and running with 150 more coming online soon. 

   Some caveats were having each worker having its own / filesystem. Mirroring /diskless/192.168.1.1 to all 255 /'s in the subnet was a pain in the butt and now I am afraid to do an update as the root may end up not in sync with the mounted /usr partitions. Also to get each worker to mount its own slash I had to change the pxe default config file to not specify the root partition to mount.

    Another thing I had to do was to make the system self configuring. The worker nodes will be rotated out and in and I did not want to edit the dhcp.conf file for every change. Not being a dhcp pro I ended up making a perl daemon that monitors syslog. When I see DHCPDISCOVER's that are not replied to I auto add a entry to dhcp. Once all the IP's in the subnet are full I ping old hosts and when I find one that is down I rewrite the entry for that host and restart dhcp. 

   Thanks again. This information provided a very valuable and functional solution to a large problem I was faced with.

----------

## wimalopaan

If you have a look to my gentoo diskless description

http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/diskless2.pdf

you'll find a methode using translucent filesystems (unionfs) where you have only ONE root-fs fpr ALL clients. Theres is no limiting factor. You only have to update ONE client-root for ALL your clients.

Regards,

Wilhelm

----------

## corn

Thats excellent Wilhelm I look forward to learning more about your solution but unfortunately my german does not allow me to extract any information from your documentation.

----------

## wimalopaan

I've made an english version of the howto. 

http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/diskless2_en.pdf

----------

## krani1

sorry it says 404 not found  :Sad: 

----------

## wimalopaan

should be done now

----------

## jamapii

Diskless booting seems to be broken with the latest kernels/stages or with some NFS setups.

The following information might be necessary, from this thread https://forums.gentoo.org/viewtopic-t-240592-highlight-root+dev+nfs+nfsroot.html

 *rafo wrote:*   

> A couple of days ago I wrote
> 
>  *Quote:*   I have tried to change it to use udev, but I just can't get it to work. The diskless node semi-dies at the point where /sbin/rc calls /sbin/depscan.sh (it still responds to ping), and it does not seem to be entirely reproducible: Sometimes I see the green message "Caching service dependencies", sometimes not. 
> 
> I now have it all sorted it out, I think. I have submitted a number of bug reports on the "Diskless Nodes with Gentoo" paper, September 16, 2005  (http://www.gentoo.org/doc/en/diskless-howto.xml). The Bugzilla numbers are 106525, 107258, 107260, 107262, 107263, 107264, 107271.

 

----------

## Albert_Alligator

 *wimalopaan wrote:*   

> I've made an english version of the howto. 
> 
> http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/diskless2_en.pdf

 

Hi, I like the install guide, but the fonts you use in the PDF make it hard to read the scripts you've written. Can't copy them, and much of what I think are spaces look like brackets that are lying on their backs. Also, when I copied the scripts, all the letters pasted as individual words.

Stateless.sh

```

#!/bin/bash

MODPROBE=/sbin/modprobe

IFCONFIG=/sbin/ifconfig

ahostname(){

    MYHOST="$1'$IFCONFIG eth0 | awk '/HWaddr/{print$5}' | tr -d': '' "

    echo "STATELESS: Seting Host name to $MYHOST"

    echo "HOSTNAME=\"$MYHOST\" " > /etc/conf.d/hostname

    /bin/hostname "$MYHOST"

}

getparams(){

     local cmdline=$(dmesg | grep '^Kernel command line ' | sed 's/^Kernel command line://g')

     for pp in $cmdline ; do

        echo $pp | grep '^stateless=' >/dev/null 2>&1

        if [ $? -eq 0 ]; then

                 echo $pp | sed 's/stateless=//g'

                 return 0

        fi

    done

    echo ""

    return 1

}

isset () {

          for p in $ ( getparams | tr ',' ' ') ; do

              if [ "$p" == "$1" ]; then

                          return 0

              fi

          done

          return 1

}

aunionfs () {

         isset unionfs

                if [ $? -eq 0 ]; then

                   echo "STATELESS : Loading module unionfs . . . "

                   $MODPROBE unionfs

                   while[ "$1" != "" ]; do

                           echo "STATELESS : Mounting tmpfs $1 . . . "

                           mount -n -t tmpfs -o defaults none /mnt/unionfs/$1

                           echo "STATELESS:Mounting$1 unionfs . . . "

                           mount -n -t unionfs -o dirs=/mnt/unionfs/$1=rw:/$1=ro none/$1

                           shift

                   done

                else

                   echo "STATELESS: Not using unionfs as requested . . . "

                fi

}

aunionfs etc var

ahostname stateless

exec /sbin/init

```

You need to look at this and see if it is correct, I'm not much of a shell scripter my self, at least not yet, and so I'm not sure if that is correct.

This is what they look like without me fixing them.

```

# !/ sbin / r u n s c r i p t

g e t c m d () {

        for i in  cat / proc / cmdline ; do

                if [  echo $i | grep xauto  ]; then

                        echo  $ i 

                        return

                fi

        done

        echo  x a u t o = a u t o 

}

d e p e n d () {

        before xdm

}

start () {

        # define some variables

        X O U T F I L E =/ root / xorg . conf . new

        OUTPUT =/ etc / X11 / xorg . conf

        F I R S T P A R T =/ etc / X11 / a u t o c o n f / part1 . cfg

        L A S T P A R T =/ etc / X11 / a u t o c o n f / part2 . cfg

        # lets have a look what we should do

        if [  cat / proc / c md l in e | grep xauto  ]; then

                ebegin  S t a r t i n g X a u t o c o n f i g u r a t i o n . . . 

                case  g e t c m d | tr  =    | awk  { p r i n t $2 }   in

                        1)

                        cp / etc / X11 / a u t o c o n f / xorg .1. conf $ O U T P U T

                        ;;

                        auto )

                        # Run X - c o n f i g u r e to get c o n f i g u r a t i o n

                  X - configure

                  # D e t e r m i n i n g the start of the i n t e r e s i n g s e c t i o n

                  C O U N T E R =  cat $ X O U T F I L E | wc | awk  { p r i n t $1 }  

                  u n t i l [  cat $ X O U T F I L E | tail - n $ C O U N T E R | head - n 1 | awk  { p r i n t $1 $2 }  | grep  S e c t i o n 

                             D e v i c e    ]; do

                          let COUNTER -=1

                  done

                  STARTLINES = $COUNTER

                  # D e t e r m i n i n g the end of the i n t e r e s t i n g s e c t i o n

                  C OU NT E R =1

                  u n t i l [  cat $ X O U T F I L E | tail - n $ S T A R T L I N E S | head - n $ C O U N T E R | tail - n 1 | awk  { p r i n t $1

                             }  | grep EndSection  ]; do

                          let COUNTER +=1

                  done

                  ENDLINES = $COUNTER

                  # Write new configuration

                  cat $FIRSTPART > $OUTPUT

                  cat $XOUTFILE | tail - n $STARTLINES | head - n $ENDLINES > > $OUTPUT

                  cat $LASTPART > > $OUTPUT

                  ;;

             esac

             eend 0

      else

             ebegin  S k i p p i n g X a u t o k o n f i g u r a t i o n . . . 

             eend 0

      fi

}

stop () {

      ebegin

      eend 0

}

r es ta r t () {

      ebegin

      eend 0

}

```

That's what the other one looked like before I spent some time cleaning it up...and I haven't even tried it to see if it works. (crossing my fingers).

Anyhoo, I'll be cleaning up the above one as well and post my cleaned version.....of course, you'll have make sure its correct as you are the author.

Thanks again for the tutorial, yours is the best one yet, makes use of unionfs which I had been meaning to do.

Cheers,

Al

----------

## wimalopaan

I've made the two scripts available as source:

http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/

----------

## jamapii

The diskless booting problem is with NFS wsize.

That mount option can't be changed by a remount, and it is (might be) 4096 by default. The kernel documentation is wrong (/usr/src/linux/Documentation/nfsroot.txt). If this doesn't work, it must be changed in pxelinux.

----------

## kucrut

First off, thanks for the original howto, as well as pdf howto from wimalopaan. one thing left tho, does the client use server's resources or does it use its own?

----------

## wimalopaan

If you have only one client-configuration, all is well with my description in 

http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/diskless2_en.pdf

But if have several groups of clients with slightly different configurations it is wasteful, if you use a different full client-root. It would be much easier if we could use filesystem unification. This can be done by unionfs 

http://www.fsl.cs.sunysb.edu/project-unionfs.html

but there are some issues on using unionfs over nfs on unionfs. I wrote a patch for the actual unionfs-cvs-snapshot to overcome this limitation. 

The patch disables the lookup of whiteouts in the right-most branch and 

disables the lookup of whiteouts in ro-branches before the check if the 

whiteout must be renamed. You can't use unionfs directly as an underlying fs 

for unionfs, but you can use a nfs-exported unionfs-filesystem as a 

ro-branch. Now, unionfs sees a nfs-branch, which is allowed, and the unneeded 

lookups aren't done anymore. This was a major drawback, because the 

whiteout-lookup in the ro-branches causes the underlying unionfs to return an 

'Operation not permitted'. With the patch, this doesn't happen anymore.

This problem is typical in the case of diskless-clients with different 

configurations. In this scenario you have a base client-root-directory 

(say /tftproot/client_A) and some derived client-roots 

(e.g. /tftproot/client_B), which are something like 

/tftproot/client_B dirs=/tftproot/client_Bdiff=rw:/tftproot/client_A=ro. 

These /tftproot/client_* can be ro-exported via nfs. On the client-side you 

can use them as a rightmost ro-branch for the client-root and the other 

branches tmpfs, e.g.

 

mount -t tmpfs none /tmpfs/etc

mount -t unionfs none /etc -o dirs=/tmpfs/etc=rw:/etc=ro

We found this very useful in our environment.

----------

## wimalopaan

Sorry, but the link to the patch wasn't included in my previous posting:

http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/gdxs.html

----------

## kyron

First, obvious thanks go to wimalopaan for his document.

I jsut wanted to tell you all that I am trying to sum up a wiki version of this document. I wrote an e-mail to wimalopaan for explicit permission to do so. As I stated in my e-mail, I am only using my personnal wiki as a "sandbox" before posting a more complete version of the document on the (non-) official Gentoo Wiki  :Wink: 

My motives for wikifying the document:

1- easy update

2- easy copy-paste of scripts

----------

## simon78

Thanks for the howto! I installed a mythfrontend using this guide a year or so ago. I updated the server (dhcp/nfs/tftp) to the new baselayout and udev. Now it seems like tftp doesn't work. The diskless recieves its IP from dhcp:

```

Jan 25 12:55:39 simons dhcpd: DHCPDISCOVER from 00:04:61:50:3b:06 via eth2

Jan 25 12:55:39 simons dhcpd: DHCPOFFER on 192.168.2.2 to 00:04:61:50:3b:06 via eth2

Jan 25 12:55:41 simons dhcpd: DHCPDISCOVER from 00:04:61:50:3b:06 via eth2

Jan 25 12:55:41 simons dhcpd: DHCPOFFER on 192.168.2.2 to 00:04:61:50:3b:06 via eth2

Jan 25 12:55:45 simons dhcpd: DHCPDISCOVER from 00:04:61:50:3b:06 via eth2

Jan 25 12:55:45 simons dhcpd: DHCPOFFER on 192.168.2.2 to 00:04:61:50:3b:06 via eth2

Jan 25 12:55:54 simons dhcpd: DHCPDISCOVER from 00:04:61:50:3b:06 via eth2

Jan 25 12:55:54 simons dhcpd: DHCPOFFER on 192.168.2.2 to 00:04:61:50:3b:06 via eth2

```

But, it does not get te kernel image. When i try to manully download it works, and i get log entries from tftp that says that a file was downloaded. But from the diskless, nothing!

tftpd is running.

Any hints?

----------

## kyron

 *simon78 wrote:*   

> [...snip...]
> 
> Any hints?

 

Check the LTSP wiki, these guys are my inspiration to all this and it's LTSP that gave me a good start for such a platform.

----------

## BonesToo

I've been running a diskless gentoo client now for 2 months with help from this thread and the diskless doc on gentoo.org.  All is running good.  Although I have one issue that I haven't worked out yet and I can't seem to find an answer for it anywhere.

When I reboot/shutdown the machine with "reboot", "shutdown -h now", or "halt" everything shuts down correctly except at the very end of the process my hardware is still running.  It never actually powers off or reboots to the bios, it just waits with the following text on the screen:

```

 * Unmounting filesystems ...                                        [ok]

 * Remounting remaining filesystems readonly ...                     [ok]

 * fsck will be skipped on next startup

Power down.

/etc/init.d/shutdown.sh: line 8: /sbin/halt: Network is unreachable

```

My theory is that since the root NFS mount was unmounted it can't get to the actual halt command to turn the hardware off?  I could be completely wrong.  I need some input on what is going on here.  Is anyone else seeing something like this???

Could it be that my hardware just doesn't support software shutdown?  Im running a PC/104 single board computer with VIA C3 Nehemiah CPU.  I can post more specs if needed.

Any ideas?    :Sad: 

----------

## kyron

 *BonesToo wrote:*   

> 
> 
> [...snip...]
> 
> My theory is that since the root NFS mount was unmounted it can't get to the actual halt command to turn the hardware off?  I could be completely wrong.  I need some input on what is going on here.  Is anyone else seeing something like this???
> ...

 

sounds like a fair assumption. Try mdifying the disklesse's /etc/init.d/shutdown.sh scritp to:

```
/sbin/poweroff -f
```

Note that this is a "nasty" shutdown and should only be used on "stateless" machines (ie: all processes and their temporary information rely on RAM to store their states). This usually impliest that /etc and /var is mostly in some sort of (RAM|UNION)fs

----------

## BonesToo

I'll try changing script to /sbin/poweroff -f.  But I don't know how this will be any different.

You bring up a good point.  Using a ramfs to store the files needed during the shutdown process.  This is what I've been spending most of my time on.  But I am having trouble getting my initramfs cpio image compiled into the kernel.  I followed gentoo doc instructions and did the following:

1) Copied my initramfs cpio compressed image to /usr/src/linux/usr/initramfs_data.cpio.gz

2) execute: touch /usr/src/linux/usr/initramfs_data.cpio.gz

3) make the kernel as normal

But it still seems to make a new cpio image from the initramfs_list.  Any help here?  I'm going to look around on the forums for more info on this.

```

...

  HOSTCC  usr/gen_init_cpio

  CHK     usr/initramfs_list

  CPIO    usr/initramfs_data.cpio

  GZIP    usr/initramfs_data.cpio.gz

  AS      usr/initramfs_data.o

  LD      usr/built-in.o

...

```

But the question is, is this the correct approach?  

Will unpacking a cpio image into a ramfs with /sbin/halt in it be what I need to shutdown the machine correctly? 

Does a root ram fs maintain itself after the nfs root is mounted?

Will this ramfs be available at shutdown?

I'm at a lose here.

EDIT: My pxelinux config looks like this?  Am I missing a /dev/ram0 in here?

```
DEFAULT kernel15.nofb

LABEL kernel15.nofb

KERNEL bzImage.15.nofb

APPEND ip=dhcp root=/dev/nfs nfsroot=192.168.1.102:/root/diskless/smokey,rsize=8192,wsize=8192,timeo=14

```

----------

## Per Olav

 *Quote:*   

> If you are happy, like me, with this, post a message here to gratulate me

 

Thank you! Worked great for me!

I ran into one problem only.. but it's not really related to the Howto I guess. 

Here goes:

I needed to connect some samba shares on my diskless client, and then use the netmount init script. When I shut down the client,

it umounts them  but also the root filesystem which is connected via NFS. This causes a lot of errors during the rest of the shutdown.

Is there a simple solution to this?

Edit: typoLast edited by Per Olav on Sat Apr 08, 2006 7:17 pm; edited 1 time in total

----------

## emuller

 *Quote:*   

> When I shut down the client, it umounts them but also the root filesystem which is connected via NFS. This causes a lot of errors during the rest of the shutdown. Is there a simple solution to this?
> 
> 

 

On my machine it doesn't unmount root but mounts it read-only upon failing its unmount attempt (umount -r option used in netmount implies this behavior) ... which causes a lot of errors for the rest of the shutdown ... 

I put a "return 0" as the first line in stop() of the netmount script.  This effectively disables stopping netmount.  Since I have /usr, /opt and /home as seperate NFS mounts (appart from /) I didn't feel like figuring out any other clean way to keep them from unmounting...

I guess one could clean up the stop code in netmount:

```
stop() {

        local ret

        ebegin "Unmounting network filesystems"

        [ -z "$(umount -art ${NET_FS_LIST// /,} 2>&1)" ]

        ret=$?

        eend ${ret} "Failed to simply unmount filesystems"

        ...

```

to fit your needs.  

Any other ideas? I'd be interested to know what solution you arrive at.  

Now that I think about it a "return 1" might be better as the first line of stop() ... so that it always errors?

e.

----------

## kyron

FYI, there is some work being done specifically for the NFS root mount/unmount issues:

https://bugs.gentoo.org/show_bug.cgi?id=99682

And the following is also interesting:

https://bugs.gentoo.org/show_bug.cgi?id=112059

----------

## wimalopaan

Hi,

for anyone who is interested:

as described in a previous posting our diskless setup includes the use of unionfs on the client and on the server. On the server we can then maintain relatively simple a set of diskless client roots for different purposes, e.g. one setup for a diskless rich client (the user logs into the client and the KDE-session starts on the client) and one setup for a diskless thin client (the user logs into the server and the KDE-session starts on the server), and so on.

Especially if you have diskless thin clients it is neccessary to transport the local devices of the thin client to the server. We experimented with several solutions (nfs back to the server, samba, ...) but we came up with the adoption of the LTSP solution what is called ltsp-fs. 

We provide ebuilds for the ltsp-fs and some (little) documentation on

http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/gdxs.html

and

http://mozart.informatik.fh-kl.de/download/Software/GentooDiskless/ltspfs.html

Please note that you need to use the provided overlay ebuild for pmount as well (mainly for security reasons). Also note that you have to setup two kde autostart scripts as described in the output of the ebuilds.

----------

## wescott

Thanks for the howto.  This would work great for the set-top boxes I want to develop (for expandability, extra boxes that boot this way).

----------

## cloc3

 *wimalopaan wrote:*   

> I have made a new version of the diskless gentoo client guide (see below). 

 

I've also write a unionfs gentoo diskless guide, before reading this topic, and I've put it on the wiki, to be modify by someone more practiced then me.

If you like, have a look to it.

----------

## petrasl

Hi,

I'm trying to setup Gentoo on a diskless laptop following the wiki, but I have some trouble:

The server is ready and the laptop boot on « pxe » mode ; the grub's menu is printed.

The image of the kernel is loaded, but the system hangs when once the kernel gives the hand to « init ». 

With the last kernel's log, I see that the interface « eth0 » is ready and has the good settings:  IP / rootserver / rootpath

- The nfs root file system is mounted correctly

- devfs is mounted on /dev (kernel-2.4)

- INIT: version 2.86 booting 

This is part of my kernel configuration:

```

CONFIG_PACKET=y

CONFIG_UNIX=y

CONFIG_INET=y

CONFIG_IP_MULTICAST=y

CONFIG_IP_PNP=y

CONFIG_IP_PNP_DHCP=y

CONFIG_IP_PNP_BOOTP=y

CONFIG_NFS_FS=y

CONFIG_NFS_V3=y

CONFIG_NFS_DIRECTIO=y

CONFIG_ROOT_NFS=y

CONFIG_NFSD=y

```

It's a 2.4 series kernel, so I done:

```

emerge --unmerge udev ; emerge devfsd

```

/etc/fstab :

```

/dev/cdroms/cdrom0 /mnt/cdrom iso9660 noauto,ro 0 0

proc /proc proc defaults 0 0

tmpfs /dev/shm tmpfs nodev,nosuid,noexec 0 0

```

Finally I emerged the lasted staffs like syslog-ng and wixie-cron.

Anybody konw what should'I try?

----------

## erik258

I am doing the same install method.  All seems to go OK at boot, until 

```
CLIENT MAC ADDR: 00 50 70 56 2E CA      GUID: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF

CLIENT IP: 192.168.10.100       MASK: 255.255.255.0     DHCP IP: 192.168.10.1

GATEWAY IP: 192.168.10.1

Probing...[RTL8139] - The PCI BIOS has not enabled this device!

Updating PCI command 0006->0007. pci_bus 00 pci_device_fn 00

PCI latency timer (CFLT) is unreasonably low at 0. Setting to 32 clocks.

ioaddr 0x0000, addr 00:80:40:55:AA:2A 10Mbps full-duplex

```

comes up on the screen.  the computer halts there, crtl+alt+del ineffective.

Does anyone recognize this?  

The logs from the root show...

```
Dec 10 15:32:55 [dhcpd] DHCPDISCOVER from 00:50:70:56:2e:ca via eth1

Dec 10 15:32:55 [dhcpd] DHCPOFFER on 192.168.10.100 to 00:50:70:56:2e:ca via eth

1

Dec 10 15:33:03 [dhcpd] DHCPREQUEST for 192.168.10.100 (192.168.10.1) from 00:50

:70:56:2e:ca via eth1

Dec 10 15:33:03 [dhcpd] DHCPACK on 192.168.10.100 to 00:50:70:56:2e:ca via eth1

Dec 10 21:33:03 [in.tftpd] RRQ from 192.168.10.100 filename pxegrub_

Dec 10 21:33:03 [in.tftpd] tftp: client does not accept options_

Dec 10 21:33:03 [in.tftpd] RRQ from 192.168.10.100 filename pxegrub_
```

It seems that dhcpd and in.tftpd are working, if misconfiguredly.  

Any help is greatly appreciated

----------

## mikek

 *erik258 wrote:*   

> I am doing the same install method.  All seems to go OK at boot, until 
> 
> ```
> CLIENT MAC ADDR: 00 50 70 56 2E CA      GUID: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF
> 
> ...

 

I got that same message trying to use pxegrub.  I think it came from grub not having support for my ethernet hardware.  I switched to pxelinux (part of syslinux) and was able to get the kernel booted.

Now, I'm having problems with the kernel just "hanging" after mounting the rootfs.  It's still alive (I can ping it), but no progress is being made.  If you hit that and figure out a solution, let me know!  :Smile: 

mike

----------

## erik258

I think i may have had a similar problem.   i will try to help if you like; i'd like to see the kernel boot command line and the last line on the screen, though.  My guess would be that either the mount is failing but thinks its succeeding or succeeds but the directory is specified incorrectly and there isn't a system rooted there, or perhaps you forgot to add support for root fs on nfs?  I wanted to quote menuconfig to help you out but couldn't find it ; (.  Sorry, you'll have to consult an online howto.  

I think you're right about the pxegrub thing; looks like emerge doesn't encorporate these configuration flags yet.  I mean to get it working some time, but since none of my diskless machines have bootable NICs, it seems pointless to bother now.  At the moment, I have them booting from floppy instead.  ; )

----------

## erik258

I got a box of nice PXE-enabled 3C905s and am back on the project now.   I also got a whole new message after booting grub!  This time i dowloaded grub-0.97 and patched (won't compile w/o), then, copied grub over and booted as previously. 

Here's the last lines.

```

CLIENT MAC ADDR: 00 50 70 56 2E CA      GUID: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF

CLIENT IP: 192.168.10.100       MASK: 255.255.255.0     DHCP IP: 192.168.10.1

GATEWAY IP: 192.168.10.1 

Probing pci nic...

[3c905c-tpo]

3C90X Driver 2.00 Copyright 1999 LightSys Technology Services, Inc. 

Portions Copyright 1999 Steve Smith

Provided with ABSOLUTELY NO WARRANTY

---------------------------------------------------------------------------------------------------------

MAC Address =

Connectors present: 10Base-T / 100Base-TX

```

that's all. It looks a lot more promising, but nothing actually loads (i left it sitting overnight just to be sure).

I'd much prefer pxegrub to pxelinux... anyone have any suggestions for me?

----------

## erik258

evidently PXE booting with grub is now broken.  I recommend PXELinux as a replacement.  Looks like i should have listened to everyone else, my kernel is booting fine now. 

here's the howto i found useful:

http://www.gentoo.org/doc/en/diskless-howto.xml

----------

## jesse_kahtava

Hello.

I've followed this howto as well as http://www.gentoo.org/doc/en/diskless-howto.xml and everything but nfs as root seems to be working. Here's the bottom bit of output:

```
eth0: link up, 100Mbps, full-duplex, lpa 0x45E1

Sending DHCP requests ., OK

IP-Config: Got DHCP answer from 192.168.1.10, my address is 192.168.1.11

IP-Config: Complete:

        device=eth0, addr=192.168.1.11, mask=255.255.255.0, gw=192.168.1.1,

        host=192.168.1.11, domain=hornpipe, nis-domain=(none),

        bootserver=192.168.1.10, rootserver=192.168.1.10, rootpath=

Looking up port of RPC 100003/2 on 192.168.1.10

Looking up port of RPC 100005/1 on 192.168.1.10

VFS: Mounted root (nfs filesystem) readonly.

Freeing unused kernel memory: 144k freed

nfs_stat_to_errno: bad nfs status return value: 45

Warning: unable to open an initial console.

nfs_stat_to_errno: bad nfs status return value: 45

nfs_stat_to_errno: bad nfs status return value: 45

nfs_stat_to_errno: bad nfs status return value: 45

nfs_stat_to_errno: bad nfs status return value: 45

Kernel panic - not syncing: No init found. Try passing init= option to kernel.
```

Also, here's my kernel command line options:

```
default 0

timeout 30

title=Diskless Gentoo

root (nd)

kernel /mystic/boot/linux-2.6.17-beyond4 ip=dhcp root=/dev/nfs nfsroot=192.168.1.10:/diskless/mystic
```

Has anyone else encountered this error? I'm able to mount nfs fine inside of a working system.

I tried a different kernel, but got the same problem. I know for certain that NFS for root is enabled.

Oh, and in response to the above post; PXE Grub is working well for me

----------

## erik258

I've encountered that error once.  It means either that init itself (/sbin/init) can't be found, or can't be executed, or else (this was my problem) the libraries it needs can't be found.

 *http://www.mail-archive.com/unionfs@mail.fsl.cs.sunysb.edu/msg02088.html wrote:*   

> NFS4 still fails, thus adding the kernel boot option: nfsroot=...,nfsver=3 helped.

 

try nfs v 3?  Although, i didn't need to do this.  Only thing I can think of, you haven't installed nfs-utils.  Do you have nfs software installed?

----------

## jesse_kahtava

I've already completely disabled NFSv4 support in the kernel so I expect it's using NFSv3.

Also, I checked that init is in the right place and it executes within a chroot. 

The error value of 45 is mainly what's concerning me since it spits out a bunch of them before finally giving a kernel panic.

I don't think it's able to read from the NFS mount at all.

Oh, and nfs-utils is installed. As I said though, i'm under the impression that it's completely unable to read from NFS.

----------

## erik258

 *Quote:*   

> Also, I checked that init is in the right place and it executes within a chroot. 

 

within a chroot?  It needs access to /lib to properly run!  

although, i agree with your concern about the NFS error being more severe.  But perhaps the chroot explains that too.

----------

## jesse_kahtava

yes, within a chroot. the nfs mount contains a complete filesystem (with /lib and all) that i had created for the computer before deciding i wanted it diskless. After mounting the nfs volume i was able to chroot into it and execute init. It fails with a timeout, not because it can't be found or because of any sort of seg fault which gives me a good idea that it's there and should be working under normal circumstances. and as i said before, it is in the standard location (/sbin/init)

----------

## jesse_kahtava

I found the problem! It seems that Reiser4 and NFS don't cooperate well when mounting NFS root. I made a new partition, formatted it with Reiser3, copied the full root contents from the Reiser4 partition over, set up /etc/exports to point to the new mount point with Reiser3 and everything works fine. After realizing this was a problem I googled around a bit and found a handful of references to Reiser4 not working 100%.

!!!As of this writing Reiser4 does not work being exported and used as a NFS root!!!

(Well, not for me anyway)

----------

## erik258

that explains it.  I have mine on reiser3.  I've thought about switching my workstation over, what do you think?

----------

## Mgiese

hi there,

just a short question, i finished the setup of my first diskless pc (pentium4-256mb) but now i just wonder why unpacking GCC4.1.1 took nearly 30 minutes or even more, to apply all the patches already took 30 minutes so far and its not even finished...  :Sad:  any suggestions ? how do i speed up that ? thx a lot

----------

## erik258

your network is probably maxing out at 12.5mb/sec one way through much of that time.

of course, gcc is one of the biggest packages you're likely to install.

----------

## wimalopaan

To all how are using unionfs for building diskless clients:

actual versions of unionfs produce various kernel-oopses.

aufs (another unionfs) is a direct replacement for unionfs, but cleanly built from scratch.

If you want to give it a try, heres an ebuild:

https://bugs.gentoo.org/show_bug.cgi?id=160065

----------

## Mgiese

is it possible to use nvidia-drivers on a diskless node ? the xorg modul "nv" is working fine, but i can not "modprobe nvidia". when installing the diskless pc i have read i should put all modules into kernel, because i am unable to load them later. but i need dri on the diskless node. i am not using grub for the diskless pc. thanks in advance

----------

## erik258

I don't see any reason why you wouldn't be able to load any module you wanted.  Nothing cares as far as i can tell whether the root fs is on nfs or a disk, usb or anything else.

----------

## Mgiese

thanks for your info.. i compiled now agpgart and i2c modules into the kernel, lets see if it helps

when setting up the client i used the following guide : http://www.gentoo.org/doc/en/diskless-howto.xml

```
About the slave kernel

It is recommended that you compile the slave kernel without any modules, since loading and setting them up via remote boot is a difficult and unnecessary process. Additionally, the slave kernel should be as small and compact as possible in order to efficiently boot from the network. We are going to compile the slave's kernel in the same place where the master was configured.

```

so i maybe just misinterpreted that info : its not impossible - its just difficult... THX A LOT !!

----------

## conbart

 *Quote:*   

> 
> 
> aufs (another unionfs) is a direct replacement for unionfs, but cleanly built from scratch. 
> 
> If you want to give it a try, heres an ebuild: 
> ...

 

There is now an ebuild for aufs at the gentoo-sunrise overlay  :Smile: 

----------

## Rumit

https://forums.gentoo.org/profile.php?mode=activate&u=159900&act_key=uXUwhrvcMWEDxrDWxI2Eg47mek20fG

----------

## mbar

Sorry for digging up this thread, but I did not want to create new one  :Wink: 

I'm planning to connect diskless node (i686 architecture, namely P3) to my x86_64 server. Gentoo diskless guide assumes that both (master and slave) architectures are the same.

Are there any quirks or incompatibilities when trying to connect i686 slave to x86_64 master?

----------

## erik258

I don't forsee any such difficulties.

----------

## WereCatf

 *mbar wrote:*   

> Sorry for digging up this thread, but I did not want to create new one 
> 
> I'm planning to connect diskless node (i686 architecture, namely P3) to my x86_64 server. Gentoo diskless guide assumes that both (master and slave) architectures are the same.
> 
> Are there any quirks or incompatibilities when trying to connect i686 slave to x86_64 master?

 

Just remember that the files being served to the slave must be compiled as 32-bit binaries, not 64-bit ones  :Smile: 

----------

## brot

 :Smile:  As requested in the howto, thank you (the author  :Wink:  ) for that howto, that is exactly what i was searching...

----------

## selig

Nice... thanks a lot for the quick (not counting compile time but hey, it's Gentoo  :Smile: ) & easy diskless setup. The only glitch I've run into is that my "client does not accept options" so I had to issue the commands in pxegrub manually, it does not display a menu. But since I am using this as a rescue solution (so far it looks like at least the partition table got wiped out, along with GRUB - maybe more) for my laptop which has not got a CD-drive, it is not a problem.

----------

