# Diskless client, NFS root stays mounted ro

## dusanc

Hello

When using Gentoo/OpenRC for a diskless client root fs stays mounted RO when booting.

Clients fstab:

```
krshina3 /diskless/shared # cat etc/fstab | grep -v '#'

192.168.1.3:/diskless/shared  /            nfs  noatime,sync,hard,intr,rw,nolock,noacl    0 0

192.168.1.3:/usr/portage      /usr/portage nfs  noatime,sync,soft,intr,rw,nolock,noacl    0 0

none            /tmp            tmpfs   defaults        0       0

none            /var/run        tmpfs   defaults        0       0

none            /var/tmp        tmpfs   defaults        0       0

shm                     /dev/shm        tmpfs           nodev,nosuid,noexec     0 0

```

When going to interactive boot /etc/init.d/root and /etc/init.d/mtab say that they can't start because they need /etc/init.d/localmount but /etc/init.d/localmount has dependency that it has to start after /etc/init.d/root so that boot process looks like:

root - Error, no localmount

mtab - Error, no localmount

localmunt - OK

Nonroot NFS mounts get mounted rw without problems.

/etc/exports

```
krshina3 /diskless/shared # cat /etc/exports

# /etc/exports: NFS file systems being exported.  See exports(5).

/diskless/shared 192.168.1.64/26(sync,rw,no_root_squash,no_all_squash,fsid=0,no_subtree_check)

/usr/portage 192.168.1.64/26(sync,rw,no_all_squash,no_subtree_check)

```

----------

## krinn

root doesn't depend on nfsclient, but if your root is nfs, it should depends on nfsclient (that's something to handle).

i see no reason for root to depend on localmount, same for mtab, i don't see a reason for it to depend on localmount ; and i see nothing in root or mtab that set a dependency on localmount.

localmount do depends on mtab and root.

so how you endup with root and mtab depending on localmount?

----------

## NeddySeagoon

dusanc,

What krinn said ... and post 

```
ls -lR /etc/runlevels/
```

from the client please.

----------

## dusanc

 *krinn wrote:*   

> root doesn't depend on nfsclient, but if your root is nfs, it should depends on nfsclient (that's something to handle).
> 
> i see no reason for root to depend on localmount, same for mtab, i don't see a reason for it to depend on localmount ; and i see nothing in root or mtab that set a dependency on localmount.
> 
> localmount do depends on mtab and root.
> ...

 

I agree on everything you said, that's why I didn't understand what is happening so I asked here for help.

Diskless enviroment was built by the regular gentoo amd64 handbook and this problem started after first successful boot to NFSroot. 

Now I'm going to setup syslog-ng for remote logging so I can get more info.

Untill then :

```
 krshina3 / # ls -lR /etc/runlevels/

/etc/runlevels/:

total 3

drwxr-xr-x 2 root root 18 Dec 29 00:00 boot

drwxr-xr-x 2 root root  6 Jan  6 23:35 default

drwxr-xr-x 2 root root  3 Dec 29 00:00 nonetwork

drwxr-xr-x 2 root root  5 Dec 29 00:00 shutdown

drwxr-xr-x 2 root root  8 Dec 29 00:00 sysinit

/etc/runlevels/boot:

total 0

lrwxrwxrwx 1 root root 18 Dec 15 22:50 binfmt -> /etc/init.d/binfmt

lrwxrwxrwx 1 root root 20 Dec 15 22:50 bootmisc -> /etc/init.d/bootmisc

lrwxrwxrwx 1 root root 16 Dec 15 22:50 fsck -> /etc/init.d/fsck

lrwxrwxrwx 1 root root 20 Dec 15 22:50 hostname -> /etc/init.d/hostname

lrwxrwxrwx 1 root root 19 Dec 15 22:50 hwclock -> /etc/init.d/hwclock

lrwxrwxrwx 1 root root 19 Dec 15 22:50 keymaps -> /etc/init.d/keymaps

lrwxrwxrwx 1 root root 22 Dec 15 22:50 localmount -> /etc/init.d/localmount

lrwxrwxrwx 1 root root 20 Dec 15 22:50 loopback -> /etc/init.d/loopback

lrwxrwxrwx 1 root root 19 Dec 15 22:50 modules -> /etc/init.d/modules

lrwxrwxrwx 1 root root 16 Dec 15 22:50 mtab -> /etc/init.d/mtab

lrwxrwxrwx 1 root root 18 Dec 15 22:50 procfs -> /etc/init.d/procfs

lrwxrwxrwx 1 root root 16 Dec 15 22:50 root -> /etc/init.d/root

lrwxrwxrwx 1 root root 16 Dec 15 22:50 swap -> /etc/init.d/swap

lrwxrwxrwx 1 root root 18 Dec 15 22:50 sysctl -> /etc/init.d/sysctl

lrwxrwxrwx 1 root root 24 Dec 15 22:50 termencoding -> /etc/init.d/termencoding

lrwxrwxrwx 1 root root 19 Dec 15 22:50 urandom -> /etc/init.d/urandom

/etc/runlevels/default:

total 0

lrwxrwxrwx 1 root root 17 Dec 15 22:50 local -> /etc/init.d/local

lrwxrwxrwx 1 root root 20 Dec 15 22:50 netmount -> /etc/init.d/netmount

lrwxrwxrwx 1 root root 16 Jan  6 23:35 sshd -> /etc/init.d/sshd

lrwxrwxrwx 1 root root 21 Jan  6 23:34 syslog-ng -> /etc/init.d/syslog-ng

/etc/runlevels/nonetwork:

total 0

lrwxrwxrwx 1 root root 17 Dec 15 22:50 local -> /etc/init.d/local

/etc/runlevels/shutdown:

total 0

lrwxrwxrwx 1 root root 21 Dec 15 22:50 killprocs -> /etc/init.d/killprocs

lrwxrwxrwx 1 root root 20 Dec 15 22:50 mount-ro -> /etc/init.d/mount-ro

lrwxrwxrwx 1 root root 21 Dec 15 22:50 savecache -> /etc/init.d/savecache

/etc/runlevels/sysinit:

total 0

lrwxrwxrwx 1 root root 17 Dec 15 22:50 devfs -> /etc/init.d/devfs

lrwxrwxrwx 1 root root 17 Dec 15 22:50 dmesg -> /etc/init.d/dmesg

lrwxrwxrwx 1 root root 29 Dec 28 23:47 kmod-static-nodes -> /etc/init.d/kmod-static-nodes

lrwxrwxrwx 1 root root 17 Dec 15 22:50 sysfs -> /etc/init.d/sysfs

lrwxrwxrwx 1 root root 16 Dec 29 00:00 udev -> /etc/init.d/udev

lrwxrwxrwx 1 root root 24 Dec 29 00:00 udev-trigger -> /etc/init.d/udev-trigger

```

----------

## krinn

enable rc.log would better help, you have no problem from kernel, but a problem with openrc that complain localmount is need for mtab and root

you still have other issue, but that something for later (you lack net.lo in boot runlevel, you lack to put vers=3 in fstab or you try to use nfvs4 with a bad nfvs4 mount instruction, and your export file is invalid for nfsv4, which should have fsid=0 for root dir, and no other entry out of that directory)

----------

## NeddySeagoon

dusanc,

```
/etc/runlevels/default:

total 0

lrwxrwxrwx 1 root root 17 Dec 15 22:50 local -> /etc/init.d/local

lrwxrwxrwx 1 root root 20 Dec 15 22:50 netmount -> /etc/init.d/netmount

lrwxrwxrwx 1 root root 16 Jan  6 23:35 sshd -> /etc/init.d/sshd

lrwxrwxrwx 1 root root 21 Jan  6 23:34 syslog-ng -> /etc/init.d/syslog-ng 
```

Where do you configure and start your network interface for userspace?

The kernel knows about it as its used to mount root but user space can't see that.

I suspect that allowing udev to rename it will cause the root filesystem to vanish.

It needs to have the same name and set up as the kernel was given before user space started.

Root stays read only because rootfsck, which runs in userspace, can't see the network.

----------

## dusanc

 *krinn wrote:*   

> enable rc.log would better help, you have no problem from kernel, but a problem with openrc that complain localmount is need for mtab and root
> 
> you still have other issue, but that something for later (you lack net.lo in boot runlevel, you lack to put vers=3 in fstab or you try to use nfvs4 with a bad nfvs4 mount instruction, and your export file is invalid for nfsv4, which should have fsid=0 for root dir, and no other entry out of that directory)

 

I enabled rc.log but now I have the problem that /var/log is ro and init doesn't parse /etc/fstab

As for net.lo, yeah I probably missed that from handbook.

Is this OK, I took it from /etc/init.d/root and tried when logged in client?

```
mount -n -o remount,rw /

mount.nfs: an incorrect mount option was specified
```

----------

## NeddySeagoon

dusanc,

Not net.lo, net.eth0 if the interface is wired.

Its 

```
mount -o remount,rw / 
```

should be enough.  It will fail as the network is not up until net.eth0 is started.

----------

## dusanc

 *NeddySeagoon wrote:*   

> dusanc,
> 
> ```
> /etc/runlevels/default:
> 
> ...

 

Networking works, I can ping www.google.com

Even when I added net.eth0 everything is the same.

```
krshina3 /etc/init.d # cat /etc/conf.d/net

config_eth0="noop"

```

```
 krshina3 /etc/init.d # rc-update show

               binfmt | boot

             bootmisc | boot

                devfs |                                 sysinit

                dmesg |                                 sysinit

                 fsck | boot

             hostname | boot

              hwclock | boot

              keymaps | boot

            killprocs |                        shutdown

    kmod-static-nodes |                                 sysinit

                local |      default nonetwork

           localmount | boot

             loopback | boot

              modules | boot

             mount-ro |                        shutdown

                 mtab | boot

             net.eth0 |      default

             netmount |      default

               procfs | boot

                 root | boot

            savecache |                        shutdown

                 sshd |      default

                 swap | boot

               sysctl | boot

                sysfs |                                 sysinit

            syslog-ng |      default

         termencoding | boot

                 udev |                                 sysinit

         udev-trigger |                                 sysinit

              urandom | boot

```

----------

## dusanc

 *NeddySeagoon wrote:*   

> dusanc,
> 
> Not net.lo, net.eth0 if the interface is wired.
> 
> Its 
> ...

 

Even with 

```
mount -o remount,rw /
```

I get same message

```
mount.nfs: an incorrect mount option was specified
```

----------

## dusanc

Ok I emerge -1 openrc in chroot with:

```
root #mkdir /lib64/rc/init.d

root #ln -s /lib64/rc/init.d /run/openrc

root #touch /run/openrc/softlevel

root #emerge --oneshot sys-apps/openrc 
```

In the end I got this:

```
>>> Installing (1 of 1) sys-apps/openrc-0.34.11::gentoo

 * The OpenRC dependency data has already been migrated.

Service `consolefont' needs non existent service `localmount'

Service `consolefont' needs non existent service `termencoding'

Service `ip6tables' needs non existent service `localmount'

Service `iptables' needs non existent service `localmount'

Service `net-online' needs non existent service `sysfs'

Service `net.eth0' needs non existent service `sysfs'

Service `net.eth0' needs non existent service `localmount'

Service `nfsmount' needs non existent service `netmount'

Service `numlock' needs non existent service `localmount'

Service `opentmpfiles-setup' needs non existent service `localmount'

Service `s6-svscan' needs non existent service `localmount'

Service `udev-settle' needs non existent service `udev'

 * Caching service dependencies ... [ ok ]

>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * GNU info directory index is up-to-date.

```

```
krshina3 /diskless/shared # ls etc/init.d/

agetty            consolefont   hostname   killprocs          modules       net.eth0   nfsmount            procfs      rpc.pipefs  s6-svscan  sysctl        udev-settle

binfmt            devfs         hwclock    kmod-static-nodes  modules-load  net.lo     numlock             pydoc-2.7   rpc.statd   savecache  sysfs         udev-trigger

bootmisc          dmesg         ip6tables  local              mount-ro      netmount   opentmpfiles-dev    pydoc-3.5   rpcbind     sshd       syslog-ng     urandom

busybox-ntpd      fsck          iptables   localmount         mtab          nfs        opentmpfiles-setup  root        rsyncd      swap       termencoding

busybox-watchdog  functions.sh  keymaps    loopback           net-online    nfsclient  osclock             rpc.idmapd  runsvdir    swclock    udev

```

I really don't understand what is happening.

----------

## dusanc

Ok I fixed it.

Although the files in /etc/init.d existed they were all empty.

Only stupid thing I did that could do that is to run rc-status in chrooted enviroment.... Or something else, dunno.

emerge -1 openrc && dispatch-conf crated back files in /etc/init.d/ and solved everything.

Thanks a lot for your help

Dusan

PS Maybe it's not fixed as /etc/init.d/udev is still 0 in size, probably if I get it back things will stop working again. Gonna sleep on it.

----------

## krinn

 *dusanc wrote:*   

> PS Maybe it's not fixed as /etc/init.d/udev is still 0 in size, probably if I get it back things will stop working again. Gonna sleep on it.

 

There's a know bug with zfs 0 files, if you use zfs, you know what keyword to search for the fix.

----------

