# mounts troubles

## richard.langly

I've got a machine that boots via nfs and / is being mounted as readonly. The other partitions listed in the fstab aren't mounting at all. 

I found the following in 'dmesg' output.

```

Root-NFS: unknown option: rw

...

VFS: Mounted root (nfs filesystem) readonly.

Freeing unused kernel memory: 436k freed.

```

Here's it's fstab.

```

10.0.0.10:/diskless/10.0.1.1    /       nfs     sec=sys,sync,hard,intr,rw,nolock,rsize=8192,wsize=8192,nfsvers=3        0       0

10.0.0.10:/opt                  /opt    nfs     sync,hard,intr,rw,nolock,rsize=8192,wsize=8192  0       0

10.0.0.10:/usr                  /usr    nfs     sec=sys,sync,hard,intr,ro,nolock,rsize=8192,wsize=8192,nfsvers=3        0       0

10.0.0.10:/home                 /home   nfs     sync,hard,intr,rw,nolock,rsize=8192,wsize=8192  0       0

10.0.0.10:/var/log              /var/log nfs    sync,hard,intr,rw       0 0

```

I am curious why / in the fstab above shows nfsvers=3, yet /proc/mounts below shows /dev/root as vers=2 ... is that right?

And here's /proc/mounts.

```

$ cat /proc/mounts

rootfs / rootfs rw 0 0

/dev/root / nfs ro,vers=2,rsize=4096,wsize=4096,namlen=255,hard,nointr,nolock,proto=udp,timeo=11,retrans=2,sec=sys,mountproto=udp,addr=10.0.0.10 0 0

proc /proc proc rw,nosuid,nodev,noexec 0 0

sysfs / sys sysfs rw,nosuid,nodev,noexec 0 0

udev /dev/ tmpfs rw,nosuid,size=1024k,mode=755 0 0

devpts /dev/pts devpts rw,nosuid,noexec,gid=5,mode=620 0 0

usbfs /proc/bus/usb usbfs rw,nosuid,noexec 0 0

```

Anyone have an idea about why the 'unknown option: rw' appears?Last edited by richard.langly on Wed Jan 14, 2009 3:22 am; edited 2 times in total

----------

## boerKrelis

As I understand it, you want to mount / rw but it gets mounted ro? Try 'rootflags=rw' as a kernel parameter.

Some mountoptions need to be the same in fstab and the kernel parameters. For instance, if you want to use quota on your /, you should specify 'rootflags=quota' (for XFS) as a kernel parameter in grub.conf, otherwise the remount (with the options specified in fstab) that Gentoo's init does won't work. So try this, it might help.

----------

## richard.langly

My node is diskless, which is why I'm mounting it via NFS. The master node which is the only one that uses grub mounts fine as rw. So, so I still add this rootflags=rw as a kernel parameter somewhere's? 

Also, my filesystem is JFS.

----------

## boerKrelis

 *richard.langly wrote:*   

> 
> 
> My node is diskless, which is why I'm mounting it via NFS. The master node which is the only one that uses grub mounts fine as rw. So, so I still add this rootflags=rw as a kernel parameter somewhere's?
> 
> 

 

Depends on how you boot it. I've used Etherboot with ISC DHCP (so no TFTP!) on diskless nodes in the past, with Etherboot you have to specify kernel parameters with the '--append=' option of mknbi, see http://etherboot.berlios.de/doc/html/mknbi.html. I haven't used PXE.

 *richard.langly wrote:*   

> 
> 
> Also, my filesystem is JFS.
> 
> 

 

From the POV of the client, whose mount options we're discussing, the filesystem is NFS. Whether the NFS server keeps its stuff on XFS, JFS or EXT3 doesn't matter for the mountoptions the client has to use.

----------

## richard.langly

I am using PXE and I see that I already have the 'rw' in the append option.

```

DEFAULT /bzImage

APPEND ip=dhcp rootfstype=nfs root=/dev/nfs nfsroot=10.0.0.10:/diskless/10.0.1.1,nolock,rw

```

However, what I don't understand is why I'm seeing the 'Root-NFS: unknown option: rw' in my dmesg output as listed above. So I think my client is seeing the 'rw', but it's not accepting it.

Another thing I see when it boots is the following ...

```

(can't see/copy what's on the screen at this point, then I see the following)

/lib64/udev/shell-compat.sh: line 6: /dev/null: Read-only file system

/lib64/udev/shell-compat.sh: line 6: /dev/null: Read-only file system

/lib64/udev/shell-compat.sh: line 6: /dev/null: Read-only file system

/lib64/udev/shell-compat.sh: line 6: /dev/null: Read-only file system

* Mounting /dev ...                                                     [ ok ]

* Starting udevd ...                                                    [ ok ]

* Populating /dev with existing devices through uevents                 [ ok ]

* Waiting for uevents to be processed ...                                

udevd-event][1426]: rename_netif: error changing netif name eth0 to eth2: Device or resource busy

                                                                        [ ok ]

Mounting devpts at /dev/pts ...                                         [ ok ]

* Skipping /etc/mtab initialization (ro root?)

* Updating modules.dep ...

FATAL: Could not open /lib/modules/2.6.26-gentoo-r4/modules.dep.temp for writing: Read-only file system

rm: cannot remove '/fastboot': Read-only file system

* Failed to start /etc/init.d/checkfs

* One or more critical sstartup scripts failed to start!

* Please correct this, and reboot ...

Give root password for maintenance

(or type Control-D to continue): 

```

Any help much appreciated.

----------

## boerKrelis

I think the following:

unlike I previously thought: initially, / should be mounted ro - change your append from rw to ro

running the gentoo initscript 'netmount' would remount your rootfs rw - but init doesn't reach the netmount script

so you have to remount your / rw sometime earlier

so make yourself an initscript that does that and add it to the runlevel 'boot', make sure it runs before any other bootscripts that depend on / being mounted rw. Or change the 'localmount'-script to do your remount for you, that might work too.

In fact, try to remount your / rw after giving your root password for maintenance, to get your options right.

(at the very least 'mount -o remount,rw /').

----------

## richard.langly

After giving the root password for maintenance, I can manually remount /, and then mount each of the other partitions as listed in fstab.

I changed the append to ro in my nodes pxelinux.cfg scripts, but got the same error as with rw. Then I changed this line in my nodes pxelinux.cfg scripts ...

```

APPEND ip=dhcp rootfstype=nfs root=/dev/nfs nfsroot=10.0.0.10:/diskless/10.0.1.1,nolock,rw

```

to this line, and I was actually able to boot all the way through ...

```

APPEND ip=dhcp rw rootfstype=nfs root=/dev/nfs nfsroot=10.0.0.10:/diskless/10.0.1.1,nolock

```

Note the location of 'rw'. Anyone understand what's going on with that? So, at least / is mountingc correctly, however, none of the other partitions do.

I did this because I was confused on why I should have to add something to the initscripts which I thought is something that gentoo should already do correctly.

----------

## boerKrelis

That's weird. What does 'rc-status' and 'rc-update show boot' show you?

----------

## richard.langly

Here are rc-status and rc-update just after a boot, / is rw, yet other partitions haven't mounted so it still hasn't booted cleantly.

```

$ rc-update show boot

/sbin/rc-update: line 123: find: cmomand not found

/sbin/rc-update: line 123: find: command not found

bootmsic | boot

checkfs | boot

checkroot | boot

clock | boot

consolefont | boot

hostname | boot

keymaps | boot

localmount | boot

modules | boot

net.lo | boot

rmnologin | boot

urandom | boot

```

```

$ rc-status

/bin/rc-status: line 124: find: command not found

```

And since that wasn't much to go on ...

```

$ ls -l /etc/runlevels/boot

bootmisc -> /etc/init.d/bootmisc

checkfs -> /etc/init.d/checkfs

checkroot -> /etc/init.checkroot

clock -> /etc/init.d/clock

consolefont -> /etc/init.d/consolefont

hostname -> /etc/init.d/hostname

keymaps -> /etc/init.d/keymaps

localmount -> /etc/init.d/localmount

modules -> /etc/init.d/modules

net.lo -> /etc/init.d/net.lo

rmnologin -> /etc/init.d/rmnologin

urandom -> /etc/init.d/urandom

```

And here's my /etc/exports

```

/diskless/10.0.1.1      10.0.1.1(sync,rw,no_subtree_check,no_root_squash,no_all_squash)

/opt    10.0.0.0/16(sync,rw,no_subtree_check,no_root_squash,no_all_squash)

/usr    10.0.0.0/16(sync,ro,no_subtree_check,no_root_squash,no_all_squash)

/home   10.0.0.0/16(sync,rw,no_subtree_check,no_root_squash,no_all_squash)

/var/log 10.0.0.0/16(sync,rw,no_subtree_check,no_root_squash,no_all_squash)

```

----------

## boerKrelis

Shouldn't 'netmount' be in the boot runlevel? AFAIK, 'localmount' will only mount local filesystems, so you need 'netmount' too.

----------

## richard.langly

I looked at my old diskless configuration (which hadn't been updated in a few years and worked fine), and it shows netmount being in default also, but I did move it to boot, but it didn't help.

And after a boot, I try to mount one of the partitions listed in the diskless nodes fstab and get ...

```

# mount /usr

mount.nfs: rpc.statd is not running but is required for remote locking.

mount.nfs: Either use '-o nolock' to keep locks local, or start statd.

```

And more information ...

```

# ./netmount start

/etc/init.d/rpc.statd: line 20: killall: comand not found

* Starting NFS statd ...

statd: Coud not chdir: No such file or directory                    [ !! ]

* ERROR: cannot start netmount as rpc.statd could not start

```

Just some more info to see when it boots ...

```

Cannot open font file default8x16

Cannot open font file default8x16

Cannot open font file default8x16

Cannot open font file default8x16

Cannot open font file default8x16

* Failed to set user font                                                                       [ !! ]

* Please edit /etc/conf.d/ntp-client

* Unable to locate the client command ntpdate!

* Starting portmap ...                                                                          [ ok ]

/etc/init.d/rpc.statd: line 20 : killall: command not found

* Starting NFS statd ...

statd: Could not chdir: No such file or directory                                      [ !! ]

* ERROR: cannot start netmount as rpc.statd could not start

/etc/init.d/sshd: line 30: /usr/sbin/sshd: No such file or directory

* starting vixie-crong ...

/bin/start-stop-daemon: stat /usr/sbin/cron: No such file or directory (No such file or directory)  [ !! ]

* Starting local ...                                                                                [ ok ]

```

Something I'm curious about when / is mounted, I cat /proc/mounts and see vers=2 on /dev/root ... does that mean nfs version 2? Because AFAIK, I'm only using 3 which I build the kernel with.

```

$ cat /proc/mounts

rootfs / rootfs rw 0 0

/dev/root / nfs rw,vers=2,rsize=4096,wsize=4096,namlen=255,hard,nointr,nolock.proto=udp,timeo=11,retrans=2,sec=sys,mountproto=udp,addr=10.0.0.10 0 0

```

----------

## boerKrelis

There's a bit of a catch-22 with netmount relying on services that are relying on /usr (and /var) being mounted. You can see that when you look at /etc/init.d/netmount.

However, there's also an /etc/init.d/nfsmount. It seems that it needs 'net'. I'm not sure your 'net' is up. And it will try to start rcp.statd, which will fail (according to your boot output).

To me it seems that it will take quite some work to get the initscripts to play nice with netmounted /usr and /var.

Putting /usr on the same mount as / will save you a lot of trouble.

You could make an initscript yourself, though, that does the minimum work to mount mount /var and /usr - without 'need net' in the depend().

Maybe something like this:

```

#/sbin/runscript

depend() {

need portmap

}

start{

mount -t nfs -o sync,hard,intr,rw,nolock,rsize=8192,wsize=8192 10.0.0.10:/opt /opt

#etc etc etc

}

stop {

#Just a stub.

eend 0

}

```

Try it, it might work. Stick it in your boot runlevel. There might be a more 'official' approach available, but I don't know it.

----------

## richard.langly

 *boerKrelis wrote:*   

> I'm not sure your 'net' is up. 

 

What exactly do you mean by 'net' being up? I'd like to check this to make sure and see where this leads me.

----------

## boerKrelis

'net' is some sort of pseudo-service which other services can depend on. Try

```

grep 'need net' /etc/init.d/*

```

and

```

grep 'provide net' /etc/init.d/*

```

to see for yourself.

Now, when is 'net' considered to be up? That's determined in /etc/rc.conf.

```

# Do we allow any started service in the runlevel to satisfy the depedency

# or do we want all of them regardless of state? For example, if net.eth0

# and net.eth1 are in the default runlevel then with rc_depend_strict="NO"

# both will be started, but services that depend on 'net' will work if either

# one comes up. With rc_depend_strict="YES" we would require them both to

# come up.

rc_depend_strict="YES"

```

With "YES", every service that has a 'provide net' needs to be up for 'net' to be considered up. With "NO", any successfully started service that has a 'provide net' can cause 'net' to be considered up.

The thing with your situation is that you have network connectivity (kernel-level autoconfiguration) BEFORE init takes over. That's great - it's the very essence of netbooting - but the initscripts don't know about that.

You could try to make a minimalistic initscript with a 'provide net' in it, set rc_depend_strict="NO", and make it start really early in init (you can do that by not listing any dependencies -> no "use foo" or "need bar". That way, you can make sure 'net' is up.

See also http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=4#doc_chap4.

It would be nice if there was some tool to print the init.d/* dependency graph. Anyone know of any?

----------

