# NFS4 much slower than NFS3 (when using subversion)

## Alan Turing

I have a file server (using kernel 3.11.7-hardened-r1) on which an nfs server is running.

I have a desktop computer (using kernel 3.10.25-gentoo) which is mounting some folders which are exported by the server.

I am using subversion, the repository is somewhere in the internet, my local checkout is on the fileserver. This local checkout is also exported via nfs and sometimes mounted by the client.

Unfortunately this is very slow. An  "svn up"  done on the client takes around 135s. If I export the local checkout via samba, an "svn up" takes only around 20s.

Annoying: The "svn up" on the nfs filesystem seems to be slow for no reason. Neither on the client nor on the server I can see high activity with top or iotop.

However, mounting a filesystem via samba has some other disadvantages, and I live with this slow-svn-over-nfs problem for a few years now.

A few month ago, I switched from nfs3 to nfs4 and guess what? This made it even slower.

Meanwhile, I was too annoyed and made the following benchmarks.

First, I copied a 900MB file from the client to the nfs share.

With both, nfs3 and nfs4 this took around 9s. 100MB/s (over gigabit ethernet) - nothing to complain!

Furthermore, I copied the clients /usr/portage/dev-* (33102 files, together 100MB) to the nfs share.

With nfs3 this took 95s with nfs4 this took 135s.

Finally, I also compared the "svn up" on my local checkout. (For the case where there are no changes in the repository)

With nfs3 this took 135s with nfs4 this took 260s.

The /etc/exports on my server contains the line

```
/storage             192.168.0.42(rw,async,no_root_squash,no_subtree_check)
```

The /etc/fstab on my client contains the line

```
192.168.0.23:/storage  /mnt/nfs   nfs  noauto,nfsvers=4,retry=0,noatime,nocto,nolock,local_lock=all    0 0
```

I also tried different mount options and different export options. Using other options, the performance was slightly worse but there was always a big difference between nfs3 and nfs4.

Do you have any comments or suggestions for me?

----------

## 666threesixes666

"Do you have any comments or suggestions for me?"  suggestion...  svn2git & git....  sftp & sshfs-fuse....  nfs, out the window, svn out the window...  there are many ways to do things, and you're giving your self more means to the same ends.....Last edited by 666threesixes666 on Wed Mar 12, 2014 9:55 am; edited 1 time in total

----------

## Goverp

I'm in no way an NFS or SVN expert, but I've noticed that NFS can run slowly if you don't have the correct setup for handling ID mapping; I presume it's using some slow work-around in such circumstances.  Hope that helps.

----------

## Alan Turing

I tested the sshfs fuse filesystem (using its default options)

Copying the 900MB file took 30s.

Three times as much as nfs3. But both cores of the servers CPU where at 100%.

Copying the 33102 files from /usr/portage/dev-* took 14m9s

Nine times as much as nfs3. But no CPU was used more than 10%.

Doing my "svn up" took 55s.

Faster than nfs3 !

Using git to access the svn repository?

That is an option than I am going to check. Might be complicated, because I want to use a client that is integrated in the eclipse IDE.

Problem with ID mapping?

Sounds reasonable, since with nfs4 there came this idmapd daemon. I am going to check this.

----------

## krinn

nfsv4 define a root in exports, if it's not there, you aren't doing it right. I'm not sure if newer implementations has simply drop this or not, but older server was accepting it , the v4 standard explicit the need of root, the implementation without one was out of the standard and all distros was doing what they want. Result was full of bugs.

----------

## 666threesixes666

svn2git, as in exporting your repo to git.  then accessing git via git

 *Quote:*   

> dev-vcs/svn2git
> 
> Tool for one-time conversion from svn to git.

 

so was sshfs fuse a good thing?  im just throwing ideas out there.  ssh is encrypted so certainly more secure than nfs.  git by default uses ssh to push and pull commits.

https://www.eclipse.org/egit/

i guess its supported via egit?  i dont use eclipse so its just a possibility being stated as a possibility.

its all moot to me i haven't looked into any bench marked comparisons.

gentoo's git wiki's starting to get pretty saucy...  https://wiki.gentoo.org/wiki/Git

i like git because linus made it, and it revolves around diffs and is ultimately a patching system.  i like git because it doesnt require svn repo upstream, you can git a folder on  your desktop up and that is the upstream...  git flow of git 2 git instead of client svn-srv

----------

## Hu

For converting a Subversion repository to Git, I prefer using git svn, which ships with core Git and is available with USE=subversion.  However, OP may be happy with his legacy centralized VCS and prefer to solve the NFS slowness.  Older Subversion was very hard on inodes due to how the shadow information was stored.  Newer versions seem to be better, but may still be too heavy for use over NFS.  It would be useful to know why Subversion performs so poorly here.  OP, could you post a breakdown of number of each type of NFS request issued?

----------

## Alan Turing

I also defined a root in /etc/exports. I think the only difference is that you add fsid=0 to your root export in /etc/exports and use paths relative to this root in your mount options on the client.

So far I never noticed a difference between exporting a root via nfs or defining the exports in the old nfs3 style.

If the sshfs for me a good thing? I am indecisive. As written above, it was not always faster but it was also not always slower.

Converting the repository to GIT is not an option for me. I am neither the admin nor the only user of the repository.

I tried to obtain the NFS requests using nfsstat. Therefore, I rebootet the client to reset the counting of requests, did one "svn up" directly afterwards, and executed "nfsstat -l"

Using the following line in /etc/exports on the server

```
/storage             192.168.0.42(fsid=0,rw,async,no_root_squash,no_subtree_check)
```

and the following line in /etc/exports on the client

```
192.168.0.23:/  /mnt/nfs    nfs         noauto,nfsvers=4,retry=0,noatime,nocto,nolock,local_lock=all            0 0
```

I obtained the following output

```

bionseY ~ # nfsstat -l

nfs v4 client        total:    47171 

------------- ------------- --------

nfs v4 client         read:    16551 

nfs v4 client        write:       29 

nfs v4 client       commit:        2 

nfs v4 client         open:       17 

nfs v4 client    open_conf:        1 

nfs v4 client        close:        8 

nfs v4 client       fsinfo:        5 

nfs v4 client        renew:        3 

nfs v4 client    setclntid:        3 

nfs v4 client      confirm:        3 

nfs v4 client         lock:     6594 

nfs v4 client        locku:     6573 

nfs v4 client       access:     3272 

nfs v4 client      getattr:     3395 

nfs v4 client       lookup:     3260 

nfs v4 client  lookup_root:        2 

nfs v4 client       remove:        7 

nfs v4 client     pathconf:        3 

nfs v4 client       statfs:        8 

nfs v4 client      readdir:     4144 

nfs v4 client  server_caps:        8 

nfs v4 client  rel_lkowner:     3283 

```

Afterwards I did the same with nfs3 and obtained the following output.

```

bionseY ~ # nfsstat -l

nfs v3 client        total:    30318 

------------- ------------- --------

nfs v3 client      getattr:     3321 

nfs v3 client       lookup:     3256 

nfs v3 client       access:     3260 

nfs v3 client         read:    16561 

nfs v3 client        write:       27 

nfs v3 client       create:        7 

nfs v3 client       remove:        7 

nfs v3 client  readdirplus:     3870 

nfs v3 client       fsstat:        4 

nfs v3 client       fsinfo:        2 

nfs v3 client     pathconf:        1 

nfs v3 client       commit:        2 

```

----------

