# NFS - write perfomance changes between 50Mb/s and 0Kb/s

## Finswimmer

Hi,

I've got a network with various Linux computers. They use nfs to exchange data.

When I read from the server the speed is very good, about 4-5Mbyte/s. But if I try to write to the server, the process starts with a speed of about 20Mbyte/s or even more and then drops down to zero. After a half a minute it starts the same from the beginning.

The main problem is that over all it is very slow and that this behaviour leaves my desktop pc nearly unusuable.

I've tried to search for this problem, but I didn't succeed. The problem is the same no matter if I use cp or konqueror.

Do you have any ideas? I've also tried to change the mount-options on my local pc but it didn't help.

I think that there is some kind of buffer which is used, but I don't want it. I'd prefer a constant speed of maybe 4Mbyte/s rather than these various speeds.

Thanks in advance for any ideas and comments.

Regards Finswimmer

P.s.:

I don't know if it matters, but:

Server: 2.4; Desktop: 2.6[/list]

----------

## fctk

mmh... i had a similar problem but i used samba and cifs... https://forums.gentoo.org/viewtopic.php?p=2600169

anyway, it could be a bug in the kernel, so you can try to update/downgrade it...

----------

## luisfelipe

Try searching for information on the flags you can use when mounting the partition (they are specified

on /etc/fstab). Some of them can make a big change on performance. 

This link should help :

http://nfs.sourceforge.net/nfs-howto/performance.html

----------

## dev-urandom

@Finswimmer, what are the nfs mount options that you use?

----------

## Finswimmer

Hi,

thanks for your answers.

I tried various mount options, but none of them helped:

rsize=8192,wsize=8192,wdelay,tcp

I tried them one by one and together, but no difference.

For me it seems that first a local buffer is used, which would explain why the copying process starts with up to 50Mbyte/s and then the real copying process begins. I could prove this idea by using iptraf. The write performance to the server is always about 3-4Mbyte/s, no matter what the KDE file dialog or cp says.

Any ideas left?  :Smile: 

Regards, Finswimmer

----------

## dev-urandom

Your options look ok, a lot of servers run amazingly good with them ... 

What is the server? Is it running NFS v4 or something exotic?

Is your network crowded? Do you generally get packet drops? Mainly does your server drop packets?

is iptables doing anything funny? ( I dont think this is the case, but you never know  :Wink:  )

Can you try copying some small file to see whether it makes it through.  It could be a permissions problem, combined with some gui goofups.

 *Finswimmer wrote:*   

> Hi,
> 
> thanks for your answers.
> 
> I tried various mount options, but none of them helped:
> ...

 

----------

## Finswimmer

Hi,

actually I receive a lot of RX errors, but if I copy with SMB the speed is ok. And I can see with iptraf that the nfs speed is also ok.

The only problem is, that there is a cache which is fully written and then copied and so on. And this is the behaviour, I don't want.

Small files behave the same: High speed at first and then the speed drops down to zero at 100% copied data. Then it waits for some seconds and the copy process is completed.

Regards Finswimmer

----------

## dev-urandom

Hmm,

Haven't really seen this issue, so i can't pin point it.

Hope that your server isn't heavily loaded.  Just your one machine facing this problem right?

I remember having read about some issues with NFS v4 and locking thats similiar to what you say.  

Can you try with only NFS v3 if you happen to use NFS v4?  I cant garuntee that this will solve your issues, but its worth  a shot.  

Maybe someone here might come up with something better  :Very Happy: 

----------

## Caustiq

Hello, I'm experiencing the same write behavior with NFSv3, (kernel v2.6 for both client and server).

My issues are just as Finswimmer explains, downstream is steady at about 4MB/s the entire transfer, upstream begins skyrocketing up to 20MB/s and even higher, then drops right down to near zero, then some time later (about 30s) shoots up, then back down, etc, etc.   :Confused: 

Here are the relevant mount options I am using: rsize=8192,wsize=8192,timeo=14,intr,nfsvers=3,tcp,rw

/etc/exports line on server: /home/caustiq darktower(async,no_subtree_check,rw)

This happens on both of my NFS servers on the LAN (which no one uses except myself). Both NFS servers are configured the same.

Client hard drive performance:

 *Quote:*   

> darktower ~ # hdparm -tT /dev/sda
> 
> /dev/sda:
> 
>  Timing cached reads:   3812 MB in  2.00 seconds = 1906.29 MB/sec
> ...

 

Server hard drive performance:

 *Quote:*   

> 
> 
> shardik ~ # hdparm -tT /dev/hda
> 
> /dev/hda:
> ...

 

Client ethtool info:

 *Quote:*   

> darktower ~ # ethtool eth0
> 
> Settings for eth0:
> 
>         Supported ports: [ TP ]
> ...

 

Server ethtool info:

 *Quote:*   

> shardik ~ # ethtool eth0
> 
> Settings for eth0:
> 
>         Supported ports: [ TP MII ]
> ...

 

This is really frustrating. I searched this forum very well and could not find any one with similiar issues until I found this thread.

----------

## DarkMind

same here....

when i trie to write to the server, the write is very slow (190 kbps  :Shocked:  ) (with a 100 mbps network/full duplex)

----------

## Flamie

Howdy fellas!

I had the same problem, and the mount option async solved it and after some more reading I decided to stick with this that has worked like a charm ever since ;)

async,hard,intr,rw,nolock,rsize=8192,wsize=8192

----------

## Caustiq

I solved this issue I was having by enabling DMA on the hard drives of the problematic servers. I thought for sure I had checked DMA was enabled and working, must've had too much coffee that night or something.  :Smile:  Now I'm getting a solid 11-12MB/s (or about 95% of the full 100Mb link) as I should. 

Edit: man hdparm to learn how to enable DMA.

----------

## jamapii

 *Flamie wrote:*   

> async,hard,intr,rw,nolock,rsize=8192,wsize=8192

 

I solved NFS problems by setting rsize=1024,wsize=1024. But my symptoms were different, NFS just hung forever. It seems this applies more often if you have more than 1 network card in one of the PCs.

----------

## vf1sveritech

I am having the same problem with my nfs and writing, with super high speeds at first and than stalling out, and when it stalls it causes reading to stall as well. For instance playing music or movies and trying to write causes the media to stop, and the programs will just sit their until it resumes which can be very short or several minutes.

However FTP and writing works at full speed and stays constant 10MB/s AND does not disrupt streaming media.

I checked ethtools on both machines and both are full duplex and have auto-negotiation on. Both machine's kernels have server and client support for v3 and nfs-utils installed, and portmapping on and everything. I have reset the switch on many occasions, remounted, restarted portmapping, and nfs, and nothing seems to fix it.

----------

## depontius

Back in late August there was a discussion about NFS stalls on the Linux Kernel Mailing List.  It appears to have kicked in with 2.6.25, and there was some bisection work done that I believe it was somehow related to the ACL code.  The stalls aren't merely write related, they're just plain NFS, itself.

I have no idea what the status of this is.  I occasionally search LKML for "nfs stall" and find the original thread, but nothing new.  One of these days I'm going to get the guts to post on LKML and ask if there has been any resolution to this problem checked in.

----------

