# no gigabit performance

## snIP3r

hi all!

i have some issue with my gigabit network connection. if i copy files from my gentoo box to a client (e.g. win xp) via samba, i get only about 20mb/s read or write. ftp performance is 30mb/s server-client and 10mb/s client-server. all network hardware is gigabit capable. first i tried netio to test the max. possible network performance:

```

netio gentoo server, client winxp, tcp

Packet size  1k bytes:  62801 KByte/s Tx,  64156 KByte/s Rx.

Packet size  2k bytes:  66139 KByte/s Tx,  64863 KByte/s Rx.

Packet size  4k bytes:  101237 KByte/s Tx,  65537 KByte/s Rx.

Packet size  8k bytes:  110993 KByte/s Tx,  72451 KByte/s Rx.

Packet size 16k bytes:  112051 KByte/s Tx,  74262 KByte/s Rx.

Packet size 32k bytes:  112765 KByte/s Tx,  76289 KByte/s Rx.

netio server winxp, client gentoo server, tcp

Packet size  1k bytes:  68246 KByte/s Tx,  48891 KByte/s Rx.

Packet size  2k bytes:  68599 KByte/s Tx,  54008 KByte/s Rx.

Packet size  4k bytes:  69586 KByte/s Tx,  82032 KByte/s Rx.

Packet size  8k bytes:  72196 KByte/s Tx,  89593 KByte/s Rx.

Packet size 16k bytes:  73426 KByte/s Tx,  95149 KByte/s Rx.

Packet size 32k bytes:  73528 KByte/s Tx,  99191 KByte/s Rx.

```

this test tells me, that higher transfer speeds are possible. but i also have done some tests to test io systems on both sides:

```

iozone winxp c: (truecrypt aes, 256 bit, xts)

C:\Programme\benchmarks\Iozone 3.321>iozone -s 5120m -r 64 -i 0 -i 1 -t 1

        Iozone: Performance Test of File I/O

                Version $Revision: 3.321 $

                Compiled for 32 bit mode.

                Build: Windows

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins

                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss

                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,

                     Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,

                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,

                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.

        Run began: Sat Sep  5 16:40:09 2009

        File size set to 5242880 KB

        Record Size 64 KB

        Command line used: iozone -s 5120m -r 64 -i 0 -i 1 -t 1

        Output is in Kbytes/sec

        Time Resolution = 0.000000 seconds.

        Processor cache size set to 1024 Kbytes.

        Processor cache line size set to 32 bytes.

        File stride size set to 17 * record size.

        Throughput test with 1 process

        Each process writes a 5242880 Kbyte file in 64 Kbyte records

        Children see throughput for  1 initial writers  =   46491.52 KB/sec

        Parent sees throughput for  1 initial writers   =   46470.68 KB/sec

        Min throughput per process                      =   46491.52 KB/sec

        Max throughput per process                      =   46491.52 KB/sec

        Avg throughput per process                      =   46491.52 KB/sec

        Min xfer                                        = 5242880.00 KB

        Children see throughput for  1 rewriters        =   49502.93 KB/sec

        Parent sees throughput for  1 rewriters         =   49333.97 KB/sec

        Min throughput per process                      =   49502.93 KB/sec

        Max throughput per process                      =   49502.93 KB/sec

        Avg throughput per process                      =   49502.93 KB/sec

        Min xfer                                        = 5242880.00 KB

        Children see throughput for  1 readers          =   68432.46 KB/sec

        Parent sees throughput for  1 readers           =   68411.34 KB/sec

        Min throughput per process                      =   68432.46 KB/sec

        Max throughput per process                      =   68432.46 KB/sec

        Avg throughput per process                      =   68432.46 KB/sec

        Min xfer                                        = 5242880.00 KB

        Children see throughput for 1 re-readers        =   62831.66 KB/sec

        Parent sees throughput for 1 re-readers         =   62807.11 KB/sec

        Min throughput per process                      =   62831.66 KB/sec

        Max throughput per process                      =   62831.66 KB/sec

        Avg throughput per process                      =   62831.66 KB/sec

        Min xfer                                        = 5242880.00 KB

iozone winxp e: (no encryption)

E:\iozone>iozone -s 5120m -r 64 -i 0 -i 1 -t 1

        Iozone: Performance Test of File I/O

                Version $Revision: 3.321 $

                Compiled for 32 bit mode.

                Build: Windows

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins

                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss

                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,

                     Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,

                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,

                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.

        Run began: Sat Sep  5 16:40:09 2009

        File size set to 5242880 KB

        Record Size 64 KB

        Command line used: iozone -s 5120m -r 64 -i 0 -i 1 -t 1

        Output is in Kbytes/sec

        Time Resolution = 0.000000 seconds.

        Processor cache size set to 1024 Kbytes.

        Processor cache line size set to 32 bytes.

        File stride size set to 17 * record size.

        Throughput test with 1 process

        Each process writes a 5242880 Kbyte file in 64 Kbyte records

        Children see throughput for  1 initial writers  =   46491.52 KB/sec

        Parent sees throughput for  1 initial writers   =   46470.68 KB/sec

        Min throughput per process                      =   46491.52 KB/sec

        Max throughput per process                      =   46491.52 KB/sec

        Avg throughput per process                      =   46491.52 KB/sec

        Min xfer                                        = 5242880.00 KB

        Children see throughput for  1 rewriters        =   49502.93 KB/sec

        Parent sees throughput for  1 rewriters         =   49333.97 KB/sec

        Min throughput per process                      =   49502.93 KB/sec

        Max throughput per process                      =   49502.93 KB/sec

        Avg throughput per process                      =   49502.93 KB/sec

        Min xfer                                        = 5242880.00 KB

        Children see throughput for  1 readers          =   68432.46 KB/sec

        Parent sees throughput for  1 readers           =   68411.34 KB/sec

        Min throughput per process                      =   68432.46 KB/sec

        Max throughput per process                      =   68432.46 KB/sec

        Avg throughput per process                      =   68432.46 KB/sec

        Min xfer                                        = 5242880.00 KB

        Children see throughput for 1 re-readers        =   62831.66 KB/sec

        Parent sees throughput for 1 re-readers         =   62807.11 KB/sec

        Min throughput per process                      =   62831.66 KB/sec

        Max throughput per process                      =   62831.66 KB/sec

        Avg throughput per process                      =   62831.66 KB/sec

        Min xfer                                        = 5242880.00 KB

iozone gentoo server / partition with dm-crypt aes-lrw-benbi, 256 bit, xfs filesystem, hardware raid5 4drives wd 3200ys, 3ware 9650se with 256mb cache

area52 stuff # iozone -s 6144m -r 64 -i 0 -i 1 -t 1

        Iozone: Performance Test of File I/O

                Version $Revision: 3.242 $

                Compiled for 64 bit mode.

                Build: linux-AMD64

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins

                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss

                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,

                     Randy Dunlap, Mark Montague, Dan Million,

                     Jean-Marc Zucconi, Jeff Blomberg,

                     Erik Habbinga, Kris Strecker, Walter Wong.

        Run began: Sat Sep  5 16:43:07 2009

        File size set to 6291456 KB

        Record Size 64 KB

        Command line used: iozone -s 6144m -r 64 -i 0 -i 1 -t 1

        Output is in Kbytes/sec

        Time Resolution = 0.000001 seconds.

        Processor cache size set to 1024 Kbytes.

        Processor cache line size set to 32 bytes.

        File stride size set to 17 * record size.

        Throughput test with 1 process

        Each process writes a 6291456 Kbyte file in 64 Kbyte records

        Children see throughput for  1 initial writers  =   60889.49 KB/sec

        Parent sees throughput for  1 initial writers   =   59922.82 KB/sec

        Min throughput per process                      =   60889.49 KB/sec

        Max throughput per process                      =   60889.49 KB/sec

        Avg throughput per process                      =   60889.49 KB/sec

        Min xfer                                        = 6291456.00 KB

        Children see throughput for  1 rewriters        =   78183.15 KB/sec

        Parent sees throughput for  1 rewriters         =   72808.38 KB/sec

        Min throughput per process                      =   78183.15 KB/sec

        Max throughput per process                      =   78183.15 KB/sec

        Avg throughput per process                      =   78183.15 KB/sec

        Min xfer                                        = 6291456.00 KB

        Children see throughput for  1 readers          =   50457.01 KB/sec

        Parent sees throughput for  1 readers           =   50454.23 KB/sec

        Min throughput per process                      =   50457.01 KB/sec

        Max throughput per process                      =   50457.01 KB/sec

        Avg throughput per process                      =   50457.01 KB/sec

        Min xfer                                        = 6291456.00 KB

        Children see throughput for 1 re-readers        =   48836.97 KB/sec

        Parent sees throughput for 1 re-readers         =   48836.87 KB/sec

        Min throughput per process                      =   48836.97 KB/sec

        Max throughput per process                      =   48836.97 KB/sec

        Avg throughput per process                      =   48836.97 KB/sec

        Min xfer                                        = 6291456.00 KB

```

here we can also see, that more speed is possible.

my reelvant smb.conf settings:

```

socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192

use sendfile = no

```

can someone perhaps help me with this? what can i do to improve performance?

thx in advance

snIP3r

----------

## poly_poly-man

samba tends to be crazy slow... can you do tests with ftp? then we might be able to help your samba performance.

----------

## snIP3r

 *poly_poly-man wrote:*   

> samba tends to be crazy slow... can you do tests with ftp? then we might be able to help your samba performance.

 

hi!

i already did some tests and these are the results:

```

ftp performance is 30mb/s server-client and 10mb/s client-server

```

this is very crazy cause read performance (getting file from server) is good, but write performance (saving file to server) is very poor...

greets

snIP3r

----------

## poly_poly-man

that sounds about right for a half-duplex 100mbps line... can you give ifconfig -a on the linux machine (and the equivalent on the windows machine)?

----------

## snIP3r

 *poly_poly-man wrote:*   

> that sounds about right for a half-duplex 100mbps line... can you give ifconfig -a on the linux machine (and the equivalent on the windows machine)?

 

ifconfig -a does not show the info you want. i assume you mean ethtool:

 *Quote:*   

> 
> 
> area52 ~ # ethtool eth1
> 
> Settings for eth1:
> ...

 

and the relevant windows infos: http://area52.kicks-ass.org/sniper/images/yukon.jpg

MTU is set to 1500 on both eth devices.

thx for help

snIP3r

EDIT: i saw someone accessing the pic with error code 403 - this was mod security with message Request Missing a

 User Agent Header. mod_security is disabled temporarily  :Wink: 

----------

## snIP3r

*bump*

i still no idea how to increase network performance for filetransfers win-linux...   :Question: 

----------

## Mousee

Unfortunately there's far too many "what if's" with what we currently know about your network situation. Examples:

Are all of the computers connected using, at minimum, CAT5e cabling?

Does the router/switch you're using support Jumbo Frames? Do all of the network cards in your network support Jumbo Frames?

Do you have Jumbo Frames enabled and set to their max size (9K is typical) on all of your computers, specifically the Windows computer?

Are there any services, hardware faults, etc that may be causing these slow speeds that we are currently unaware of?

These are things you need to check into first.

It could also be a faulty network card or driver (including the one built-in to the kernel) that is causing the slow speeds.

Anyways, I would begin with testing hardware first and then possibly even try booting both machines with a LiveCD (ie. SystemRescueCD) and attempt to transfer files between the two machines from there. That way you'll know if it's a hardware of software bottleneck.

----------

## snIP3r

hi!

thx for spending time with my issue. here are some answers to your "if's":

1. yes, all cables are >= cat5e (they are cat6)

2. as far as i can get it from the dokus all components support jumbo frames (nics are intel pro1000mt, marvell yukon 88e8053 and switch is dlink dgs-1008d)

3. i am sure that there is no hardware fault (already changed the intel marvell drivers with no better results). the service i use is samba, getting its files from an dm encrypted filesystem using xfs (speed test results in first post).

thx for help in advance

snIP3r

----------

## Cyker

The problem is Samba itself; I have the same issue, and I've tried optimizing it up the proverbial!

Even SFTP is faster!  :Sad: 

There ain't much in the way of tunables you can play with to improve performance and bring it up to the speed of, say, netcat  :Razz: .

You've done the main things; The other one is enabling asynchronous io and setting the aio read/write sizes to 1, but tbh it made very little difference for me,

I have tried optimizing my RAID filesystem but even with massive buffering, the bottleneck still seems to be Samba...

I'm still using a relatively old version of Samba tho'. 3.2 and up still ain't stable...

----------

## snIP3r

 *Cyker wrote:*   

> The problem is Samba itself; I have the same issue, and I've tried optimizing it up the proverbial!
> 
> Even SFTP is faster! 
> 
> There ain't much in the way of tunables you can play with to improve performance and bring it up to the speed of, say, netcat .
> ...

 

hi cyker!

thx for the reply. how about using nfs server - i think there might be a way to enable nfs client in winxp...

thx

snIP3r

----------

## Cyker

I've never tried but I'm pretty sure you can as my BSD-using friend uses NFS (And often mocks me for using Samba and also ripping off their Ports system  :Razz: ).

My main problem is I'm too lazy to figure out how to set it up  :Razz: 

----------

## gentoo_ram

 *snIP3r wrote:*   

>  *Cyker wrote:*   The problem is Samba itself; I have the same issue, and I've tried optimizing it up the proverbial!
> 
>  
> 
> thx for the reply. how about using nfs server - i think there might be a way to enable nfs client in winxp... 

 

Unfortunately, NFS totally sucks on XP.  I've tried it, it wasn't a good experience.  Samba is going to be the most compatible way to share files with Windows.  I haven't found a good way to get around its performance issues.

I've gotten 50-60 MB /second over NFS (on Linux) and only 25MB/sec max over Samba between my server and my main gaming machine.

----------

