# Poor (?) speed in samba when running gigabit network

## MrWorf

I have a fileserver with the following specs:

```
2.4Ghz Celeron

512MB ram

6 x 7200rpm IDE drive, each on their on IDE channel (all are running as masters)

Intel 875 chipset

Intel 1Gb CSA integrated networkcard

Linux 2.6.7

Samba 3.0.4-r1
```

My workstation has the following specs:

```
2.8Ghz P4

1GB ram

120GB 7200rpm IDE drive

Intel 865 chipset

Intel 1Gb CSA integrated networkcard

WinXP Pro SP1

Latest drivers and updates
```

The network is Cat5e based (shielded) with a D-Link 8port gigabitswitch at the heart,

My linux fileserver runs five of the six drives as a raid5 array. This array is capable of saturating the PCI bus (bonnie++ tests shows a sustained 60-90MB/s read/write performance).

However, when transfering from my WinXP box to the linuxbox I only get 25MB/s ... And when doing it, my XP machine tops out at 80-90% CPU usage while my fileserver runs at a low 30-40%. I know the harddrive in my workstation is capable of ~40MB/s.

So why this "poor" performance, why this difference in CPU usage?

To my understanding the CSA bus does not use the PCI bus and should thus not interfere with IDE drive performance (which is PCI bound).

I've done tests with pure network performance tools between these machines and they have no problems at all with pushing 900-990Mbit/s in either direction, and the CPU usage wasn't that high (however, I've missplaced the program and testresults when writing this   :Confused:  so no results can be posted at this time, will try to get some for tomorrow ... and it was a few weeks since I did the tests  :Rolling Eyes:  so I might remember incorrectly  :Embarassed: )

Jumboframes was an idea to increase the speed (maybe the XP's CPU usage is due to the massive amount of interrupts/packets?) but this lead to severe linux problems (see https://forums.gentoo.org/viewtopic.php?t=197950), so I haven't been able to test this theory yet.

Any ideas, clues, suggestions, (solutions?  :Shocked:   :Very Happy:  ) would be appreciated.

----------

## nobspangle

25MB/s sounds about right for a windows PC with a single IDE harddrive

----------

## Monkeywrench

not trying to derail the thread... but I have a kind of related question: how come transfering files over a LAN requires CPU time? When I transfer files via SSH, my main desktop (athlon xp 2000+) goes to something like 40%, and my other headless folding@home box (pentium 3 450mhz) goes to 100%. I asked a friend about this, and he suggested it was the encryption. But I didn't think Samba had encryption?

And another little thing: how far from the theoretical maximum do transfer speeds usually fall? MrWorf is only getting 25 MB/s over a gigabit network. Is this normal performance?  :Embarassed: 

----------

## nobspangle

CPU time is required for everything, especially accessing IDE harddrives, SSH transfers will be worse as they have to deal with encryption as well.

Mr. Worf is a long way off the theoretical maximum of his network because he is writing to a single IDE harddrive. If he was writing to a large SCSI RAID array he would more than likely achieve over 100MB/s

----------

## MrWorf

 *nobspangle wrote:*   

> Mr. Worf is a long way off the theoretical maximum of his network because he is writing to a single IDE harddrive. If he was writing to a large SCSI RAID array he would more than likely achieve over 100MB/s

 

Uhm, you need to read my post correctly:

 *MrWorf wrote:*   

> However, when transfering from my WinXP box to the linuxbox I only get 25MB/s ... And when doing it, my XP machine tops out at 80-90% CPU usage while my fileserver runs at a low 30-40%. I know the harddrive in my workstation is capable of ~40MB/s. 

 

and

 *Quote:*   

> My linux fileserver runs five of the six drives as a raid5 array. This array is capable of saturating the PCI bus (bonnie++ tests shows a sustained 60-90MB/s read/write performance). 

 

If it was the otherway around, the answer would be logical, but this, however, is not   :Wink: 

----------

## nobspangle

sorry I did miss read your post I thought you were transfering the other way.

However I still think that expecting more than 25MB/s on average from a single IDE harddisk in Windows XP is pushing things. If you had two disks with your system/swap on one and you were doing read writes to the other you might see a small improvment. However IDE is fairly flawed since only one drive can be read/written to at any one time.

Also there is a big difference in the performance of a software IDE RAID5 array and a hardware SCSI RAID5 array.

----------

## MrWorf

I don't know what to do to convince you that the systems are more than able to handle the speed I want.

Here is an output from bonnie++ running all benchmarks with a 1GB filesize to avoid memorycache on my fileserver (only 512MB)

```
Using uid:0, gid:0.

Writing with putc()...done

Writing intelligently...done

Rewriting...done

Reading with getc()...done

Reading intelligently...done

start 'em...done...done...done...

Create files in sequential order...done.

Stat files in sequential order...done.

Delete files in sequential order...done.

Create files in random order...done.

Stat files in random order...done.

Delete files in random order...done.

Version  1.03       ------Sequential Output------ --Sequential Input- --Random-

                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--

Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP

nerv             1G 15748  86 59520  20 27151  11 18424  87 82762  17 434.9   2

                    ------Sequential Create------ --------Random Create--------

                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--

              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP

                 16  1575  79 +++++ +++ 20072  18  1994  98 +++++ +++  4661  95

```

This benchmark was taken when the fileserver was running normally, so they would probably be a bit higher if I stopped samba and nfs (but this ought to be more correct). This benchmark shows a 59MB/s writespeed with 20% cpu usage (remember, this is raid5, so the XOR operations is probably account for some of the cpu use here). Reading gives a massive 82MB/s with a cpu usage of 17%.

So I'd say that my IDE based "slow"  :Rolling Eyes:  raid5 is quite up to speed for receiving 40+ MB/s and delivering 60+ MB/s over the PCI bus  :Very Happy: 

The real limit of how far I can push this is probably more due to PCI bus bandwidth limitations and/or driver issues, than the five 120GB 7200rpm IDE drives.

Unfortunatly, I'm at work, so a similar benchmark för WinXP's driveperformance can not be done at the moment. But I will try and make those benchmarks later tonight when I get home. However, swap writes during the transfer from my WinXP box should not be an issue since I wasn't running any extra programs during the measurements and I have a gig of ram (physical ram, not including swap) so that render swap unused during filetransfer (never heard of swap being used when transfering files).

 *Quote:*   

> However IDE is fairly flawed since only one drive can be read/written to at any one time. 

 

This only holds true when you put two IDE drives on one cable. When you run a single drive per channel, you get very good speed(tm)   :Wink:   This is why two sata drives will beat two pata drives if the pata drives are on one cable (master, slave configuration).

 *Quote:*   

> Also there is a big difference in the performance of a software IDE RAID5 array and a hardware SCSI RAID5 array.

 

No denying that, but todays IDE drives are alot faster than they used to be. The main problem with IDE is twofold:

1) They aren't as reliable

2) They use more CPU cycles since the host has to do more processing

The main advantage is that they are cheap  :Very Happy:  ... and I mean dirtcheap  :Razz:   :Cool: 

----------

## nobspangle

I don't deny that 5 drives in a RAID 5 array are more than capable of swamping a 33MHz PCI bus. My point is that a single IDE harddrive running under windows XP will only give 25-30MB/s maximum in a real world situation (not a benchmark)

----------

## MrWorf

 *nobspangle wrote:*   

> I don't deny that 5 drives in a RAID 5 array are more than capable of swamping a 33MHz PCI bus. My point is that a single IDE harddrive running under windows XP will only give 25-30MB/s maximum in a real world situation (not a benchmark)

 

That might be true, but a large filecopy (such as 1GB or so) would probably come close to what the benchmark does. ie, read a large file as fast as possible while squirting it down the ethernet pipe   :Wink:  But I'll see if I can run bonnie++ on WinXP aswell, atleast it should give some indication of what speed it should muster.

But there is another interesting thing about this. And that is that my fileserver has a lower cpu utilization than my workstation when sending files from the workstation. Even if the drive is the limiting factor on the workstation, one would think that the raid5 software raid on the linux make for it having a higher cpu utilization.

Ohwell....

----------

## nobspangle

you have a workstation running windows and a server running linux and you're suprised that the workstation has higher cpu utilization.

----------

## MacMasta

My professional opinion is that a single IDE drive, esp. in windows, is going to have a hard time pushing more than about 30 meg/sec, and that windows' implementation of SMB is so bad that it's no suprise at all that it's sucking on the CPU.

Out of curiosity, how badly fragmented is the windows drive? It might not be able to maintain one read; seek times could put a real hurt on your transfer speed.

~Mac~

----------

## MrWorf

 *nobspangle wrote:*   

> you have a workstation running windows and a server running linux and you're suprised that the workstation has higher cpu utilization.

 

Well, yes  :Smile:  Since the workstation has a P4 instead of a Celeron, 2.8Ghz instead of 2.4Ghz, 1GB ram instead of 512MB. And the server has to do software raid5 which adds some overhead (parity calculations, etc).

 *MacMasta wrote:*   

> Out of curiosity, how badly fragmented is the windows drive? It might not be able to maintain one read; seek times could put a real hurt on your transfer speed. 

 

The file in question (a vobfile) is not fragmented (checked with the defragger on WinXP).

 *Quote:*   

> My professional opinion is that a single IDE drive, esp. in windows, is going to have a hard time pushing more than about 30 meg/sec, and that windows' implementation of SMB is so bad that it's no suprise at all that it's sucking on the CPU. 

 

I guess you're right. But I still find it a bit sad. I was expecting a bit more from it than this. Ohwell... I guess I need to go raid on WinXP too to increase drive speeds  :Wink: 

----------

## Scotty49

Have you ever tried to disable the QoS Packet Planner (at least in German Windows XP it's called like this) in your Windows Network Config? For me this brought a giant speed increase.

Scotty

----------

## MrWorf

 *Scotty49 wrote:*   

> Have you ever tried to disable the QoS Packet Planner (at least in German Windows XP it's called like this) in your Windows Network Config? For me this brought a giant speed increase.

 

Yepp, already did that. Also tried to use jumboframes, but when turning that on on the server, it starts to produce strange paging errors and processes start to hang (not crash) like never before.

----------

## think4urs11

try the transfers with

a) SMB

b) NFS

c) FTP

d) SCP/SFTP

most probably you'll see significant differences in terms of transfer speed.

(i'd say  c->b->a->d  from fast to slow)

FTP should give you around 85-90% of theoretical maximum at least.

----------

