# SMP problem on Core2 Duo and Dual Core EM64T

## paziu

Hello,

I am running a dual Xeon 5060 ( total 4 cores ) system. amd64 2006.1 mini install cd was used to install it. Both processor family processor types (generic x86_64 and amd64) was tried to have this system running.. ( SMP enabled, ARCH=nocona or amd64 ) - everything looks good.

  My kenel module uses netfilter, when runnning on the Xeon system, I see all CPUs active and processig the packets ( I am using a packet generator to send/receive packets to/from Intel PRO/1000 dual port PCI-E network interface(s) )

This is the problem with the Core 2 Duo system ( 2 cores - single CPU ) : 

When I run the same kernel ( 2.6.18-r2 ) with the same configuration of the kernel, also with the same network interfaces, when packet generator is being active, I show only CPU 0 ( core 0 ) active. the CPU1 is 100% idle. the affinity on the IRQs is set the to 3 ( CPU0 & 1 ) The only way of making the other CPU busy is setting the affinity on the IRQ being used by one of the network interfaces to 2 ( CPU1 ) - On the Xeon system, it automatically uses all the cores ( 4 cores with hyperthreading = 8 CPUs shown in top )

I have the same problem with the dual core Intel EM64T cpu system

It seems to me that the Xeon is the only one at this moment that is able to use all CPUs at a time without manually assigning the IRQs to a core

Thank you very much for all your suggestions and comments,

Mike Fiedler

----------

## teapot

maybe the problem is that you use the same kernel config for both systems?

you need to tell us more...

does this only happen when simulating network traffic or is the second core 100% idle no matter which programs you run?

----------

## teapot

I don't think you're supposed to use Hyperthreading scheduling on Core 2 Duo. I don't do that and both my cores are being used...

----------

## paziu

Hello and thank you for your reply,

I am using "same CPU/System profile" and same gcc flags since both are 64bit. The rest of the kernel config is also almost the same. ( different  SATA controller / USB ) nothing really else ( i do have sound disabled on both )

During Compilation I see several instances of gcc compiler running at the same time ( MAKEOPTS = j5 on the xeon and -j3 on the Core2 Duo )

All user space processes on both systems are running on all available CPUs good, I do not see any problem with this - only when kernel is active ( when using the packet generator ) on the Xeon I see all CPUs running and are < 100 idle ( 20-40% and more in use ) but on the Core2 Duo and dual core em64t there is only CPU0 being used when the kernel/drivers busy processing the packets sent from a different machine running an UDP packet generator.

The 2nd core is 100% idle unless I change the affinity manually on the IRQ being used by one of the interfaces ( 2 dual Intel Pro/1000 PCI-E 4x)

The user space processes are running on all cores on all systems, but only the xeon system is able to handle kernel on all available CPUs

It happens pnly when simulating the network traffic ( using a different system to generate )

Thank you and please let me know what other information/config files would you like me to provide (if), 

Mike

 *teapot wrote:*   

> maybe the problem is that you use the same kernel config for both systems?
> 
> you need to tell us more...
> 
> does this only happen when simulating network traffic or is the second core 100% idle no matter which programs you run?

 Last edited by paziu on Fri Nov 24, 2006 4:45 pm; edited 1 time in total

----------

## paziu

Hi,

Both cores are being used fine when user space processes are running - when the kernel becomes loaded, it consumes only one core on the core2 duo and em64t systems  -  xeon system uses all the cores all the time, when user space processes are running and when the kernel becomes busy.

It becomes quite busy when sending 100000 packets/sec to each 1 Gbit interface which are 1024-600 bytes in size 

The core2 duo and em64t have no HyperThreading on the CPUs, but I think I have it enabled in the kernel config.

Thank you,

Mike

 *teapot wrote:*   

> I don't think you're supposed to use Hyperthreading scheduling on Core 2 Duo. I don't do that and both my cores are being used...

 

----------

## teapot

I'm not really into kernel scheduling. 

This is the only help I can give you. 

Try to disable hyperthreading support and read about multicore and multi-cpu scheduling in kernel 2.6

and which options in the kernel they relate to.

If you post your kernel config we get more details to work with.

----------

## paziu

Hello,

I did compress .config / make.conf / and output from /proc/cpuinfo on the dual core em64t box - here is the link :

http://depositfiles.com/en/files/402802

In the meantime I did change the kernel a few times, no difference - only 1 core active unless affinity changed manually.

Thank you!

Mike

 *teapot wrote:*   

> I'm not really into kernel scheduling. 
> 
> This is the only help I can give you. 
> 
> Try to disable hyperthreading support and read about multicore and multi-cpu scheduling in kernel 2.6
> ...

 

----------

## teapot

hmm... I'm confused. 

You say you have a Core 2 duo , but cpuinfo says otherwise: 

 *Quote:*   

> model name      :               Intel(R) Pentium(R) D CPU 3.20GHz
> 
> stepping        : 4
> 
> cpu MHz         : 3192.092

 

If it is a core duo. It is heavily overclocked...

The only thing I can come up with is to change the I/O scheduler under "Block Devices" 

Try that...

----------

## paziu

Hello,

This zip contains files from Dual Core EM64T system ( Pentium D 3.2 GHz ),

The second system I have problems with is Croe 2 Duo, both cannot handle SMP right ( the kernel usage is seen only on the first core )

Thank you,

Mike

 *teapot wrote:*   

> hmm... I'm confused. 
> 
> You say you have a Core 2 duo , but cpuinfo says otherwise: 
> 
>  *Quote:*   model name      :               Intel(R) Pentium(R) D CPU 3.20GHz
> ...

 

----------

## teapot

Did you change the I/O scheduler?

----------

## paziu

Yes, I did try NOOP, deadline, cfq... no chanege on the CPU usage,

I had also Hyperthreading scheduler tested while enabled and disabled...

Still Core0 ( CPU0 ) active upto 100% the second core usually 0% unless several user processes started....

The only way of getting both to work is manually setting the smp affinity to 01 02 ( cpu0 cpu1 ) on the IRQs occupied by the NICs...

By default it is set to 3 ( both CPUs ) - unfortunately it only uses CPU0 for some reason.

Thank you,

MIke

 *teapot wrote:*   

> Did you change the I/O scheduler?

 

----------

## teapot

It might be something wrong with the NIC driver. 

I'm out of ideas. 

If you don't find anyone to help you out in here I suggest you go to the drivers project site.

http://sourceforge.net/projects/e1000/

They can probably give you a good answer...

----------

## Joseph K.

This is quite a late reply, but I've finally managed to answer my own questions and thought others would like to know too.  What I mean is, if you're at all confused about whether or not your Intel CPU (Core 2 Duo in my case) supports Hyper-Threading Technology, read this: http://www.intel.com/products/ht/hyperthreading_more.htm

Thus, the Gentoo Wiki is correct in recommending not to enable HT Technology for a Core 2 Duo.  Could/should that be forced in the kernel configuration, to avoid confusion?  I haven't seen 2.6.21, but in 2.6.20 CPU selection it is a combined "Core 2 / newer Xeon" option.  If Core/Core 2 was distinct from Xeon, then HT could be made unavailable for it and remain available for Xeon.

For a technical description of SMP scheduling in Linux on Intel architecture, try this: http://oss.intel.com/pdf/mclinux.pdf

Sorry I can't help with your specific problems.  Cheers.

----------

## kernelOfTruth

disable irqbalance in kernel & try kernels with sd- or cfs-schedulers, (cfs- seems to balance load on several core better than sd [that's what gnome-system-monitor shows])

good luck

----------

