# cpusets

## redwood

Anybody using app-admin/cpusets from dev-zero overlay?

I first used eselect to select python-2.7, then emerged cpusets,

but there seems to be a path problem:

 # cset shield --cpu=7

cset: **> [Errno 2] No such file or directory: '/cpusets//cpus'

But I have /cgroup and /cpusets

# ls /

bin   cgroup   dev  home  lib32  lost+found  mnt  proc  sbin  tftpboot  usr

boot  cpusets  etc  lib   lib64  media       opt  root  sys   tmp       var

And for good measure I set the default path per "man cset" instructions

# cat /etc/cset.conf 

mountpoint = /cpusets

I've been trying to fix some intermittent dahdi/asterisk problems whereby calls from the PSTN trunk through my Digium TDM400P card occasionally have a choppy sound. 

I noticed that one user had multiple firefox tabs open with process CPU usage bouncing from 25%-80%. So I had her close out her firefox tabs and the choppiness seemed to disappear.

I have an 8-core AMD Magna-Cours (family 10h) processor which should be amply fast to run a pbx.

Previously I had dahdi/asterisk/freepbx running on an old AMD Athlon Thunderbird without problems --

but then the memory/cpu faulted and so I put the cards in the 8-core server. Things were running fine for the past year until  recent upgrades of dahdi/asterisk. I've now upgraded to dahdi-2.6 and asterisk-10.1.0

The PSTN->asterisk->sip phone problem is only intermittent, which is why when I noticed the high cpu usage on the firefox process, I thought that might be the problem. And killing firefox seemed to work. But most of my 8-cores are way underutilized:  

```

ATOP - apps               2012/02/20  18:52:14             2536 seconds elapsed

PRC | sys  20.52s | user  20.24s | #proc    202 | #zombie    0 | #exit      0 |

CPU | sys      1% | user      2% | irq       0% | idle    791% | wait      6% |

cpu | sys      0% | user      0% | irq       0% | idle     98% | cpu001 w  1% |

cpu | sys      1% | user      1% | irq       0% | idle     98% | cpu000 w  1% |

cpu | sys      0% | user      1% | irq       0% | idle     97% | cpu004 w  2% |

cpu | sys      0% | user      0% | irq       0% | idle     99% | cpu007 w  1% |

cpu | sys      0% | user      0% | irq       0% | idle     99% | cpu006 w  0% |

cpu | sys      0% | user      0% | irq       0% | idle     99% | cpu002 w  0% |

cpu | sys      0% | user      0% | irq       0% | idle    100% | cpu003 w  0% |

cpu | sys      0% | user      0% | irq       0% | idle    100% | cpu005 w  0% |

CPL | avg1   0.06 | avg5    0.22 | avg15   0.14 | csw  1199174 | intr 6471477 |

MEM | tot    7.8G | free    7.0G | cache 233.0M | buff  132.5M | slab   78.3M |

SWP | tot   20.0G | free   20.0G |              | vmcom 963.9M | vmlim  23.9G |

DSK |         sda | busy      5% | read   23779 | write   3054 | avio    4 ms |

DSK |         sdb | busy      4% | read   21997 | write   3054 | avio    4 ms |

DSK |         sdc | busy      0% | read     391 | write     12 | avio    1 ms |

NET | transport   | tcpi  347231 | tcpo  362253 | udpi    3165 | udpo    2987 |

NET | network     | ipi   350607 | ipo   354544 | ipfrw      0 | deliv 350511 |

NET | eth0     0% | pcki  351068 | pcko  365558 | si  120 Kbps | so  161 Kbps |

NET | br0    ---- | pcki  350691 | pcko  365322 | si  100 Kbps | so  156 Kbps |

NET | lo     ---- | pcki     503 | pcko     503 | si    0 Kbps | so    0 Kbps |

```

I thought maybe I could use cpusets to isolate dahdi/asterisk from misbehaving user tasks.

The app-admin/cpuset python wrapper is supposed to ease managing /sys/fs/cgroup/cpuset/ and /cpusets

Thanks in advance if anyone has used cpusets.Last edited by redwood on Tue Feb 21, 2012 3:36 am; edited 2 times in total

----------

## Hu

If CPU competition is the problem, it might be sufficient to give Asterisk a more favorable nice value.

----------

## redwood

I'm using the following  NICE value:

# grep ASTERISK_NICE /etc/conf.d/asterisk 

ASTERISK_NICE="-15"

I tried setting up cpusets manually, 

according to directions in /usr/src/linux/Documentation/cgroups/cpusets.txt

#CGROUP CPUSETS

PID=`cat /var/run/asterisk/asterisk.pid`;

CSET="/sys/fs/cgroup/cpuset/asterisk"

mkdir $CSET

/bin/echo 1   >  $CSET/cpuset.cpu_exclusive

/bin/echo 7   >  $CSET/cpuset.cpus

/bin/echo 0-1 >  $CSET/cpuset.mems

/bin/echo $PID > $CSET/tasks

/bin/echo 1 > $CSET/cpuset.memory_migrate

But I couldn't remove the cpuset, e.g. commands such as the following 

resulted in "no space left on device" error message:

/bin/echo "" >$CSET/cpuset.cpus

So I had to reboot server to remove cpuset.

Then I found a SUSE pdf "Quick Start SUSE Linux Enterprise Real Time Extension 11 SP1"

detailing use of cpuset. 

So I looked for a Gentoo ebuild and eventually I found an ebuild for cpuset from dev-zero overlay.

BTW, my 8-core systems has 8G ram:

# numactl -H

available: 2 nodes (0-1)

node 0 cpus: 0 1 2 3

node 0 size: 4093 MB

node 0 free: 3212 MB

node 1 cpus: 4 5 6 7

node 1 size: 4096 MB

node 1 free: 3668 MB

node distances:

node   0   1 

  0:  10  20 

  1:  20  10

----------

## redwood

Found a typo in my /etc/cset.conf, 

the mountpoint should have been (for 3.2.1-gentoo-r2)

mountpoint = /sys/fs/cgroup/cpuset

instead of cpusets

My /etc/fstab mountpoint for cgroup is:

cgroup                  /sys/fs/cgroup          cgroup          rw              0 0

When the kernel mounts /sys/fs/cgroup during bootup,

it automatically populates /sys/fs/cgroup/{cpuset, cpuacct, memory, devices, net_cls, cpu, freezer}

which causes a problem for cset:

```

# cset set -r

mount: none already mounted or /sys/fs/cgroup/cpuset busy

cset: **> mount of cpuset filesystem failed, do you have permission?

```

If I  "umount /sys/fs/cgroup"

then

```

# cset set -r

cset: **> [Errno 2] No such file or directory: '/sys/fs/cgroup/cpuset'

```

cset.py apparently tries to mount cpuset if it determines that it is not already mounted.

But  kernel-3.2 automatically populates /sys/fs/cgroup/

as shown by the following single entry:

```
# grep cgroup /proc/mounts 

cgroup /sys/fs/cgroup cgroup rw,relatime,net_cls,freezer,devices,memory,cpuacct,cpu,cpuset 0 0
```

So I filed a bug report http://code.google.com/p/cpuset/issues/detail?id=12

BTW I did find a good tutorial on cset

http://www.suse.com/documentation/slerte_11/slerte_tutorial/data/slerte_tutorial.html

----------

## Ant P.

If it's not CPU-intensive then you may be able to fix it by setting Asterisk's CPU affinity to keep it on 1 core. Low-latency media stuff tends to not like being bounced between processors.

----------

## redwood

Well, in /etc/conf.d/asterisk

I tried using the asterisk option "-p" for pseudo-realtime thread support

as well as the ASTERISK_NICE value of "-19"

#ionice -p $ASTERISK_PID

unknown: prio 0

So far I haven't noticed any improvement. 

Some conversations are as perfectly clear, random others are still choppy.

Sometimes I do notice users' firefox/opera webbrowsers and adobe-flash having high CPU usage.

Sometimes killing those user processes helps. But since the voice problems are random, it's hard

to diagnose. 

I'm using cfq scheduler:

# for d in /sys/block/sd[a-z]/queue/scheduler; do echo "$d => $(cat $d)" ; done

/sys/block/sda/queue/scheduler => noop deadline [cfq] 

/sys/block/sdb/queue/scheduler => noop deadline [cfq] 

/sys/block/sdc/queue/scheduler => noop deadline [cfq]

I'm about ready to try switching from asterisk to freeswitch.

I've installed freeswitch-1.0.6  

along with both bluebox and fusionpbx (rev. 1877)  for graphically editing/managing 

freeswitch's dialplan configuration files in /etc/freeswitch/

2600hz's Bluebox is supposed to be the successor to FreePBX,

but it still doesn't have an option to add a PSTN (dahdi) trunk.

Maybe I can manually edit /etc/freeswitch/freetdm.conf 

and get a dahdi trunk working with freeswitch even if BlueBox doesn't know about the trunk?

----------

## finalturismo

how can i download app-admin/cpusets?

I cant seem to find the package.

----------

