RT like operation - CPU isolation for special processes


#1

Hi all,

here is a simple manual/recipe for some optimization of your Zynthian.
Please note: THIS IS BETA! DON’T USE IF YOU DON’T KNOW WHAT YOU ARE DOING!

What we are doing today is to try to divide processes into nrt (non-realtime) and rt (realtime). Why should we do this?
The standard preemptive Linux kernel is really good for audio processing - jack2 (our audio server) works very well also on high loaded systems. But on top we can say: I want to use 2 or 3 of my cores only for audio and the rest should work for the GUI, etc.

A very nice manual describes this and from there I also noticed there is a good tool for making everything easier: https://github.com/OpenEneaLinux/rt-tools.

Ok, let’s go: We want to device our 4 core of a Raspi-2/-3 (every step is in a script in https://github.com/dcoredump/zynthian-recipe/recipre/rt-tools.sh):

  1. Add the following to /boot/cmdline.txt: cgroup_enable=cpuset isolcpus=1,2,3. The line shoud look like this:
    dwc_otg.lpm_enable=0 console=tty1 elevator=noop root=/dev/mmcblk0p2 rootfstype=ext4 fsck.repair=yes cgroup_enable=cpuset isolcpus=1,2,3 rootwait
    We are telling the kernel only to use CPU0 (ans not CPU1,2,3). Also we enable cgroup - a feature for advising processes to a CPU-set.
  2. Now clone, build and install https://github.com/OpenEneaLinux/rt-tools.git
  3. Reboot

If we now look at our used CPUs (e.g. with htop) there should only CPU0 be used. This CPU is for our nrt-processes. The other 3 CPUs are for the rt-processes. Now we can use partrt for creating and advising processes to the CPUs.

  1. First calculate a bitmask. The mask for CPU2,3,4 is 0xe (for CPU2,3: 0xc).
  2. Create a CPU-set for rt: partrt create 0xe
  3. Now you can move a process (you have to know the PID) to the rt-CPU-set: partrt move <PID> rt or you can start a new process directly on the rt-CPU-set: partrt run -f 99 rt /usr/local/bin/jackd -R -P99 -d alsa -dhw:0 -r48000 -p256 -n2 -Xraw (here I have also used the option -f99 as FIFO schedular with priority 99).

I tuned my Zynthian for using 2 CPUs for rt and changed the systemd-scripts so they start jack2 and mod-host on the rt-CPU-set.

Note: Every new started process (compiler, …) is located per default on the nrt-CPU-set and cannot advise threads anymore to more CPUs than located in nrt (or rt)! So: Compiling on 4 CPUs is not doable anymore!

And a nice trick for getting some more CPU cycles for other things:

sysctl vm.dirty_writeback_centisecs=1500

This avoids too fast writebacks to the SD card (if I had understand right). This can simply be added add the end of /etc/sysctl.conf.

Regards, Holger


#2

This is a great work, @C0d3man!! :trophy::trophy:
I will test it ASAP :wink:
Do you have some ideas about measuring the “performance” of these changes? I think it wont be easy …

Regards,


#3

For jack and mod-host it is simple to implement. I have made some service files for systemd:
https://github.com/dcoredump/zynthian-recipe/zynthian.stage/etc/systemd

For the engines you have to go into your Python scripts and change the start behaviour of the engines - that’s nothing for me… you know: my python-skills… :wink:

On the mentioned web-page they also have some simple examples for tests (cyclictest).

Regards, Holger