Realtime Kernel for Raspi

Update, there is kernel 4.14 now. I need to get RealtimePi to work with it, there is a commit that breaks patching which I need to revert.

Yet another update

  1. I got kernel 4.14 patched, its being built
  2. So did the Rpi foundation , they pushed a realtime kernel branch, asked them on the forum what is going on: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=206750&p=1306688#p1306688
1 Like

Did we abandon plans to use a realtime kernel in zynthian? I would have thought it would be beneficial.

From the JackAudio documentation:

http://jackaudio.org/faq/realtime_vs_realtime_kernel.html

Of course, i’ve tested RT kernels (RT_PREEMPT patch) with zynthian, but i hadn’t see any measurable improvement. Moreover, by using the RT_PREEMPT code, you will reach CPU limits more easily because less CPU power is available for doing useful tasks (audio preccessing).

For understanding how RT-kernel can help us, you have to think in a “jackish” way. Jack Audio is the key part and if having a RT-kernel doesn’t improve jack performance, i don’t see any reason to change.

Of course, you are free to do your tests and share your conclusions. I’m not closed to changing to a RT-kernel, but i wouldn’t like to do it without having clear (with numbers!) the benefits and drawbacks.

Regards,

Thanks @jofemodo - all understood. I thought I had seen reference to using RT so was looking for its implementation. (That is why I installed ZynthianOS.) I will have a play and see if I can build it and do some analysis. My previous experience with RT was in minimising latency. It will be good to see if we can get similar performance with lower latency with RT. I will report back.

1 Like

You will get lower performance with same latency (although probably more accurate, aka less jitter). Still lower performance with lower latency :wink:

The good part is you can get lower latency & jitter, the drawback is that you will have less performance.

Regards,

Finally got RaspberryPi 4 kernel working Realtime: https://github.com/guysoft/RealtimePi/issues/20

1 Like

Do you think it would be easy to generate a “dual-kernel” image? :wink:
So you could choose what kernel do you want to use from the webconf tool. Not everybody want a RT kernel. Lower latency has a price: lower performance.

Regards!

1 Like

Yes, its a matter of arranging files in the right way, and then setting the right ```cmdline.txt`` setting.
There is an export of all the kernel files in the release.
There is a tar.gz with all the files to pull and arrange.

2 Likes

Are the current Zynthian images using RT_PREEMPT by default? If not, is there any location to download the real-time version?

It’d be nice to quantify this. What magnitude are we talking about? 1, 3, 10, 30%?

It depends the engine, but reading this analysis:

i would say that the performance reduction is about 8-10%, what is not too much.

But if you take a look to the latency metrics, you can see a very interesting thing:

  • maximum kernel latency for “standard” kernel: 300us
  • maximum kernel latency for “RT” kernel: 90us

what makes no difference for “our” audio use-case, given that we are running jackd at 5-10ms of latency, the little difference on kernel latency only gives a marginal advantage on reducing jitter, but, from my point of view, not enough for justifying the 10% loss on performance.

I’ve read and thought quite a bit about this subject and always reach the same conclusion:

We don’t need a RT kernel on zynthian. Simply it’s not the right use-case for that.
A RT-kernel only is needed when your application has strict latency requirements of <1ms.

I other words, by using a RT-kernel, you won’t avoid or reduce the XRuns, and you won’t get a lower latency with jackd. You only will get a slighty better jitter and 10% less CPU power.

Regards,

1 Like

That makes total sense. Thank you so much for the added detail.

I compiled a RT kernel last week for Raspberry Pi 4 and it was unusable due to xruns. I don’t think I should have seen such large impact so there may have been other optimisation possible but I didn’t have time to investigate further. Although the latency wouldn’t improve there is an advantage that RT may provide: you can get more reliable / consistent results because you can guarantee that JACK gets serviced at expected times. With standard kernel this could be delayed by some other process, e.g. screen refresh might cause xrun even though CPU isn’t overloaded. This is a perennial topic and I believe we should review our findings regularly but current evidence is that RT does not offer sufficient benefit.

Please, take a look to the analysis i posted above. The difference is in the hundred of useconds, not relevant for our use-case, unless some driver is not working nicely. The only advantage would be a marginal reduction on jitter.

Regards

The purpose of a real-time kernel is to guarantee the time an activity is executed. Although under ideal conditions a non-real-time kernel may give excellent results there is the chance that an activity may be delayed by an arbitrary period so we are relying on other processes playing nicely including (and probably most pertinent) device drivers which may run atomic functions with may wait for response before releasing processor. Things like storage controllers were particularly prone to such delays. I completely agree with @jofemodo that RT gives insufficient benefit unless maybe we find an expert that can do all the optimization and prove empirically that there is tangible benefit. I would welcome that if I got consistent, reliable behaviour without detriment to performance but that seems rather unlikely. There is a cost in performance which we don’t want to pay at (at the moment). @jofemodo knows best :grin::clap:

I would like to add that the current kernel is a “low latency” kernel and it is already mostly-preemptive. Most of kernel IRQ code is not blocking. Of course, there is some blocking code yet, but those blocks have been balanced for being as short as possible, while having the minimum lost of performance, just the middle point we want for our use-case on zynthian. The current “low latency” kernel have Real Time processes (i.e. jackd) that can interrupt almost any kernel thread at any time, except those little blocking fragments. Of course, there is the risk of a badly-programmed driver … but we shouldn’t use this kind of drivers. :wink:

Regards,

The raspeberrypi/linux Realtime branches are maintained a little erratically, so its not really possible to serve an up to date raspberrypi realtime kernel in each release.
Also they tend to be buggy on the USB side, because that is the main driver that the Rpi foundation maintain in their fork. So I think if we want realtime as something constant and reliable we might need to switch devices, but then it creates other problems.