Thanks for this insight @ricard, your continual effort to improve your wonderful softsynth andyour commitment to ensuring it works well on zynthian.
In theory, NanoVG should work better than Cairo on Raspberry Pi (or at least be similar), but only if V3D OpenGL driver is enabled. As discussed in another thread, this is not enabled in zynthian and indeed, was found to (possibly) trigger an issue - although that may have been a misinterpretation of the cause/effect. We should probably try to enable the V3D driver if it can help with plugin native GUIs.
SDL2 may provide better CPU usage than Cairo, especially if combined with OpenGL ES and V3D driver.
The most likely reason for Osirius + OpenGL apps clashing is because of the shared CPU usage and the OpenGL fallback to CPU. The benefits of OpenGL are lost and probably actually make things worse by adding a GPU emulation layer in CPU.
VNC rendering though is a significant CPU load. There is the framebuffer copying, video encoding and transport which all add to the CPU load. There are some things that can be done to reduce CPU load in VNC which include:
Reduce screen resolution. (There is now a screen resolution switcher widget in vangelis.)
Lower colour depth.
Use a lightweight video encoder.
Reduce screen redraws, e.g. hide apps.
If we were to move to PipeWire (for audio and video) we could benefit from its efficiency and use WebRTC for remote desktop access. There are many benefits to this but it is substantial work to enable. Maybe we could look at this in a future “optimisation” development cycle.
I appreciate that most of this post is not MiMi-d specific so it could move to its own thread, but I didn’t want to start the ball rolling on such a large body of work, so here it is as a reply to your specific research.
Yes, I agree the discussion about graphic backends and VNC rendering would deserve its own thread - interesting about the V3D OpenGL driver, I’ll have to look that up. I was also thinking that perhaps it might be possible to enable GPU backed rendering in VNC by having the engine GUI surface routed to one of the HDMI outputs - but that would be a larger undertaking than just getting a plugin to run smoothly.
The strange thing I’ve found is that in a snapshot with Osirus + MiMi-d (2.3.0 with nanoVG), adjusting MiMi-d parameters with consequent parameter redrawing caused audio glitches, but adjusting Osirus parameters does not. Indeed, redraws as a result of parameter changes in Osirus don’t seem to use a lot of CPU either.
Furthermore, in a snapshot with 8 other plugins (OB-Xd, JV-880 and others), even though MiMi-d still showed large CPU spikes, there were no audible glitches. So it seems to me that Osirus has some sort of issue which makes it extra sensitive to the CPU loading. The real-time thread in Osirus does not consume that much CPU, and the split of priorities between realtime and non-realtime threads should ensure that the audio is clean even though the video may be sluggish.
Resizing the MiMI-d window now with the Cairo backend is obviously a CPU consuming endeavor as it can be pretty sluggish over VNC - yet the audio doesn’t glitch at all.
Although I am, alas, no software architecture expert (which one day with more time for myself I would love to become, at least partially), I have an understanding, from the devs Discord, that some TUS emulators have been ingeniously made to work, with some effort, also on the ARM64 CPU platform, however they were originally conceived for stronger hardware resources.
I think that this makes them particularly reactive, to fluctuations of the computational load when they run on Pi5, in essentially limit operational conditions, considering that, as we all now, they aren’t synthesiser models, but exact low-level software replicas of digital circuitry.
Yes as far as I understand, essentially the TUS plugins run the original code in an emulated DSP environment. Considering that the original code has 100% access to the DSP, there certainly must be some feat involved to running it in a buffer-at-a-time environment in the form of a plugin, but I’ve never studied the inner workings of the TUS plugins, interesting as that may be.
Enabling v3d (dtoverlay=vc4-kms-v3d) on RPi5 seems fine but doing so on RPi4 breaks the UI. There is no output to a V4 screen and VNC shows a large vangelis background image with small UI in top left corner.
This is an aspect of DSP-based synthesis on most hardware synthesisers, especially from the 90s and early 2000s, which is often overlooked. That is the deterministic (non-statistic) and constant availability of the full computational power and audio bandwidth, for the digital signal processing code written in the IC.
The implicit one or two samples chunk size, close to the nominal limit of the DSP-DAC data pipeline, is the main reason for hardware synthesisers with a DSP (bank) at their core being so inherently reliable and acoustically solid: no potential signal dropouts caused by audio buffering, simply because there is no interrupt request-based sharing of the CPU cycles, in order to support a computer OS besides running the synthesis program in real-time.
,