My initial benchmarking shows this may not be viable (in its current form) on zynthian. Running GuitarMidi-lv2 with a light-weight synth, like MDA e-piano consumes a lot of CPU. It looks like it is using 200% CPU - that is based on each core providing 100%, so 50% of available CPU, just to run the audio to MIDI converter. I appreciate the processing it is doing is substantial but I don’t think it would work on zynthian very well. I look forward to seeing if there is any improvement in this.
[Edit] I enabled dtoverlay=vc4-kms-v3d on V5 to see if GPU might help… it didn’t. Same high CPU usage.
I also suggest you document the parameters. There are 4 params that may be adjusted but it is not clear how to use them. I tried to systematically adjust them all but couldn’t get the note detection under control.
Thanks for testing. Have you also tried it on linux on a pc?
I managed to get it running on the MOD Dwarf, at least in principle. It needs heavy optimizations though and I’m not quit sure how low I can get the latency. I think that it will be enjoyable when used with synths loaded with slower patches.
Have you tried lowering the input stage to 1.7v max???
Doesn’t matter for PC, but Zynthian runs on Raspberry and I don’t know if it could be relevant or not, but many versions do not make use of any external USB-audiocard device.
In the next 2-3 weeks Im optimizing GuitarMidi for cpu performance across pc and mod dwarf. This should benefit the zynthian as well. I’ll get me an extra rpi 5 to test on that too
I dont know what you mean, since I dont own a zynthian. The high cpu usage is also observable on other arm devices and even older intel based laptops. Im on it!
I won’t promise anything but I think so. The plugin has two stages: the filterbank and the neural net.
Both have roughly around the same amount of operations, flops.
The neural net is loaded by the highly optimized tflite lib. The filter bank is in part my own code and totally non optimal. Im working on that part so that it is equal or even faster than the neural net. Then the plugin will run in full polyphony
I’ve improved the plugins runtime performance on the cpu so that it runs on the mod dwarf without xruns
However latency is still very high ~50ms. This is due to the neural net having ~800k parameters. I cant release it this way. I have thought about the implementation of a monophonic but that is an entirely different model that needs to be designed, trained and tested. taking time away from my task to provide rock solid polyphony for linux and linux-powered devices. And in some way this move feels backward since the core tenet of the plugin is that you should be able to seemlessly switch from solos to chords and this I have achieved.
There is hope though. I have ordered the coral usb accelerator Google Coral USB Accelerator - Edge TPU ML - ARM Botland - Robotic Shop for which tensorflow lite has support, I think this is the way forward, my plan is anyway to support new guitar gear that have tensor processing units. Thanks for your understanding
As far as Zynthian, all you need is a Pi - you won’t have the full V5 UI available, but as far as testing your software’s function, you’re good to go, there’s a screen-only mode.
Zynthian runs LV2 headless and presents the parameters consistently in the zynthian UI. If the LV2 has a native GUI and VNC is enabled and it is not explicitly disabled in zynthian code (some native UIs cause problems) then you can also access the native UI via VNC.
I compiled the LV2 from source. (There are no release packages for arm64.) Its detection is much better. Using the bridge pickup on a Squire Telecaster it tracks fairly well from low E to 11th fret high E string. It accurately detects chords through this region too. That is all very impressive. It does driver all 4 cores hard with each core sitting about 70% and the fan being driven on to full power. There is too much latency in the detection though. I have not measured but it feels like about 100ms, too slow for almost all uses.
I tried this again because I watched some very good videos by lofileif, including Best MIDI Guitar solution in 2026 - Fretboard scanning vs Pitch-to-MIDI which gives a good insight to MIDI guitar options. I rebuilt my Roland hex pickup MIDI guitar and compared it with this software neural network solution. The hardware still wins, but it is amazing what can be done in software. A few years ago you would have been burned as a witch!
[Edit] When in bypass, 3 cores reduce to near zero but the 4th core goes to 100%. Is this a bug in the plugin?
Thanks a lot for testing. The current version, while working well on PC, is hefty for the pi, I am aware of that. I am looking into some options on bringing down latency on weaker CPUs. Have you tried it on PC?
I do not use a PC. I tried bypassing other LV2 plugins on zynthian and did not see the same behaviour. Zynthian has a complex bypass mechanism using a forked version of jalv so you may not be triggering it in the same way.
