My initial benchmarking shows this may not be viable (in its current form) on zynthian. Running GuitarMidi-lv2 with a light-weight synth, like MDA e-piano consumes a lot of CPU. It looks like it is using 200% CPU - that is based on each core providing 100%, so 50% of available CPU, just to run the audio to MIDI converter. I appreciate the processing it is doing is substantial but I don’t think it would work on zynthian very well. I look forward to seeing if there is any improvement in this.
[Edit] I enabled dtoverlay=vc4-kms-v3d on V5 to see if GPU might help… it didn’t. Same high CPU usage.
I also suggest you document the parameters. There are 4 params that may be adjusted but it is not clear how to use them. I tried to systematically adjust them all but couldn’t get the note detection under control.
Thanks for testing. Have you also tried it on linux on a pc?
I managed to get it running on the MOD Dwarf, at least in principle. It needs heavy optimizations though and I’m not quit sure how low I can get the latency. I think that it will be enjoyable when used with synths loaded with slower patches.
Have you tried lowering the input stage to 1.7v max???
Doesn’t matter for PC, but Zynthian runs on Raspberry and I don’t know if it could be relevant or not, but many versions do not make use of any external USB-audiocard device.
In the next 2-3 weeks Im optimizing GuitarMidi for cpu performance across pc and mod dwarf. This should benefit the zynthian as well. I’ll get me an extra rpi 5 to test on that too
I dont know what you mean, since I dont own a zynthian. The high cpu usage is also observable on other arm devices and even older intel based laptops. Im on it!
I won’t promise anything but I think so. The plugin has two stages: the filterbank and the neural net.
Both have roughly around the same amount of operations, flops.
The neural net is loaded by the highly optimized tflite lib. The filter bank is in part my own code and totally non optimal. Im working on that part so that it is equal or even faster than the neural net. Then the plugin will run in full polyphony
I’ve improved the plugins runtime performance on the cpu so that it runs on the mod dwarf without xruns
However latency is still very high ~50ms. This is due to the neural net having ~800k parameters. I cant release it this way. I have thought about the implementation of a monophonic but that is an entirely different model that needs to be designed, trained and tested. taking time away from my task to provide rock solid polyphony for linux and linux-powered devices. And in some way this move feels backward since the core tenet of the plugin is that you should be able to seemlessly switch from solos to chords and this I have achieved.
There is hope though. I have ordered the coral usb accelerator Google Coral USB Accelerator - Edge TPU ML - ARM Botland - Robotic Shop for which tensorflow lite has support, I think this is the way forward, my plan is anyway to support new guitar gear that have tensor processing units. Thanks for your understanding