I noticed that lilv the library that a few DAW’s use to load lv2 plugins has python api bindings
Instead of using jalv can we get zynthian to just shove them in using python-lilv?
Yes, we could get it to work, and certainly it would improve things, but i’m not sure it’s “the way” because all MIDI-learning/CC-routing/automation would be still done in python.
I’m finishing the merge of riban’s “chainification refact” and after this, i would like to focus in improving sequencer. One of the many interesting features we would like to add is “controller-tracks”, so we could automate volume-fades in/out, ping-pong panning or “kind-of-LFO” on any engine parameter or mixer fader, etc.
With the current state of things, all this massive stream of control messages is going to pass thru the sloooow python MIDI dispatcher function (and worst! from there, they will go thru the CLI interfaces of (mostly) Jalv LV2 engines!!). I’m afraid this is going to be a real bottleneck when we start adding “automation” on every sequence. And we want to do it, of course.
@riban, i know you understand me 
All the best!
3 Likes
Hi @guys!
You seem to have had a long and interesting chat.
Well, indeed zynthian dependency of WiringPi is not so big. Most complex code is user-space i2c stuff for managing MCP23017. We could move to kernel’s i2c-dev library quite easily. For the native GPIO stuff, moving to libgpiod seems to be the way.
Yes. I also found some of this LV2 plugins and i suspect the number will go up, so yes, we need to support this “LV2-parameters”. Do you know, @riban, if the “grouping” mechanism is also implemented for “LV2-parameters”? I couldn’t found any info about it. Well, indeed, the documentation about “parameters” is also scarce and too-cryptic. As someone told me, LV2 docs are clear after you understand the subject of the documentation 
The best!
2 Likes
But if you replace jalv with something else surely you have the same problem, if you use python-lilv wont it all be in python. Python becomes the lv2 host.
1 Like
But python is slow for processing MIDI in real time. I would like to implement all the MIDI processing in a faster language, and this would include managing MIDI-learning of engine parameters.
Anyway, we need to put numbers to this and really know how many MIDI CC messages can we manage with python without being a bottle neck or consuming too many CPU resources.
Regards,
I pulled these posts out of the ZynthClub topic as they are developing into a discussion of a specific subject.
We already use Python lilv bindings for interacting with ttl (LV2 configuration files). Some operations can be painfully slow, like the iteration required to search for plugins and configs due to the tree like structure of LV2 config - it has to pull detail from each branch which is both slow and memory comsuming. (I think there is a bug / limitation of lilv or the Python bindings.)
It may be possible to launch the plugins using pylilv and control and monitor them directly in Python. This might not even have significant impact on performance because we are already filtering MIDI CC through Python. We want to improve this part in the future which may see us move this message processing to a lower level language at which point anything we may have done with hosting LV2 could go with it. Of course a change like that proposed by @Baggypants could involve subatantial effort which might be better targetted elsewhere. It may be more appropriate to wait until the refactor of CC processing occurs to consider such a change. It kinda depends on who is feeling like doing what and when…
3 Likes
I would ideally want to replace jalv with a LV2 host that can:
- Instantiate plugins without running their UI
- Present control ports and parameters for monitoring and control
- Allow the naitive UI to be launched and stopped independently
This may be available in existing code or with a little modification otherwise it is a daunting task to write a LV2 host from scratch!
LV2 Parameters
The basic LV2 mechanism for controls is lv2:ControlPort, inherited from LADSPA. Control ports are problematic because they are not sample accurate, support only one type (float), and require that plugins poll to know when a control has changed.
Parameters can be used instead to address these issues. Parameters can be thought of as properties of a plugin instance; they are identified by URI and have a value of any type. This deliberately meshes with the concept of plugin state defined by the LV2 state extension. The state extension allows plugins to save and restore their parameters (along with other internal state information, if necessary).
Parameters are accessed and manipulated using messages sent via a sequence port. The LV2 patch extension defines the standard messages for working with parameters. Typically, only two are used for simple plugins: patch:Set sets a parameter to some value, and patch:Get requests that the plugin send a description of its parameters.
Taking JC303 as an example…
The dsp.ttl file defines a set of parameter types, e.g.
plug:cutoff
a lv2:Parameter ;
rdfs:label "Cutoff" ;
rdfs:range atom:Float ;
lv2:default 0 ;
lv2:minimum 0 ;
lv2:maximum 1 .
It then defines instances of these types as writable (a parameter that the host can set) and readable (a parameter that the host can read that may be changed internally), e.g.
patch:writable
plug:waveform ,
plug:tuning ,
plug:cutoff ,
plug:resonance ,
plug:envmod ,
plug:decay ,
plug:accent ,
plug:volume ;
patch:readable
plug:waveform ,
plug:tuning ,
plug:cutoff ,
plug:resonance ,
plug:envmod ,
plug:decay ,
plug:accent ,
plug:volume ;
A relatively simple implementation might be to detect writable parameters in our lv2 module and jalv engine alongside control ports then present these in the UI.
The lilv library does not expose parameters as obviously as it does control ports. We need to dig into the data model further. I am yet to find example code for this so may need to look at jalv source (which does seem to find the parameters and present them as controls with names prefixed by underscore, e.g.
_accent = 0.000000
_cutoff = 0.000000
_decay = 0.000000
_envmod = 0.000000
_resonance = 0.000000
_tuning = 0.000000
_volume = 0.000000
_waveform = 0.000000
2 Likes
That would greatly improve the sequencer’s expressiveness, and allow for interesting multi-synth layered patches, like polyrhythmic arps with cross-fading components and articulated or sequenced pads/strings/brass.
2 Likes
I will add, “managing MIDI learning and automation”, so we have sample-accurate control of parameters, allowing precise automation and avoiding to overload python code with “real-time” tasks.
Also, “preset management”, that is already implemented (and we extended!) in jalv.
Regards,
1 Like
I was reading Jalv code and trying to find a “fast way” to expose these parameters to zynthian, but it’s more complex than i thought. Anyway, it’s a “low-enough hanging fruit” that we should collect for having “parameters” working with zynthian in the next weeks/months.
Regards,
2 Likes
import lilv
world = lilv.World()
world.load_all()
plugins = world.get_all_plugins()
plugin = plugins.get_by_uri("http://github.com/midilab/JC303/")
plugin.get_num_ports()
writable_uri = world.new_uri("http://lv2plug.in/ns/ext/patch#writable")
readable_uri = world.new_uri("http://lv2plug.in/ns/ext/patch#readable")
lv2_properties = world.find_nodes(plugin.get_uri(), writable_uri, None)
for lv2_property in lv2_properties:
lv2_property.get_path()
world.ask(plugin.get_uri(), writable_uri, lv2_property)
world.ask(plugin.get_uri(), readable_uri, lv2_property)
This gets the properties but then controls need to be created based on each properties’ parameters, e.g. rdfs_label. I am stuck there!
1 Like
Well I guess this just serves us right.
2 Likes
Each time I tried to host a .so plugin, python crashed. I don’t think it would be a great idea anyway because the audio would need to be transferred from the Python host to jack.
I have looked at jalv and mod-host source code. There is a lot there to decipher. I am not sure which (if any) would be best as the basis for our LV2 host. We already have integration with jalv but are considering different integration so this may be of limited benefit. Indeed, we have a single engine class that hosts all of our LV2 so migrating to another host would mean writing one engine which isn’t too much work. We also use jalv to show native GUIs but that has its own issues and does not allow seperate hosting of plugin and GUI which we desire. mod-host does not handle native GUI but is controlled via OSC (or command line) so it may offer some advantages. It acts as a single host for multiple LV2. Maybe we would consider using one host per chain (although there are complications of routing when there are non-LV2 engines in the chain).
[Edit] mod-host does have code to show external UI. I haven’t tested to see if this works, where it is shown and what is meant by “external UI” but it looks promising. There is not code to hide external UI which may be useful. The OSC command is show_external_ui %i with an integer parameter indexing the plugin effect id within mod-host instance.
2 Likes
I tried launching external (native) UI from mod-host and mostly it does not work. Some plugins (like samplv1) do show the container for a UI but do not show content - maybe something related to update. So there is the possibility of making this work but… not working out of the box.
I have tested mod-host a lot and it provides a lot of the features we want. There are a lot of issues, some of which I have reported and submitted PRs to resolve. The external UI isn’t useful. Very few plugins support it, it is discouraged by the LV2 author and the implementation in mod-ui is fundamental flawed. I am now looking to completely separate the DSP hosting (which mod-host does very well) and the UI hosting which probably needs a new dedicated app (that I have started working on).
This approach has many advantages including:
- Leveraging existing codebase
- Maintaining common code base with upstream
- Benefiting from upstream enhancements
- Contributing to upstream development
- Robust DSP that is not impacted by UI
- Separate UI that can be launched on demand
- Feature rich support of LV2, including parameter and atom access which gives better integration with more plugins
- Clean interface between Zynthian and LV2 host
- Provides effect bypass
We probably don’t want to start migration to mod-host until we have a working UI launcher because otherwise we would lose functionality but I have a PoC app that is moving towards something useful…
5 Likes
Unfortunately, Suil does not come to the rescue. It purports to host any plugin in any toolkit but in fact the combination is very limited. I wrote a host in GTK3 and Suil only supports GTK3 & X11 plugins. There are a lot of GTK2 and QT5 plugins so this is of little help. I could write seperate libs that each host plugins in their native toolkit: X11, GTK2, GTK3, QT4, QT5. This is effectively reimplementing Suil but completing it. I could look at Suil to see if I can expand support (at least for the toolkit I choose - which was GTK3 initially because I thought it made sense but could be any of these).
This is frustrating because I had hoped that Suil would provide the solution. I can’t really see what Suil offers if it does not provide many combinations of hosting. The current list is:
| Host toolkit | X11 | GTK2 | GTK3 | QT5 | COCOA | WIN |
|---|---|---|---|---|---|---|
| X11 | X | |||||
| GTK2 | X | X | X | X | ||
| GTK3 | X | X | ||||
| QT5 | X | X | X | |||
| COCOA | X | X | ||||
| WIN | X |
LV2 developers are being encouraged to not implement GTK2 UIs but there are a lot of them already developed and I doubt many devs will revisit the code to redesign UIs - possibly the least interesting part of their code.
For our use I would probably ignore plugins with only Windows and Mac (Cocoa) UIs and conentrate on X11, GTK & QT. I will look at the Suil source and try to find a way to engage with the author, David Robillard. There don’t seem to be any forums for discussion. I may need to go on an irc channel… I knew those 56K modems would come in use!
6 Likes
I might be completely wrong but all distributions have slowly migrated to Pipewire… I know I know jack is very reliable, has an ecosystem, we have all the code, little motivation, few resources, etc.
AFAICT pipewire has native support for LV2. Filter Chain module does allow for including LADSPA and LV2 plugins in the pipewire graph, there’s also a way to control the parameters of the plugin
I have started with this reddit post:
https://www.reddit.com/r/linuxaudio/comments/y19dp3/pipewire_spa_plugin_api_and_lv2ladspa_support/?rdt=41446
Then I have read that almost all LV2 params can or will be controlled setting lv2:Parameter .
Forgive me if I am on the wrong track 
PS: we have pipewire-jack-audio to be compatible
3 Likes
Thanks @pau! I didn’t know that pipewire has native support of LV2 in its graph. That is really interesting and something I look forward to playing with. Both pipewire and it’s LV2 support are still in their infancy and I worry about the issues we may encounter if we migrated to them. Pipewire aims to provide solutions to many requirements we don’t have. The desktop integration, video, etc. are things that are irrelevant to our embedded product and it’s multi-card (resampling) is fantastic but possibly resource hungry. The lack of parameter control is an example of it’s infancy.
I definitely want to investigate pipewire and particularly benchmark against jack. I’m not sure whether now is the right time? If so then it might save some effort - not investigating enhancements to / replacement of jalv. I am nervous that pipewire isn’t quite ready for us yet…
1 Like
It looks like Suil is not the miracle solution I had hoped for. Indeed it may not support GTK2 for much longer. I ran a script to check what UI toolkits are supported by the plugins installed on Zynthian (which is most of the available open source plugins) and got this result:
- GTK2: 157
- GTK3: 0
- QT4: 0
- QT5: 4
- X11: 446
So most of the plugins with native UI support X11 with a substantial proportion supporting GTK2 and only a handful supporting X11. None yet support GTK3 (so building my host with GTK3 seems to have been a mistake 
.)
Whatever the solution we chose for hosting the LV2 DSP, we may want some mechanism for displaying the native UI and the current situation is that support for X11 and GTK2 covers almost all plugins with just those 4 outliers. FYI the QT5 UIs are for:
- abGate
- QMidiArp Arpeggiator
- QMidiArp Step Sequencer
- QMidiArp LFO
GTK2 is no longer supported and we shouldn’t be building new stuff with it but I am sceptical we will be able to persuade the authors of 157+ plugins to refactor their UIs overnight (or ever!) so if we want native UI support for these plugins then we will have to support GTK2. The only real way to do that is to use an app that hosts the GTK2 UI natively. We could write / adapt wrappers for each UI toolkit and launch them as external processes from the host app. I am told that trying to host GTK2 inside GTK3 is a challenge that is probably not worth the effort!
Why am I investigating this now? We have a lot of plugins being written with JUCE / DPF which use “LV2 Parameter” for control. We currently use jalv (and its native UI supporting relatives) to host LV2. jalv does not support LV2 Parameter. We have to choose which version of jalv to run for each LV2 based on the GUI toolkit it supports. We don’t want to run versions of jalv with UI support if we don’t use the native UI but there is no way to open the native UI separately to the DSP host. So we want a way to host LV2 that supports Parameter control and ideally, allows separation of DSP and native UI. The first of these two requirements is the driving force to allow us to provide full integration of modern LV2 plugins.
This will not be in Oram! We have much work to do in this field and it may be quite disruptive. Talking about it here will hopefully engage interested users and at least be a repository for my thoughts and findings (so I don’t need to keep them in my failing memory…)
5 Likes
Update: jalv.gtk in Oram (Debian 12) is actually a symlin to jalv.gtk3. There is no longer support for GTK2 UI which means the 157 LV2 that have GTK2 UI no longer support native UI in Zynthian. This is a decision by upstream jalv maintainer. If we want / need GTK native UI we will have to implement and maintain ourselves. It is plausible that with sufficient encouragement (i.e. removal of support) that LV2 plugin authors may revisit their code and add GTK3 or QT6 (yes, that is now supported in jalv) UI but I wonder how many have the time or inclination to do so?
1 Like