Custom UI ambitions, or maybe some day the controller grid page

Hi,

Here I come again, after reading a lot of zynthian-UI code and thinking about it a bunch, and realizing this is a big task BUT I do have the experience to pull it off, I just need a deeper understanding of the codebase and to iron out some quirks and learn some stuff.

Maybe this could be a midiFX layer added on top of the synth, that has presets for the layers that you specify, this could simplify things a lot.

The main idea is to be able to make a “controller grid” page, a 2 by 8 (or X by Y) grid with the names (or abbreviated names) of parameters and their values, read a certain midi value for each parameter (regardless of the actual layer midi mappings), with the objective of being able to use 1 midi controller (think nanokontrol or launchcontrol XL) as more of a physical synth interface, by displaying in our already existing display what we would in a controller display (if that makes any sense)

If you’re not understanding me think of the faderfox ec4
image
Where the Zynthian takes on the upper part of the device, making an otherwise “dumb” and cheap midi controller with 8/16 controls into a potentially infinite number of them (4 pages might be enough).

Other points:

  • The midi channel of the controller grid page is attached to the midi channel of the layer used.
  • There can be more than 1 controller grid page for each layer(maybe switch between them with a specific cc?) the page stays as the last one accessed.
  • This might add unnecessary complexity (and it does) to anybody that does not use it, so making it adjustable either through the admin panel or webconf might be of interest.
  • some layers might not need it at all, so make it so that anybody could build their own(maybe a JSON file related to the layer).

Open a feature request I hear y’all screaming, but wait, I have my reasons not to (yet).

I think this feature is pretty huge, especially after reading some of the code and trying to understand the architecture behind the controller pages, so I don’t want to open such a wide, ambitious, and specific feature request without a PoC of this working in my computer with jack, Alsa and midi.

As so this post is to be used as a place to discuss the stupidity of this idea, the uselessness of such a feature and the many ways I have gotten it wrong, and also to help me along as I’m building this for DAAAAAAAM SURE as I’ve basically sold all my hardware synths and this is my only option.

PD: Huuuuuge missed oportunity with “Zyntherface” :zynface:

4 Likes

I want to understand…

Currently Zynthian displays 4 parameters at a time with each adjustable by one of the 4 rotary encoders. Are you suggesting displaying more than the four parameters with each adjustable by MIDI CC? So it would be the same as MIDI learn for all parameters of the selected engine (which is currently possible) plus a display that shows the values but not allow direct control from the Zynthian UI. (I guess we could add navigation and control from the UI as well as direct control from MIDI CC.)

If that is what you mean then I see it as a good idea. It is currently painful to edit parameters on Zynthian. By the time I have navigated through dozens of pages several times and eventually found that elusive parameter the artistic spark has gone and I am a programmer again! I want to grab a knob and twiddle.

Yes, very much that’s the idea, we “lose” the zynthian encoders for parameter manipulation, but we use that space for just displaying more parameters and maybe as you said midi learn. Maybe by using select and layer encoders as page up and page down button we could reduce the number of menus that one goes through before finding said parameter. (I know this is not in the ui design guidelines but i miss it so bad in the layer controls view)

What you just said is exactly the reason why I was thinking of buying a faderfox, but it’s expensive, it has no faders, and, by using the zynthian as the screen, we enable the use of multiple controllers with this features, making all synth parameters much more usable from an external controller.

This thread is a bit of a brainstorming one, I think that you do have something more concrete then I do in mind.

The hardware I’m gonna be using for this purpose is an original nanokontrol, and i would totally recommend it over the nanokontrol2 (particularly for the 4 layers and the extra knob and fader that you get)

If you bold-press SELECT you can then use the SELECT encoder to move the page selection up and down to select page which makes navigating pages of parameters faster.

There is already a precedent for using encoders to navigate a grid. This is implemented in the step sequencer so maybe there is scope to have a view that shows all the parameters of an engine in a grid which can be navigated with encoders (x/y) and adjusted with an encoder. I would like this view as most of the time I need to adjust one parameter. Also, the current layout of parameters means that if one wants to adjust two parameters simultaneously they may not be on the same page. It may be advantageous to allow the normal view of 4 controls per page to be configured by the user.

I know about the bold click on SELECT to go a page up but i find it counter intuitive and kinda cumbersome, I click once to go forward and have to click for long time + scroll to go back. I do not find as much use for the SNAP click ever really (this could be due to my own use case) so i wonder maybe using it as a previous page button would make more sense.

As I’ve been thinking about it it would be a grid of a set number of rows/columns (maybe set by the user).
It would divide into pages like it currently does, but what you said about navigation using encoders does sound like a nice option.

We could make this the default interface, having many more controls on each page. We could allow use to expose desired controllers on a, “performance” screen which acts like the current UI, 4 controls per page. This would change the current implementation to allow custom views counting the desired controllers.

Yeah, that would be great, and maybe this allows to finally sound design in the Zynthian without a computer or tablet.

I found another device that does something similar as what i was hinting at:
image
(the image is a link to their website)

Obviously i am not talking about the full feature set but the way it handles parameter setting, and the interface seem like a good inspiration

I am ready to get working into this feature but I find myself kinda lost in the mists of the github repo, and would really like to know i you @riban have something in mind that you could maybe explain to me what that would be and where i could start looking into this.

Zynthian GitHub has several repositories that each contain elements of the system. The one relevant to UI is https://github.com/zynthian/zynthian-ui which is cloned to the Zynthian at /zynthian/zynthian-ui.

Each screen / view is created with a Python class whose file is witihn the zyngui subdirectory. Screens are derived from a superclass, zynthian_gui_base. Most screens derive from a subclass called, zynthian_gui_selector which implements the list and controllers for rotary encoders. There are various calls made to these objects like refreshing screen, handling encoders and switches, etc. If you are looking for inspiration or example, you are probably best avoiding the step sequencer screens which use a more complex / unorthodox / dissident UI implementation which does not align with the wider ethos and whose implementation is likely to change.

We are in need of a coding style guide but try to infer style from existing code. (I struggle to do so due to long standing preferences but am getting there!)

The main application file is /zynthian/zynthian-ui/zynthian_gui.py. I tend to test things by:

  • ssh to the Zynthian with X-forwarding: ssh -Y root@zynthian.local
  • stopping the zythian service: systemctl stop zynthian
  • Changing to the directory: cd /zynthian/zynthian-ui
  • Starting zynthian: ./zynthian_gui.py

@jofemodo has told me that is suboptimal but it works for me.

I use keyboard bindings to drive it observing display on my local machine.

I have recently started using Geany (apt intall geany on Zynthian), a simple IDE.

If you are going to submit pull requests you will want to fork the repro then point your zynthian at your fork.

I am not traditionally a Python programmer so fairly new to it. (There are high-brow exchanges occasionally regarding programming languages - all in good jest. I prefer lower level stuff but Python allows fast coding, suitable for many purposes.) Feel free to ask more questions as you proceed. Be aware that just because we build it does not mean it will be acceptable to the project. If you want to avoid spending too long working on an ultimately rejected idea it is a good plan to garner opinion (like you are doing here) and an ounce of approval from the boss goes a long way :wink:. (I am not (nor no where near) the boss!) I tend to find my efforts appreciated but generally need to rework them when very good feedback and advice is provided. (I don’t always agree but it is wise to allow others to be wrong occasionally.)

Good luck.

2 Likes

Nice, that’s exactly the advice i was looking for! Thanks!

Yeah, I think I’ll tinker around the code today to try and understand it, and then draw some diagrams and sketches of how I think this could work, and post them here so everyone can voice their opinions (including Mr Bossman’s). I am aware that, without contribution guidelines and coding style it’s kinda risky but even if i have to live in a UI fork I can always call my fork Zyntherface and be a happy weirdo.

One of the seven Zynthian dwarves.

Or maybe 2 of them together?

Oooookay, after playing around with tkinter and hating it a bit, I jumped into the UI design to understand some of the process (i’ve not done this nor anything like it before).
I got the core UI design concepts done and also some of the data structures, let’s start with the UI:

There is a main grid with an X/Y grid of values (i find that for the official 3.2 inch screen 4x4 works, but it’s already the max) and Z pages (this can be arranged manually) that keeps each parameter binding in it’s own dict, and organizes them.

For each parameter there are 4 control types:

  • Slider: a normal slider, akin to what we have today
  • Pan slider: same, but with the 0 value in the center
  • Switch: pretty self explanatory, defaults to on/off but can be changed
  • List: Has a list of values that you choose from, I’m still debating if this should be shown in the screen in the order you scroll through as a popup with all the options or keep it simple and tiny.

Also for each parameter there is the following data:

  • label (defaults to lv2-label)
  • ctrl_label (user defined label, optional, helps with weird controller layouts)
  • lv2_parameter_out (still researching about this)
  • cc_in
  • channel_in
  • type (the ones listed above)
  • options (if type = list or type=switch)

Now for the BIG question:

Should I build this as a separate application that would fit into the midi effects category, with it’s own UI and maybe set a new standard for “modules” or “plugins” for zynthian that have their own UI (this would also allow this “virtual midi controller” to run on any system that uses linux) using Kivy or tkinter (still deciding on this)?

OR

Should I build on top of what we already have, creating an option within the layers to show this interface if there is the json file for it?

Either of those would require me to write an editor application, which I would love to be able to have integrated in the webconf tool or run as a web app since that means cross compatibility (and then you copy your json file over to a certain folder.

Thanks for your help so far @riban, here’s hoping to become one of the dwarfs

2 Likes

I’d stay away from Kivy, unless you are familiar with it. It looked like the perfect answer to a python programmer who generalizes everything, but It seemed to have a lot of characteristics when I tried it. Perhaps it’s better now.

The thing about tkinter is its everywhere python is ( idle uses it, and that 's pretty much the reference python project). IT’s got a lot of critics but it works at a base level and most of the zynthian infrastructure tends to be at that level.

Consider the different screen sizes that get used. Certainly the kits are the canonical source but people do lots of weird and wacky things with open source big and small screens and also remember that mouse and touchscreen are considered part of the zynthian infrastructure.

If your code is available as a branch I’ll load it up and have a play, cos until my zynthian sentence is passed, I’m committed to this sort of stuff. . . :slight_smile:

I’ve said it before but I love to see what people do with all this stuff. It really lifts the soul to see what interest and personal drive bring to something, it really is grater than the sum of it’s parts .

:slight_smile:
Oh by the way, if you think writing an alternative front end relieves you of the :face_with_monocle: then dream on baby …

This turned out to be the way I keep notes in this proyect, so a lil update, mostly about decisions taken and some clarifications.

The idea:
A virtual midi controller - wait, what?

You all know the applications for configuring the controls of a midi controller, for example the Korg nanokontrol2.
I the app you specify the following for each knob/slider/button:

  • midi channel
  • midi cc
  • range (sometimes)

The idea is, by using the screen, creating a configurable midi controller grid, like this one

For each parameter there are 4 control types:

  • Slider: a normal slider, akin to what we have today
  • Pan slider: same, but with the 0 value in the center
  • Switch: pretty self explanatory, defaults to on/off but can be changed
  • List: Has a list of values that you choose from, I’m still debating if this should be shown in the screen in the order you scroll through as a popup with all the options or keep it simple and tiny.

So, basically, to create a midi router with a live display of parameters that can be controlled with (in the case of the picture above) a 16 control hardware midi controller but control (16 parameters x 4 pages) 64 parameters of a synth engine.
Kinda what the encoder controllers do, in that you have 4 controls for each layer parameter page, and then you have an N number of pages for the total number of parameters of the engine, but user configurable , addressable with midi (without interfering with the navigation controls) and with the option to show more parameters.

The idea (for the default config at least) is that the hardware midi controller is configured with a static midi config (ie midi controller knob no1 is on channel 1 cc 1, knob 2 on channel 1 cc2, etc), and, by changing pages, you can change what each control does. If you where to exit the screen entirely, the controls stay mapped to the last page you had selected, so that you can edit the engine even if it’s not in sight.

Also for each parameter there is the following data:

  • label (defaults to lv2-label)
  • ctrl_label (user defined label, optional, helps with weird controller layouts)
  • lv2_parameter_out (still researching about this)
  • cc_in
  • channel_in
  • type (the ones listed above)
  • options (if type = list or type=switch)

Regarding how the tecnical details it would be a 3 dimension array (x/y/page) of the parameter data structure explained above, and that would act as the midi routing matrix. The idea is to use only one midi channel in for each instance of the control grid( be able to use current layer for example) so you could have 2 of the same hardware midi controller controlling 2 different synths by using 2 different midi channels

Now for the BIG question:

Should I build this as a separate application that would fit into the midi effects category, with it’s own UI and maybe set a new standard for “modules” or “plugins” for zynthian that have their own UI (this would also allow this “virtual midi controller” to run on any system that uses linux) using Kivy or tkinter (still deciding on this)?

OR

Should I build on top of what we already have, creating an option within the layers to show this interface if there is the json file for it?

I’ve settled on the following:
Developing a portable python app, and, when it’s functional, try to either make it into an lv2 plugin and integrate the interface with zynthian OR keep it standalone and integrate it with zynthian (yes, aswering a EITHER/OR with another EITHER/OR, I’m sure that’ll go well) In a way, zynthian already does this, but with the four encoders, so I don’t know if that’ll help with the integration efforts.

Either of those would require me to write an editor application, which I would love to be able to have integrated in the webconf tool or run as a web app since that means cross compatibility (and then you copy your json file over to a certain folder.

This would make so if you have a bigger controller or screen and want more/less parameters on view, you are able to control that and the order in wich parameters appear, and also make it possible to arrange the parameters in pages with titles like FILTER, OSC, LFO, ENV etc so that they make more sense and are grouped together in a more meaningful way.

All this to say, I’ve been thinking a lot on this but not developing much. I hope I can bring you some :face_with_monocle: from this interface sooner rather than later.

PD: I would be really grateful for any kind of feedback, particularly from the more seasoned (and grounded) zynthian developers @jofemodo @riban @wyleu, and sorry to bother you, please tell me If this is too intrusive. I just want to make sure to be aligned with something that is, at least in principle, useful and zynthianic before I sink a bunch of time into it.

8 Likes

Hi Pastitas, are you still working on this? I love this idea.

2 Likes

Hi! Yes, i’ve actually just created the repo where i’ll put some code, and i’m in the process of reverse engineering the controller screens in zynthian to see if i can integrate this with that (a video chat with @riban or @jofemodo would help with that a lot actually)

I’ve managed to run vscode as a server and you can run debugging on the pi from the computer, wich is something I definitely have to post here in detail

I’m glad to hear about your progress. I’m looking forward to trying out your concept!

VS Code server is memory hungry. It will stop my Pi3 working after a while but is better on the Pi4 with more memory.

Defo tell us how you are debugging - sounds very useful.

I have a task (which I haven’t yet started) to design a mechanism / framework for community contributed plugins / screens / modules for Zynthian. Currently it can seem daunting. There is a base class from which to derive screens. I’m not at a PC and can’t remember details. (Old man!)