VNC UI weird click behaviour

Using Zynthian
1

Hello community,

I just started with zynthian on a brand new Raspi 4 B with the official power supply. I planned to play with it headless before I buy the set. But I have a tough time exploring the features, because all of the clicks or “touches” in the VNC UI lead to completely different actions. E.g. clicking on “audio levels” shortly highlights that menu item but then “layers” is opened! I can use the arrow keys to navigate, but no matter where I click it just selects/opens what currently is highlighted.

I then wanted to disable VNC in order to try X11: Disabling VNC in webconf and clicking “save” always activates VNC again. It’s not possible to disable it.
Ok, then to X11: I managed to see the zynthian UI on my Windows PC, can click once or twice and then the X Server freezes (I tried two). I had to kill the X server processes in order to close the window.

Btw. I did plenty of restarts and I updated all software. This behaviour is reproducable on all my devices (Windows, Android, Chrome OS).

I still hope I missed something obvious. Do you have any ideas or advice?

2

Precisely how i it all set up?
I’ve seen similar when driving Zynthian derived vnc with a mouse, touch on vnc may be similarly derived.
The ‘solution’ is ensure that the cursor and the highlight are located on the same select-able entity. I think one looks active but the other selects or something similar.
That way you will be ‘fairly’ confident you’ve got the right selection.

Hopefully this is relevant or nothing more than something I should have reported when I first worked out a solution.

Can you try with an hdmi monitor ( if you have access to one)? and try with a mouse plugged into the PI and see how well behaved it is.

3

Hi @ey77

Welcome to the community. You have already encountered our most demanding member :wink: so it is all plain sailing and fun from here on! (You will find out about his demands later - I am sure…)

You have experienced a bug in the current (stable) version whereby clicking anywhere on the screen selects the currently highlighted item. You may work around this by using the computer keyboard.

The simplest way to access the UI remotely is via webconf INTERFACE->VNC-UI. This presents the Zynthian user interface within the web browser. Driving Zynthian with touch / mouse can be challenging. Some of the interface does not work well. I prefer to use the computer keyboard.

X-forwarding is not advised. It seems to have a much larger overhead than VNC and the instructions to use it are now obsolete. (If they remain on the wiki they should be removed - let me know where you saw that.)

4

Wow, before I could even say thank you for your quick replies, you’ve already fixed it! Amazing, thank you!

I have updated the software, and now it works better. But still long and bold clicks do not work. I have to touch and move the finger over the screen towards the target item. And then the time before lifting the finger determines, whether a regular, bold or long “click” is triggered. It’s not possible to directly touch an item, hold it for the “bold” or “long” period and then lift it.

It works using a mouse, but not using touch input devices. I tried the no vnc web client as well as a native Android VNC client. I tried it on Windows, Chromeos and Android.

By the way, after the update every patch I loaded had a portamento effect. I then opened the portamento setting of an arbitrary patch and the on/off control had “Err” as its value. I could change that value to “off” and after that, all patches were working normally again. Is this a “glitch” or should this be documented as another bug?

Thank you!

5

Touchscreen devices often have their own layer of handling touch, e.g. Windows 10 work will interpret a long touch in a particular way. I’m not sure we can do much about this. It is certainly frustrating to have to move your finger a little after landing it.

6

Ok, I see. I guess I’ll get around using mouse/keyboard when configuring zynthian and setting up MIDI commands to control it from a MIDI controller live.

7

I’ve found plugging the mouse into the zynth, turning on the cursor then using vnc in read only mode is the most reliable configuration. Does require you to be near the zynth but that kind f makes sense a lot of the time.

1 Like
8

I find using keyboard / mouse on the client machine connected via VNC is fine. Touch (with some OS) on the client machine is flawed as described above.

9

I am resurrecting this old thread in order to garner some support from interested parties to either resolve some issues with touch via VNC, or make the decision to not support it…

The problem is that touch events do not necessarily make it to zynthian intact, leading to undesirable behaviour. Consider this event chain:

Touchscreen → OS → Web Browser → VNC → zynthian

There are several hops that the event must make and often the OS (Windows, Android, etc.) and browser (Chrome, etc.) convert long press to gestures, e.g. right click, context menu, etc. Zynthian depends on receiving mouse press, motion and release events to perform its actions.

The experience is that short touch often works fine but bold and long press will fail, sometimes with some side-effects, like accessing OS functionality (context menu, etc.)

The question to you is, do you want to get involved in trying to resolve this? Maybe there are some workarounds available in the OS, browser, VNC, etc. that can be documented. Maybe there is some code improvement in zynthian that could facilitate the required functionality. Maybe this is too much hassle with too little reward. I seldom experience this but occasionally I use an Android tablet or a laptop with touch screen and observe the issue. It would be nice to be able to reuse old tablets I have lying around as remote control for zynthian, without the need for an OSC client (like the brilliant work that @ronsum did a couple of years ago).

So, the floor is yours. Help if you want. Or ignore this topic to send it back into obscurity…

1 Like
10

Hi,

To double check if this is the same issue.

Can anybody get the V5 on screen buttons (eg Alt, Back etc) to respond via VNC on a tablet or phone?

They don’t seem to respond, tested on Android phone and iPad, both with RVNC app and with VNC started via webconf.

I am aware of the use of a mouse connected to Pi / Zynthian as a practical workaround.

Tested with latest vangelis.

UPDATE: Interestingly, I see the button being ‘pushed’ in this recording, but nothing happens. (Recorded RVNC session on Android)

The UI log shows the button is pressed, but not ‘resolved’

Mar 05 20:25:15 pisound startx[871]: IndexError: list index out of range

Mar 05 20:25:18 pisound startx[871]: ERROR:zynthian_gui.cuia_thread_task: CUIA ‘zynswitch’ failed with params: [‘6’, ‘R’]

Mar 05 20:25:18 pisound startx[871]: Traceback (most recent call last):

Mar 05 20:25:18 pisound startx[871]: File “/zynthian/zynthian-ui/zyngui/zynthian_gui.py”, line 2564, in cuia_thread_task

Mar 05 20:25:18 pisound startx[871]: if zynswitch_cuia_ts[i] is None:

Mar 05 20:25:18 pisound startx[871]: ~~~~~~~~~~~~~~~~~^^^

Mar 05 20:25:18 pisound startx[871]: IndexError: list index out of range

UPDATE 2: the buttons weren’t found because my config had -1,-1,-1,-1,0 configured for my custom zynth (where 0 is mapped tot a hardware button). Adding extra -1 dummies enabled extra buttons.

I’ll try using all -1s of a fake V5, and then add an extra 0 for my single hardware button.