Receive SysEx-Data

Using Zynthian
1

Hi,
is it possible to receive SysEx-Data, store it like a midi-capture and send it back to the Zynthian?
My Novation Impulse has only 25 Patches and I like to use them for edit purposes I need for setBfree, OBX or others. If it’s possible, i had endless paches without changing cables to another computer. :+1::rainbow:

1 Like
2

Hi @Micki,

AFAIK there is currently a SysEx blocking inside zyncoder. There are bad machines in the world who send sometime “strange” SysEx commands, so there was a filter implemented.

I have a local branch where the filter is disabled, because I liked to test if I can send SysEx from an editor to Dexed. Perhaps you like to test this?

Regards, Holger

3

Hallo Holger,
thanks for the quick answer! That would be great!

I’ve tested the bulk transfer with midi-ox. The impulse sends bulks with a length of 288 bytes per patch and receives them savely. I think it will work.

But please send informations how to install the update. I’m new with the raspberry and the possibilities of it. I don’t want to kill my SD :face_with_monocle::smirk::rainbow:

4

Hallo Norbert,

In fact it’s a question of how familar you are with using Unix/Linux && tools. So it’s easy for one who knows how to use ssh, git and compilers. I would recommend to make a copy of the SD card before testing.

Here we go (from Putty or MobaXTerm if you are on Windows):

ssh root@zynthian.local # Login into your Zynthian
cd /home/pi/zyncoder/ # change to the zyncoder source directory
sed -i.orig 's/ || ev.buffer\[0\]==SYSTEM_EXCLUSIVE//' zynmidirouter.c # remove the part of the code which is ignoring SYSEX 
./build.sh # build the new zyncoder 
reboot # reboot system with new zyncoder

To switch back:

ssh root@zynthian.local # Login into your Zynthian
cd /home/pi/zyncoder/ # change to the zyncoder source directory
mv zynmidirouter.c.orig zynmidirouter.c # restore the original code
./build.sh # build  zyncoder 
reboot # reboot system with original zyncoder

HtH, Holger

1 Like
5

Thank you very much. Yesterday i made a copy of my sd😂
I’ll try it tomorrow. Thanx again for helping :rainbow::+1::bouquet::notes:

6

Hi Holger,
Shit happens…
I decided to test your SysyExIn with the new zynthian-image. But after writing and verifying this with Etcher I had a checksum-error.
I want to solve this before i go further steps. I can copy an image with Etcher perfectly but I cannot write an image from the internal drive…
Perhaps you or anyone has an idea :face_with_monocle:

7

I discussed a sysex tool with jofemodo.

perhaps we should put this under the wished feature list in the tracker.

2 Likes
8

wonderful :star_struck:

9

Sorry, but I am a Linux guy… I can only use Google to find something which may work on Windows. Perhaps Win32 Disk Imager?

Regards, Holger

10

Hi Micky.
You Can try with another tool like UsbImager wich does the same than etcher but us much more lightweight see here:

If still no Luck:
Your image or your sd card Can be corrupted.

11

Hi!

I tried disabling sysex filtering from the midi routing and while I seem to be able to send sysex from dexed on a w10 machine to zynthian (at least the midi received symbol flashes). Dexed on the zynthian doesnt react to sent programs or sysex parameter changes triggered from the remote dexed gui. Does the sysex receive on zynthians dexed need to be enabled or something?

1 Like
12

I’m afraid there are still SYSEX bugs in the code - I had started to remove them a few weeks ago, but then didn’t have enough time to finish it - sorry. In MicroDexed the SYSEX code is running. I still need to port it to Dexed.lv2. I hope I will find time for it soon.

Regards, Holger

2 Likes
13

I have been looking at sysex handling in Zynthian and there are several issues.

  • jack2 MIDI buffers need to be cleared on each cycle but this may intefere with USB MIDI which has fixed size buffers and hence limited data transfer, e.g. 256 size buffer must be sent like a packet before being repopulated
  • a2j seems to use a 1024 byte buffer and limit messages to this size, truncating sysex messages
  • direct control of alsa midi ports is limited when a2j manages the ports
  • ttymidi does not (by default) pass sysex messages so the physical 5 pin MIDI connectors don’t support sysex

I have been able to send sysex to USB MIDI via alsa and receive via alsa and jack2 when a2j is not running. I have sent via jack2 but I haven’t managed to do that reliably due to the way jack handles MIDI buffers - it will resend the message on each cycle unless you clear the buffer and if you do clear the buffer on each cycle then my USB MID interface (midisport 2x2) only seems to get the first 256 bytes.

I have been testing an interface with Behringer FCB1010 foot pedal so that we may have an editor on Linux but am struggling to get it to work when Zynthian is running its services. The actual interface took about a day to put together but making it work with the various MIDI layers in Zynthian has taken several more and it still isn’t there yet! (Frustrating!)

I think we need to take a low-level look at how Zynthian manages MIDI and consider if there is benefit from altering it to support sysex. I think there may be. It may allow load/save of engine data and/or load/save of whole Zynthian state (snapshot). The latter would require use of a MIDI manufacturer ID which may have a significant and ongoing cost.

14

FYI, currently zynthian MIDI router blocks ALL sysex data. It’s hardcoded here:

As @riban says, adding support for SysEx is far from trivial and must be carefully thought. I would like to see this implemented, but I don’t have much experience with SysEx and we would need a bunch of hardware devices and use-cases for testing. Of course, sooner than later, this will be implemented. You know how things work at zynthianland :wink:

If time is now, then we should define the use-cases we want to support:

1.) Receive SysEx from external devices
2.) Send SysEx to external devices
3.) Route SysEx from Input to Output
4.) Send SysEx to internal engines

We need good examples and probably we have to put some limits, so we need to find the values for these limits too.

Finally, we need a crew of beta-testers with devices supporting SysEx.

Regards,

2 Likes
15

I am happy to test as well as help with design and implementation. I have a few devices that send/receive sysex.

I think we should consider how we manage MIDI routing in general. There are a lot of things that Zynthian could do and users have struggled to understand how routing works and perform routing they require. If we have a robust, easy to use and understand MIDI routing system then we can ensure sysex can flow as required.

I would suggest offering the ability to filter/block some traffic over such routing.

3 Likes
16

I poked around in this area last summer and remember running into this wall of layers that would need this or that for it all to work. Which is kinda annoying when bare ALSA can manage all the routing and SysEx with no problems at all.

The ability to route to/from arbitrary ports would indeed seem like an important piece of the SysEx puzzle. Also from what I recall, mod-ttymidi doesn’t support receiving SysEx at all (but sending worked I think), so implementing that would be a prerequisite, otherwise it ends up being too much of a quirk in the system.

17

I’ve got a tubload of stuff that sysexes things around.

18

I think we should consider how we manage MIDI routing in general. There are a lot of things that Zynthian could do and users have struggled to understand how routing works and perform routing they require. If we have a robust, easy to use and understand MIDI routing system then we can ensure sysex can flow as required.

This i think is a very very necessary feature, as midi routing is non-trivial in the user nor the technical level, It can be really hard as a user to understand the way zynthian routes midi and treats different midi inputs and outputs. One way that I’ve been thinking about this particular issue is a change in the UI (zynth or webconf) and try and have more of a graphical view of what’s going on, as today it’s very hard to integrate zynthian in complex midi setups.

One idea that’s been floating in my head is a list (or table) of ports with cables, having inputs, outputs and the zynthian in the middle (kinda like patchage, but uglier and simpler) so you either route an input to an output directly, to the zynthian or through the zynthian and to the output. I am aware that this can be done as of today but I think the current interface makes it fairly complicated to understand.

This could also allow MPE support by limiting the MPE controller’s output to a specific layer on the zynthian, although I think that this would require extra modifications in the midi router.

19

I worry that a wiring type UI may become confusing quite quickly. As soon as wires start to overlap it can look confusing. None of us want tangled cables.

I think we should have context relevant connection like we have now in the engine configuration menu. We should also have a global view which could be a X/Y grid similar to how Ardour does it.

3 Likes
20

Also it needs to be something that displays easily on the gui at different scree sizes the nord modular (here he goes again) used cables but gave you the option to remove them because it quickly became a modular mess which made it difficult to see the modules. Possibly a use for the second hdmi connector on pi4s?