# API Proposal

**URL:** <https://discourse.zynthian.org/t/api-proposal/6481>\
**Category:** Development\
**Created:** [December 4, 2021, 8:58am UTC](https://discourse.zynthian.org/t/api-proposal/6481 "2021-12-04T08:58:20Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![riban](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/riban/32/2331_2.png) [@riban](https://discourse.zynthian.org/u/riban)\
**Post date:** [December 4, 2021, 8:58am UTC](https://discourse.zynthian.org/t/api-proposal/6481/1 "2021-12-04T08:58:20Z")

</div>

Hi

There have been a few topics here discussing implementing an Application Programming Interface (API) but none have progressed too far. I have been working on this a bit lately and have a high-level proposal for the kind of approach and type of monitoring and control that we may want / need to implement. I have intentionally avoided considering the low-level technology to implement this because I don’t want the high-level design to be driven or constrained by preconceptions of specific technology.

The aim is to separate user interface from core functionality, providing a core accessible via the API and allowing different user interfaces to be designed. These may include the current GUI with 4 encoders, simple stomp boxes, web interfaces (like webconf), etc.

Here is the initial high-level proposal which I publish here for comment but should probably live somewhere (like GitHub). Please take a look if you are interested in using an API and comment on errors, omissions, etc. We will soon be looking at the low level technology to implement this. The API _may_ replace CUIA and other access methods, e.g. direct OSC monitoring and control of mixer.

* * *

**Zynthian API proposal**

_ **Purpose** _

Zynthian-core is (will be) the module that provides core functionality for Zynthian. This includes managing audio and MIDI processing engines, signal routing, monitoring, etc. The API will provide control and monitoring of zynthian-core allowing different user interfaces to be connected.

The current implementation of Zynthian (2109) does not separate UI from core functionality. This will be refactored to separate zynthian-core from zynthian-ui (and other elements of ZynthianOS). An interim API may be implemented by wrapping 2109 zynthian-ui to present the API as if only zynthian-core is running. There is an overhead to this process which will give reduced performance.

_ **High Level Design** _

A model-view-controller approach is proposed. Zynthian-core will contain a _model_ that holds the current configuration and data. This is equivalent to the current snapshot, containing all data to describe the state of the system. It will also contain much of the _controller_, allowing the control of the _model_ although some elements of the controller may be implemented in the user interface modules. Interaction with users will be implemented by the _view_ which may be any user interface. The UI may also implement some controller elements to enhance functionality, e.g. add extra features not embedded within the _controller_ / _model_. The API will provide Create/Read/Update/Delete (CRUD) type access to the _model_ and functions of zynthian-core as well as real-time functions that do not fit the CRUD access model. The API will be extensible to allow addition of data without breaking the API and use major.minor version to indicate breaking / non-breaking changes.

**Create** - Clients may request elements be added to the _model_.

**Read** - Clients may request (almost) any data within the _model_. This may be polled and/or accessed via a subscription + publish mechanism to allow registration by API clients for data to be dynamically published to the client as the data changes. (These may be considered the pull and push mechanisms.) These two approaches allows for static or slow changing data to be requested / polled whilst more urgent or dynamic data to be published in a timely manner or for an event driven client. Not all data will be available to the subscription + publish model to avoid excessive code and overhead - not all data is required.

**Update** - Clients may request changes to data / states.

**Delete** - Clients may request elements be removed from the _model_.

There may be a permissions scheme applied for clients to allow / restrict some operations. This is not high priority but may prove advantageous, e.g. allowing tightly bound modules fuller access than remote control devices.

_ **Detailed Design** _

The specific technology will be chosen to suit requirements. This will consider overhead, latency, security, likely (forecast) clients, etc.

Client connections may be session or non-session based. It should be possible for a client to make a request without expecting a response. It should also be possible for a client to validate a request has succeeded. This may be via a session based connection, a data subscription or a poll.

**Sessions** - A session is a bi-directional connection between client and server. The client connects to the server and remains connected throughout the session. Requests made by the client can be responded to by the server over the connection. The server may know about current sessions hence be able to send dynamic data to connected clients.

**Connectionless** - Connectionless is a uni-directional transmission of data. A sender may send a request or data to a receiver. There may not be a concept of connection so any dynamically updated data may need a keep-alive and timeout mechanism to avoid dead clients being sent data. A client may subscribe for data then request a change and be notified of the change through the subscription publication. It will also be notified via the same mechanism for changes made by other clients or the core.

Choice of session or connectionless based API will be considered with use cases and likely (forecast) clients. One or both may be adopted. The higher-level concepts of subscription + publish, etc. will be implemented and selected lower-level data transfer mechanism(s) will implement this.

_ **Specific Data** _

The following is a non-exhaustive list of data that may be implemented in the _model_ / API access methods and real-time messages. The suffix in square braces identifies how the data may be manipulated by the API [CRUD].

- Mixer - Multiple channels of audio mixed to a main stereo mix-bus

- Chain - Group of interconnected engines

- Engine - Instance of an engine class

- Engine Class - Types of audio / MIDI processors / generators

- Routing Graph - graph of MIDI and audio routes

- Presets - Saved configuration of an engine

- Snapshots - Full model data

- Physical UI - Switches, encoders, potentiometers, etc

- Step Sequencer - Manipulation of patterns, sequences, etc.

- Real Time Messaging - Interface for low latency, real-time messages \*Some other messages may need to be reduced latency, e.g. mixer control and monitoring though maybe not real-time, i.e. there may be different levels of latency message type

- System - Monitor and control the overall system

- GUI - Zynthian specific monitoring and control of GUI, extensible / optional to allow system specific implementation

[Edit] This text is updated to reflect discussion and decisions made below.

---

<div class="post-metadata">

**Author:** ![Pastitas](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/pastitas/32/8243_2.png) [@Pastitas](https://discourse.zynthian.org/u/Pastitas)\
**Post date:** [December 4, 2021, 8:01pm UTC](https://discourse.zynthian.org/t/api-proposal/6481/2 "2021-12-04T20:01:53Z")

</div>

that’s a great high-level definition for such a complex system, good job!!  
It will be interesting how this implementation might enable the community to better integrate their own new spin into things, such a system reminds me of the community behind monome norns and how It’s (admittedly with a steep learning curve) more accessible approach to a code-based DSP environment has had user-developed content become part of the original project and device, making the it all way bigger then the sum of it’s parts.

Here’s to zynthian’s bright future, with such great minds at the wheel who know’s where we’ll go.

---

<div class="post-metadata">

**Author:** ![jofemodo](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/jofemodo/32/2_2.png) [@jofemodo](https://discourse.zynthian.org/u/jofemodo)\
**Post date:** [December 5, 2021, 11:10am UTC](https://discourse.zynthian.org/t/api-proposal/6481/3 "2021-12-05T11:10:29Z")

</div>

Hi @riban !

Great work, mate! The time is coming … in fact it’s almost now. Let’s do it!

Thanks!

---

<div class="post-metadata">

**Author:** ![notmart](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/notmart/32/7267_2.png) [@notmart](https://discourse.zynthian.org/u/notmart)\
**Post date:** [December 6, 2021, 9:27am UTC](https://discourse.zynthian.org/t/api-proposal/6481/4 "2021-12-06T09:27:12Z")

</div>

Just a clarification:  
what you call “nodes” there, corresponds pretty much to what in the old ui and ui code is called “layers” ?

---

<div class="post-metadata">

**Author:** ![riban](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/riban/32/2331_2.png) [@riban](https://discourse.zynthian.org/u/riban)\
**Post date:** [December 6, 2021, 11:48am UTC](https://discourse.zynthian.org/t/api-proposal/6481/5 "2021-12-06T11:48:51Z")

</div>

No, a node within this description is an engine. I have renamed “Layer” as “Chain”. (In fact, “Layer” is used ambiguously within current Zynthian code but what a user would think of as a Layer is now Chain - at least in this iteration of the API.)

[Edit] I may change “node” to “engine” or similar to release the word “node” for use as a point in the connection graph, e.g. audio output, etc.

---

<div class="post-metadata">

**Author:** ![jofemodo](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/jofemodo/32/2_2.png) [@jofemodo](https://discourse.zynthian.org/u/jofemodo)\
**Post date:** [December 6, 2021, 5:01pm UTC](https://discourse.zynthian.org/t/api-proposal/6481/6 "2021-12-06T17:01:40Z")

</div>

> [@riban](#):
>
> a node within this description is an engine

I would say a node is “an engine instance” inside a “chain” ;-b  
Of course, you can have a single “chain” with a single “engine” instance, for instance if you only want to use Pianoteq.

Best Regards,

---

<div class="post-metadata">

**Author:** ![jofemodo](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/jofemodo/32/2_2.png) [@jofemodo](https://discourse.zynthian.org/u/jofemodo)\
**Post date:** [December 6, 2021, 11:10pm UTC](https://discourse.zynthian.org/t/api-proposal/6481/7 "2021-12-06T23:10:56Z")

</div>

Regarding the protocol, i would suggest json-rpc:

> **[JSON-RPC](https://en.m.wikipedia.org/wiki/JSON-RPC)**
>
> JSON-RPC is a remote procedure call protocol encoded in JSON. It is similar to the XML-RPC protocol, defining only a few data types and commands. JSON-RPC allows for notifications (data sent to the server that does not require a response) and for multiple calls to be sent to the server which may be answered asynchronously.
> JSON-RPC works by sending a request to a server implementing this protocol. The client in that case is typically software intending to call a single method of a remote system....

or gRPC:

> **[gRPC](https://grpc.io/)**
>
> A high-performance, open source universal RPC framework

Both are lightweight and easy to implement on almost any language.

gRPC is probably better, but i only have experience with json-rpc, so we should do a deeper research.

Any other proposal?

Regards

---

<div class="post-metadata">

**Author:** ![riban](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/riban/32/2331_2.png) [@riban](https://discourse.zynthian.org/u/riban)\
**Post date:** [December 7, 2021, 8:33am UTC](https://discourse.zynthian.org/t/api-proposal/6481/8 "2021-12-07T08:33:03Z")

</div>

I think we need a low-level interface that allows close binding so that core functionality such as the main UI can be integrated with minimal overhead and latency. I suggest C/C++ and Python bindings. We can then add higher level remote interfaces such as the two that @jofemodo mentions.

---

<div class="post-metadata">

**Author:** ![le51](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/le51/32/1706_2.png) [@le51](https://discourse.zynthian.org/u/le51)\
**Post date:** [December 7, 2021, 10:38am UTC](https://discourse.zynthian.org/t/api-proposal/6481/9 "2021-12-07T10:38:05Z")

</div>

Hi,  
On the “view” side, is it planned to use the actual graphical toolkit ?

---

<div class="post-metadata">

**Author:** ![Baggypants](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/baggypants/32/1911_2.png) [@Baggypants](https://discourse.zynthian.org/u/Baggypants)\
**Post date:** [December 7, 2021, 12:31pm UTC](https://discourse.zynthian.org/t/api-proposal/6481/10 "2021-12-07T12:31:49Z")

</div>

Of you have an engine with effects are the effects then nodes along the chain?

---

<div class="post-metadata">

**Author:** ![riban](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/riban/32/2331_2.png) [@riban](https://discourse.zynthian.org/u/riban)\
**Post date:** [December 7, 2021, 7:13pm UTC](https://discourse.zynthian.org/t/api-proposal/6481/11 "2021-12-07T19:13:53Z")

</div>

@le51 The view is the GUI so reimplementing the current UI would put the tkinter code in the view. Other UI may be implemented using different toolkits or even hardware, e.g. a foot pedal might be considered a view of it included some configuration element. (Switches themselves are part of the controller.)

@Baggypants Any engine is a node (or maybe an engine) whether that be a source like a synth or a processor like an effect. A chain is the combination of engines in a serial and/or parallel chain. The effects within an engine are not treated separately, e.g. if a synth has reverb then that is part of the synth engine/node.

---

<div class="post-metadata">

**Author:** ![le51](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/le51/32/1706_2.png) [@le51](https://discourse.zynthian.org/u/le51)\
**Post date:** [December 7, 2021, 11:43pm UTC](https://discourse.zynthian.org/t/api-proposal/6481/12 "2021-12-07T23:43:55Z")

</div>

> [@riban](#):
>
> The view is the GUI so reimplementing the current UI would put the tkinter code in the view. Other UI may be implemented using different toolkits or even hardware, e.g. a foot pedal

I’ve looked at your code for the ZynMixer patch, it’s clear and commented 👍 but the “view” part is the trickyest.

Regarding the API

I’ve just found this maybe it could be of some interest.

> Mentat is a HUB / Conductor for OSC / MIDI capable softwares. It aims to centralize all controls in one place, manage their state and create routings.  
> Mentat is a module for [python 3]

They are engines, modules, routes …

[Mentat doc](https://jean-emmanuel.github.io/mentat/)  
[Github repo](https://github.com/jean-emmanuel/mentat)

---

<div class="post-metadata">

**Author:** ![riban](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/riban/32/2331_2.png) [@riban](https://discourse.zynthian.org/u/riban)\
**Post date:** [December 8, 2021, 8:27am UTC](https://discourse.zynthian.org/t/api-proposal/6481/13 "2021-12-08T08:27:48Z")

</div>

I would like to maintain an OSC interface and expand / rationalalise it to match the new API. We can also add other remote access interfaces. We could farm out each interface to different developers. We shouldn’t spread ourselves too thinly. We don’t want lots of half finished interfaces.

I think we should start with c / c++ and python bindings then add a remote interface, maybe OSC first as that is already partially implemented, simple and we’ll understood by some of us.

---

<div class="post-metadata">

**Author:** ![jofemodo](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/jofemodo/32/2_2.png) [@jofemodo](https://discourse.zynthian.org/u/jofemodo)\
**Post date:** [December 8, 2021, 10:07am UTC](https://discourse.zynthian.org/t/api-proposal/6481/14 "2021-12-08T10:07:43Z")

</div>

> [@riban](#):
>
> I suggest C/C++ and Python bindings.

Of course. My proposal is for “remote API” access.

---

<div class="post-metadata">

**Author:** ![jofemodo](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/jofemodo/32/2_2.png) [@jofemodo](https://discourse.zynthian.org/u/jofemodo)\
**Post date:** [December 8, 2021, 11:07am UTC](https://discourse.zynthian.org/t/api-proposal/6481/15 "2021-12-08T11:07:03Z")

</div>

> [@riban](#):
>
> Any engine is a node (or maybe an engine) whether that be a source like a synth or a processor like an effect. A chain is the combination of engines in a serial and/or parallel chain.

I like using “node” for an “engine instance” inside a “chain”. Please, don’t use “engine” alone for meaning a “node”, because it would create confusion. We could use “engine instance”, but “node” is shorter and cleaner 😉

So let’s define what a chain should be. It could be like this:

- Every chain is composed of “nodes”, that are audio/MIDI processing units (engine instances!). Nodes are interconnected inside the chain. We should discuss the details of this “interconnection”. FMPOV, this is one of the hottest points 😉

- Every chain is attached to a mixer channel ( **mixchan** ). This could be the chain’s ID, what do you think @riban? Or do you think it could be useful to assign several chains to the same mixer chain?

- Every chain receive MIDI messages from a **list of MIDI sources**. Each **midisrc** is defined by a **mididev** and/or a set of **midichans**. Mididev or midichans could be empty, meaning that it applies to all devs or MIDI channels. Of course, the **midisrc** list could be empty, what means this chain doesn’t receive MIDI from any source. Please, note this doesn’t affect the MIDI learning mechanism!  
Of course, we have to rethink (and rename!!) the “stage-mode” with all this on mind. @riban suggested to flag input devices for including/excluding them into the “stage mode mechanism”. I think it’s a good idea.

- Every chain have a standard MIDI input filter that allows:

- Every chain sends MIDI messages to a **list of MIDI destinations**. Each **mididest** is defined by a **mididev** and/or a **midichan**. An empty mididev means it sends to all output devices. If midichan is set for a mididest, it means messages will be translated to this channel before sending to the output device.

I would like to extract some keywords for including into our “developers dictionary” and avoid further confusion while increasing efficiency on communication:

- **engine**
- **node** (engine instance)
- **chain**
- **mixchan**
- **midichan**
- **mididev**
- **midisrc**
- **mididest**

Ideas? Suggestions?

Enjoy!

---

<div class="post-metadata">

**Author:** ![riban](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/riban/32/2331_2.png) [@riban](https://discourse.zynthian.org/u/riban)\
**Post date:** [December 9, 2021, 7:17am UTC](https://discourse.zynthian.org/t/api-proposal/6481/16 "2021-12-09T07:17:30Z")

</div>

> [@jofemodo](#):
>
> I like using “node” for an “engine instance” inside a “chain”. Please, don’t use “engine” alone for meaning a “node”, because it would create confusion. We could use “engine instance”, but “node” is shorter and cleaner

A node (or vertex) is a point within a graph and an interconnect (or edge) is a connection between nodes. It makes sense for us to map the jack graph into Zynthian data model so I propose we reserve the word **node** for a node within the jack graph, i.e. an individual audio or MIDI input or output.

We have types or classes of engine from which we create instances. I would prefer to use **engine class** to describe the type of engine and **engine** to describe the instance. If we compare the analogy, A Land Rover Freelander may have a TD4 engine. We refer to the **engine** in each car (which is an instance of a TD4 class of engine).

> [@jofemodo](#):
>
> Every chain is attached to a mixer channel ( **mixchan** ). This could be the chain’s ID, what do you think @riban?

Actually this may not be true. A purely MIDI chain will not connect to an audio mixer channel. There may be an argument for handling audio and MIDI chains separately.

> [@jofemodo](#):
>
> Or do you think it could be useful to assign several chains to the same mixer chain?

I don’t think this gives significant benefit and adds complexity which can lead to confusion. One mixer channel per audio chain.

> [@jofemodo](#):
>
> Every chain receive MIDI messages from a **list of MIDI sources**.

This is similar to the graph in that a chain may receive MIDI messages from zero or more MIDI source nodes. We should consider how this is optimised (routed in jack / filtered within zynthian module). Should we limit to this or do we allow routing MIDI to any engine within a chain? If there is a constraint that a maximum of one engine in a chain benefits from MIDI input this would be fine, e.g. only one synth engine exists to be triggered by MIDI events or only the first MIDI effect receives MIDI input. But what if we want more complex configuration, e.g. a chain contains multiple manuals of an organ with each expecting a different MIDI controller input?

> [@jofemodo](#):
>
> Each **midisrc** is defined by a **mididev** and/or a set of **midichans**.

I don’t understand what a **mididev** refers to. @jofemodo please explain.

> [@jofemodo](#):
>
> Every chain have a standard MIDI input filter

Yes but it should be bypassed by default and when the settings are neutral, i.e. if a user sets transpose to zero then there should be no processing performed. A similar thing _could_ be put into audio inputs, e.g. a gate that is only instantiated if non-neutral setting selected. On that point - I think the audio inputs should default to being routed to mixer channels - probably mono, i.e. input A goes to channel 1, input B goes to channel 2.

> [@jofemodo](#):
>
> some keywords for including into our “developers dictionary”

Such a dictionary or glossary should include definitions too, e.g.

| name | description |
| --- | --- |
| **engineclass** | A class or type of MIDI or audio generator or processor, e.g. Fluidsynth |
| **engine** | An instance of an engine class |
| **node** | A MIDI or audio source or destination within the jack graph |
| **interconnect** | A MIDI or audio connection between two nodes |
| **chain** | Group of nodes, and interconnects |
| **mixchan** | Channel strip in audio mixer |

I am not sure if we need midisrc and mididest if we have nodes which we might describe as **midi source node**. We need to distinguish between MIDI and audio nodes and between source and destination nodes. We either do so with modifiers like MIDI, audio, source, destination or by having 4 different entries in our glossary, e.g. **midisrc** , **mididest** , **audiosrc** , **audiodest** which are all nodes in the jack graph.

The API should avoid constraints although the implementation may have constraints which should be validated by the API, e.g. the audio mixer is currently 16 channels maximum but this may change.

[Edit] I have updated the initial post (high-level API proposal) with some of these suggestions - conversation can continue…

---

<div class="post-metadata">

**Author:** ![notmart](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/notmart/32/7267_2.png) [@notmart](https://discourse.zynthian.org/u/notmart)\
**Post date:** [December 9, 2021, 10:34am UTC](https://discourse.zynthian.org/t/api-proposal/6481/17 "2021-12-09T10:34:13Z")

</div>

ok, this clarifies things a lot, so a chain is a chain of engines, which one would be the engine main synth, the other ones the effects…

---

<div class="post-metadata">

**Author:** ![riban](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/riban/32/2331_2.png) [@riban](https://discourse.zynthian.org/u/riban)\
**Post date:** [December 9, 2021, 10:57am UTC](https://discourse.zynthian.org/t/api-proposal/6481/18 "2021-12-09T10:57:39Z")

</div>

Close enough! A chain of engines can be quite complex with audio generators (like synths) and / or effects and / or MIDI processing. For synth chains we tend to think of the synths as the root engine with a chain of MIDI processing engines feeding its input and a chain of audio processing from its output feeding the mixer. The chain can have serial and parallel paths.

---

<div class="post-metadata">

**Author:** ![notmart](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/notmart/32/7267_2.png) [@notmart](https://discourse.zynthian.org/u/notmart)\
**Post date:** [December 9, 2021, 10:59am UTC](https://discourse.zynthian.org/t/api-proposal/6481/19 "2021-12-09T10:59:33Z")

</div>

> [@riban](#):
>
> Snapshots - Full model data
> 
> - Available snapshots [CRUD] \*List of snapshots but also methods to create, update and delete snapshots
> - Name [-RU-]
> - Data [-RU-] \*Full state model

A couple of things about snapshots:  
for the functionality we need, we also had to add the function of a “partial” snapshot that encompasses just a subset of layers (chains?) (we do save a layer and everything in midi cloned to)

this to share online a particular sound with all its settings

we would need that kind of functionality as well

---

<div class="post-metadata">

**Author:** ![riban](https://discourse.zynthian.org/user_avatar/discourse.zynthian.org/riban/32/2331_2.png) [@riban](https://discourse.zynthian.org/u/riban)\
**Post date:** [December 9, 2021, 11:06am UTC](https://discourse.zynthian.org/t/api-proposal/6481/20 "2021-12-09T11:06:01Z")

</div>

Yep! The interface (which is the next task to define) will allow for loading parts of a snapshot expanding on the current functionality. This will allow, amongst other things, the ability to load / overwrite / merge a filtered selection, e.g. just chain 1. I think the high-level description which says snapshots can be created, read, updated and deleted allows for this. I noticed last night that the section on Snapshots is short but it kind of has to be otherwise it would restate everything else as it is the whole data model.

[Next page](https://discourse.zynthian.org/t/api-proposal/6481.md?page=2)