Testing / Stable

At club we discussed how much the testing branch deviates from stable during each development cycle. This topic allows us to understand why this happens and discuss how to reduce the impact of it

A while ago we moved towards a more frequent update of stable with more structure in the change process but this point in time sees us in a particularly significant period of change. The core Zynthian code is being rationalised and significant changes to UI to improve workflow. Testing needs to be bleeding edge for a while longer than we would ideally like. @jofemodo is doing far more hot-fixes to stable during this development cycle. We are also learning valuable lessons about user experience, documentation, etc. just like the subject of this discussion.

I think we should introduce a testing branch of documentation / wiki. This works allow me behaviour to be documented and that documentation be tested.

It may be appropriate to agree goals for each development cycle so that we know when it is finished and can merge into stable. That may also allow consideration of how much change is appropriate during a cycle.

1 Like

There are several ongoing threads of development and without resorting to Jira like management we do seem to exist in a hinterland between stable and testing.

Assuming a reductionist approach perhaps we could start by listing these separate areas and giving a current state on each in the Wiki to allow easy sight of objectives and milestones?
Suggestions, requests and demands, according to personality could perhaps be collected as well. It’s obvious there is much going on in the background but some element of interface stability would mean that exceedingly useful features like the testing Audio Mixer interface could be used in anger, whilst something like the audio recorder/playback functionality or zynseq don’t dangle as very useful but incomplete components within the new environment.
The listing of stable functionality, for instance, would at least ensure functionality isn’t being lost.

Code needs to be documented, at some stage. Ideally before the code get’s writ, but I don’t think I’ve ever actually seen it happen for real. Because the new technical innovations come from bleeding edge excitement being joyously presented to management, thus sparking the cash that scales the hastily constructed prototype to the first great success problem ( @wyleu see passim)

With objectives we get testable concepts, and realistically a robust API requires tests. The Raspberry Pi world would seem to be bottom up , and we are learning project management with an innocence and success that’s quite breathtaking!

Base Concepts, Blobs, areas of Interest, Nice to haves,classes, snapshots, must haves, hand written suggestions attached to a raven ( dead)…

An early list in no particular order.

Object Summary Comments Software Links
Audio Mixer GUI Main Interface in zynv2 Introduction of a snapshot instantiated mixer component -
ZynSeq Zynthian sequencer component Step sequencer introduced as first major addition to Zynthian GUI -
Audio Recorder Minimalist audio sample recorder Integrated with Audio Player -
Audio Playback Minimalist Audio sample playback Consumes audio samples from zynthian file structure -
Tuner GUI Tuner component Added after thread discussion Displaying control outputs on UI? - #32 by jofemodo
Layer visualisation Display a current GUI map of the Audio Server structure Zynthian specific patchage replacement Patchage
Webconf LV2 interface Web based Zynthian Interface for maintaining LV2 components - -
Keyboard Control of Interface Qwerty keyboard operation Ensure functionality defined, tested and documented Zynthian UI Users Guide - ZynthianWiki
OSC Control OSC Control Ensure functionality defined, tested and documented -
MIDI Control of Interface MIDI Interface control Ensure functionality defined, tested and documented -
Mouse control of UI interface USB Mouse control Ensure functionality defined, tested and documented -
Zynface The CV & Trigger interface - -

IF you want to play cut and paste on this, then this is a good read.

It’s probably best kept on a thread to develop and kicked around. A chattier place. Can’t see a way of embedding images in the table and, sadly, because real progress takes massive time, if we do transfer it to a wiki environment it will need a restructure but we would at least believe we have a list worth keeping.

Perhaps the list should be maintained as a json object?, no hold on, what if we developed it as a zynthian GUI LV2 component?

I suggest it’s renderered up by a zynthian at the start of every zynth club meeting.

Share and enjoy!

Actually I stopped using Testing branch after some bad experiences.
The major disadvantage is that it is not guaranteed that switching back from testing to stable works. In the end you have to burn a new image if you need a new stable version.
Imho you guys are doing a great job inventing new features but neglecting those who are using zynthian as an instrument.
There should be a primary directive: Never break snapshot. Provide an upgrade recipe, if you change the schema. And you need regression tests.

1 Like

The regression tests could be presented as an lv2 object that one loads as a snapshot.
We embody this philosophy by including MIDI & Audio test in admin.

The only difference would be asking the user if the zynth did or didn’t do what it described.

Sorry about my perpetually LV2 obsessed position, but it strikes me that the boundary between zynthian and this descriptive world is where we need clarity. To allow the wide grammar we have constructed to be efficiently defined.

In Testing we display the current snapshot as default as this is what’s in the namespace, testing would seem an obvious first alternative.

Loading the testing snapshot would provide an easy grip of life start up to test the machine which might not require a screen to run.

If we can launch a definition from within our environment that matches the functionality described in the documentation and performs the expected functions.

Course if it sounds good So much the better !

…scurries of to have another whirl round the turtle world.

I wasn’t around, when you decided to introduce the testing branch, so that I could have spoken up back then.
But I am not convinced, that this paradigm works at all.
I understand, that you chose this approach because you wanted to keep the “master” stable.
Now you have a testing branch that contains all small uncritical changes and the ones of the big new features.
And stable doesn’t show much progress. It’s rather the same as in the old days when we had a new image every year. Only worse because testing is quite unusable for months and small changes like new config values or new Pianoteq instruments cannot be used altough it is fixed for months.
Having a testing branch means, that you can push commits and hope, that somebody else is finding the bugs. But as not many people are using it, stable and testing are diverging with every commit.
The “advantage” of pushing into “master” is, that the 80% of non-professionals are testing it and finding a bug soon. Who is really using testing right now? Less than 10% of the users ?
The professional musicians or those who are preparing for a gig, could use a stable branch that is derived from master.
I would suggest 3 stages. The master where we push the normal little changes and which is bound to the zynthians by default.
A testing for the big-brian-changes. (doesn’t look like you have a big bmi though)
And a stable branch to which we could cherry-pick the commits that are older than X months.
If the 10% percent of the cutting edge power users are happy with their testing branch, it is merged into master (and hence x months later to stable)

Just my 2ct

and a little “funny” story, that isn’t so funny.
I was having rehearsal with big orchestra last thursday. Everything was ok…with confidence using two “original” zynthians and not those self-made ones like the years before. After approx. an hour suddently the Pianoteq instrument changed pitch. c became a b-flat. The bass player and I looked at each other and began to “argue”…everybody was playing the right note. until i played one on my second instrument and was convinced :slight_smile: A restart of the zynthian solved it…so much for stable build…sometimes I think, we need a “lock”-switch. So that only program change and volume works. No accidental save of snapshots possible or parameter changes of presets. I am using a Keylab 61 and the sliders can be mapped on 3 layers. It can happen, that when I am an pane B, I would change the sound although I wanted to change the volume, which is on pane A. But I have no clue, what the transpose of the Pianoteq could have triggered. Happened at home before as far as I remember but accused the testing build which I was running back then.

Hi @mheidt !

When we splitted branches, we wanted to keep testing “not so far” from stable, trying to make disrupting development in other branches, so you could switch-to and switch-back from testing without problems. We managed to do it like this for a time, but we didn’t calculate very well the impact of integrating the new audio mixer and here we are, with a testing branch that is disrupting and not what we planned for the testing branch. I’m really sorry about the current situation but we’ve decided to look forward, take a deep breath and try to move the zynthian software to the next level. We have good reasons to do so. I can only ask you to be patient and keep using stable for the music stuff, while having an alternate SD-card for testing.

Anyway, i will try to fix the snapshot disruption one-way, so you can use your stable snapshots on testing branch. Switching-back from testing to stable is going to be more challenging and i can’t say if it will be ever possible until we merge current testing branch on stable.

Best Regards,

2 Likes

That will never be possible because you have changes that cannot be reverted like system updates.
That’s why I said, that you need 3 levels as I described.

2 Likes

We already have three levels:

  • Stable should be the most tested and stable version with fewest bugs and most consistent behaviour, i.e. no feature changing between releases with only major bugs fixed
  • Testing provides the feature changes targeted for next stable release and will vary in features and stability
  • Development branches are branched from testing (or occasionally stgable) which implement specific features or bug fixes and are not for end users except where a specific bug / feature needs validating before being merged to testing / stable

We also have a period leading up to a new stable release when release candidates are issued to allow for wider testing.

We are on a continuous improvment cycle with the development process, learning lessons from our mistakes. At the moment there is too much change in testing and we are working to get to a state where we can consolidate that with stable but at the moment, if you are a musician, treat the testing branch as broken and stick with stable.

Major issues reported against stable will be triaged, and resolved as swiftly as we can, i.e. if you are on stable and stay there things shouldn’t get worse - only better.

We will endevour to ensure migration from current stable to next stable is as frictionless as possible and we should try to implement some form of roll-back in case of issues.

Perhaps we should list functionality in the Zynthian Stable branch:-

This is by no means complete, but please feel free to add elements.

Object Summary Comments Software Links
Audio Test Plays test wav to system output using alsa player via Admin -
MIDI Test Plays take 5 using current engines Requires engines to be selected via Admin -
Audio Recorder Records Audio from output - -
MIDI Recorder REcords MIDI from MIDI zynthian - -
Playback Audio Plays back a recorded audio wav file via Jack mechanism -
Playback MIDI Plays back MIDI file - -
Upload WAV file Allow upload of WAV file to zynthian directory Available via webconf -
Upload MIDI file Allows upload of MIDI File to zynthian directory Available via webconf -
Generate a layer Add a specified engine to a layer - -
Remove a Specific Layer - -
Modify A specific layer Substitute a selected layer for the current layer - -
Add an additional layer Allow two layers to be specified - -
Load a Snapshot Load a zss file replacing existing layers - -
Save a Snapshot Save the current layer config as a zss file - -
Clean All Remove all current layers - -
Panic All notes off should silence zynthian -

I think what @mheidt is driving at is something like this:

  • Master: Where bugfixes and minor changes are merged as soon as possible, so they can be further tested by a wide audience.
  • Stable: Lags master by a certain a mount of time (i.e. commits), after master has been thoroughly tested. Possibly commits need to be cherry-picked from master (e.g. bugfixes are cherry-picked faster than feature upgrades, depending on the nature of the original problem that called for the bugfix).
  • Testing: Development of new features such as the audio mixer etc. Features are merged to master once they are complete.

In addition, there could be development branches like you mentioned @riban, but that would be in addition to the three levels above.

Indeed, on my Zynthian, I maintain a ‘zyn-updates’ branch for zynthian-ui, where I’ve brought in bugfixes from ‘testing’ for the UI together with small features affecting the synth engines I use (mostly for Zynaddsubfx, since I’m using master there).

The problem as I see it is that someone would have to maintain the ‘stable’ branch on a more regular basis than today and decide when things are brought over from ‘master’, which would take time from other development work.

yes @ricard, you followed my thoughs.

this could be scripted. Lets say every 3 months we cherry-pick (scripted) all commits that are older than 3 months to a “prestage” branch and get this branch one week to be tested by us before it goes stable.
The cherry-pick-script shouldn’t take date but jofemodo decides, which commit 3 months ago would be a good candidate. We don’t want to split commits, that belong together.
Now we would have 4+(x developer) branches @riban.