At club we discussed how much the testing branch deviates from stable during each development cycle. This topic allows us to understand why this happens and discuss how to reduce the impact of it
A while ago we moved towards a more frequent update of stable with more structure in the change process but this point in time sees us in a particularly significant period of change. The core Zynthian code is being rationalised and significant changes to UI to improve workflow. Testing needs to be bleeding edge for a while longer than we would ideally like. @jofemodo is doing far more hot-fixes to stable during this development cycle. We are also learning valuable lessons about user experience, documentation, etc. just like the subject of this discussion.
I think we should introduce a testing branch of documentation / wiki. This works allow me behaviour to be documented and that documentation be tested.
It may be appropriate to agree goals for each development cycle so that we know when it is finished and can merge into stable. That may also allow consideration of how much change is appropriate during a cycle.
There are several ongoing threads of development and without resorting to Jira like management we do seem to exist in a hinterland between stable and testing.
Assuming a reductionist approach perhaps we could start by listing these separate areas and giving a current state on each in the Wiki to allow easy sight of objectives and milestones?
Suggestions, requests and demands, according to personality could perhaps be collected as well. Itās obvious there is much going on in the background but some element of interface stability would mean that exceedingly useful features like the testing Audio Mixer interface could be used in anger, whilst something like the audio recorder/playback functionality or zynseq donāt dangle as very useful but incomplete components within the new environment.
The listing of stable functionality, for instance, would at least ensure functionality isnāt being lost.
Code needs to be documented, at some stage. Ideally before the code getās writ, but I donāt think Iāve ever actually seen it happen for real. Because the new technical innovations come from bleeding edge excitement being joyously presented to management, thus sparking the cash that scales the hastily constructed prototype to the first great success problem ( @wyleu see passim)
With objectives we get testable concepts, and realistically a robust API requires tests. The Raspberry Pi world would seem to be bottom up , and we are learning project management with an innocence and success thatās quite breathtaking!
Ensure functionality defined, tested and documented
-
Zynface
The CV & Trigger interface
-
-
IF you want to play cut and paste on this, then this is a good read.
Itās probably best kept on a thread to develop and kicked around. A chattier place. Canāt see a way of embedding images in the table and, sadly, because real progress takes massive time, if we do transfer it to a wiki environment it will need a restructure but we would at least believe we have a list worth keeping.
Perhaps the list should be maintained as a json object?, no hold on, what if we developed it as a zynthian GUI LV2 component?
I suggest itās renderered up by a zynthian at the start of every zynth club meeting.
Actually I stopped using Testing branch after some bad experiences.
The major disadvantage is that it is not guaranteed that switching back from testing to stable works. In the end you have to burn a new image if you need a new stable version.
Imho you guys are doing a great job inventing new features but neglecting those who are using zynthian as an instrument.
There should be a primary directive: Never break snapshot. Provide an upgrade recipe, if you change the schema. And you need regression tests.
The regression tests could be presented as an lv2 object that one loads as a snapshot.
We embody this philosophy by including MIDI & Audio test in admin.
The only difference would be asking the user if the zynth did or didnāt do what it described.
Sorry about my perpetually LV2 obsessed position, but it strikes me that the boundary between zynthian and this descriptive world is where we need clarity. To allow the wide grammar we have constructed to be efficiently defined.
In Testing we display the current snapshot as default as this is whatās in the namespace, testing would seem an obvious first alternative.
Loading the testing snapshot would provide an easy grip of life start up to test the machine which might not require a screen to run.
If we can launch a definition from within our environment that matches the functionality described in the documentation and performs the expected functions.
Course if it sounds good So much the better !
ā¦scurries of to have another whirl round the turtle world.
I wasnāt around, when you decided to introduce the testing branch, so that I could have spoken up back then.
But I am not convinced, that this paradigm works at all.
I understand, that you chose this approach because you wanted to keep the āmasterā stable.
Now you have a testing branch that contains all small uncritical changes and the ones of the big new features.
And stable doesnāt show much progress. Itās rather the same as in the old days when we had a new image every year. Only worse because testing is quite unusable for months and small changes like new config values or new Pianoteq instruments cannot be used altough it is fixed for months.
Having a testing branch means, that you can push commits and hope, that somebody else is finding the bugs. But as not many people are using it, stable and testing are diverging with every commit.
The āadvantageā of pushing into āmasterā is, that the 80% of non-professionals are testing it and finding a bug soon. Who is really using testing right now? Less than 10% of the users ?
The professional musicians or those who are preparing for a gig, could use a stable branch that is derived from master.
I would suggest 3 stages. The master where we push the normal little changes and which is bound to the zynthians by default.
A testing for the big-brian-changes. (doesnāt look like you have a big bmi though)
And a stable branch to which we could cherry-pick the commits that are older than X months.
If the 10% percent of the cutting edge power users are happy with their testing branch, it is merged into master (and hence x months later to stable)
and a little āfunnyā story, that isnāt so funny.
I was having rehearsal with big orchestra last thursday. Everything was okā¦with confidence using two āoriginalā zynthians and not those self-made ones like the years before. After approx. an hour suddently the Pianoteq instrument changed pitch. c became a b-flat. The bass player and I looked at each other and began to āargueāā¦everybody was playing the right note. until i played one on my second instrument and was convinced A restart of the zynthian solved itā¦so much for stable buildā¦sometimes I think, we need a ālockā-switch. So that only program change and volume works. No accidental save of snapshots possible or parameter changes of presets. I am using a Keylab 61 and the sliders can be mapped on 3 layers. It can happen, that when I am an pane B, I would change the sound although I wanted to change the volume, which is on pane A. But I have no clue, what the transpose of the Pianoteq could have triggered. Happened at home before as far as I remember but accused the testing build which I was running back then.
When we splitted branches, we wanted to keep testing ānot so farā from stable, trying to make disrupting development in other branches, so you could switch-to and switch-back from testing without problems. We managed to do it like this for a time, but we didnāt calculate very well the impact of integrating the new audio mixer and here we are, with a testing branch that is disrupting and not what we planned for the testing branch. Iām really sorry about the current situation but weāve decided to look forward, take a deep breath and try to move the zynthian software to the next level. We have good reasons to do so. I can only ask you to be patient and keep using stable for the music stuff, while having an alternate SD-card for testing.
Anyway, i will try to fix the snapshot disruption one-way, so you can use your stable snapshots on testing branch. Switching-back from testing to stable is going to be more challenging and i canāt say if it will be ever possible until we merge current testing branch on stable.
That will never be possible because you have changes that cannot be reverted like system updates.
Thatās why I said, that you need 3 levels as I described.
Stable should be the most tested and stable version with fewest bugs and most consistent behaviour, i.e. no feature changing between releases with only major bugs fixed
Testing provides the feature changes targeted for next stable release and will vary in features and stability
Development branches are branched from testing (or occasionally stgable) which implement specific features or bug fixes and are not for end users except where a specific bug / feature needs validating before being merged to testing / stable
We also have a period leading up to a new stable release when release candidates are issued to allow for wider testing.
We are on a continuous improvment cycle with the development process, learning lessons from our mistakes. At the moment there is too much change in testing and we are working to get to a state where we can consolidate that with stable but at the moment, if you are a musician, treat the testing branch as broken and stick with stable.
Major issues reported against stable will be triaged, and resolved as swiftly as we can, i.e. if you are on stable and stay there things shouldnāt get worse - only better.
We will endevour to ensure migration from current stable to next stable is as frictionless as possible and we should try to implement some form of roll-back in case of issues.
I think what @mheidt is driving at is something like this:
Master: Where bugfixes and minor changes are merged as soon as possible, so they can be further tested by a wide audience.
Stable: Lags master by a certain a mount of time (i.e. commits), after master has been thoroughly tested. Possibly commits need to be cherry-picked from master (e.g. bugfixes are cherry-picked faster than feature upgrades, depending on the nature of the original problem that called for the bugfix).
Testing: Development of new features such as the audio mixer etc. Features are merged to master once they are complete.
In addition, there could be development branches like you mentioned @riban, but that would be in addition to the three levels above.
Indeed, on my Zynthian, I maintain a āzyn-updatesā branch for zynthian-ui, where Iāve brought in bugfixes from ātestingā for the UI together with small features affecting the synth engines I use (mostly for Zynaddsubfx, since Iām using master there).
The problem as I see it is that someone would have to maintain the āstableā branch on a more regular basis than today and decide when things are brought over from āmasterā, which would take time from other development work.
this could be scripted. Lets say every 3 months we cherry-pick (scripted) all commits that are older than 3 months to a āprestageā branch and get this branch one week to be tested by us before it goes stable.
The cherry-pick-script shouldnāt take date but jofemodo decides, which commit 3 months ago would be a good candidate. We donāt want to split commits, that belong together.
Now we would have 4+(x developer) branches @riban.
A restart of the zynthian solved itā¦so much for stable buildā¦sometimes I think, we need a ālockā-switch. So that only program change and volume works. No accidental save of snapshots possible or parameter changes of presets. I am using a Keylab 61 and the sliders can be mapped on 3 layers. It can happen, that when I am an pane B, I would change the sound although I wanted to change the volume, which is on pane A. But I have no clue, what the transpose of the Pianoteq could have triggered. Happened at home before as far as I remember but accused the testing build which I was running back then.