Issue reporting and tracking

[riban]

There has been discussion in other threads about how the project is managed and how issues are reported and tracked. I am opening this topic to allow us to discuss the issue tracking methodology and (hopefully) agree a process.

@jofemodo manages the GitHub project and has set up repositories (with issue trackers), projects and kanban boards. I think we should have:

• Process for reporting and tracking issues
• Process for updating issue status (kanban board)
• Identify roles
• Describe what each issue (kanban) status means
• Specify which roles nominally perform which tasks

I started to draft some of this out for discussion. I have created flowcharts for the kanban (which I think needs an extra column for backlog) and initial triage process which are attached to this post.

I think we need these roles:

• Project Manager - overall control of the project
• Defect Manager - responsible for overall issue progress
• Team Member - anyone with permission to alter issue status
• User - anyone who reports issues or contributes to issue resolution

Note that roles are not necessarily mapped to individuals, i.e. one person may have multiple roles and one role may be performed by multiple people.

The kanban flowchart shows the role that is nominally responsible for moving tickets between columns. The rectangular blocks represent kanban columns.

It is possible for tickets to be moved back to Backlog, e.g. if progress has stopped, maybe due to a team member being unavailable or deciding they are unable to progress the issue.

The initial triage process shows how a new ticket is reviewed and added to the kanban board. Colour indicates who performs each task.

I think we should have these types of ticket:

• Defect / Bug - An issue with the product or supporting tools which impairs normal functionality
• Feature / Enhancement - A change to functionality
• Other - Anything that may not directly require change to the product or supporting tools, e.g. a discussion topic - though these should really be directed to Discourse

I think we should have issue status that aligns with the kanban columns. We should be able to mark a ticket as resolved / done (I prefer resolved) to indicate that the team has offered a resolution to the original reporter (or community at large).

It may be advantageous to have a resolution type which might include:

• Fixed
• Invalid
• Won’t Fix
• Duplicate
• Cannot reproduce (which may be covered by Invalid if we want to avoid subtle differences)

GitHub only really provides tags for indicating status so team members will need clear guidance on what tags represent what status and which tags are mutually exclusive, e.g. there should only be one resolution type.

I know this kind of thing can look complex / labour intensive / process heavy but may I assure you that having a consistent set of processes improves quality of issue reports and improves productivity. The trick is to get the minimal process that works and to adjust (and update docs) in light of experience.

[mheidt]

Could someone finally give him a real job

[riban]

I’m always open to offers .

One of the complications I have been trying to resolve is tying together the issue tracking of separate but related repositories. Do we need separate issue trackers for each repository? Can we just use one issue tracker for the whole Zynthian product? I cannot identify the requirement for separate issue trackers. Please enlighten me. Here are some advantages of a single issue tracker:

• Single point of access for users simplifies reporting
• Single tracker for all issues simplifies issue tracking
• Judicial use of tags allows association of issues with modules (repos)
• Common issue numbering system removes complexity of cross linking / referencing
• Issue tracking process simplified by one issue tracker linked to one kanban board (although you could have more boards linked to the same tracker if required)
• Clear picture of state of project without the need to cross reference separate issue trackers
• Common location for everyone to see status, users, developers, project management, etc.

[wyleu]

The existing system requires an almost prescient knowledge of where a problem may lie.

[riban]

I suggest using Labels to define which version/s an issue affects and Milestones to define which version resolution is scheduled, e.g. I report an issue and observe it occurs in Gorgona so I add the GORGONA label to the issue. The project manager decides this will be fixed in Aruk so assigns the Milestone ARUK. If the project manager thinks this should also be fixed in Gorgona then they also assign the Milestone GORGONA.

I have done some more flowcharts.This one shows the Project Manager’s triage process, that is the decision making on what to do with issues:

This one shows progress of ticket to close:

[wyleu]

Good to set this down

[jofemodo]

Your analysis is really interesting and complete @riban! I think we can take a good bunch of ideas from it and for sure that it will help to improve zynthian’s development workflow. Thanks a lot!!

Of course, we must avoid too much “burocracy”, because our development resources are precious and we shouldn’t spend too much time doing management tasks. Not to mention that programming is more funny

So, trying to take the key ideas from the last days discussion, while keeping the “low burocracy mantra” in my mind, i have a decent proposal to improve the current issue tracking:

zynthian/zynthian-issue-tracking

As you can see, i removed repo’s kanbans. Now we have only the global kanban, that can be filtered easily. We also have a global issue list, and per-repo issue lists.

The workflow should be:

1. Issue creation: Only from the zynthian-issue-tracking repo!
2. Triage: Assign initial tags and move to the right code-repo for deeper analysis and hopefully, solving.
3. Resolution: Awaiting => Analysis => Work in Progress => Testing => Done

Note that:

• The new kanban have these columns:

  • New
  • Freezer
  • Awaiting
  • Analysis
  • Work in Progress
  • Testing
  • Done

• Issue templates are used for creatiing “bug reports” and “feature requests”. Questions are redirected to forum.

Thanks

[wyleu]

Can we hoist this up on the Wiki?
Perhaps we could also adopt a BDD sort of questioning… ?

As a:
I want to:
So that:

[jofemodo]

@wyleu

Can we hoist this up on the Wiki?

We should. Your task But let’s wait a little bit to stabilize the thing …

@wyleu

Perhaps we could also adopt a BDD sort of questioning… ?

Uhmmmm! I normally prefer the more formal 3th person approach, but i’m not closed to this. Why do you think it’s a better approach?

Regards,

[wyleu]

Later automated analysis.

It forces a more semantically passable detection string and gets mere humans to compose statements in a much less conversational style.

…and that’s coming from me. . .

Put both; it’s really just name value stuff but make one mandatory( if we had infinite time to do such a thing) and put it first, worth doing it whilst we examine the concept.

[riban]

I am still not convinced that separate issue trackers for some (but not all) repos has benefit but let’s see how this goes. I have reported some bugs and features this morning so I can now assess how well the process works for a user which is a key reason for this - it should be easy for a user to report and follow an issue. (I have already submitted a feature request to improve that process.)

[jofemodo]

@riban

I am still not convinced that separate issue trackers f

I’m thinking about it. I’m not convinced too

The problem is the “transition” to the new model. We have an issue history on each repository and if i close the issue tracker for each repository, we lost access to this history. Of course we can move all issues to the new issue-tracking repo, but i’m not sure of how this would work regarding commit’s references and so on. We should study carefully this “transition”.

Let’s try like this and see …

Regards,

[wyleu]

@jofemodo

We should. Your task But let’s wait a little bit to stabilize the thing …

Ok. Place holder built.
Observation; as a music producing zynthezier would it not seem sensible to allow playing of musical extracts without downloading them first…?

All I’ve found so far is …
mediawiki TimedMediaHandler. . . .

[riban]

Oy! @wyleu - stay on topic!!! (First and final warning sergeant.)

[wyleu]

@wyleu gets allocated a mission . . .

https://youtu.be/KUd1OxPbKk4?t=3304 . . . . .

As far as issue tracking goes, I think there is a recognizable division that could be represented with at least a tag.

1/ That which actually contributes to the zynthian core product ( hifi berry card, all in one encoder 3.5 " screen

2/ that which is specifically outside of that product sphere ( 7" touchscreen implementation).

[riban]

@jofemodo

We should study carefully this “transition”

Indeed! I think that moving issues between issue trackers may retain their history but does not retain tags, etc. so it could prove irksome to do so.

A bit of feedback from using the current system:

• As issues are transferred from tracker to repo the issue number changes. This may result in confusion when cross referencing issues. I just did a cross reference and used url to link to each ticket.
• Keeping track of own tickets initially seems daunting when they change location but using the Global List then clicking on “Created” shows all the issues created by the currently logged in user (you / me) for that filter, i.e. everything I reported on the linked Zynthian projects.This is a nice feature which we should identify in any user guides we produce.
• We should get a feel for how well this works and when satisfied produce a simple guide for users
• The Bug and Feature request templates are not great. I tested these before and found the out-of-box templates to be inappropriate. We should configure the templates to request all and only the required info and present that info in a good format. I find I am replacing elements in the template or being inconsistent.
• Having reported the first issue from the template button I am tempted to report subsequent issues from the “New Issue” button that is displayed after an issue is added.
• If you follow a ticket that has been moved to a repo issue tracker then you may be tempted to report a new issue from there which is inappropriate. I wonder if this can be restricted.
• It is still not clear who manages each stage of a ticket life. I guess that will be @jofemodo though other contributors may be dealing with some tickets if they are the primary support for a module, e.g. webconf is mostly supported by @mheidt? Everyone managing tickets should use consistent workflow.
• The kanban columns are still not explained and may be interpreted differently by users. I am guessing the “Freezer” is like a backlog where items are put when not being actively worked on.

[jerash]

hi, could you share how you feel about this centralized issue tracking after a month ?
@jofemodo could you share how you made this work ? I don’t understand how issues are assigned to a repository or another

[riban]

I have looked at the kanban board and worked through some of the issue tracking workflows and have this feedback:

• Separate issue trackers for each project adds complexity with little tangible benefit
• Moving issues between projects retains history but not tags
• The quantity, description and automation of columns feels too high / complex
• Issues are automatically moved to Testing when they are closed but this has resulted in lots of tickets in the Testing column that are not being progressed

My suggestions for improvement are:

• Reduce quantity of columns to:
  • New (all new tickets created here)
  • In Progress (tickets that have been assigned to a developer or back to user for more info)
  • Freezer (tickets that have been triaged and deemed insufficiently urgent to assign for progress at this time)
  • Testing (tickets that have been resolved by developer but await validation that fix is tested)
  • Closed (tickets that have been tested or closed as won’t fix)
• Move all tickets to tracker project and disable reporting issues against individual repositories (if possible)
• Update metadata for each ticket moved to align with ticket handling process…
• Define ticket handling process, e.g. (roles are not necessarily individual people)
  • New ticket created by user
  • Defect manager reviews new tickets and moves to Freezer or In Progress
  • Defect manager assigns any In Progress tickets including questions back to reporter
  • Test manger moves ticket to Testing when development branch is merged with Testing branch, adding “Ready For Test” tag
  • Automation (or defect manager) moves ticket to Done when closed (ideally closed by reporter after testing)
  • Test manager moves ticket to In Progress if fails testing
  • Project or Defect manager moves tickets from Freezer to In Progress when appropriate to start progress, e.g. at start of a development sprint

We are constrained by GitHub limitations on issue management but can do much of what we need within these constraints will little effort. The roles mentioned above may be flexible and performed by one or more people:

• Project Manager (or Product Owner) is responsible for the overal project - @jofemodo
• Defect Manager is responsible for the issue tracking process
• Testing Manager ensures sufficient testing is performed to provide confidence in operational state of testing and stable branches.
• Release Manger ensures release occurs with minimal disruption, is supportable, is advertised, has sufficient rollback provision, etc.

These roles interact and may be performed by different people depending on availability or context.

Testing should be supported by test scripts, etc. Releases should be supported by documentation, etc.