Changing the workflow.

Development archive
#1

Hello coding RIOTers, After I needed to rebase several PRs multiple times last week I’m really fed up with our current workflow. Furthermore, I discovered three things that show that we are in need of a more established workflow.

  1. our community is growing
  2. our master branch is changing constantly and has features in it that are sometimes thrown out the window weeks after they where introduced (e.g. radio_types.h or the older versions of netapi/netdev)
  3. we introduced feature-centric task forces to our communities

This is why I propose we change to a slightly adapted topic branch workflow (also known as feature branch) workflow [1]:

  • the main RIOT-OS/RIOT repository will get the following branches
  • master: points to the stable version of our latest release
  • one stable branch for every major release (?)- devel: points to current development version (what is currently our master), hot fixes can be cherry-picked to the master from here.
  • branching from devel: feature branch for every task force that is currently in active development, current changes will be merged regularly from devel by the branches maintainers- Pull Requests will be made either to devel (default) or the corresponding feature branch
  • feature branches will be merged into devel if the feature reaches sufficient stability.
  • devel will be merged into master, when a new release is out and master’s new HEAD tagged with the release number.

I observed two extra benefits from that:

  • accidentally merged “SQUASH ME” commits can be squashed further upstream, before we add the commits to master or devel
  • we can make maintenance of features more granular (and maybe solve our maintenance problem) by enacting the “Dictator and Lieutenants Workflow”, where the “Dictator(s)” are responsible for master and devel, and the feature branches can be maintained by the central persons of the Task Force (the Lieutenants). What do you think?

Cheers, Martine

[1] http://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows#Topic-Branches [2] http://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows#Dictator-and-Lieutenants-Workflow

#2

Hi,

This is why I propose we change to a slightly adapted topic branch workflow (also known as feature branch) workflow [1]:

   - the main RIOT-OS/RIOT repository will get the following branches       - master: points to the stable version of our latest release          - one stable branch for every major release (?)

(I understand "stable" to denote non-changing APIs, and a "stable branch" to denote a branch which is stable in this sense, and gets bugfixes, and possibly new features.)

We don't have enough people to support this at the moment. Also, we would definitely need to make sure the releases are sensible for this to make sense. Let's keep this one open until there is a need, time and a codebase that makes sense to put this effort into.

      - devel: points to current development version (what is currently our       master), hot fixes can be cherry-picked to the master from here.

According to my first remark there is no need for a devel branch, we just keep using master.

      - branching from devel: feature branch for every task force that is       currently in active development, current changes will be merged regularly       from devel by the branches maintainers

"will be merged regularly" sounds like the amount of merging will increase in total. I'd rather leave it to the respective maintainers how they work on their feature branch.

   - feature branches will be merged into devel if the feature reaches    sufficient stability.

What does "stability" mean in this context?

I observed two extra benefits from that:

   - accidentally merged "SQUASH ME" commits can be squashed further    upstream, before we add the commits to master or devel

Actually, that is not the case as feature branches will also need to be PR'd against master, and might need fixing as well.

Also, we could as well "fix" these accidental commits in our master branch.

   - we can make maintenance of features more granular (and maybe solve our    maintenance problem) by enacting the "Dictator and Lieutenants Workflow",    where the "Dictator(s)" are responsible for master and devel, and the    feature branches can be maintained by the central persons of the Task Force    (the Lieutenants).

We could just as well use the decentralized hierarchical development model as performed by the Linux community. https://www.kernel.org/doc/Documentation/SubmittingPatches This would mean less visibility of the "feature branches" (in possession of a "Lieutenant" in your terminology), but also less stress on the "Dictators", because they don't constantly see what doesn't need to concern them. At least I guess the GitHub interface would make it much easier to only follow repositories one is concerned with.

Cheers, Ludwig

#3

Hello,

Hi,

This is why I propose we change to a slightly adapted topic branch workflow (also known as feature branch) workflow [1]:

   - the main RIOT-OS/RIOT repository will get the following branches       - master: points to the stable version of our latest release          - one stable branch for every major release (?)

(I understand "stable" to denote non-changing APIs, and a "stable branch" to denote a branch which is stable in this sense, and gets bugfixes, and possibly new features.)

We don't have enough people to support this at the moment. Also, we would definitely need to make sure the releases are sensible for this to make sense. Let's keep this one open until there is a need, time and a codebase that makes sense to put this effort into.

I agree with Ludwig.

      - devel: points to current development version (what is currently our       master), hot fixes can be cherry-picked to the master from here.

According to my first remark there is no need for a devel branch, we just keep using master.

      - branching from devel: feature branch for every task force that is       currently in active development, current changes will be merged regularly       from devel by the branches maintainers

"will be merged regularly" sounds like the amount of merging will increase in total. I'd rather leave it to the respective maintainers how they work on their feature branch.

   - feature branches will be merged into devel if the feature reaches    sufficient stability.

What does "stability" mean in this context?

I observed two extra benefits from that:

   - accidentally merged "SQUASH ME" commits can be squashed further    upstream, before we add the commits to master or devel

Actually, that is not the case as feature branches will also need to be PR'd against master, and might need fixing as well.

Also, we could as well "fix" these accidental commits in our master branch.

   - we can make maintenance of features more granular (and maybe solve our    maintenance problem) by enacting the "Dictator and Lieutenants Workflow",    where the "Dictator(s)" are responsible for master and devel, and the    feature branches can be maintained by the central persons of the Task Force    (the Lieutenants).

We could just as well use the decentralized hierarchical development model as performed by the Linux community. https://www.kernel.org/doc/Documentation/SubmittingPatches This would mean less visibility of the "feature branches" (in possession of a "Lieutenant" in your terminology), but also less stress on the "Dictators", because they don't constantly see what doesn't need to concern them. At least I guess the GitHub interface would make it much easier to only follow repositories one is concerned with.

Another possible improvement to the current situation might be to refactor the tags in the PR tracker to allow for finer granularity of Issues and PRs. IMHO, some of these tags today have confusing names and there may even be some duplicates right now.

Best regards, Joakim

#4

Hi,

#5

Hi Martine!

In general I like the concept and I think various people (including myself) proposed a similar workflow already in the past, but it was always rejected (IMO for good reasons), because of the introduced overhead and missing personal resources.

I don't get this argument: With my workflow the only new task is the merging of stable and devel and the maintenance of hot-fixes (which can be labeled as such and are thus easily spotable). The merging of devel into stable will happen at every release, and the merging of feature branches into devel if a milestone for the feature is reached.

Cherry-picking of "hot-fixes" (which would probably occur quite often in current state of RIOT) might be some work - particular, since there will be probably a lot of conflicts.

> > - devel: points to current development version (what is currently > our > > master), hot fixes can be cherry-picked to the master from here. > > According to my first remark there is no need for a devel branch, we > just keep using master. >

Sure we first need a stable version to have a stable branch, but my thinnking introducing it now was to avoid confusion we often encountered in the whole transceiver/netdev/ng_netdev situation.

I don't think this would help. Who would be willing to fix a problem in master with e.g. the transceiver interface while everyone's working on NSTF? Let's wait for RIOT 1.0, before we think about a "stable" master branch again.

<snip>

I still like the idea of feature branches.

> We could just as well use the decentralized hierarchical development > model as performed by the Linux community. > https://www.kernel.org/doc/Documentation/SubmittingPatches > This would mean less visibility of the "feature branches" (in > possession of a "Lieutenant" in your terminology), but also less > stress on the "Dictators", because they don't constantly see what > doesn't need to concern them. At least I guess the GitHub interface > would make it much easier to only follow repositories one is concerned > with. >

When you look at the text I've linked: they also give the Linux kernel as an example for the Dictator/Lieutenant workflow ;-). The terms branch and repository are more or less interchangable in this context (at least with Git). I guess in the end its a style choice if we decide for task force repositories or feature branches. I would welcome the visibility of development in the feature branch solution, but understand that some might not be interested in the development of some minor task force.

I'm not in favor of a hierarchical development model. One important advantage of the current flat maintainer hierarchy in RIOT is that we're not dependent on a single person - and no one of us has enough time to be the "Dictator". I'm fine with having one, two, three people being responsible for a feature branch or repository (I think branches would be fine at current point of time), but the "Dictator" should stay the community.

Cheers, Oleg

#6

Just for add a bit of documentation for helping discovering the best workflow:

http://nvie.com/posts/a-successful-git-branching-model/

The post contains some good points in my opinion, for example the guideline that a feature branch have to exists in developer repo and not in origin.

ciao

Attilio

#7

Hi Attilio,