Introduction

This document describes how SonarK works with the upstream project it is forked from, which is Kibana. This is in an attempt to make the upgrading process more manageable by a human being.

Problems With “Bulk Upgrade”

Up until the writing of this document, upgrades have been performed once every few months when we decide to keep up with upstream Kibana. This process involves cherry-picking/merge/rebase months worth of upstream code, which is easily thousands of commit. An amount of complexity that makes it inevitable to generate a lot of regression and errors.

Incremental Upgrade

The proposed change to this process is, instead of cherry-picking/merge/rebase a big bulk of new upstream items, do it incrementally instead. Something a human would be able to manage. 

Mirror Branching Model

To begin, we should mirror upstream branching model. The following diagrams show how we might go about doing this. The first diagram contains the upstream branching model of kibana. They have:

• Major Version Development Branch: 7.x
• Minor Version Development Branch: 7.2, 7.4
• Release Tags/Branch: v7.2.0, v7.2.1, v7.4.0, v7.4.1. These are specific commits along the Minor Version Development Branch.

The following diagram is our mirror of kibana branching model.

Notice how Major Version Development Branch, Minor Version Development Branch, and Release Tags are all prepended with SonarK-. The local sonark repo where the upgrade is being performed will need to track both our kibana fork (SonarK) and upstream in order to pull changes from upstream to our fork. For instance, for v7.2.1, my local repo’s git remote -v returns:

origin git@github.com:jsonar/kibana.git (fetch)

origin git@github.com:jsonar/kibana.git (push)

upstream https://github.com/elastic/kibana.git (fetch)

upstream https://github.com/elastic/kibana.git (push)

Where origin is our fork and upstream is the kibana repository. Thus in order to avoid namespace collision, we have SonarK- prefix.

• TODO
  • How to treat minor branches.
  • How tag/branch are interchanged with release tags.

Daily Upstream Merge

Suppose our we just upgrade to SonarK-v7.2.1, and SonarK-7.x is currently the merge-base of upstream v7.2.1 and 7.x, let’s call that point 7.x–v7.2.1. These are illustrated with the diagram below.

From here onwards, merge next commits from upstream after the 7.x–v7.2.1 commit to our SonarK-7.x, our Major Version Development Branch. We don’t touch Minor Version Development Branch or Release Branches.

We slowly do this until SonarK-7.x reach the merge-base of our next desired version. Say this version is 7.4.1. Then our merge-base would be 7.x–v7.4.1, which is the merge-base between upstream 7.x and 7.4.1.

How Much Commits Per Day?

Eran said someone shouldn’t spend more than an hour. So whatever fits in an hour, which is more than days worth of commits from my experience.

After Merge Activity

We should have a Google Doc that we edit with the summary of notable/major changes that are merged that day.

Pull Requests

Pull requests are now done in 2 branches. Major Version Development Branch and Release Branch.

Caveats and TODOs

Not An Excuse To Non-Modular Code

We still make our code touch upstream code the least amount as possible. Even when a merge had just one file in conflict, if we made a lot of changes in that upstream code, it will consume a lot of time.

Track Upstream Master

The image below is a fuller picture of the upstream branching model. They have a master that tracks changes for the next major version, namely, 8.x. Now that we are still testing this process, the jump between SonarK-7.x and future SonarK-8.x is going to be difficult. Once this process is proven, then we will spend more effort and start tracking master. 

Worst case, we will have a 1 month of upgrade only when transitioning between major versions, instead of every minor version.