As we all know, anywhere there is a business need, we developers show up. With ideas… Great ideas! In the past, for one of our customers, it happened quite often that features we started even at the beginning of sprint, were not ready for release due to complexity or change in requirements. This frustrated developers, as it required time-consuming and error-prone rollbacks of the code. It also impacted other teams, as the code was shared. At some point, the customer requested controlled delivery of new features. So we redeveloped our delivery pipeline in cooperation with the client. We based it on Git's feature branches.
- Name of product (Foo)
- Type of branch (story, bug, release or maintenance)
- Work item id (unique identifier taken from TFS)
As the code is shared and many check-ins are done daily (2), it would be very difficult to find out what’s happening without proper history. Therefore, every commit has a reference to work item id, for example: ′#123456 #987654′ refers to user story #123456 and task #987654. In addition, our developers refresh story branch during work (3), ideally on a daily basis, to minimize number of merge conflicts at the end of work.
Once finished, story branch is merged back to develop branch by pull request (4). Develop branch is a main branch for product (and we have many of them, because we as a company develop many products on the same platform for our customer). Pull requests are made by developer, but reviewed by technical leader or architect – this way we ensure good quality of code.
When we come closer to release time, we derive release branch from develop branch and stabilize it. Ready for deployment code is then made available to customer through another pull requests created by our technical architect. Once reviewed by client’s architect, it is merged to master and installed on production. And here the cycle ends.
After every release, we tag master branch to allow easy access for maintenance needs. Every merge up is accompanied by merge down to alleviate integration conflicts. When the branch is no longer needed, it should be removed by its owner. As we human tend to forget about clean ups, we have a small tool, which generates reminders. Thanks to that we don’t have stale branches in the repository.
Build and Deployment
As mentioned at the beginning, Git is only a part of the solution. To guarantee high quality of products, you also need stable build and deployment process. We use TeamCity as a Continuous Integration server, supported by our open-sourced PSCI library, which allows to express configuration as a code. TeamCity is configured per each product separately, with independent steps for compilation and packaging to shorten feedback loop.
Every product is divided on multiple build components (UI, web services or database) – see image above. As a final step, all component outputs are merged and used to prepare installation package. What's even more important, TeamCity builds every branch independently, thanks to that as an output we have separate packages for every branch.
Thanks to PSCI, we can build application once and deploy it many times (to various environments with different configuration). Deployment is super easy, as it is just another process in TeamCity. Testers really like it, because they can choose what feature (branch), where (environment) and when they want to have it done – see image above.
And there’s more than that – you can even have many different features (branches) installed on one server. So there is no need for additional hardware to support this new delivery pipeline. Quite simple and powerful thanks to use of branch name in application Urls (e.g. http://foo-test-server.local/Foo-story-334455).
Our deployment is very sophisticated. It’s possible to deploy application on fresh server, and all application’s dependencies (Windows features, IIS, SQL Server etc.) will be automatically installed. Thanks to that we have really flexible solution, which is scalable and maintainable.
Described approach is not always trouble-free. Sometimes we have problems e.g. during merge one developer can by mistake overwrite someone else’s changes. Fortunately, we have tests to the rescue. All kinds of tests: unit, integration, automated, etc. Despite this, in my opinion we have really good delivery process and happy customers.
Here are the types of branches, we have:
- Master – main branch, which stores info about release history.
- Develop – in our case is the main branch per product. It’s created from master branch and contains stories finished for a product.
- Story/bug – branch is created when developer starts working on some user story/bug. It’s created from develop branch and contains all commits required to finish the entire story. This branch contains also fixed bugs related to user story.
- Release – branch contains all finished user stories in sprint. It’s created from develop branch by the technical architect. On this branch, application is stabilized and prepared for release. Pre-release bugs are fixed in this branch.
- Maintenance – branch is created in special case when bug appeared on production. It comes from release tag on master and contains only bug-related fix.