Starting point

The client had an old virtual machine-based Buildbot CI/CD system. It worked well but did not track all the applications and libraries the client was developing. Moreover, it was hard to extend beyond its initial, limited design.

Project

Wide operating system coverage was very important for the client, so Buildbot continued to be a very good choice. So, we redesigned the Buildbot environment on several levels. First, data (“what to build”) and code (“how to execute builds”) was separated. This easily allowed building of all the applications and libraries the client required and defining build permutations flexibly. The virtual machine-based workers were also converted to Docker workers launched on demand.

End result

At the end of the project there were two main Buildbot instances - one for production and one for staging. Developers could create private Buildbot instances for testing and development purposes on demand. Each Buildbot could be configured differently to support multiple use cases. Every commit to a tracked application launched 20-400 builds across a wide range of containerized operating systems. Some of those builds triggered a full integration test suite, whereas some built Debian, Ubuntu or MSI packages.

Main technologies

• Buildbot
• Docker