Team members:

Business Situation

Client was encountering everyday build failures because there were differences in build dependencies on separate Jenkins agents hosts. Need for unified build and standardisation across dev-build machines arose. On top of that - applications were built by proprietary build system based on msbuild targets which were not viable after most of new services were built on dotnet (ASP.NET Core). 

Solution

Migration from Jenkins freestyle builds to multibranch Jenkins pipelines (or Gitlab CI) which opened more possibilities out of the box. Build machines were transformed from manually installed virtual machines to docker images in 2 phases. 

In the first phase docker was managed by Microsoft Service Fabric (azure service on-premises), so there were constantly running build containers(already based on docker) prepared to serve requests. In the second phase it was allowed to run builds with on-demand containers which started at the beginning of the build and closed after built was performed and artifacts were stored.

Results

Failing builds because of differences between build machines went down to zero (excluding failures on npm/nuget server timeouts). Elasticity of the solution is also a big plus. Part of the solution was also Selenium cluster for running E2E test of frontends with video recording of failed tests for later analysis.