How core to your business is your CI/CD tool?

If you’re like me, you started with CruiseControl as your CI system sometime in the late 2000’s. You probably installed the operating system yourself and maybe even mounted the server in the rack that was running it. Some things have changed since then and some haven’t. One important question to ask is how core to your business, assuming you are in the software business, your CI/CD tool is?

Traditionally it was pretty close to my core competency, software development, and we always wanted to keep the CI/CD system under our control. I went from CruiseControl to Jenkins->TeamCity->Jenkins again in 2016 with Jenkins’ pioneering Jenkinsfile approach. At the same time, source code has gone from being self-hosted in a lot of cases into Github/Bitbucket/etc. in the cloud and more “SAAS CI/CD” offerings have appeared. Github Actions, CircleCI, Travis, and Azure Devops have introduced low barriers to entry for getting up and running.

If you have to spend 10% of your time fussing with Jenkins in a year, that could easily end up in $20k of labor that could be spent on other things. Commercial options through Cloudbees do not necessarily make that cost go away either. I think we have arrived at the point where starting with the SAAS options first makes sense. I can think of a few exceptions/edge cases where that might not be the case:

  • Really strict security requirements for who/what can get access to secrets to deploy code (or the code itself) and a third party service is too risky or their on-prem offerings are too expensive

  • Large AWS presence (more than 1 EKS cluster and a couple EC2 VMs) such that the spot pricing and instance capabilities of tools like Jenkins’ AWS plugin can save you a lot of money and provide some control

  • Significant investment in value add Jenkins plugins that would be expensive to port away from

The last point leads to a warning to any CI/CD platform regardless of vendor: Be wary of magic of CI/CD vendors and relying too much on their integrations. While their integrations can add value, they also can make it difficult to replicate unexpected behavior in local environments. Generally you should let your build tools (that can run locally) do most of the work (infrastructure tools like Terraform count as ‘build tools’ in this context). Try to rely on the CI/CD platform more as an orchestrator and elegant observer of your build process rather than a lower level participant in it.

Where does that leave us? If you’re someone like me that has difficulty letting go of Jenkins, it might make sense to consider other options, especially as new projects come up.

Brady Wied