Pipeline as code in Jenkins
Pipeline as code is a technique which defines that how we can write CI/CD build or deployment pipeline with code which can be fed to make pipeline automatically and which can even be version controlled.
Before talking about how to follow this technique to make it more efficient and easy to adapt, let's just understand why we want to do it? Jenkins, GO CD, Team City or Circle CI are few tools to do continuous integration and continuous delivery. These tools help in continuously integrating the source code and act as a safety net for the developer as the code gets tested by the pipeline after every check-in to the code. It makes a developer life easier as they do not have to worry about a buggy code getting through to production and code is always ready to be deployed.
In Jenkins, making pipeline is easy as one has to either use Jenkins syntax or write groovy scripts in the pipeline. Generally, teams follow this approach to build a new pipeline as it is easy and fast. But as the team size increases, the number of pipelines also increases as they have to build/deploy more projects. Building a pipeline in such a scenario can be easy using the traditional way of building pipelines manually. But modification, testing or maintenance can become a nightmare if the team has hundreds of pipeline. As some pipelines like building source code (Builds) can be similar, so to create a new build pipeline one can simply copy paste the pipeline script in new pipelines for all other similar build pipelines, but what if they want to add a new stage in all build pipeline? It will require them to copy that stage pipeline script in every pipeline and if there are hundreds of them then it will require a lot of manual effort to be done and will be error-prone, so there is a need for templatization of similar pipelines such that it can make the modification of the pipeline easy. As we talked about modification it also comes with an overhead of maintenance. If there is a change in pipeline script no one will get to know what got changed or who changed it, that makes the pipeline script buggy and testing of the script is also not possible in such scenario, and if it is a breaking change there is no way to revert back to a stable version as the pipeline script is not versioned.
Due to these problems of scaling similar builds (templatization), testing pipeline code, and to make the pipeline code version controlled we use this technique called ‘Pipeline as code’.
There are various ways of following this technique :
- Jenkins files
- Jenkins job builder tool
In this blog, I will be talking about the Jenkins Job Builder tool. To read about how to do Pipeline as Code using Jenkins file you can check out this link.
Jenkins Job Builder
Jenkins job builder is a tool which takes yaml or json configuration and uses them to configure Jenkins pipeline. A developer can write the configuration for the Jenkins pipelines and with that configuration JJB(Jenkins Job builder) tool will configure specific pipelines in Jenkins.
Let us see how to create a pipeline using JJB(Jenkins job builder).
job-template can be used if you have several jobs that are similar, as all similar jobs can point to a single template by specifying the name of the template which is mentioned in the project configuration. For example, a normal build job that can be similar for all teams will have similar build stages, so all the build jobs can point to the same template. Any modification to the template will automatically reflect in all pipelines sharing the template.
A project is like a template which is referred by the job-template, the project collects all the relatable jobs together and gets the values passed from the job. It takes all the job-templates configured with a specific job name and the templates are then applied to configure new Jenkins pipeline.
To make a pipeline in Jenkins one has to run the JJB tool with these yaml configurations as its parameters and the tool will populate all the pipelines mentioned in the jobs. Example command
jenkins-jobs --conf etc/jenkins_jobs.ini-sample update jobs/samplejobs.yaml:templates/sampletemplates.yamlJenkins Job Builder updates, deletes & creates pipeline on the basis of the configuration provided. This configuration can easily be stored in a version controlled system like git and a developer can run the tool over every check-in in the code.
JJB also makes the modification of pipelines very easy as for adding one stage in multiple pipelines one just need to do changes in the template which gets reflected in every other job which is using that template and with the modification being done, it can also be tracked like all the changes in the pipeline code is done with code check-in which can be reverted easily in case if the code breaks or if there is an issue with the code. Debugging issues also become very easy and we can even write tests for all the pipeline code which was untested before. The most important benefit that pipeline as code brings is if one wants to shift to a new instance of Jenkins it can easily be done as the code resides in a git repository and can easily be applied to make new Jenkins pipeline instantly.
Pros of using ‘Pipeline as Code’
- Easy to create, update & delete pipelines
- Version controlled (can detect which version had the bug)
- Test pipeline code
- Easy to configure similar pipelines
- Easy to migrate to new Jenkins instance
This technique helps in scaling CI/CD and make it more reliable. To read more about JJB refer to this link.
