How to CI/CD with Jenkins?

by Mikołaj Młodzikowski

Jenkins - who is he ?

  • Jenkins has started as project Hudson in 2004 at Sun Microsystem, developed by Kohsuke Kawaguchi in java
  • Jenkins as we know has started in 2011 as Hudson fork (while Hudson name belonged to Orcale after buying Sun) and Kohsuke Kawaguchi received O’Reilly Open Source Award
  • Features used in current Jenkins development appeared in 2014 as Jenkins 2.0 introducing pipelines as standard

Extensions

  • Plugins - base part of Jenkins ecosystem, java API provided by plugins is accessible on most of Jenkins abstraction layers - DSL, freestyle jobs and pipelines
  • Libraries - the best way to add custom steps exposed as JVM libraries into your pipelines
  • Agents - core part of each jenkins architecture

Groovy

  • used for programming and scripting with features similar to those of Python or Ruby, what makes it very easy to use non-native groovy idioms
  • Groovy source code file can be executed as an (uncompiled) script - what means errors during runtime, it could be also compiled statically, but this is not Jenkins case
  • object-oriented, functional, java syntax and libraries compatible

Groovy

  • is able to mix features of dynamically 📌 and statically typed language at the same time
  • Groovy implicitly generates getters and setters, and allows to construct object with a field -> value map
  • operators overloading and the safe navigation operator ?.

Groovy

  • native syntax for lists and associative arrays (maps) and closures
  • native support for regular expressions, polymorphic iteration, string interpolation
  • has kind of cool groovyshell ;)

Most used job types

  • Freestyle - manual or DSL setup where flow is configured via XML file which uses steps and plugins
  • Pipelines - Jenkinsfile or groovy script in job configuration
  • Maven Project - Java orientated extension of freesytle job

Pipeline types

  • Scripted - closures to define the flow have no order and are defined by progammer (low level)
  • Declarative - clousers have documentated structure (high level)
  • Single pipeline - only one job is created and could be independent from repository
  • Multibranch pipeline - project is assigned to the repository and creates multiple jobs depending on discovery strategy

Jobs triggers

  • Poll SCM - Jenkins actively checks source code and triggers jobs if any changes where introduced
  • Internal - upstream/downstream relations to the other jobs (e.g. build pipeline -> deployment pipeline)
  • Webhooks - generic hooks, VCS providers, Jira
  • Jenkins Crons Syntax - TZ=UTC H 8 * * *

Triggers tricks - when conditions

Triggers tricks - when conditions

Triggers tricks - when conditions

Triggers tricks - when conditions

Parameters

  • Parameters are available during build for executor shell
  • If parameters is defined with lower case letters then it will be available in shell with UPPER_CASE name
  • Parameters are also available in global variable params for pipelines

Discarding builds

  • Too many builds and artifacts are heavy on jenkins operating memory
  • You can set specific number of days you want to keep builds and/or artifacts
  • You can set specific number of builds and/or artifacts you want to keep

Jobs configuration schools

  • Manual - of course :)
  • DSL - Domain specific language
  • Plugins / tools using API - for instance job-builder plugin
  • Pipeline closures - requires initial run

DSL - Domain specific language

  • Allows to configure jobs as Groovy scripts
  • Available under env.JENKINS_URL/plugin/job-dsl/api-viewer/index.html
  • Requires DSL plugin
  • Could be configured as cron to overwrite manual changes made by users

Libraries

  • vars catalog - contains groovy code with steps where each file is the name of step and has call method
  • Each step underhood is an object which helper methods could be accessed like stepName.helperMethod()
  • call method for each step could be overloaded
  • src catalog - contains Java-like code which could be imported in steps / pipelines

Libraries

  • resource catalog - contains helper filer which could be loaded (bash scripts, json files etc)
  • libraries could be versioned - by branch, tag or commit
  • you can load multiple libraries at once
  • you can define full pipelines within libraries

Agents types

  • SSH Agents - connected to Jenkins via SSH connection performed from Master to Agent
  • JNLP (Java Network Launch Protocol) Agents - connected to Jenkins via socket connection performed from Agewnt to Master
  • Dynamic agents - they are mostly JNLP agents and they are created dynamically on Master’s request - supported bo JCloud

SSH Agents

  • Recommended by CloudBees when using their plugin :)
  • Has less software requirements what comes in handy when using dynamic agents
  • Has issues with non-linux servers

JNLP Agents

  • Comes in handy when you need to codify your Jenkins Agents (jar could be
  • Has less software requirements what comes in handy when using dynamic agents
  • Has issues with non-linux servers

Kubernetes agents

  • Under the hood the Pod object is formed within side container which contains JNLP agent
  • Jenkins master has to be reachable from K8s cluster
  • Pod template could be defined in Jenkins configuration with label or could be created inline

Docker software isolation

  • Workspace could be mapped into docker container within the agent if it has docker software
  • This requires mapping container user to user running jenkins process on agent
  • Private registry and dockerfile are supported too

Dynamic VMs

  • Supported clouds: Digital Ocean, GCP, AWS, Azure, etc…
  • Best settings are retention time and number of executors set to 1
  • It is required that VM image contains java

UI Lifting

  • Green Balls plugin - replaces blue color with green introducing Western culture context
  • Simple Theme Plugin - Allows to install out of the box skins or introduce custom CSS - https://github.com/jenkinsci/simple-theme-plugin
  • Groovy Postbuild plugin - Codified edition of HTML in Jenkins UI

UI Lifting - Simple changes

UI Lifting - Material

http://afonsof.com/jenkins-material-theme/

UI Lifting - groovy extra description

UI Lifting - groovy extra description - code

UI Lifting - Adding icons

https://github.com/jenkinsci/jenkins/tree/master/war/src/main/webapp/images/48x48

UI Lifting - Adding icons - code

UI Lifting - HTML raports

UI Lifting - summaries

Plugins management - installation

Plugins management - configuration

  • All Jenkins configurations - including plugins and jobs are XML - backup them!
  • You can use JCasC - Jenkins Configuration as Code (tool from Praqma)
  • Try to use Jenkins Credentials for storing secrets for plugin configurations - same as for Jobs ;)

sources