What does CI/CD try to solve?

So, what is CI/CD anyway?

And what is a pipeline?

LAB: Create a pipeline for a golang app

• A running Jenkins instance. This could be a cloud instance, a virtual machine, a bare metal one, or a docker container. It must be publicly accessible from the internet so that the repository can connect to Jenkins through webhooks.
• Image registry: you can use Docker Registry, a cloud-based offering like ECR or GCR, or even a custom registry.
• An account on GitHub. Although we use GitHub in this example, the procedure can work equally with other repositories like Bitbucket with minor changes.

Step 01: The application files

package main

import (
   "log"
   "net/http"
)

type Server struct{}

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
   w.WriteHeader(http.StatusOK)
   w.Header().Set("Content-Type", "application/json")
   w.Write([]byte(`{"message": "hello world"}`))
}

func main() {
   s := &Server{}
   http.Handle("/", s)
   log.Fatal(http.ListenAndServe(":8080", nil))
}

package main

import (
   "log"
   "net/http"
)

type Server struct{}

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
   w.WriteHeader(http.StatusOK)
   w.Header().Set("Content-Type", "application/json")
   w.Write([]byte(`{"message": "hello world"}`))
}

func main() {
   s := &Server{}
   http.Handle("/", s)
   log.Fatal(http.ListenAndServe(":8080", nil))
}

FROM golang:alpine AS build-env
RUN mkdir /go/src/app && apk update && apk add git
ADD main.go /go/src/app/
WORKDIR /go/src/app
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o app .

FROM scratch
WORKDIR /app
COPY --from=build-env /go/src/app/app .
ENTRYPOINT [ "./app" ]

apiVersion: v1
kind: Service
metadata:
  name: hello-svc
spec:
  selector:
    role: app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
      nodePort: 32000
  type: NodePort

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-deployment
  labels:
    role: app
spec:
  replicas: 2
  selector:
    matchLabels:
      role: app
  template:
    metadata:
      labels:
        role: app
    spec:
      containers:
      - name: app
        image: "{{ image_id }}"
        resources:
          requests:
            cpu: 10m

- hosts: localhost
  tasks:
  - name: Deploy the service
    k8s:
      state: present
      definition: "{{ lookup('template', 'service.yml') | from_yaml }}"
      validate_certs: no
      namespace: default
  - name: Deploy the application
    k8s:
      state: present
      validate_certs: no
      namespace: default
      definition: "{{ lookup('template', 'deployment.yml') | from_yaml }}"

Step 02: Install Jenkins, Ansible, and Docker

1. Log in to the Jenkins instance
2. Install Python 3, Ansible, and the openshift module:

   sudo apt update && sudo apt install -y python3 && sudo apt install -y python3-pip && sudo pip3 install ansible && sudo pip3 install openshift

3. By default, pip installs binaries under a hidden directory in the user’s home folder. We need to add this directory to the $PATH variable so that we can easily call the command:

   echo "export PATH=$PATH:~/.local/bin" >> ~/.bashrc && . ~/.bashrc

4. Install the Ansible role necessary for deploying a Jenkins instance:

   ansible-galaxy install geerlingguy.jenkins

5. Install the Docker role:

   ansible-galaxy install geerlingguy.docker

6. Create a playbook.yaml file and add the following lines:

- hosts: localhost
  become: yes
  vars:
    jenkins_hostname: 35.238.224.64
    docker_users:
    - jenkins
  roles:
    - role: geerlingguy.jenkins
    - role: geerlingguy.docker

7. Run the playbook through the following command:

   ansible-playbook playbook.yaml.

   Notice that we’re using the public IP address of the instance as the hostname that Jenkins will use. If you are using DNS, you may need to replace this with the DNS name of the instance. Also, notice that you must enable port 8080 on the firewall (if any) before running the playbook.
8. In few minutes, Jenkins should be installed. You can check by navigating to the IP address (or the DNS name) of the machine and specifying port 8080:
9. Click on the Login link and supply “admin” as the username and “admin” as the password. Note that those are the default credentials set by the Ansible role that we used. You can (and should) change those defaults when using Jenkins in production environments. This can be done by setting the role variables. You can refer to the role official page.
10. The last thing you need to do is install the following plugins that will be used in our lab: git, pipeline, CloudBees Docker Build and Publish, GitHub.

Step 03: Configuring Jenkins User To Connect To The Cluster

<code class="  language-yaml">$ sudo cp ~/.kube/config ~jenkins/.kube/
$ sudo chown <span class="token punctuation">-</span>R jenkins<span class="token punctuation">:</span> ~jenkins/.kube/</code>

Step 04: Create The Jenkins Pipeline Job

• We used the Poll SCM as the build trigger; setting this option instructs Jenkins to check the Git repository on a periodic basis (every minute as indicated by * * * * *). If the repo has changed since the last poll, the job is triggered.
• In the pipeline itself, we specified the repository URL and the credentials. The branch is master.
• In this lab, we are adding all the job’s code in a Jenkinsfile that is stored in the same repository as the code. The Jenkinsfile is discussed later in this article.

Step 05: Configure Jenkins Credentials For GitHub and Docker Hub

Step 06: Create the Jenkinsfile

pipeline {
   agent any
   environment {
       registry = "magalixcorp/k8scicd"
       GOCACHE = "/tmp"
   }
   stages {
       stage('Build') {
           agent {
               docker {
                   image 'golang'
               }
           }
           steps {
               // Create our project directory.
               sh 'cd ${GOPATH}/src'
               sh 'mkdir -p ${GOPATH}/src/hello-world'
               // Copy all files in our Jenkins workspace to our project directory.               
               sh 'cp -r ${WORKSPACE}/* ${GOPATH}/src/hello-world'
               // Build the app.
               sh 'go build'              
           }    
       }
       stage('Test') {
           agent {
               docker {
                   image 'golang'
               }
           }
           steps {                
               // Create our project directory.
               sh 'cd ${GOPATH}/src'
               sh 'mkdir -p ${GOPATH}/src/hello-world'
               // Copy all files in our Jenkins workspace to our project directory.               
               sh 'cp -r ${WORKSPACE}/* ${GOPATH}/src/hello-world'
               // Remove cached test results.
               sh 'go clean -cache'
               // Run Unit Tests.
               sh 'go test ./... -v -short'           
           }
       }
       stage('Publish') {
           environment {
               registryCredential = 'dockerhub'
           }
           steps{
               script {
                   def appimage = docker.build registry + ":$BUILD_NUMBER"
                   docker.withRegistry( '', registryCredential ) {
                       appimage.push()
                       appimage.push('latest')
                   }
               }
           }
       }
       stage ('Deploy') {
           steps {
               script{
                   def image_id = registry + ":$BUILD_NUMBER"
                   sh "ansible-playbook  playbook.yml --extra-vars \"image_id=${image_id}\""
               }
           }
       }
   }
}

1. Build is where we build the Go binary and ensure that nothing is wrong in the build process. 
2. Test is where we apply a simple UAT test to ensure that the application works as expected.
3. Publish, where the Docker image is built and pushed to the registry. After that, any environment can make use of it.
4. Deploy, this is the final step where Ansible is invoked to contact Kubernetes and apply the definition files. 

Testing our CD pipeline

1. Add our files:

   git add *

2. Commit our changes:

   git commit -m "Initial commit"

3. Push to GitHub:

   git push

4. On Jenkins, we can either wait for the job to get triggered automatically, or we can just click on “Build Now”.
5. If the job succeeds, we can examine our deployed application using the following commands.
6. Get the node IP address: 

kubectl get nodes -o wide
NAME                                          STATUS   ROLES    AGE   VERSION          INTERNAL-IP   EXTERNAL-IP     OS-IMAGE                             KERNEL-VERSION   CONTAINER-RUNTIME
gke-security-lab-default-pool-46f98c95-qsdj   Ready    <none>   7d    v1.13.11-gke.9   10.128.0.59   35.193.211.74   Container-Optimized OS from Google   4.14.145+        docker://18.9.7

7. Now, let’s initiate an HTTP request to our app:

$ curl 35.193.211.74:32000
{"message": "hello world"}

package main

import (
   "log"
   "net/http"
)

type Server struct{}

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
   w.WriteHeader(http.StatusOK)
   w.Header().Set("Content-Type", "application/json")
   w.Write([]byte(`{"message": "Hello World!"}`))
}

func main() {
   s := &Server{}
   http.Handle("/", s)
   log.Fatal(http.ListenAndServe(":8080", nil))
}

$ git add main.go
$ git commit -m "Changes the greeting message"                                                                                                       
[master 24a310e] Changes the greeting message
 1 file changed, 1 insertion(+), 1 deletion(-)
$ git push
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 319 bytes | 319.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/MagalixCorp/k8scicd.git
   7954e03..24a310e  master -> master

TL;DR

• CI/CD is an integral part of any modern environment that follows the Agile methodology.
• Through pipelines, you can ensure a smooth transition of code from the version control system to the target environment (testing/staging/production/etc.) while applying all the necessary testing and quality control practices.
• In this article, we had a practical lab where we built a continuous delivery pipeline to deploy a Golang application.
• Through Jenkins, we were able to pull the code from the repository, build and test it using a relevant Docker image. 
• Next, we Dockerize and push our application - since it passed our tests - to Docker Hub.
• Finally, we used Ansible to deploy the application to our target environment, which is running Kubernetes.
• Using Jenkins pipelines and Ansible makes it very easy and flexible to change the workflow with very little friction. For example, we can add more tests to the Test stage, we can change the version of Go that’s used to build and test the code, and we can use more variables to change other aspects in deployment and service definitions. 
• The best part here is that we are using Kubernetes deployments, which ensures that we have zero downtime for the application when we are changing the container image. This is possible because Deployments use the rolling update method by default to terminate and recreate containers one at a time. Only when the new container is up and healthy does the Deployment terminate the old one.