Build and Deploy Docker Images to Oracle Cloud Infrastructure Container Engine for Kubernetes(OKE) using CircleCI

With the growing adoption of the micro-services architecture, modern day applications are being developed as micro-services with the ability to scale the individual services independently. This loose-coupling provides the power for developer to make changes to the individual components and push the changes to production independently.

When it comes to underlying infrastructure to deploy the micro-services, docker containers are most popular. At the enterprise level we need container management and orchestration solution to manage these containers. Kubernetes is the front-runner in this space since the time when it became open-source. Almost all the cloud providers are providing the kubernetes service in their cloud now.

Container Engine for Kubernetes(generally referred as OKE) provides the production-grade container solution for Kubernetes with the highly predictable performance of the Oracle Cloud Infrastructure. With the fully managed service and providing the High Availability for the cluster, customers can concentrate on the business rather than maintaining infrastructure.

For more info, please go through the link : https://cloud.oracle.com/containers/kubernetes-engine

With the few clicks we can create entire OKE cluster along with the required number of worker nodes. It has integration with the Oracle Cloud Infrastructure Registry(OCIR) which is the docker registry in the Oracle Cloud Infrastructure.

To create OKE cluster and deploy sample app :

Creating a Cluster with Oracle Cloud Infrastructure Container Engine for Kubernetes and Deploying a…

We can use Oracle Container Pipelines(aka Wercker) to automate the Continuous Integration/Continuous Delivery of the docker images to OKE clusters. Below is the tutorial for the same.

Integrating Oracle Cloud Infrastructure Container Engine For Kubernetes and Registry with Wercker

Also OKE is not tightly coupled to any specific tools(Oracle DevCS, wercker) for CI/CD integration. We can use Jenkins, CircleCI or any other tools. In this tutorial we will integrate Container Engine for Kubernetes(OKE) with CircleCI.

Prerequisites :-

1. Running OKE cluster.
2. OCIR registry access.
3. Github account.

CircleCI documentation : https://circleci.com

Let us see how we can setup the same:

Sign-up for free to CircleCI using github/bitbucket account.

Add the github project that you want to build and deploy, to your CircleCI account. To build project using CircleCI, you need to include a file config.xml under the folder “.circleci” in root directory.

GitHub Repo used in this setup : https://github.com/Shasthri/CircleCI-OKE.git

Below are the high level steps that are executed by config.yml.

As a first step, build environment is selected as docker image(circleci/node).

docker: 
      - 
        image: "circleci/node:7.10"

Then checkout the code to the build environment.

steps: 
      - checkout

Install OCI CLI on the build environment.

run: 
          name: "Install OCI CLI"        
          command: |
              sudo apt-get install -y libssl-dev libffi-dev python-dev build-essential
              bash -c "$(curl -L https://raw.githubusercontent.com/oracle/oci-cli/master/scripts/install/install.sh) --accept-all-defaults" 
              /home/circleci/bin/oci --help
              sudo cp -f  /home/circleci/bin/oci /usr/bin/oci
              oci --help
              oci -version

Install docker client to run the docker commands.

run: 
          name: "Install Docker client"        
          command: |
              set -x
              VER="17.03.0-ce"
              curl -L -o /tmp/docker-$VER.tgz https://download.docker.com/linux/static/stable/x86_64/docker-$VER.tgz
              tar -xz -C /tmp -f /tmp/docker-$VER.tgz
              sudo chown -R $(whoami) /usr/bin
              mv -f /tmp/docker/* /usr/bin
              docker --version

Install kubernetes client which is required to create Kubernetes resources in cluster.

- run:
           name : Install kubectl for running kubernetes commands
           command: |
                set -x
                curl -LO curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
                chmod +x ./kubectl
                mv ./kubectl /usr/bin/kubectl

Also configure the access to the Oracle Kubernetes Engine(OKE) cluster by downloading the kubeconfig after providing the Cluster-id.

- run:
          name: configure access to Oracle Kubernetes Engine cluster
          command: |
                set -x
                mkdir -p $HOME/.kube
                export ENDPOINT="containerengine.us-ashburn-1.oraclecloud.com"
                mkdir -p $HOME/.oci
                cp ociconfig ~/.oci/config
                echo '-----BEGIN RSA PRIVATE KEY-----' > $HOME/.oci/oci_api_key.pem
                echo $OCI_API_KEY >> $HOME/.oci/oci_api_key.pem
                echo '-----END RSA PRIVATE KEY-----' >> $HOME/.oci/oci_api_key.pem
                chmod 600 $HOME/.oci/oci_api_key.pem
                # cat $HOME/.oci/oci_api_key.pem
                sed -i -e "s/OCI_USER/$OCI_USER/g" ~/.oci/config
                sed -i -e "s/OCI_FP/$OCI_USER_FP/g" ~/.oci/config
                sed -i -e "s/OCI_TENANCY_ID/$OCI_TENANCY_ID/g" ~/.oci/config
                sed -i -e "s/OCI_REGION/$OCI_REGION/g" ~/.oci/config
                ls -ltr $HOME/.oci
                # cat ~/.oci/config
                touch $HOME/.kube/config
                # cat $HOME/.kube/config
                chmod +x ./get-kubeconfig.sh
                ./get-kubeconfig.sh $OKE_CLUSTER_ID > $HOME/.kube/config
                #cat $HOME/.kube/config
                kubectl get nodes

Install dependencies using yarn and run the test cases.

restore_cache: 
          keys: 
            - "v1-dependencies-{{ checksum \"package.json\" }}"
            - v1-dependencies-
      - 
        run: "yarn install"
      - 
        save_cache: 
          key: "v1-dependencies-{{ checksum \"package.json\" }}"
          paths: 
            - node_modules
      - 
        run: "yarn test"

Here we have used Mocha & chai framework (test/mocktest.js)to run sample test cases as shown below:

var assert = require('assert');
var expect  = require('chai').expect;
var request = require('request');


it('Should return 200 status', function() {
    request('http://localhost:3000' , function(error, response, body) {
        expect(response.statusCode).to.equal(200);;
        done();
    });
});

Build and push the docker image to Oracle Cloud Infrastructure Registry(OCIR).

run: 
          name: "Build Docker Image"
          command: |
              docker version          
              TAG=1.$CIRCLE_BUILD_NUM
              docker build -t  $DOCKER_REGISTRY/$TENANCY/circleci-demo-docker:$TAG .
              docker login $DOCKER_REGISTRY -u $DOCKER_USER -p $DOCKER_PASS
              docker push $DOCKER_REGISTRY/$TENANCY/circleci-demo-docker:$TAG

Deploy the image to Oracle Kubernetes Engine using kubectl configured earlier. Before deploying, replace the Image tag, docker registry and tenancy with the actual values. Since this kube-deploy.yaml file uses the docker-registry secret to pull the image from the OCIR, we need to create the secret as well using the command ‘kubectl create secret’.

- run:
         name : Deploy the docker image to Kubernetes cluster
         command: |
                set -x
                kubectl get nodes
                export TAG=1.$CIRCLE_BUILD_NUM
                echo $TAG
                sed -i -e "s/IMG_TAG/$TAG/g" kubernetes/kube-deploy.yaml
                sed -i -e "s/DOCKER_REGISTRY/$DOCKER_REGISTRY/g" kubernetes/kube-deploy.yaml
                sed -i -e "s/TENANCY/$TENANCY/g" kubernetes/kube-deploy.yaml
                cat kubernetes/kube-deploy.yaml
                kubectl create secret docker-registry regcred --docker-server=iad.ocir.io --docker-username=$DOCKER_USER --docker-password=$DOCKER_PASS  --docker-email=prasannashasthri@gmail.com || echo 'secret exists'
                kubectl apply -f kubernetes/kube-deploy.yaml
                sleep 5
                kubectl get svc

We can observe that there are environment variables used in the config.yml and kube-deploy.yaml to make sure the API keys, secrets and other sensitive information are not exposed.

To set the environment variable(Name-Value) in circleCI, Goto project settings → Environment variables (Under Build settings). Add the value for the environment variables that you have used in the config.yml file. During the build time, the actual value will get replaced as mentioned in this section.

Once this setup is completed you can observe that any changes that you push to github repository will trigger a build in CircleCI project.

Below is the steps that got executed in CircleCI window :

Execution steps in CircleCI dashboard

During the deployment, kubectl apply command is used to apply the configuration mentioned in the deployment yaml file. Once the deployment is completed, we can verify the same by accessing the application through the kubernetes service IP:Port

In this way, we can use CircleCI as CI/CD tool with the Oracle Kubernetes Engine.

Hope you learnt something!!

Follow me on twitter for interesting articles..

Also published in https://devopsguy.blogspot.com

Thanks,
Prasanna Shasthri
Linked-In : www.linkedin.com/in/prasanna-shasthri-b1a77961
Twitter : https://twitter.com/PShasthri