Jenkins CI/CD Pipeline with ArgoCD Integration

In this blog, we will walk through the process of setting up a Jenkins end-to-end CI/CD pipeline for a Java-based application. This pipeline will utilize various tools and technologies including Maven, SonarQube, Docker, ArgoCD, Shell Scripting, and Kubernetes.

Prerequisites

Before we begin, ensure you have the following:

Java application code hosted on a Git repository
Jenkins server
Kubernetes cluster
AWS Account
Argo CD

You can follow along using the GitHub repository here.

Project Workflow

a. Setting Up the Git Repository

We will start by setting up a Git repository to host the source code for our Java application. Whenever a developer commits or pulls requests for any code changes, the Jenkins pipeline will be triggered using Webhooks.

b. Building with Maven

Next, Maven will build the source code present on GitHub. Unit tests and static code analysis will be conducted during the build stage. If the build stage fails, the failure report will be sent over through email or Slack notifications.

c. Code Quality Check with SonarQube

If the build stage is successful, SonarQube will check the code quality for various security vulnerabilities. If the vulnerabilities exceed a certain threshold as per company code policies, this stage will be stopped, and the report will be sent over through email or Slack notifications.

d. Building and Uploading Docker Image

Next, we will build the Docker image for the application and upload this image to a central container registry, most preferably DockerHub. All stages will be performed by a Docker Agent.

e. Continuous Delivery with ArgoCD

Once the CI stages are complete, the continuous delivery (CD) process begins. We will use Shell scripts to update the Git Application Manifests which store YAML manifests like pod.yml, deployment.yml, and service.yml. ArgoCD, a Kubernetes controller, will then automatically sync and manage the desired state of Kubernetes resources with the state defined in the Git repository.

Application Testing using AWS EC2

Before diving into creating a CI/CD pipeline for our Java-based application, it is crucial to ensure that the application runs smoothly in a cloud environment. So, first we will run the application only, and for that, we will be using an AWS EC2 instance with t2.micro instance type.

Steps to Run the Application:

Launch an AWS EC2 Instance

Log in to your AWS Management Console.
Navigate to the EC2 Dashboard.
Launch a new instance using the t2.micro instance type.
Click on Launch Instance after creating a key pair for the instance.

Connect to Your EC2 Instance

Connect to your instance using SSH:

ssh -i path/to/your-key-pair.pem ec2-user@your-ec2-instance-ip

Clone the Application Repository

Once connected to the EC2 instance, clone the repository containing the Java application:

git clone https://github.com/nishankkoul/Jenkins-CICD.git
cd Jenkins-CICD/spring-boot-app

Install Maven

Update the package list and install Maven:

sudo apt update
sudo apt install maven -y

Verify the Maven installation:

mvn -version

Build the Application

Build the application using Maven:

mvn clean package

This command generates a packaged artifact in a .jar file stored in the target folder.

Run the Application Locally (Optional)

To avoid potential issues with local setup, Java versions, and other dependencies, it is recommended to use Docker. However, if you prefer to run the application locally, install Java 11 and execute:

sudo apt install openjdk-11-jdk -y
java -jar target/spring-boot-web.jar

Access the application at http://localhost:8080.

Install Docker

Follow the official Docker installation guide for Ubuntu: Install Docker on Ubuntu

Grant Docker Permissions

Add the ubuntu user to the docker group:

sudo usermod -aG docker ubuntu

Log out and log back in to apply the changes.

Review the Dockerfile

Here is the Dockerfile that will be used to containerize the application:

FROM adoptopenjdk/openjdk11:alpine-jre
ARG artifact=target/spring-boot-web.jar
WORKDIR /opt/app
COPY ${artifact} app.jar
ENTRYPOINT ["java","-jar","app.jar"]

This Dockerfile specifies:

Base Image: adoptopenjdk/openjdk11:alpine-jre
Build Argument: artifact with default value target/spring-boot-web.jar
Working Directory: /opt/app
Copy Artifact: Copies the JAR file into the container
Entry Point: Runs the Java application

Build the Docker Image

Execute the command to build the Docker image:

docker build -t java-web-app:v1 .

Run the Docker Container

Create and run a Docker container from the image:

docker run -d -p 8010:8080 java-web-app:v1

Verify that the container is running:

docker ps

Configure Security Groups for EC2

Edit the inbound rules of the security group associated with your EC2 instance to allow traffic on port 8010 from anywhere (IPv4).

Access the Application

Copy the public IP address of your EC2 instance.
Open a web browser and navigate to:

http://{Instance_IP_Address}:8010

Replace {Instance_IP_Address} with the IP address of your EC2 instance to access the application.

The application is successfully running!

Cleanup

Exit the SSH session and terminate the EC2 instance if it is no longer needed.

Jenkins CI/CD Project Execution

Now that we have successfully run the Java application on an AWS EC2 instance, we can proceed with creating a CI/CD pipeline for the entire project. This will ensure that any future code changes are automatically built, tested, and deployed, enhancing the development workflow and application reliability.

Step 1: Launching an EC2 Instance

To begin, we need to create an EC2 instance with Ubuntu as the operating system. For this guide, we will use a t2.large instance type. Follow these steps:

Launch the Instance: Navigate to the AWS Management Console, select EC2, and launch a new instance with Ubuntu as the OS.
Connect to the Instance: Once the instance is running, connect to it using SSH.

Step 2: Installing Jenkins

Before we install Jenkins, we need to install Java. Follow these commands:

sudo apt update && sudo apt install openjdk-11-jre
java -version

After verifying the Java installation, proceed to install Jenkins:

curl -fsSL https://pkg.jenkins.io/debian/jenkins.io-2023.key | sudo tee /usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins

Note: By default, Jenkins will not be accessible externally due to inbound traffic restrictions. Open port 8080 in the security group of your EC2 instance.

Step 3: Configuring Jenkins

Access Jenkins using the following URL: http://{your_ec2_instance_ip_address}:8080. Retrieve the administrator password with:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Use the password to log in, then:

Install the suggested plugins.
Skip and continue as admin.
Save and finish to access the Jenkins dashboard.

Step 4: Creating a Jenkins Pipeline

To create a pipeline in Jenkins:

Click on New Item, name it jenkins-cicd, select Pipeline, and click OK.
Under Pipeline Configuration, select Pipeline script from SCM.
Choose Git and enter the repository URL: https://github.com/nishankkoul/Jenkins-CICD/.
Set the script path to: java-maven-sonar-argocd-helm-k8s/spring-boot-app/Jenkinsfile.
Set the branch to main and save.

Step 5: Understanding the Jenkinsfile

The Jenkinsfile defines the CI/CD pipeline stages and uses Docker as an agent:

Build and Test: Builds and tests the project using Maven.
Static Code Analysis: Analyzes code with SonarQube.
Build and Push Docker Image: Builds and pushes a Docker image to Docker Hub.
Update Deployment File: Updates the Kubernetes deployment file with the new Docker image tag.

pipeline {
  agent {
    docker {
      image 'abhishekf5/maven-abhishek-docker-agent:v1'
      args '--user root -v /var/run/docker.sock:/var/run/docker.sock' // mount Docker socket to access the host's Docker daemon
    }
  }

  stages {
    stage('Build and Test') {
      steps {
        sh 'ls -ltr'
        // build the project and create a JAR file
        sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && mvn clean package'
      }
    }
    stage('Static Code Analysis') {
      environment {
        SONAR_URL = "http://54.210.194.59:9000/" //Change this value depending upon your VM's IP Address
      }
      steps {
        withCredentials([string(credentialsId: 'sonarqube', variable: 'SONAR_AUTH_TOKEN')]) {
          sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && mvn sonar:sonar -Dsonar.login=$SONAR_AUTH_TOKEN -Dsonar.host.url=${SONAR_URL}'
        }
      }
    }
    stage('Cleanup Docker Resources') {
      steps {
        script {
          sh '''
            docker container prune -f
            docker image prune -a -f
            docker volume prune -f
            docker network prune -f
          '''
        }
      }
    }
    stage('Build and Push Docker Image') {
      environment {
        DOCKER_IMAGE = "nishankkoul/ultimate-cicd:${BUILD_NUMBER}"
        // DOCKERFILE_LOCATION = "java-maven-sonar-argocd-helm-k8s/spring-boot-app/Dockerfile"
        REGISTRY_CREDENTIALS = credentials('docker-cred')
      }
      steps {
        script {
          sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && docker build -t ${DOCKER_IMAGE} .'
          def dockerImage = docker.image("${DOCKER_IMAGE}")
          docker.withRegistry('https://index.docker.io/v1/', "docker-cred") {
            dockerImage.push()
          }
        }
      }
    }
    stage('Update Deployment File') {
      environment {
        GIT_REPO_NAME = "Jenkins-CICD"
        GIT_USER_NAME = "nishankkoul"
      }
      steps {
        withCredentials([string(credentialsId: 'github', variable: 'GITHUB_TOKEN')]) {
          sh '''
            git config user.email "Add your email id here"
            git config user.name "Add your name here"
            BUILD_NUMBER=${BUILD_NUMBER}
            sed -i "s/replaceImageTag/${BUILD_NUMBER}/g" java-maven-sonar-argocd-helm-k8s/spring-boot-app-manifests/deployment.yml
            git add java-maven-sonar-argocd-helm-k8s/spring-boot-app-manifests/deployment.yml
            git commit -m "Update deployment image to version ${BUILD_NUMBER}"
            git push https://${GITHUB_TOKEN}@github.com/${GIT_USER_NAME}/${GIT_REPO_NAME} HEAD:main
          '''
        }
      }
    }
  }
}

Step 6: Installing Docker Pipeline Plugin

To use Docker as an agent, we first need to install the Docker Pipeline plugin. Go to "Manage Jenkins" > "Manage Plugins" > "Available" and search for the "Docker Pipeline" plugin. Select the checkbox next to the plugin and click "Install without restart" or "Download now and install after restart" depending on your preference.

After the plugin is installed, we do not need to install Maven as we will be using an image that has already Maven and Docker installed inside it.

Step 7: Installing SonarQube

To install SonarQube:

a. So, the next installation is SonarQube. Go back to "Manage Jenkins" > "Manage Plugins" > "Available" and search for "SonarQube Scanner". Select the checkbox next to the plugin and click "Install without restart" or "Download now and install after restart" depending on your preference.

b. Create a new user and download SonarQube:

sudo -i
apt install unzip
adduser sonarqube
su - sonarqube
wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-9.4.0.54424.zip
unzip sonarqube-9.4.0.54424.zip
chmod -R 755 sonarqube-9.4.0.54424
chown -R sonarqube:sonarqube sonarqube-9.4.0.54424
cd sonarqube-9.4.0.54424/bin/linux-x86-64/
./sonar.sh start

c. Open port 9000 in your security group to access SonarQube at http://{ec2_instance_ip_address}:9000.

d. Log in with admin/admin, update the password, and generate a token for Jenkins under Administrator > Security.

Step 8: Configuring SonarQube in Jenkins

Add the SonarQube token in Jenkins:

Go to Manage Jenkins > Credentials > System > Global Credentials.
Add a new credential of type Secret Text, paste the token, and name it sonarqube

Step 9: Log in to the SonarQube Server

We have successfully started the SonarQube server. To log in to the SonarQube server, enter the login ID and password as "admin". After that, update your password and click on "Update".

Step 10: Setting Up DockerHub and GitHub Credentials

To add DockerHub and GitHub credentials in Jenkins, follow these detailed steps:

Adding DockerHub Credentials

Open Jenkins: Go to your Jenkins dashboard.
Manage Jenkins: Click on "Manage Jenkins" in the left sidebar.
Manage Credentials: In the "System Configuration" section, click on "Manage Credentials".
Global Credentials: Click on the "(global)" domain to add credentials that can be used by all jobs.
Add Credentials: Click on "Add Credentials" on the left sidebar.
Select Credential Type:
- From the "Kind" dropdown, select "Username with password".
Enter DockerHub Credentials:
- Username: Enter your DockerHub username.
- Password: Enter your DockerHub password.
- ID: Enter "docker-cred".
- Description: Enter a description, for example, "DockerHub Credentials".
Save: Click on "OK" to save the credentials.

Adding GitHub Credentials

Open Jenkins: Go to your Jenkins dashboard (if not already there).
Manage Jenkins: Click on "Manage Jenkins" in the left sidebar (if not already there).
Manage Credentials: In the "System Configuration" section, click on "Manage Credentials" (if not already there).
Global Credentials: Click on the "(global)" domain to add credentials that can be used by all jobs (if not already there).
Add Credentials: Click on "Add Credentials" on the left sidebar (if not already there).
Select Credential Type:
- From the "Kind" dropdown, select "Secret text".
Enter GitHub Credentials:
- Secret: Paste your GitHub personal access token.
- ID: Enter "github".
- Description: Enter a description, for example, "GitHub Personal Access Token".
Save: Click on "OK" to save the credentials.

Make sure to switch back to the root user from the sonarqube user.

Step 11: Build the Pipeline

Start the Build:

Click on "Build Now" to initiate the pipeline execution.

Monitor the Build:

As the pipeline runs, you can monitor the build process and see each stage being executed.

Successful Execution:

The pipeline has been executed successfully.

Review SonarQube Results:

SonarQube results are available and have been processed successfully.

DockerHub Image:

DockerHub contains the new image that was built during the pipeline execution.

Update Deployment:

The image version inside the deployment.yml file has also been updated accordingly.

Step 12: Installing Minikube and Kubernetes

Create a new EC2 instance (Ubuntu) with t2.large instance type and install Minikube and kubectl:

Install Docker as the driver:

sudo apt-get update
sudo apt-get install docker.io

Grant the required permissions to both ubuntu and jenkins users:

sudo usermod -aG docker ubuntu
sudo usermod -aG docker jenkins

Install Minikube:

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube

Install kubectl:

sudo snap install kubectl --classic

Start the Minikube cluster:

minikube start --driver=docker

Check Minikube status:

minikube status

Step 13: Installing ArgoCD

Install ArgoCD on Minikube:

a. Create a namespace:

kubectl create ns argocd

b. Apply ArgoCD manifests:

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argocd/v2.5.8/manifests/install.yaml

c. Verify the installation:

kubectl get all -n argocd

Step 14: Accessing ArgoCD UI

a. Expose ArgoCD Server

By default, the ArgoCD server is not exposed outside the cluster. To access the ArgoCD UI, you can use port-forwarding.

Execute the following command to enable port forwarding:

kubectl port-forward svc/argocd-server -n argocd --address 0.0.0.0 8080:443 &

b. Allow Port 8080 in the Security Group

To access the ArgoCD UI, you need to allow port 8080 in the Security Group of the EC2 instance for All Traffic. Once done, open your browser and enter the URL:

http://{ec2_instance_ip_address}:8080

Note: You may encounter a privacy warning. Ignore the warning, click on "Advanced," and then hit "Proceed to localhost (unsafe)" to continue to the GUI interface. (Your browser settings may present a different option to continue).

c. Access the ArgoCD UI

You should now be able to access the ArgoCD UI successfully! Note that the terminal used for port forwarding will be occupied, so use another terminal for the following steps.

d. Retrieve Initial Admin Password

To log in to ArgoCD, you need the initial password for the admin user. Retrieve it using the command:

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

Enter the username as “admin” and use the password obtained from the command above. Click "Sign in."

Step 15: Creating an Application in ArgoCD

a. After logging in, click on "+New App" and enter the following configurations:

Application Name: test
Project Name: default
Sync Policy: default
Repository URL: https://github.com/nishankkoul/JenkinsCICD
Revision: HEAD
Path (path of deployment.yml): java-maven-sonar-argocd-helm-k8s/spring-boot-app-manifests
Cluster URL: https://kubernetes.default.svc/
Namespace: default

Click on "Create."

b. Sync and Deploy Application

After a short while, ArgoCD will successfully sync with the repository and deploy the pods as defined in deployment.yml.

c. Access the Application

Before accessing the application, execute the following commands to check the status of pods and services:

kubectl get pods
kubectl get svc

d. Check the service type defined in service.yml:

As per service.yml, the service is a NodePort-type service running on port 32225. To access the service, perform port-forwarding:

kubectl port-forward svc/spring-boot-app-service --address 0.0.0.0 7000:80

e. Allow Port 7000 in the Security Group

Allow port 7000 for all traffic in the Security Group of your EC2 instance. Then, enter the following URL in your browser:

http://{ec2_instance_ip_address}:7000

f. Access Your Application

Finally, you should be able to access your application, completing the deployment process. Congratulations on successfully deploying your application using ArgoCD and Minikube!

Jenkins End-to-End CI/CD Project using ArgoCD

Table of contents