Creating GitOps Pipelines Using Amazon Elastic Kubernetes Service(EKS) and GitHub Actions | Introduction to GitOps- Part 3

7 min readMar 23, 2022

In this blog, We will learn, What is GitOps, How it is different from Traditional Ops, what are the benefits of GitOps, and How to Create a GitOps Pipeline using Amazon Elastic Kubernetes Service(EKS).

Prerequisites

before we get started make sure you have access to

AWS Account having EKS Cluster Access
GitHub or GitLab Account
AWS Elastic Container Registry (ECR)
eksctl and kustomize installation on the local machine

Evolution to GitOps

Managing everyday tasks, which includes infrastructure configuration and software deployment. Infrastructure configurations mean that computer resources prepare to enable the software application to operate correctly, and software deployment is the process of taking a particular version of a software application and making it ready to run on the computing infrastructure. Managing these two processes is the core of GitOps. Before we dig into how this management is done in GitOps, however, it is useful to understand the traditional Ops and how GitOps is helping us out.

Traditional Ops

In the traditional IT Operations model, the development team is responsible for delivering new versions of a software application to a quality assurance team that tests the new version and then delivers it to an operation team for deployment. It becomes very difficult for a traditional operating model to support an increasingly compressed released cycle.
The operation team is responsible for infrastructure configuration and deployment of new software changes. The operation team’s primary focus is to ensure the reliability resilience and security of the system running the software. This requires a lot of specialized knowledge.

DevOps

DevOps is both an organizational structure and mindset change with an emphasis on automation. An operations team is no longer responsible for deployment and operations, the application
the development team takes on these responsibilities.

The benefits of DevOps include

Better collaboration between development and operations
2. Improved product quality
3. More frequently releases
4. Reducing time for new features

GitOps

GitOps is the operational framework that allows us to take the best practices used for application development to infrastructure automation. In practice, GitOps is achieved by combining Infrastructure as Code(IaC), Git Repositories, PR/MR, and CI/CD Pipelines.

GitHub (along with GitLab, Bitbucket, and so on) is central to the modern software development life cycle, so it seems natural that it is also used for systems operation and management.

Creating GitOps Pipeline Using Amazon Elastic Kubernetes Service(EKS)

Now, you got the idea why GitOps is so useful. As you know we will create a GitOps pipeline using Amazon Elastic Kubernetes Service(EKS).

Forking GitHub Repository

First of All, you need to fork this repository.

Creating AWS Account

We need to create a separate user for Kubernetes and elastic container registry so it will only access these two AWS Services. To create a new user login to your AWS Console and then go to the AWS Identity Access and Management(IAM).

Setting Up Lab

Once we created a new AWS user, it's time to set up a lab, for this, we need to install AWS CLI, eksctl, Kubectl and customize. You can setUp according to your operating system, as I did on the Windows system so I also need the chocolatey package manager to download eksctl and customize. Once you download all the required tools, then we need to configure the AWS Account on AWS CLI so we can create a new cluster using eksctl. Use the following command to configure aws CLI.

aws configureor aws configure --profile=default

once you configured the aws account, now you need to use eksctl to create a new cluster.

eksctl version // to check version eksctl create cluster //to create cluster

You need to wait until eksctl set up the cluster to the us-east-1 region and its availability zones. Once it's completed, you can check the pod using this command.

kubectl get pods -A

Create Elastic Container Registry

Now we need to create an Elastic Container Registry, for this, you can use guestbook as a naming convention.

Adding AWS Secrets to GitHub Repository

Now we need to add secrets to the GitHub Repository which we created earlier, for this you can use this command to get an account ID, also you need to have AWS Access Key and Secret Key to proceed Further. this is what you need. Go to your GitHub Repository setting and then from the left menu select the secret, then add them one by one.

AWS_ACCOUNT_ID=AWS_ACCESS_KEY_ID=AWS_SECRET_ACCESS_KEY=To get account Id use this commandaws sts get-caller-identity

Configuring GitHub Actions Workflow

Now you need to go to your GitHub Repository and then go to the .github/workflows/main.yml.Ensure that the Environment Variable naming convention is the same as your secrets.

Installing Flux

Now we need to install Flux, for this, you can install Flux according to the operating system you are currently using from the Flux Official Website. Once installed you can check the Flux and cluster requirements using this command.

flux check --pre

Creating GitHub Access Token & Flux Bootstrapping

Now we need to create a new GitHub Access token, so flux can also create a new private repository. once you are done you can export your secret token and access key using cmd.

export GITHUB_TOKEN=[your-github-token]export GITHUB_USER=[your-github-username]

for Windows you can use SET

In this step, a private repository is created and all of the controllers will also be installed to your EKS cluster. When bootstrapping a repository with Flux.

flux bootstrap github \--owner=$GITHUB_USER \--repository=fleet-infra \--branch=main \--path=[cluster-name] \--personal

for Windows User , Please Make Sure the Export GitHub User is working ,otherwise it will give you an error , in such cases you can sue this command.
flux bootstrap github --owner=adilshehzad786 --repository=fleet-infra --branch=main --path=<CLUSTER_NAME> --personal

Let's Check the Flux namespace using this command

kubectl get namespaces

Now You need to clone the newly created GitHub Repository and native to it using the Cd command.

git clone https://github.com/$GITHUB_USER/fleet-infracd fleet-infra

Now Use this command to connect the Guestbook repository with the fleet-infra repository.

flux create source git guestbook --url=https://github.com/adilshehzad786/gitops-guestbook --username=adilshehzad786 --password=<github token> --branch=main --interval=30s --export > ./<cluster-name>/gitops-guestbook.yaml

COnfigure Flux Kustomize to deploy changes to the new repo

flux create kustomization guestbook --source=gitops-guestbook --path="./deploy" --prune=true --validation=client --interval=1h --export > ./amazing-unicorn-1648047534/gitops-guestbook-sync.yaml

now, using the git command you can add the changes, commit and push to the main branch.

git add -A && git commit -m "add guestbook-gitops deploy" && git push

to view the latest change in your cluster, use the following command

flux get kustomizations — watch

In case you are getting errors or troubleshoot the error you can use the following command

flux get sources gitflux get kustomizations

Making Changes to Application

Now let's make a change to the index.html file and replace the 15 lines with the new line

<button type="button" class="btn btn-primary btn-lg btn-block" ng-click="controller.onRedis()">Submit</button>

Proper your changes to a new branch and create a Pull Request, Wait for the GitHub Actions, and then merge the changes.

Display the Guestbook application

Display the Guestbook frontend in your browser by retrieving the URL from the app running in the cluster with:

kubectl get service frontend

Cleaning up

To delete the cluster, run:

eksctl delete cluster --name [name of your cluster]

Conclusion

In this Blog, we learn, how to create GitOps Pipeline using AWS Kubernetes & GitHub Actions. feel free to ask any questions if you have.