Creating GitOps Pipelines Using Amazon Elastic Kubernetes Service(EKS) and GitHub Actions | Introduction to GitOps- Part 3
In this blog, We will learn, What is GitOps, How it is different from Traditional Ops, what are the benefits of GitOps, and How to Create a GitOps Pipeline using Amazon Elastic Kubernetes Service(EKS).
before we get started make sure you have access to
- AWS Account having EKS Cluster Access
- GitHub or GitLab Account
- AWS Elastic Container Registry (ECR)
- eksctl and kustomize installation on the local machine
Evolution to GitOps
Managing everyday tasks, which includes infrastructure configuration and software deployment. Infrastructure configurations mean that computer resources prepare to enable the software application to operate correctly, and software deployment is the process of taking a particular version of a software application and making it ready to run on the computing infrastructure. Managing these two processes is the core of GitOps. Before we dig into how this management is done in GitOps, however, it is useful to understand the traditional Ops and how GitOps is helping us out.
In the traditional IT Operations model, the development team is responsible for delivering new versions of a software application to a quality assurance team that tests the new version and then delivers it to an operation team for deployment. It becomes very difficult for a traditional operating model to support an increasingly compressed released cycle.
The operation team is responsible for infrastructure configuration and deployment of new software changes. The operation team’s primary focus is to ensure the reliability resilience and security of the system running the software. This requires a lot of specialized knowledge.
DevOps is both an organizational structure and mindset change with an emphasis on automation. An operations team is no longer responsible for deployment and operations, the application
the development team takes on these responsibilities.
The benefits of DevOps include
- Better collaboration between development and operations
2. Improved product quality
3. More frequently releases
4. Reducing time for new features
GitOps is the operational framework that allows us to take the best practices used for application development to infrastructure automation. In practice, GitOps is achieved by combining Infrastructure as Code(IaC), Git Repositories, PR/MR, and CI/CD Pipelines.
GitHub (along with GitLab, Bitbucket, and so on) is central to the modern software development life cycle, so it seems natural that it is also used for systems operation and management.
Creating GitOps Pipeline Using Amazon Elastic Kubernetes Service(EKS)
Now, you got the idea why GitOps is so useful. As you know we will create a GitOps pipeline using Amazon Elastic Kubernetes Service(EKS).
Forking GitHub Repository
First of All, you need to fork this repository.
Creating AWS Account
We need to create a separate user for Kubernetes and elastic container registry so it will only access these two AWS Services. To create a new user login to your AWS Console and then go to the AWS Identity Access and Management(IAM).
Setting Up Lab
Once we created a new AWS user, it's time to set up a lab, for this, we need to install AWS CLI, eksctl, Kubectl and customize. You can setUp according to your operating system, as I did on the Windows system so I also need the chocolatey package manager to download eksctl and customize. Once you download all the required tools, then we need to configure the AWS Account on AWS CLI so we can create a new cluster using eksctl. Use the following command to configure aws CLI.
aws configureor aws configure --profile=default
once you configured the aws account, now you need to use eksctl to create a new cluster.
eksctl version // to check version eksctl create cluster //to create cluster
You need to wait until eksctl set up the cluster to the us-east-1 region and its availability zones. Once it's completed, you can check the pod using this command.
kubectl get pods -A
Create Elastic Container Registry
Now we need to create an Elastic Container Registry, for this, you can use
guestbook as a naming convention.
Adding AWS Secrets to GitHub Repository
Now we need to add secrets to the GitHub Repository which we created earlier, for this you can use this command to get an account ID, also you need to have AWS Access Key and Secret Key to proceed Further. this is what you need. Go to your GitHub Repository setting and then from the left menu select the secret, then add them one by one.
AWS_ACCOUNT_ID=AWS_ACCESS_KEY_ID=AWS_SECRET_ACCESS_KEY=To get account Id use this commandaws sts get-caller-identity
Configuring GitHub Actions Workflow
Now you need to go to your GitHub Repository and then go to the
.github/workflows/main.yml.Ensure that the Environment Variable naming convention is the same as your secrets.
Now we need to install Flux, for this, you can install Flux according to the operating system you are currently using from the Flux Official Website. Once installed you can check the Flux and cluster requirements using this command.
flux check --pre
Creating GitHub Access Token & Flux Bootstrapping
Now we need to create a new GitHub Access token, so flux can also create a new private repository. once you are done you can export your secret token and access key using cmd.
export GITHUB_TOKEN=[your-github-token]export GITHUB_USER=[your-github-username]
for Windows you can use SET
In this step, a private repository is created and all of the controllers will also be installed to your EKS cluster. When bootstrapping a repository with Flux.
flux bootstrap github \--owner=$GITHUB_USER \--repository=fleet-infra \--branch=main \--path=[cluster-name] \--personal
for Windows User , Please Make Sure the Export GitHub User is working ,otherwise it will give you an error , in such cases you can sue this command.
flux bootstrap github --owner=adilshehzad786 --repository=fleet-infra --branch=main --path=<CLUSTER_NAME> --personal
Let's Check the Flux namespace using this command
kubectl get namespaces
Now You need to clone the newly created GitHub Repository and native to it using the Cd command.
git clone https://github.com/$GITHUB_USER/fleet-infracd fleet-infra
Now Use this command to connect the Guestbook repository with the fleet-infra repository.
flux create source git guestbook --url=https://github.com/adilshehzad786/gitops-guestbook --username=adilshehzad786 --password=<github token> --branch=main --interval=30s --export > ./<cluster-name>/gitops-guestbook.yaml
COnfigure Flux Kustomize to deploy changes to the new repo
flux create kustomization guestbook --source=gitops-guestbook --path="./deploy" --prune=true --validation=client --interval=1h --export > ./amazing-unicorn-1648047534/gitops-guestbook-sync.yaml
now, using the git command you can add the changes, commit and push to the main branch.
git add -A && git commit -m "add guestbook-gitops deploy" && git push
to view the latest change in your cluster, use the following command
flux get kustomizations — watch
In case you are getting errors or troubleshoot the error you can use the following command
flux get sources gitflux get kustomizations
Making Changes to Application
Now let's make a change to the index.html file and replace the 15 lines with the new line
<button type="button" class="btn btn-primary btn-lg btn-block" ng-click="controller.onRedis()">Submit</button>
Proper your changes to a new branch and create a Pull Request, Wait for the GitHub Actions, and then merge the changes.
Display the Guestbook application
Display the Guestbook frontend in your browser by retrieving the URL from the app running in the cluster with:
kubectl get service frontend
To delete the cluster, run:
eksctl delete cluster --name [name of your cluster]
In this Blog, we learn, how to create GitOps Pipeline using AWS Kubernetes & GitHub Actions. feel free to ask any questions if you have.