MG's Blog

Initial EKS Setup for beginners

Initial EKS Setup for beginners

Mridul Ganga's photo
Mridul Ganga
·

3 min read

Table of contents

When you create a new Kubernetes Cluster in EKS - you still need to do several things until you can start using it. This simple set of steps aims to help people create an EKS cluster in simple steps and configure the most essential services like Log forwarding to Cloudwatch, Cert Manager for automatically vending SSL certs from Letsencrypt, and Ingress controller to route external traffic to k8s services.

Create Kubernetes Cluster on EKS

  1. Install awscli (https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)

  2. Install aws-iam-authenticator (https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html)

  3. Run aws configure and setup your root user's access key and secret key.

  4. Install eksctl (https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html)

  5. Create a cluster.yaml file in a nice folder.

# contents of cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: basic-cluster
  region: ap-south-1

# enable control plane logging
cloudWatch:
  clusterLogging:
    enableTypes: ["*"]

# enable oidc for irsa
iam:
  withOIDC: true

nodeGroups:
  - name: ng-1
    instanceType: t3.small
    desiredCapacity: 2
    volumeSize: 80
  - name: ng-2
    instanceType: t2.small
    desiredCapacity: 2
    volumeSize: 100
  1. Create the cluster using eksctl (make sure you have awscli and creds setup already)
eksctl create cluster -f cluster.yaml
  1. Create kubeconfig to access the cluster
mkdir ~/.kube
eksctl utils write-kubeconfig --cluster=basic-cluster --kubeconfig=~/.kube/config

More info at https://eksctl.io/usage/creating-and-managing-clusters/

Adding vpc-cni addon

vpc-cni addon allows EKS to provision more pods on a node. In the below code, replace the service account ARN with the role from your AWS IAM and change basic-cluster with your cluster name.

eksctl create addon --name vpc-cni --version 1.11.4-eksbuild.1 --cluster basic-cluster \
    --service-account-role-arn arn:aws:iam::111122223333:role/AmazonEKSVPCCNIRole --force

More info at https://docs.aws.amazon.com/eks/latest/userguide/managing-vpc-cni.html

Deploy Nginx ingress controller on the cluster

The following command will create the required resources for Nginx Ingress Controller and also create a LoadBalancer Service. The Load Balancer should create an External Load Balancer on AWS and point it to our ingress controller.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.5.1/deploy/static/provider/aws/deploy.yaml

BONUS:

  1. When you deploy nginx and created the load balancer, get the endpoint for the same using kubectl get svc --namespace=nginx-ingress

  2. You can now point your domain to this endpoint using CNAME records (ex - *.k8s.mridulganga.dev -> aaa71bxxxxx-11xxxxx10.us-east-1.elb.amazona..).

  3. To validate the above configuration, you can curl to the endpoint and expect to see a 404 error from the Nginx curl xyz.k8s.mridulganga.dev

More info at https://kubernetes.github.io/ingress-nginx/deploy/#aws

Deploy cert-manager & create cluster issuer

  1. Follow the installation instructions on the cert-manager website https://cert-manager.io/docs/installation/

  2. Create ClusterIssuer Save the following content in a file letsencrypt-staging.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-staging
    solvers:
      - http01:
          ingress:
            class: nginx

Create the resource using kubectl kubectl apply -f letsencrypt-staging.yaml

NOTE: use staging when you are first trying out certs, when you are comfortable with the cert vending process - you can move to letsencrypt-prod by creating a new Issuer using the acme server: https://acme-v02.api.letsencrypt.org/directory. Check the rate limits letsencrypt.org/docs/rate-limits

Enable Cloudwatch log forwarding using Fluent Bit

If you want to forward logs from your pods to cloudwatch, then run the following command to enable that using fluent bit daemons. Replace the variables in the command with your cluster info.

ClusterName=<my-cluster-name>
RegionName=<my-cluster-region>
FluentBitHttpPort='2020'
FluentBitReadFromHead='Off'
[[ ${FluentBitReadFromHead} = 'On' ]] && FluentBitReadFromTail='Off'|| FluentBitReadFromTail='On'
[[ -z ${FluentBitHttpPort} ]] && FluentBitHttpServer='Off' || FluentBitHttpServer='On'
curl https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/quickstart/cwagent-fluent-bit-quickstart.yaml | sed 's/{{cluster_name}}/'${ClusterName}'/;s/{{region_name}}/'${RegionName}'/;s/{{http_server_toggle}}/"'${FluentBitHttpServer}'"/;s/{{http_server_port}}/"'${FluentBitHttpPort}'"/;s/{{read_from_head}}/"'${FluentBitReadFromHead}'"/;s/{{read_from_tail}}/"'${FluentBitReadFromTail}'"/' | kubectl apply -f -

More info at https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-EKS-quickstart.html

Image Credits - opensourceforu.com/2018/06/container-orches..

DevopsKubernetesAWSEKS