Getting Started

ECS Fargate Container CI/CD Pipeline

The goal of this article is to introduce and explore the DevOps methodology for the  deployment of microservices in the managed cluster. This article also focuses on maintaining rapid frequent delivery in the production.


Introduction to GitHub Actions


GitHub Actions make it easy to automate all your software workflows. Github Actions  let you build, test, and deploy your code right from GitHub.  The easiest way is to think of Github Actions as their continuous integration system.

GitHub Actions

Apart From the mentioned, GitHub actions have many edges over any native vendor  pipeline tools like:

  • Self-hosted Workers 
  • Support for multiple branches 
  • Built-in Pipeline as Code 
  • Parallelism 
  • Distributed builds 

Github WorkFlow


Workflows run in Linux, macOS, Windows, and containers on GitHub-hosted machines,  called ’Runners’ GitHub Actions work by using configured workflows That contain actions and their  relationships with each other.

Github Actions Keywords


  • Action:​ Individual tasks that you combine as steps to create a job. 
  • Artifact: ​Artifacts are the files created when you build and test your code. 
  • Event: ​A specific activity that triggers a workflow run. (Push, Pull). 
  • Job: ​A set of steps that execute on the same runner. 
  • Runner: ​Any machine with the GitHub Actions runner application installed. 
  • Step: ​A step is an individual task that can run commands or actions. 
  • Workflow: ​A configurable automated process that you can set up in your repository to  build, test, package, release, or deploy any project on GitHub. 

Deployment Steps


Step 1 - Prepare The Application

  • Dockerfile For The Image:
FROM kumar1996/stack:latest  
WORKDIR /app  
COPY app.py /app  
COPY templates /app/templates  
EXPOSE 5000  
ENTRYPOINT ["python"]  
CMD ["app.py"] 

NOTE​: Here I am using My Base Image of flask hosted in the public Docker Hub  Repository. 

Step 2 - Infrastructure Deployment


  • ​Elastic Container Service - Create an ECR repository to store your images.(I will be creating it with Terraform  Github Action Workflow) 
  • Create an ECS Cluster - I will be using ECS Fargate Type Cluster.

ECS Cluster

  • Create an ECS task definition -
    a. Make Sure To save the Json Template In the Git Repository.  b. Our GitHub Action Will refer to this Json file while Creating tasks.  c. Our Github Action will also create revision accordingly. 
  • We will need the URI of the ECR that will be created by the terraform template.
  • The URI should look something like this: awsAccountID.dkr.ecr.aws-Region.amazonaws.com/RepositoryName
  • Create a New role for ECS and Attach Policy to it with all the required  permissions and note down the ARN of the same. 
  • The output JSON for the task definition should look something like this:
{ 
    "family": "mycluster",  
    "executionRoleArn": arn:aws:iam::XXXXXXXXX:role/ecstask", 
    "networkMode": "awsvpc", 
    "containerDefinitions":[ 
        "name": "mycluster",  
        "image": ”XXXXXXX.dkr.ecr.us-east-1.amazonaws.com/mycluster:latest", 
        "Mappings":[ "containerPort":5000, "hostPort":5000, "tcp" ],
        "essential":true ,  
        "requiresCompatibilities":[   "FARGATE"   ],  
        "cpu": "512",   
        "memory": "1024"
    ]
}  
  • We need to Create​ ECS service a Load Balancer to make the application publicly accessible.
  • Create an Application Load Balancer with no backend target.(Targets will be added by the Pipeline).
  • Now create The​ Service​ in the Cluster Section.

ECS Service

  • In the Service Section, Choose the Load Balancer and Under Container to load balancer section Select the TaskDefinitionName:5000.
Our Basic Infrastructure is ready, Let’s Move ahead.

Github Action SetUp

Prerequisites


  1. Create A user and assign the required policy to this user. (AWS ECS FULL  ACCESS). 
  2. Create two Secrets and Store the Access Key And Secret Key in the GItHub  Secrets. 
  3. Create 1 API Token from Terraform Cloud and Store It in the GitHub Secrets.

Configuring GitHub Action Workflow for Terraform


  • Step 1: Go to the Repository in the Github and click on Actions. 
  • Step 2: Click on setup this workflow yourself. 
  • Step 3: The Action will have Multiple Steps as described below -  
name: 'Terraform'

on:
  push:
    branches:
    - master
  pull_request:

jobs:
  terraform:
    name: 'Terraform'
    runs-on: ubuntu-latest

    # Use the Bash shell regardless whether the GitHub Actions runner is ubuntu-latest, macos-latest, or windows-latest
    defaults:
      run:
        shell: bash

    # Checkout the repository to the GitHub Actions runner
    steps:
    - name: Checkout
      uses: actions/checkout@v2

    # Install the latest version of Terraform CLI and configure the Terraform CLI configuration file with a Terraform Cloud user API token
    - name: Setup Terraform
      uses: hashicorp/setup-terraform@v1
      with:
        cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}

    # Initialize a new or existing Terraform working directory by creating initial files, loading any remote state, downloading modules, etc.
    - name: Terraform Init
      run: terraform init

    # Checks that all Terraform configuration files adhere to a canonical format
    - name: Terraform Format
      run: terraform fmt -check
      continue-on-error: true

    # Generates an execution plan for Terraform
    - name: Terraform Plan
      run: terraform plan

    - name: Terraform Apply
      if: github.ref == 'refs/heads/master' && github.event_name == 'push'
      run: terraform apply -auto-approve

Configuring GitHub Action Workflow for AWS ECS


deploy:
    name: Deploy
    runs-on: ubuntu-latest
    needs: [terraform]
    steps:
    - name: Checkout
      uses: actions/checkout@v2

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_KEY }}
        aws-region: us-east-1

    - name: Login to Amazon ECR
      id: login-ecr
      uses: aws-actions/amazon-ecr-login@v1

    - name: Build, tag, and push image to Amazon ECR
      id: build-image
      env:
        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
        ECR_REPOSITORY: mycluster
        IMAGE_TAG: latest
      run: |
        # Build a docker container and
        # push it to ECR so that it can
        # be deployed to ECS.
        docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
        echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"
    - name: Fill in the new image ID in the Amazon ECS task definition
      id: task-def
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: task-defination.json
        container-name: mycluster
        image: ${{ steps.build-image.outputs.image }}

    - name: Deploy Amazon ECS task definition
      uses: aws-actions/amazon-ecs-deploy-task-definition@v1
      with:
        task-definition: ${{ steps.task-def.outputs.task-definition }}
        service: service
        cluster: mycluster
        wait-for-service-stability: true

The Pipeline Will Work Like this


ECS Service

The GitHub Action we Built will create end-to-end CI/CD pipeline.

Subscribe to Developer Stack

Get the latest posts delivered right to your inbox