Goals

In this module, we will use this starter template - aws-webhook-series-part-1 as a starting point.

By the end of this module, you should:

✅ Be able to setup AWS and Github Actions CI/CD using Open ID Connect
✅ Be able to setup Github actions using our AWS Lambda setup
✅ Have a functional CI/CD pipeline for our AWS Lambda setup

In our previous module, we performed these steps manually, in this module our focus will be on automating all these steps.

Note: If you are just looking to setup AWS Open ID Connect using AWS Lambda and Github actions, I have higlighted the sections with a ⭐️!

Content

Introduction
Setting up the CI/CD Infrastructure ⭐️
1. Add the AWS Open ID connect provider
2. Create the custom IAM trust policy
3. Create the custom role then attach the IAM permissions
4. Apply the infrastructure
Setting up the Github Actions ⭐️
Test out the infrastructure
1. Make a change in both of the functions
2. Using AWS cli invoke the lambda function
Conclusion

Introduction

Just a quick recap of the CI/CD, the deployment process consist of the following steps:

The initial trigger (We push our code)
Creating the zip file (Install packages, test and build)
Uploading the zip file to S3
Updating our Lambda and publishing new version
Updating our Lambda Alias with the new version

In addition to the CI/CD process setup, we will also harden the security of how our Github Actions manage our AWS credentials.

Instead of uploading the to Github’s secrets store, we will just assign a role to our Github Actions and limit the access to a particular branch (ie main).

There are few benefits:

The temporary AWS Credentials are short lived
- If they were leaked, the hacker likely won’t be able to do much in that short time
The permissions will be fine-grained (The scope of the permissions on these tokens are limited)
If needed, revoking access is as simple as removing or denying these permissions on that specific role

Here is a quick presentation slide to better understand the Open ID connect flow with AWS and Github Actions:

If you’d like learn more about how AWS, Github Actions and Open ID connect all work together, I encourage you to read my original post where I go through this subject in detail - Security harden Github Action deployments to AWS with OIDC.

Setting up the CI/CD infrastructure

Let’s start setting this up.

1. Add the AWS Open ID connect provider

main.tf changes

Add the following to your main.tf.

data "tls_certificate" "github" {
  url = "https://token.actions.githubusercontent.com/.well-known/openid-configuration"
}

### Github OIDC for Lambda
resource "aws_iam_openid_connect_provider" "github_actions" {
  client_id_list  = var.client_id_list
  thumbprint_list = [data.tls_certificate.github.certificates[0].sha1_fingerprint]
  url             = "https://token.actions.githubusercontent.com"
}

variables.tf changes

Add the following to your variables.tf.

variable "client_id_list" {
  default = [
    "sts.amazonaws.com"
  ]
}

Note: By dynamically generating the TLS certificate for Github Actions, we ensure we always have the latest thumbprint when Github rotates the keys whenever we apply our terraform infrastructure.

2. Create the custom IAM trust policy

In this step, we are establishing a trust policy that limits access defined by the Github respository and branch.

We also need to update the variables.tf with the details like our repository limit the access by.

Note: The default branch we limit by is `master` but that can be changed to any other branch of your preference.

main.tf changes

Add the following to your main.tf.

data "aws_caller_identity" "current" {}

data "aws_iam_policy_document" "github_actions_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRole"]

    principals {
      type        = "AWS"
      identifiers = [
        format(
          "arn:aws:iam::%s:root",
          data.aws_caller_identity.current.account_id
        )
      ]
    }
  }

  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    principals {
      type = "Federated"
      identifiers = [
        format(
          "arn:aws:iam::%s:oidc-provider/token.actions.githubusercontent.com",
          data.aws_caller_identity.current.account_id
        )
      ]
    }
    condition {
      test     = "StringEquals"
      variable = "token.actions.githubusercontent.com:sub"
      values   = ["repo:${var.repo_name}:ref:refs/heads/master"]
    }
  }
}

variables.tf changes

Add the following to your variables.tf.


variable "repo_name" {
  type    = string
  # For Example - "Jareechang/aws-webhook-series-part-1"
  default = "<name-of-your-repository>"
}

3. Create the custom role then attach the IAM permissions

Now, we have to create our role used in our CI/CD with specific IAM Permissions.

These permissions are needed for our script to access to specific resources within our AWS environment.

These include:

Access to S3
Access to Lambda

main.tf changes

Add the following to your main.tf.

resource "aws_iam_role" "github_actions" {
  name               = "github-actions"
  assume_role_policy = data.aws_iam_policy_document.github_actions_assume_role_policy.json
}

data "aws_iam_policy_document" "github_actions" {
  statement {
    actions = [
      "s3:GetObject",
      "s3:ListBucket",
      "s3:PutObject",
    ]
    effect = "Allow"
    resources = [
      aws_s3_bucket.lambda_bucket.arn,
      "${aws_s3_bucket.lambda_bucket.arn}/*"
    ]
  }

  statement {
    actions = [
      "lambda:updateFunctionConfiguration",
      "lambda:updateFunctionCode",
      "lambda:updateAlias",
      "lambda:publishVersion",
    ]
    effect = "Allow"
    resources = [
      module.lambda_ingestion.lambda[0].arn,
      module.lambda_process_queue.lambda[0].arn
    ]
  }
}

resource "aws_iam_role_policy" "github_actions" {
  name   = "github-actions"
  role   = aws_iam_role.github_actions.id
  policy = data.aws_iam_policy_document.github_actions.json
}

outputs.tf changes

Add the following to your outputs.tf.

output "role_arn" {
  value = aws_iam_role.github_actions.arn
}

4. Apply the infrastructure

let’s use terraform to apply the CI/CD infrastructure.

export AWS_ACCESS_KEY_ID=<your-key>
export AWS_SECRET_ACCESS_KEY=<your-secret>
export AWS_DEFAULT_REGION=us-east-1

terraform init
terraform plan
terraform apply -auto-approve

📝 Helpful reference:

AWS - Configuring a role for GitHub OIDC identity provider

Setting up the Github Actions

Now that we have our CI/CD infrastructure ready, let’s create the github actions.

1. Under your root directory create the folders

Create the folders and files we need for our Github Actions.

In this case, we will create a separate Github Actions for each of our functions (hence the two workflow files).

mkdir -p .github/workflows \
&& touch .github/workflows/lambda-ingestion-ci.yml \
&& touch .github/workflows/process-queue-ci.yml

2. Add the ingestion function Github Actions

We are using a custom actions provided by AWS which will handle the temporary credentials using the role with Open ID Connect.

Then, we will run the deployment script which will go through the steps we highlighted above (install, test, build, update functions etc).

Note: Be sure to update role-to-assume and lambda-s3-bucket

name: lambda-ingestion-ci

on:
  push:
    branches:
      - master
      - main
    paths:
      - functions/ingestion/**
      - .github/**
      - scripts/**

permissions:
  id-token: write
  contents: read

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Configure
        uses: actions/checkout@v2
      - uses: pnpm/action-setup@646cdf48217256a3d0b80361c5a50727664284f2
        with:
          version: 6.10.0
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@master
        with:
            role-to-assume: "<role-to-assume>"
            aws-region: "us-east-1"
      - run: MODULE="@function/ingestion" ./scripts/deployment.sh
        env:
          AWS_LAMBDA_FUNCTION_NAME: "Ingestion-function"
          AWS_LAMBDA_ALIAS_NAME: "ingestion-dev"
          AWS_S3_BUCKET: "<lambda-s3-bucket>"

3. Add the process queue function Github Actions

This is a pretty much the identical configuration as above except it runs the deployment scripts in a different folder.

Note: Be sure to update role-to-assume and lambda-s3-bucket

name: lambda-process-queue-ci

on:
  push:
    branches:
      - master
      - main
    paths:
      - functions/process-queue/**
      - .github/**
      - scripts/**

permissions:
  id-token: write
  contents: read

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Configure
        uses: actions/checkout@v2
      - uses: pnpm/action-setup@646cdf48217256a3d0b80361c5a50727664284f2
        with:
          version: 6.10.0
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@master
        with:
            role-to-assume: "<role-to-assume>"
            aws-region: "us-east-1"
      - run: MODULE="@function/process-queue" ./scripts/deployment.sh
        env:
          AWS_LAMBDA_FUNCTION_NAME: "Process-Queue-function"
          AWS_LAMBDA_ALIAS_NAME: "process-queue-dev"
          AWS_S3_BUCKET: "<lambda-s3-bucket>"

Testing out the infrastructure

Once we have our infrastructure and Github Actions definition all ready, the final step is to give it a try to make sure its all working!

1. Make a change in both of the functions

within the src/index.ts in both our functions/ingestion and functions/process-queue make a change to the event.

You can change it to anything, the idea is to make a change so our CI/CD can deploy that new change and we can see it in our logs.

For example:

import {
  APIGatewayProxyEvent,
  APIGatewayProxyResult
} from 'aws-sdk';

// Default starter
export const handler = async(
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  console.log('Event : ', JSON.stringify({
    messages: 'Changed from deployment webhook part 2',
    event,
  }, null, 4));
  let responseMessage = 'default message from ingestion 1';
  return {
    statusCode: 200,
    body: JSON.stringify({
      message: responseMessage,
    }),
  }
}

You can make this change or make a different change in both of the functions.
⚠️ Important: Make sure the Github Repository name matches the repo_name variable in your terraform otherwise you will run into errors in the deployment!

2. Using AWS cli invoke the lambda function

aws lambda invoke \
    --function-name Ingestion-function:ingestion-dev \
    --invocation-type Event \
    --payload 'eyAibmFtZSI6ICJCb2IifQ==' \
    response.json

If it was successful, you should see a output of:

{
    "StatusCode": 202
}

To verify it, you can also check the AWS Cloudwatch logs where within our function we are logging out the event.

This should be under CloudWatch > Log Groups > /aws/lambda/Ingestion-function:

Note: Even though the steps above illustrated for the Ingestion Function , the steps are very similar for the Process Queue Function , you just have to change the aws cli parameters.

📝 Helpful reference:

AWS CLI - lambda invoke

Conclusion

Here is the link to the completed version of this module - aws-webhook-series-part-2.

And that’s it! Now, we have a fully functional CI/CD for our lambda function, we can start to build on top of the infrastructure.

Just to recap:

We added AWS and Github Actions CI/CD using Open ID Connect
We added custom permissions for our CI/CD for our IAM role
We added the Github actions configuration for both of our functions

The next module, we will be start to wiring up our API Gateway with SQS, DLQ and more!

See you in the next module!

Jerry Chang

Go Back

How To Secure Your AWS Deployments Using Github Actions and OIDC In 7 Easy Steps

Enjoy the content ?