Build/Push Runner Image using CodeBuild

Tony Tannous

/

You are viewing article number 5 of 12 in the series Scalable Self-Hosted GitHub Runners on AWS Cloud

Scalable Self-Hosted GitHub Runners on AWS Cloud

This blog post covers setting up a CodeBuild project to build and push the GitHub runner docker image to ECR.

To ensure that the version of the GitHub runner agent within the image remains current, the Codebuild project includes a check to compare the installed version with the latest official release. An EventBridge rule will need to be established to trigger the build on a daily basis using a cron expression.

The project name of github-actions-self-hosted-runner-debian-build is used for the sample commands which follow. AWS account references have been replaced with xxxxxxxxxxxx. Be sure to replace these references with your target account.

All commands were executed using the AWS CLI.

Project IAM Service Role

Create Role

File $HOME/github-actions-role-trust-policy.json below contains the trust policy definition required for the role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "codebuild.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Create the role, with the trust policy attached:

aws iam create-role \
    --path "/service-role/" \
    --role-name github-actions-self-hosted-runner-debian-build-role \
    --assume-role-policy-document file://$HOME/github-actions-role-trust-policy.json

Output sample:

{
    "Role": {
        "Path": "/service-role/",
        "RoleName": "github-actions-self-hosted-runner-debian-build-role",
        "RoleId": "AROARZPUZDIKD3RBBD2BX",
        "Arn": "arn:aws:iam::xxxxxxxxxxxx:role/service-role/github-actions-self-hosted-runner-debian-build-role",
        "CreateDate": "2023-10-10T05:26:51.539000+00:00",
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": "codebuild.amazonaws.com"
                    },
                    "Action": "sts:AssumeRole"
                }
            ]
        },
        "MaxSessionDuration": 3600,
        "RoleLastUsed": {}
    }
}

IAM Policy

Create the file $HOME/github-actions-role-iam-policy.json as shown below.

{
    "Version": "2012-10-17",
    "Statement": [{
            "Effect": "Allow",
            "Action": [
                "codebuild:StopBuild",
                "codebuild:StartBuild",
                "codebuild:RetryBuild"
            ],
            "Resource": "arn:aws:codebuild:us-east-1:xxxxxxxxxxxx:project/github-actions-self-hosted-runner-debian-build"
        }, {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:CreateLogGroup",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:codebuild:us-east-1:xxxxxxxxxxxx:build/github-actions-self-hosted-runner-debian-build:*"
        }, {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "ecr:GetAuthorizationToken"
            ],
            "Resource": "*"
        }, {
            "Effect": "Allow",
            "Action": [
                "codebuild:BatchPutCodeCoverages",
                "codebuild:BatchPutTestCases",
                "codebuild:CreateReport",
                "codebuild:CreateReportGroup",
                "codebuild:UpdateReport",
                "codecommit:GitPull",
                "codecommit:GitPush",
                "ecr:BatchCheckLayerAvailability",
                "ecr:BatchDeleteImage",
                "ecr:BatchGetImage",
                "ecr:CompleteLayerUpload",
                "ecr:DescribeImages",
                "ecr:GetDownloadUrlForLayer",
                "ecr:InitiateLayerUpload",
                "ecr:ListImages",
                "ecr:PutImage",
                "ecr:PutImageScanningConfiguration",
                "ecr:PutImageTagMutability",
                "ecr:UploadLayerPart",
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": [
                "arn:aws:codebuild:us-east-1:xxxxxxxxxxxx:report-group/github-actions-self-hosted-runner-debian-build-*",
                "arn:aws:codecommit:us-east-1:xxxxxxxxxxxx:github-actions-self-hosted-runner-debian",
                "arn:aws:ecr:us-east-1:xxxxxxxxxxxx:repository/github-actions-self-hosted-runner-debian",				
                "arn:aws:logs:us-east-1:xxxxxxxxxxxx:log-group:github-actions-self-hosted-runner-debian-codebuild",
                "arn:aws:logs:us-east-1:xxxxxxxxxxxx:log-group:github-actions-self-hosted-runner-debian-codebuild:*"
            ]
        }
    ]
}

Attach the policy to the role created in previous section:

aws iam put-role-policy \
    --role-name github-actions-self-hosted-runner-debian-build-role \
    --policy-name github_actions_self_hosted_runner_policy \
    --policy-document file://$HOME/github-actions-role-iam-policy.json

ECR Repository Policy

Since our CodeBuild project needs to pull/push the GitHub Runner image from ECR, a policy allowing it to do so will be required.

Create the file $HOME/github-actions-ecr-policy.json with definition as shown below.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CodeBuildAccessPrincipal",
      "Effect": "Allow",
      "Principal": {
        "Service": "codebuild.amazonaws.com"
      },
      "Action": [
        "ecr:BatchCheckLayerAvailability",
        "ecr:BatchGetImage",
        "ecr:GetDownloadUrlForLayer"
      ],
      "Condition": {
        "StringEquals": {
          "aws:SourceAccount": "xxxxxxxxxxxx",
          "aws:SourceArn": "arn:aws:codebuild:us-east-1:xxxxxxxxxxxx:project/github-actions-self-hosted-runner-debian-build"
        }
      }
    }
  ]
}

Apply the policy to the ECR repository github-actions-self-hosted-runner-debian that was created in previous post:

aws ecr set-repository-policy \
    --repository-name github-actions-self-hosted-runner-debian \
    --policy-text file://$HOME/github-actions-ecr-policy.json

Create CodeBuild Project

The sample CLI input file $HOME/github-actions-self-hosted-runner-debian_codebuild_project.json is used to create the project.

{
    "name": "github-actions-self-hosted-runner-debian-build",
    "description": "Build Github runner Docker image from codecommit repository",
    "source": {
        "type": "CODECOMMIT",
        "location": "https://git-codecommit.us-east-1.amazonaws.com/v1/repos/github-actions-self-hosted-runner-debian",
        "buildspec": "./buildspec.yml"
    },
    "sourceVersion": "refs/heads/master",
    "artifacts": {
        "type": "NO_ARTIFACTS"
    },
    "environment": {
        "type": "LINUX_CONTAINER",
        "image": "aws/codebuild/standard:5.0",
        "computeType": "BUILD_GENERAL1_SMALL",
        "environmentVariables": [{
                "name": "AWS_DEFAULT_REGION",
                "value": "us-east-1",
                "type": "PLAINTEXT"
            }, {
                "name": "AWS_ACCOUNT_ID",
                "value": "xxxxxxxxxxxx",
                "type": "PLAINTEXT"
            }, {
                "name": "IMAGE_TAG",
                "value": "latest",
                "type": "PLAINTEXT"
            }, {
                "name": "IMAGE_REPO_NAME",
                "value": "github-actions-self-hosted-runner-debian",
                "type": "PLAINTEXT"
            }
        ],
        "imagePullCredentialsType": "CODEBUILD",
        "privilegedMode": true
    },
    "serviceRole": "arn:aws:iam::xxxxxxxxxxxx:role/service-role/github-actions-self-hosted-runner-debian-build-role",
    "timeoutInMinutes": 60,
    "queuedTimeoutInMinutes": 60,
    "tags": [{
            "key": "codebuild-project",
            "value": "github-actions-self-hosted-runner-debian-build"
        }
    ],
    "badgeEnabled": false,
    "logsConfig": {
        "cloudWatchLogs": {
            "status": "ENABLED",
            "groupName": "github-actions-self-hosted-runner-debian-codebuild",
            "streamName": "github-actions-self-hosted-runner-debian"
        }
    },
    "buildBatchConfig": {
        "serviceRole": "arn:aws:iam::xxxxxxxxxxxx:role/service-role/github-actions-self-hosted-runner-debian-build-role",
        "combineArtifacts": true,
        "restrictions": {
            "maximumBuildsAllowed": 100,
            "computeTypesAllowed": [
                "BUILD_GENERAL1_SMALL",
                "BUILD_GENERAL1_MEDIUM"
            ]
        },
        "timeoutInMins": 60
    }
}

Run the following to create the project:

aws codebuild create-project --cli-input-json file://$HOME/github-actions-self-hosted-runner-debian_codebuild_project.json

Build Trigger Prerequisites

First, we start by creating IAM roles and policies required for the build trigger.

EventBridge Role and Trust Policy

File $HOME/github-actions-role-events-trust-policy.json below contains the trust policy definition required for the role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "events.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Create the role with a name of github-actions-codebuild-trigger-role :

aws iam create-role \
    --path "/service-role/" \
    --role-name github-actions-codebuild-trigger-role \
    --assume-role-policy-document file://$HOME/github-actions-role-events-trust-policy.json

IAM Policy for Trigger

Create the policy json $HOME/github-actions-role-events-iam-policy.json as shown below:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "codebuild:StartBuild"
            ],
            "Resource": [
                "arn:aws:codebuild:us-east-1:xxxxxxxxxxxx:project/github-actions-self-hosted-runner-debian-build"
            ]
        }
    ]
}

Attach the policy to the EventBridge role created in previous section:

aws iam put-role-policy \
    --role-name github-actions-codebuild-trigger-role \
    --policy-name github_actions_role_events_iam_policy \
    --policy-document file://$HOME/github-actions-role-events-iam-policy.json

Create Build Trigger

Create a cron-based expression rule run daily at 6:00AM UTC time:

aws events put-rule \
    --name run-github_actions-self-hosted-runner-debian-codebuild \
    --schedule-expression 'cron(0 6 * * ? *)' \
    --role-arn arn:aws:iam::xxxxxxxxxxxx:role/service-role/github-actions-codebuild-trigger-role

Set the target for the rule to be our CodeBuild project:

aws events put-targets \
    --rule run-github_actions-self-hosted-runner-debian-codebuild \
    --targets '
[
    {
        "Id": "trigger_codebuild",
        "Arn": "arn:aws:codebuild:us-east-1:xxxxxxxxxxxx:project/github-actions-self-hosted-runner-debian-build",
        "RoleArn": "arn:aws:iam::xxxxxxxxxxxx:role/service-role/github-actions-codebuild-trigger-role"
    }
]
'

Testing the Build

There are two scenarios to cover for testing:

  • ECR image has an outdated runner agent installed
  • ECR image has the latest runner agent installed

ECR Image has an Outdated Runner Agent Installed

For this test case, the image is expected to be rebuilt using the latest agent version before being pushed to ECR.

  • Choose “Start build with overrides”
Image 3
  • Choose the following options to configure and start the build
Image 1
  • The following shows Phase details for a successful build
Image 4
  • Check the Build logs
Image 5
  • From output above:
    • the Github runner agent version installed within the ECR image is: 2.309.0
    • a check of the latest runner agent version that’s currently available for download from the official repository is: 2.310.2
Image 6
  • Since a newer release of the agent has been released , the image is rebuilt with the newer version, and pushed to ECR

ECR Image has the Latest Runner Agent Installed

When the image already contains the latest runner agent version, build and push to ECR should be skipped.

Rerunning the build will allow us to verify that the build/push steps to ECR are skipped.

The following is the output log produced after a rerun:

Image 7

As expected, since the image was already packaged with the latest agent, the build/push to ECR steps were skipped.

Series Navigation<< Hosting the Runner Docker Artifacts on CodeCommitScalable ECS Cluster >>