Implementing AWS PrivateLink for ECS Fargate Tasks

AWS PrivateLink enables VPCs and other supported services to connect using private IP addresses. Network traffic in a PrivateLink configuration remains confined to the Amazon Cloud network, without ever traversing the public internet.

In this blog post, we examine the execution of a sample ECS Fargate task within two separate VPC configurations, i.e.

VPC without AWS Private Link
VPC with AWS Private Link

The aim is to gain an appreciation of the benefits offered by implementing AWS PrivateLink.

Sample Fargate Task

Key details for our sample Fargate task:

single container:
- Name: sample-fargate-logger
- ECR Registry: 123456789012.dkr.ecr.us-east-1.amazonaws.com
- ECR Repository: my-ecr-repo/app-image
- Image tag: latest
- environment variable
  - name: LOG_MESSAGE
    value: "sample-fargate-logger says hello"

writes logs to CloudWatch log group
- /ecs/sample-fargate-task

the full task definition is shown below:

{
  "containerDefinitions": [
      {
          "name": "sample-fargate-logger",
          "essential": true,
          "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/my-ecr-repo/app-image:latest",
          "environment": [
              {
                  "name": "LOG_MESSAGE",
                  "value": "sample-fargate-logger says hello"
              }
          ],
          "healthCheck": {
              "command": [
                  "CMD-SHELL",
                  "echo alive || exit 1"
              ],
              "interval": 30,
              "timeout": 5,
              "retries": 3
          },
          "logConfiguration": {
              "logDriver": "awslogs",
              "options": {
                  "awslogs-create-group": "true",
                  "awslogs-group": "/ecs/sample-fargate-task",
                  "awslogs-region": "us-east-1",
                  "awslogs-stream-prefix": "ecs"
              }
          }
      }
  ],
  "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
  "family": "sample-fargate-task",
  "networkMode": "awsvpc",
  "requiresCompatibilities": [
      "FARGATE"
  ],
  "runtimePlatform": {
      "operatingSystemFamily": "LINUX"
  },
  "cpu": "256",
  "memory": "512"
}

Task Execution without AWS Private Link

The following diagram shows the task running in a non-PrivateLink scenario.

Assuming a Fargate platform version >= 1.4.0, the task would receive a single ENI:

the ENI is “attached” to the VPC private subnet
all network traffic flows through this ENI
the task would access the following services over a public internet connection (i.e. through the NAT and IGW):
- ECR -> to pull the ECR image my-ecr-repo/app-image
- S3 -> to access ECR image layers
- Cloudwatch logs -> to write logs to log group /ecs/sample-fargate-task

For the sake of simplicity, we’ll assume the task does not depend on any other resources sourced over an internet connection.

Task Execution with AWS Private link

Going on the basis that our task does not depend on any other services apart from ECR, S3 and CloudWatch logs, the public subnet will be omitted for the sake of convenience.

The following diagram represents the task’s execution within a PrivateLink-enabled architecture.

The task no longer requires an internet connection to access ECR, S3 and CloudWatch Logs. These services would be accessed through the VPC Endpoints located within the AWS cloud.

To enable PrivateLink, VPC Endpoints described in the next section would be required.

VPC Endpoints

Target	Service Name	Reason	Type
S3	com.amazonaws.`region`.s3	This endpoint will be required to retrieve ECR image layers from S3	Gateway
ECR DKR	com.amazonaws.`region`.ecr.dkr	Accessed by Docker Registry APIs through docker client commands, including `push` and `pull` use this endpoint. Note: “When you create this endpoint, you must enable a private DNS hostname. To do this, ensure that the Enable Private DNS Name option is selected in the Amazon VPC console when you create the VPC endpoint.“	Interface
ECR API	com.amazonaws.`region`.ecr.api	Accessed by ECR API to execute API calls, including `DescribeImages` and `CreateRepository`. Note: “When this endpoint is created, you have the option to enable a private DNS hostname. Enable this setting by selecting Enable Private DNS Name* in the VPC console when you create the VPC endpoint*.“	Interface
CloudWatch Logs	com.amazonaws.`region`.logs	Required for the task to write to CloudWatch log group. Note: Private DNS Names would also be enabled for this endpoint.	Interface

private DNS Names for the interface endpoints must be enabled
before doing so, private hosted zone support must be enabled for the target VPC by modifying the VPC attributes mentioned below

aws ec2 create-vpc-endpoint .... --private-dns-enabled | --no-private-dns-enabled (boolean)

(Interface endpoint) Indicates whether to associate a private hosted zone with the specified VPC…

To use a private hosted zone, you must set the following VPC attributes to true :
enableDnsHostnames and enableDnsSupport .

Use ModifyVpcAttribute to set the VPC attributes.

creation of security groups and ingress rules for interface endpoints to allow incoming connections on port 443 from the private subnet of the VPC

Physical Implementation of PrivateLink-Enabled Solution

To clarify the theory covered so far, it’s time to run through the activities required to implement the PrivateLink-enabled architecture, as presented in the diagram from the previous section.

Once complete, we can run our sample task to ensure everything works as expected.

Note:

AWS CLI will be used to provision the infrastructure
AWS Profile name = aws_acct_admin
AWS ACCT ID = 123456789012
AWS Region = us-east-1
ECS Cluster name = fargate-cluster

Create VPC

VPC CIDR: 10.0.0.0/22
Region: us-east-1

Create the VPC and store the its corresponding ID in environment variable VPCID:

 $ VPCID="$(aws ec2 create-vpc \
    --profile aws_acct_admin \
    --region us-east-1 \
    --cidr-block "10.0.0.0/22" \
    --query Vpc.VpcId \
    --output text)"

Enable DNS support:

$ aws ec2 modify-vpc-attribute \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --enable-dns-support "{\"Value\":true}"

Enable DNS hostnames:

$ aws ec2 modify-vpc-attribute \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --enable-dns-hostnames "{\"Value\":true}"

VPC Default Security Group

Retrieve the default security group ID for the VPC and store it in variable VPC_DEFAULT_SEC_GROUP:

 VPC_DEFAULT_SEC_GROUP="$(aws ec2 describe-security-groups \
    --profile aws_acct_admin \
    --region us-east-1 \
    --filters Name=vpc-id,Values=${VPCID} \
    --query SecurityGroups[].GroupId \
    --output text)"

Create Private Subnet

Create subnet using:

Private Subnet CIDR: 10.0.0.0/23
Availability zone: us-east-1a

Store the subnet ID in variable PRIV_SUBNET:

$ PRIV_SUBNET="$(aws ec2 create-subnet \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --cidr-block "10.0.0.0/23" \
    --availability-zone us-east-1a \
    --query Subnet.SubnetId \
    --output text)"

Custom Route Table

Create route table and store its ID in variable CUST_RTB:

$ CUST_RTB="$(aws ec2 create-route-table \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --query RouteTable.RouteTableId \
    --output text)"

Create Route Table Association

Create a route table and associate it with the private subnet. Store the route table association ID in variable RTB_ASSC:

$ RTB_ASSC="$(aws ec2 associate-route-table \
    --profile aws_acct_admin \
    --region us-east-1 \
    --subnet-id ${PRIV_SUBNET} \
    --route-table-id ${CUST_RTB} \
    --query AssociationId \
    --output text)"

Security Group for Interface Endpoints

Create security group for ECR DKR endpoint using the following details,

Endpoint service name: com.amazonaws.us-east-1.ecr.dkr
Security group name: sec-ecr-dkr

Store the security group id in variable SEC_ECR_DKR_GRP:

$ SEC_ECR_DKR_GRP="$(aws ec2 create-security-group \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --group-name sec-ecr-dkr \
    --description "security group for com.amazonaws.us-east-1.ecr.dkr" \
    --query GroupId \
    --output text)"

Create security group for ECR API endpoint using the following details,

Endpoint service name: com.amazonaws.us-east-1.ecr.api
Security group name: sec-ecr-api

Store the security group id in variable SEC_ECR_API_GRP:

$ SEC_ECR_API_GRP="$(aws ec2 create-security-group \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --group-name sec-ecr-api \
    --description "security group for com.amazonaws.us-east-1.ecr.api" \
    --query GroupId \
    --output text)"

Create security group for LOGS endpoint using the following details,

Endpoint service name: com.amazonaws.us-east-1.logs
Security group name: sec-cw-logs

Store the security group ID in variable SEC_CW_LOGS_GRP:

$ SEC_CW_LOGS_GRP="$(aws ec2 create-security-group \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --group-name sec-cw-logs \
    --description "security group for com.amazonaws.us-east-1.logs" \
    --query GroupId \
    --output text)"

Create VPC Endpoints

S3 (Gateway)

com.amazonaws.us-east-1.s3

Store the VPC endpoint ID in variable VPCE_S3:

$ VPCE_S3="$(aws ec2 create-vpc-endpoint \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --service-name com.amazonaws.us-east-1.s3 \
    --route-table-ids ${CUST_RTB} \
    --query VpcEndpoint.VpcEndpointId \
    --output text)"

ECR DKR (Interface):

com.amazonaws.us-east-1.ecr.dkr

Store the VPC endpoint ID in variable VPCE_ECR_DKR:

$ VPCE_ECR_DKR="$(aws ec2 create-vpc-endpoint \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --vpc-endpoint-type Interface \
    --private-dns-enabled \
    --service-name com.amazonaws.us-east-1.ecr.dkr \
    --subnet-ids ${PRIV_SUBNET} \
    --security-group-id ${SEC_ECR_DKR_GRP} \
    --tag-specifications ResourceType=vpc-endpoint,Tags='[{Key=service,Value=ecrdkr}]' \
    --query VpcEndpoint.VpcEndpointId \
    --output text)"

ECR API (Interface)

com.amazonaws.us-east-1.ecr.api

Store the VPC endpoint ID in variable VPCE_ECR_API:

$ VPCE_ECR_API="$(aws ec2 create-vpc-endpoint \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --vpc-endpoint-type Interface \
    --private-dns-enabled \
    --service-name com.amazonaws.us-east-1.ecr.api \
    --subnet-ids ${PRIV_SUBNET} \
    --security-group-id ${SEC_ECR_API_GRP} \
    --tag-specifications ResourceType=vpc-endpoint,Tags='[{Key=service,Value=ecrapi}]' \
    --query VpcEndpoint.VpcEndpointId \
    --output text)"

CloudWatch Logs (Interface):

com.amazonaws.us-east-1.logs

Store the VPC endpoint ID in variable VPCE_LOGS:

$ VPCE_LOGS="$(aws ec2 create-vpc-endpoint \
    --profile aws_acct_admin \
    --region us-east-1 \
    --vpc-id ${VPCID} \
    --vpc-endpoint-type Interface \
    --private-dns-enabled \
    --service-name com.amazonaws.us-east-1.logs \
    --subnet-ids ${PRIV_SUBNET} \
    --security-group-id ${SEC_CW_LOGS_GRP} \
    --tag-specifications ResourceType=vpc-endpoint,Tags='[{Key=service,Value=cloudwatchlogs}]' \
    --query VpcEndpoint.VpcEndpointId \
    --output text)"

Security Group Ingress Rules

ECR DKR

Create the ingress rule for the ECR DKR security group ID, which we previously saved in variable ${SEC_ECR_DKR_GRP}.

Store the ingress rule ID in variable INGRESS_SEC_ECR_DKR:

$ INGRESS_SEC_ECR_DKR="$(aws ec2 authorize-security-group-ingress \
    --profile aws_acct_admin \
    --region us-east-1 \
    --group-id ${SEC_ECR_DKR_GRP} \
    --protocol tcp \
    --port 443 \
    --cidr ${PRIVATE_SUBNET_CIDR} \
    --query SecurityGroupRules[*].SecurityGroupRuleId \
    --output text)"

ECR API

Create the ingress rule for the ECR API security group ID, which we previously saved in variable ${SEC_ECR_API_GRP}.

Store the ingress rule ID in variable INGRESS_SEC_ECR_API:

$ INGRESS_SEC_ECR_API="$(aws ec2 authorize-security-group-ingress \
    --profile aws_acct_admin \
    --region us-east-1 \
    --group-id ${SEC_ECR_API_GRP} \
    --protocol tcp \
    --port 443 \
    --cidr ${PRIVATE_SUBNET_CIDR} \
    --query SecurityGroupRules[*].SecurityGroupRuleId \
    --output text)"

LOGS

Create the ingress rule for the LOGS security group ID, which we previously saved in variable ${SEC_CW_LOGS_GRP}.

Store the ingress rule ID in variable INGRESS_SEC_CW_LOGS:

$ INGRESS_SEC_CW_LOGS="$(aws ec2 authorize-security-group-ingress \
    --profile aws_acct_admin \
    --region us-east-1 \
    --group-id ${SEC_CW_LOGS_GRP} \
    --protocol tcp \
    --port 443 \
    --cidr ${PRIVATE_SUBNET_CIDR} \
    --query SecurityGroupRules[*].SecurityGroupRuleId \
    --output text)"

ECS Cluster

Create a cluster with name fargate-cluster:

aws ecs create-cluster \
    --profile aws_acct_admin \
    --region us-east-1 \
    --cluster-name fargate-cluster

ECR Repository

Create ECR repository with name my-ecr-repo/app-image where we will upload our task image to:

aws ecr create-repository \
    --profile aws_acct_admin \
    --region us-east-1 \
    --repository-name my-ecr-repo/app-image

Create Task Docker Image

Create directory to place files:

$ mkdir -p $HOME/fargate_ecr_image

Create the files for the docker image

$HOME/fargate_ecr_image/Dockerfile

FROM debian:latest

ENV LC_ALL C
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

COPY entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/entrypoint.sh

ENV PATH="/usr/bin:${PATH}"

ENTRYPOINT ["/bin/bash", "-c"]
CMD ["entrypoint.sh"]

$HOME/fargate_ecr_image/entrypoint.sh

#!/bin/bash

LOG_MESSAGE=${LOG_MESSAGE:-}

echo ${LOG_MESSAGE} > /dev/stdout

sleep 30

Build and tag Image:

$ cd $HOME/fargate_ecr_image
$ docker build -t 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-ecr-repo/app-image:latest .

Push Image to ECR

Authenticate to the target registry

$ aws ecr get-login-password \
    --profile aws_acct_admin \
    --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com

Push the docker image to ECR repository

$ docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-ecr-repo/app-image:latest

Create Fargate Task Definition

Task Execution Role

Create the ecsTaskExecutionRole execution role for the task by following these instructions.

Register Task Definition

$ aws ecs register-task-definition \
    --profile aws_acct_admin \
    --region us-east-1 \
    --cli-input-json "$(cat << EOF
{
  "containerDefinitions": [
      {
          "name": "sample-fargate-logger",
          "essential": true,
          "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/my-ecr-repo/app-image:latest",
          "environment": [
              {
                  "name": "LOG_MESSAGE",
                  "value": "sample-fargate-logger says hello"
              }
          ],
          "healthCheck": {
              "command": [
                  "CMD-SHELL",
                  "echo alive || exit 1"
              ],
              "interval": 30,
              "timeout": 5,
              "retries": 3
          },
          "logConfiguration": {
              "logDriver": "awslogs",
              "options": {
                  "awslogs-create-group": "true",
                  "awslogs-group": "/ecs/sample-fargate-task",
                  "awslogs-region": "us-east-1",
                  "awslogs-stream-prefix": "ecs"
              }
          }
      }
  ],
  "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
  "family": "sample-fargate-task",
  "networkMode": "awsvpc",
  "requiresCompatibilities": [
      "FARGATE"
  ],
  "runtimePlatform": {
      "operatingSystemFamily": "LINUX"
  },
  "cpu": "256",
  "memory": "512"
}
EOF
)"

Run Sample Fargate Task

$ aws ecs run-task \
    --profile aws_acct_admin \
    --region us-east-1 \
    --cluster fargate-cluster \
    --task-definition sample-fargate-task:1 \
    --launch-type="FARGATE" \
    --network-configuration "$(cat << EOF
{
    "awsvpcConfiguration": {
        "assignPublicIp": "DISABLED",
        "securityGroups": ["${VPC_DEFAULT_SEC_GROUP}"],
        "subnets": ["${PRIV_SUBNET}"]
    }
}
EOF
)"

The log group for the task should contain the following output.