EventBus and Schema Discover for Webhook Events

This entry is part 7 of 12 in the series Scalable Self-Hosted GitHub Runners on AWS Cloud

Scalable Self-Hosted GitHub Runners on AWS Cloud

Architecture Overview and Infrastructure Components

ECR Runner Image Repository

Self-hosted GitHub Runner(s) Registration Token

Hosting the Runner Docker Artifacts on CodeCommit

Build/Push Runner Image using CodeBuild

Scalable ECS Cluster

EventBus and Schema Discover for Webhook Events

ECS Runner Task Definition

Lambda Function URL

GitHub Webhook

EventBridge Rule

Testing the Final Infrastructure

Each event type received via the Github webhook will be forwarded to an EventBridge event bus where its schema will be inferred by a discoverer and stored in a schema registry.

EventBus

The following command will create an event bus named github-actions-event-bus:

aws --region us-east-1 events create-event-bus --name github-actions-event-bus

Schema Registry

EventBridge includes the default schema registry, “discovered-schemas“, which can be accessed from the AWS console:

Amazon EventBridge” > Schemas > “Discovered schema registry

Our discoverer will “scan” the event bus in order to infer the event schemas before storing them in the default registry.

Discoverer

Run the following to enable the discoverer for our event bus:

aws --region us-east-1 schemas create-discoverer \
    --source-arn arn:aws:events:us-east-1:xxxxxxxxxxxx:event-bus/git-actions-event-bus \
    --description "Schema discoverer for events on github-actions-event-bus event bus"

sample output:

{
    "Description": "Schema discoverer for events on github-actions-event-bus event bus",
    "DiscovererArn": "arn:aws:schemas:us-east-1:xxxxxxxxxxxx:discoverer/events-event-bus-git-actions-event-bus",
    "DiscovererId": "events-event-bus-git-actions-event-bus",
    "SourceArn": "arn:aws:events:us-east-1:xxxxxxxxxxxx:event-bus/git-actions-event-bus",
    "State": "STARTED",
     ...
     ...
}	

The output includes:

"State": "STARTED",

This confirms schema discovery has been enabled for the Event bus.

Scalable Self-Hosted GitHub Runners on AWS Cloud

Scalable ECS Cluster ECS Runner Task Definition