Build ARM Images on x86_64/AMD64 Hosts using Packer

Host OS Details

Architecture (AMD64)
Operating system Ubuntu 22.04 with Docker engine installed

Docker Status

The following command is meant more for the purposes of observation:

$ docker buildx ls

NAME/NODE DRIVER/ENDPOINT STATUS  BUILDKIT             PLATFORMS
default * docker
  default default         running v0.11.7+d3e6c1360f6e linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386

A key point to note from the output is the absence of support for arm* platforms.

Install Packer

Install Packer by following the instructions at installing Packer > Linux > Ubuntu/Debian.

A slightly revised version of the these are included below. The revisions account for change in GPG key download locations.

Install GPG

$ sudo apt update && sudo apt install gpg

Add the HashiCorp GPG key

$ wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg

Add HashiCorp repository

$ echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list

Update & Install packer

$ sudo apt update && sudo apt install packer

Docker Plugin for Packer

Install the Packer Docker plugin:

$ packer plugins install github.com/hashicorp/docker

Multi-Architecture Builds

One way of enabling multi-architecture builds is to leverage the clever approach, multiarch/qemu-user-static.

Quoted directly from the project’s README:

multiarch/qemu-user-static is to enable an execution of different multi-architecture containers by QEMU ¹ and binfmt_misc ². Here are examples with Docker ³.

To learn more about how multiarch/qemu-user-static enables support for additional platform variants, refer to the usage guidelines.

Enabling Multi-Architecture Builds

Run the following to enable builds for various architectures (including arm*):

$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

output:

Unable to find image 'multiarch/qemu-user-static:latest' locally
latest: Pulling from multiarch/qemu-user-static
205dae5015e7: Pull complete
816739e52091: Pull complete
30abb83a18eb: Pull complete
0657daef200b: Pull complete
30c9c93f40b9: Pull complete
Digest: sha256:fe60359c92e86a43cc87b3d906006245f77bfc0565676b80004cc666e4feb9f0
Status: Downloaded newer image for multiarch/qemu-user-static:latest
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
Setting /usr/bin/qemu-hexagon-static as binfmt interpreter for hexagon

This pulls the following image locally:

$ docker images
REPOSITORY                   TAG       IMAGE ID       CREATED         SIZE
multiarch/qemu-user-static   latest    3539aaa87393   11 months ago   305MB

How did the previous command impact local docker/buildx configuration?

Check our docker buildx configuration once again:

$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS  BUILDKIT             PLATFORMS
default * docker
  default default         running v0.11.7+d3e6c1360f6e linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386, 
                          linux/arm64, linux/riscv64, linux/ppc64, linux/ppc64le, linux/s390x, 
                          linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

Wow, the default docker builder/driver now provides support for several other platforms, including arm*
This reference to default docker builder/driver is synonymous to saying that by default, docker commands such as:

$ docker build ...
$ docker run ...

will automatically use the default builder:
NAME/NODE: default
DRIVER/ENDPOINT: docker

Testing: Use Packer to Build ARM64-based Image

The test build consists of:

Packer template, ubuntu-arm64.pkr.hcl and entrypoint.sh script, which includes the following commands:

echo "$(uname -m)"
echo "$(lsb_release -a)"

Base image used in the build is an Ubuntu arm64, sourced from Docker Hub

Final image is saved to localhost docker repository with name:tag of ubuntu-arm64-target:latest

Create Packer Build Project Files

Create project directory

$ mkdir -p $HOME/packer-build-arm64-on-amd64

Create the project files in the same directory

$HOME/packer-build-arm64-on-amd64/ubuntu-arm64.pkr.hcl

locals {
  image_name = "ubuntu-arm64-target"
  image_tag  = "latest"

}

variable "image_dir" {
  type    = string
  default = "/container_dir"
}

source "docker" "arm64" {
  image  = "docker.io/arm64v8/ubuntu:latest"
  commit = true
  pull   = true
  changes = [
    "ENTRYPOINT [\"/bin/bash\", \"-c\"]",
    "CMD [\"entrypoint.sh\"]"
  ]
}

build {
  sources = [
    "source.docker.arm64"
  ]

  provisioner "shell" {
    inline = [
      "echo set debconf to Noninteractive",
    "echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections"]
  }

  provisioner "shell" {
    execute_command = "sh -c '{{ .Vars }} {{ .Path }}'"
    inline = [
      <<EOT
      apt-get update -y \
      && apt-get upgrade -y \
      && apt-get install -y --no-install-recommends \
      lsb-release \
      && apt-get autoremove -y \
      && apt-get autoclean -y \
      && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
      EOT
    ]
  }

  provisioner "shell" {
    inline = ["mkdir ${var.image_dir}", "chmod 777 ${var.image_dir}"]
  }

  provisioner "file" {
    destination = "${var.image_dir}/"
    sources = [
      "${path.root}/entrypoint.sh"
    ]
  }

  provisioner "shell" {
    execute_command = "sh -c '{{ .Vars }} {{ .Path }}'"
    inline = [
      "cp ${var.image_dir}/entrypoint.sh /usr/bin/",
      "chmod +x /usr/bin/entrypoint.sh"
    ]
  }

  post-processors {
    post-processor "docker-tag" {
      repository = "${local.image_name}"
      tags       = ["${local.image_tag}"]
    }
  }

}

$HOME/packer-build-arm64-on-amd64/entrypoint.sh

#!/bin/bash

echo "$(uname -m)"
echo "$(lsb_release -a)"

sleep 10

exit 0

Packer Build

Run the build:

$ cd $HOME/packer-build-arm64-on-amd64
$ packer build .

Sample portions from build output:

==> docker.arm64: Creating a temporary directory for sharing data...
==> docker.arm64: Pulling Docker image: docker.io/arm64v8/ubuntu:latest
    docker.arm64: latest: Pulling from arm64v8/ubuntu
...
...
    docker.arm64: Get:1 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 libpython3.10-minimal arm64 3.10.12-1~22.04.3 [809 kB]
    docker.arm64: Get:2 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 libexpat1 arm64 2.4.7-1ubuntu0.2 [76.8 kB]
    docker.arm64: Get:3 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 python3.10-minimal arm64 3.10.12-1~22.04.3 [2242 kB]
    docker.arm64: Get:4 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main arm64 python3-minimal arm64 3.10.6-1~22.04 [24.3 kB]
    docker.arm64: Get:5 http://ports.ubuntu.com/ubuntu-ports jammy/main arm64 media-types all 7.0.0 [25.5 kB]
    docker.arm64: Get:6 http://ports.ubuntu.com/ubuntu-ports jammy/main arm64 libmpdec3 arm64 2.5.1-2build2 [89.0 kB]
    docker.arm64: Get:7 http://ports.ubuntu.com/ubuntu-ports jammy/main arm64 readline-common all 8.1.2-1 [53.5 kB]
    docker.arm64: Get:8 http://ports.ubuntu.com/ubuntu-ports jammy/main arm64 libreadline8 arm64 8.1.2-1 [153 kB]
...
...
==> docker.arm64: Provisioning with shell script: /tmp/packer-shell*********
==> docker.arm64: Committing the container
    docker.arm64: Image ID: sha256:***************************************************************
==> docker.arm64: Killing the container: ***************************************************************
==> docker.arm64: Running post-processor:  (type docker-tag)
    docker.arm64 (docker-tag): Tagging image: sha256:***************************************************************
    docker.arm64 (docker-tag): Repository: ubuntu-arm64-target:latest
Build 'docker.arm64' finished after 51 seconds 491 milliseconds.

==> Wait completed after 51 seconds 491 milliseconds

==> Builds finished. The artifacts of successful builds are:
--> docker.arm64: Imported Docker image: sha256:***************************************************************
--> docker.arm64: Imported Docker image: ubuntu-arm64-target:latest with tags ubuntu-arm64-target:latest

Check Local Docker Repository

Check the image has been saved to local repository:

$ docker images

REPOSITORY               TAG         ...                 
ubuntu-arm64-target      latest      ...

Running the Image Locally

We saw from output of the previous docker buildx command that our default docker builder now includes support for linux/arm64.

To test the image, we can pass the --platform linux/arm64/v8 argument to docker run:

$ docker run --rm -ti --platform linux/arm64/v8 ubuntu-arm64-target

which produces the following output:

aarch64

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:        22.04
Codename:       jammy