chore: bring images up to date #63
Conversation
richardcase
left a comment
richardcase
left a comment
There was a problem hiding this comment.
Thanks for this @casibbald . A few comments.
I have previously experimented with a different way to build the images using the kernel configs from FC and CH directly and then adding additional config options on top, i never quite got it working nicely: https://github.com/liquidmetal-dev/mikrolite-images/tree/main/kernel-k8s-fc
But i do think its a better
| K8S_FULL_VERSION?=$(K8S_MAJOR_MINOR).14 | ||
| UBUNTU_VERSION?=22.04 | ||
| IMAGE_NAME?=$(REGISTRY)/capmvm-k8s-ubuntu-$(UBUNTU_VERSION) | ||
| IMAGE_NAME?=$(REGISTRY)/capmvm-k8s-ubuntu-$(UBUNTU_VERSION)-amd64 |
There was a problem hiding this comment.
Do we need the ability to overwrite the amd64? Perhaps via passed in ARCH
There was a problem hiding this comment.
Actually i see there is separate Makefile later for arm :)
There was a problem hiding this comment.
@richardcase Should we merge the Makefiles?
It was a bit confusing for me with ARM initially. Maybe a single file that handles all architectures?
There was a problem hiding this comment.
So arm was experimental; I did not want to merge them initially as I was concerned it could break others' workflows; however, I think a single Makefile might be nicer. Let me know your thoughts.
@richardcase Maybe as a separate PR where we merge them?
There was a problem hiding this comment.
Yeah i think a single makefile would be easier ultimately.
kernel/Makefile
Outdated
|
|
||
| FC_KERNEL_VERSIONS ?= 4.19.215 5.10.77 # If you update this list, please remember to update the matrix in the kernel-images.yml github action! | ||
| CH_KERNEL_VERSIONS ?= 5.12 # If you update this list, please remember to update the matrix in the kernel-images.yml github action! | ||
| FC_KERNEL_VERSIONS ?= 4.19.215 5.2.21 5.10.77 # If you update this list, please remember to update the matrix in the kernel-images.yml github action! |
There was a problem hiding this comment.
We need to completely rethink these versions now. For firecracker its 5.10 and 6.1 kernels supported now.
There was a problem hiding this comment.
I will give 6.1 another stab, I could not get it to crpss build on my M1, but I have an Intel MiniPC to test with.
There was a problem hiding this comment.
updated 6.1 to use the following file:
https://github.com/firecracker-microvm/firecracker/blob/main/resources/guest_configs/microvm-kernel-ci-aarch64-6.1.config
and removed 5.2.21 as I now have an intel machine
5.2.21 - was the only version building on my M1
| FC_KERNEL_VERSIONS ?= 4.19.215 5.10.77 # If you update this list, please remember to update the matrix in the kernel-images.yml github action! | ||
| CH_KERNEL_VERSIONS ?= 5.12 # If you update this list, please remember to update the matrix in the kernel-images.yml github action! | ||
| FC_KERNEL_VERSIONS ?= 4.19.215 5.2.21 5.10.77 # If you update this list, please remember to update the matrix in the kernel-images.yml github action! | ||
| CH_KERNEL_VERSIONS ?= 5.12 5.15.12 # If you update this list, please remember to update the matrix in the kernel-images.yml github action! |
There was a problem hiding this comment.
We should also take a view of the CH kernel versions.
There was a problem hiding this comment.
Any known preference for CH versions ?
There was a problem hiding this comment.
I don't know off the top of my head. I would need to have a look around. But lets do that as a follow-up.
README.md
Outdated
| Workflows push images to the **repository’s GitHub Container Registry** by default: | ||
|
|
||
| - **Upstream** (`liquidmetal-dev/image-builder`): images go to `ghcr.io/liquidmetal-dev/...` | ||
| - **Forks** (e.g. `microscaler/liquidmetal-image-builder`): images go to `ghcr.io/microscaler/...` |
There was a problem hiding this comment.
| - **Forks** (e.g. `microscaler/liquidmetal-image-builder`): images go to `ghcr.io/microscaler/...` | |
| - **Forks** (e.g. `my-fork/liquidmetal-image-builder`): images go to `ghcr.io/my-fork/...` |
casibbald
force-pushed
the
chore/bring-images-up-to-date
branch
4 times, most recently
from
cc8c877 to
e025159
Compare
…rnel/CH/CAPMVM naming, liquidmetal-dev)
casibbald
force-pushed
the
chore/bring-images-up-to-date
branch
2 times, most recently
from
001064a to
8e592d1
Compare
casibbald
force-pushed
the
chore/bring-images-up-to-date
branch
from
8e592d1 to
4fa0c91
Compare
|
@copilot review |
There was a problem hiding this comment.
Pull request overview
This PR updates image naming, build workflows, and documentation to align with the current Liquid Metal registry/org setup and to introduce architecture-suffixed image publishing with experimental arm64 support.
Changes:
- Update kernel build system to be
ARCH-aware (amd64/arm64), including arch-suffixed image names and arch-specific kernel config paths. - Refresh GitHub Actions workflows to be fork-friendly via dynamic GHCR owner detection and add arm64 (QEMU/Buildx) build/release paths.
- Update CAPMVM image naming (amd64 suffix; experimental arm64) and refresh docs/references from
weaveworks-liquidmetaltoliquidmetal-dev.
Reviewed changes
Copilot reviewed 21 out of 29 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
kernel/firecracker/configs/arm64/linux-arm64-5.10.77.config |
Adds arm64 Firecracker kernel config for 5.10.77. |
kernel/firecracker/configs/amd64/linux-x86_64-6.1.102.config |
Adds amd64 Firecracker kernel config for 6.1.102. |
kernel/firecracker/base/Dockerfile |
Parameterizes Firecracker base build by arch/config file and switches kernel clone URL to HTTPS. |
kernel/cloud-hypervisor/base/Dockerfile |
Parameterizes CH base build by arch/config file and sets builder default image. |
kernel/README.md |
Updates kernel docs for arch-aware images, naming, and registry/org references. |
kernel/Makefile |
Introduces ARCH/platform handling, arch-suffixed image names, and buildx flow for arm64. |
experimental/arm/kernel/modules/Dockerfile |
Removes legacy experimental ARM kernel modules Dockerfile (migrated to main kernel pipeline). |
experimental/arm/kernel/firecracker/base/Dockerfile |
Removes legacy experimental ARM Firecracker base Dockerfile (migrated to main kernel pipeline). |
experimental/arm/kernel/builder/Dockerfile |
Removes legacy experimental ARM kernel builder Dockerfile (migrated to main kernel pipeline). |
experimental/arm/kernel/bin/Dockerfile |
Removes legacy experimental ARM kernel bin Dockerfile (migrated to main kernel pipeline). |
experimental/arm/kernel/README.md |
Removes legacy experimental ARM kernel README (content moved/updated under kernel/). |
experimental/arm/kernel/Makefile |
Removes legacy experimental ARM kernel Makefile (superseded by kernel/Makefile with ARCH=arm64). |
experimental/arm/github-action-runner/README.md |
Updates links from weaveworks-liquidmetal to liquidmetal-dev. |
experimental/arm/github-action-runner/Makefile |
Updates default registry/org to liquidmetal-dev. |
experimental/arm/capmvm/kubernetes/README.md |
Updates example image naming and org references for experimental ARM CAPMVM images. |
experimental/arm/capmvm/kubernetes/Makefile |
Aligns ARM CAPMVM build args/versioning with x86 flow and updates image naming. |
experimental/arm/capmvm/kubernetes/Dockerfile |
Updates Kubernetes repo/install logic to pkgs.k8s.io and new version args. |
capmvm/kubernetes/README.md |
Updates amd64 image naming example to include -amd64 suffix. |
capmvm/kubernetes/Makefile |
Updates amd64 CAPMVM image name to include -amd64 suffix. |
README.md |
Documents fork-friendly publishing via dynamic GHCR owner selection. |
.github/workflows/kernel-images.yml |
Adds PR builds, arch matrix including arm64, and fork-friendly GHCR owner handling for release. |
.github/workflows/kernel-images-manual.yml |
Adds manual arch selection and arm64 path (QEMU/Buildx) for kernel releases. |
.github/workflows/capmvm-kubernetes-manual.yml |
Makes CAPMVM publishing fork-friendly via dynamic GHCR owner env var. |
.github/workflows/capmvm-kubernetes-arm-manual.yml |
Adds new manual workflow for experimental ARM64 CAPMVM image build/release. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -33,20 +76,23 @@ jobs: | |||
| make build-fc | |||
| make push-fc | |||
There was a problem hiding this comment.
For the arm64 manual job, PUSH=--push means the build step uses docker buildx build --push and won’t leave local images behind. Running make push-fc afterwards (docker push) is likely to fail because the tags aren’t present locally. Either skip make push-fc for arm64, or change the build to --load and then push, or update the Makefile push logic for buildx.
| make push-fc |
| The builder provides firecracker-compatible images for multiple kernel versions | ||
| and **architectures** (amd64 and arm64), and cloudhypervisor-compatible images for | ||
| amd64 (`5.12`, `5.15.12`), using the [cloud-hypervisor/linux](https://github.com/cloud-hypervisor/linux) branches. | ||
|
|
||
| Builds are parameterized by `ARCH` (default `amd64`). Use `make build-fc ARCH=arm64` for Firecracker arm64 images. |
There was a problem hiding this comment.
This README now describes both amd64 and arm64 builds, but the later filename guidance still only mentions bin/vmlinux for Firecracker. For arm64, the kernel artifact is /boot/image (from make Image), so consumers typically need bin/image. Please update the documentation in this section to clearly spell out the correct filename for each arch (amd64 vs arm64) and remove/adjust any remaining statements that Firecracker only supports 5.10/4.19 if 6.1.102 is now built.
| --label "org.opencontainers.image.url"="https://github.com/weaveworks-liquidmetal/image-builder" \ | ||
| --label "org.opencontainers.image.source"="https://github.com/weaveworks-liquidmetal/image-builder/tree/main/capmvm/kubernetes" \ | ||
| --label "org.opencontainers.image.url"="https://github.com/liquidmetal-dev/image-builder" \ | ||
| --label "org.opencontainers.image.source"="https://github.com/liquidmetal-dev/image-builder/tree/main/capmvm/kubernetes" \ |
There was a problem hiding this comment.
org.opencontainers.image.source points at .../tree/main/capmvm/kubernetes, but this Makefile builds images from experimental/arm/capmvm/kubernetes. This label should reflect the actual source directory for the built image to avoid misleading provenance metadata.
| --label "org.opencontainers.image.source"="https://github.com/liquidmetal-dev/image-builder/tree/main/capmvm/kubernetes" \ | |
| --label "org.opencontainers.image.source"="https://github.com/liquidmetal-dev/image-builder/tree/main/experimental/arm/capmvm/kubernetes" \ |
| @@ -70,21 +109,23 @@ jobs: | |||
| make build-fc | |||
| make push-fc | |||
There was a problem hiding this comment.
In this job, PUSH=--push causes arm64 builds to use docker buildx build --push (see kernel/Makefile), which does not load images into the local Docker daemon. The subsequent make push-fc (docker push) will then fail for arm64. Consider either skipping make push-fc when matrix.arch == 'arm64', or switching to a buildx mode that produces local images (e.g. --load) before pushing, or updating the Makefile push targets to use buildx instead of docker push for arm64.
| make push-fc | |
| if [ "${{ matrix.arch }}" != "arm64" ]; then | |
| make push-fc | |
| fi |
| push-fc: $(addprefix push-fc-,$(FC_KERNEL_VERSIONS)) | ||
| push-fc-%: ## Push all Firecracker images | ||
| push-fc-%: ## Push Firecracker images for current ARCH | ||
| $(DOCKER) push $(FC_BASE_IMAGE_NAME):$* | ||
| $(DOCKER) push $(FC_BIN_IMAGE_NAME):$* | ||
| $(DOCKER) push $(FC_MOD_IMAGE_NAME):$* |
There was a problem hiding this comment.
push-fc-% / push-ch-% always run docker push, but for ARCH=arm64 the build is done via docker buildx build and CI sets PUSH=--push, meaning images are pushed during the build and not loaded locally. This makes the subsequent docker push step brittle for arm64. Consider making push-* conditional on $(PUSH)/$(ARCH), or standardizing on either (a) buildx --push with no separate push target, or (b) buildx --load + docker push.
| ##@ Build | ||
|
|
||
| all: build-fc build-ch ## Build all Firecracker and Cloud Hypervisor images | ||
| all: build-fc build-ch ## Build all Firecracker and Cloud Hypervisor images for current ARCH |
There was a problem hiding this comment.
all always runs build-ch, but the Makefile also states Cloud Hypervisor is amd64-only and the CH configs live under cloud-hypervisor/configs/amd64/. As written, ARCH=arm64 make all will attempt to build CH images for arm64 and likely fail. Consider making all conditional (e.g. only include build-ch when ARCH=amd64) or erroring early when ARCH!=amd64 and build-ch is invoked.
| all: build-fc build-ch ## Build all Firecracker and Cloud Hypervisor images for current ARCH | |
| ifeq ($(ARCH),amd64) | |
| all: build-fc build-ch ## Build all Firecracker and Cloud Hypervisor images for amd64 | |
| else | |
| all: build-fc ## Build all Firecracker images for current ARCH (Cloud Hypervisor is amd64-only) | |
| endif |
|
|
||
| ### Kubernetes compatibility (e.g. 1.34.x) | ||
|
|
||
| For running Kubernetes 1.34.x (and similar) inside microVMs, use **5.10.77** or **5.2.21** (Firecracker) or **5.12** or **5.15.12** (Cloud Hypervisor). These meet Kubernetes kernel requirements (PSI 4.20+, cgroup v2 5.8+, runc freezer 5.2+). Kernel **4.19.215** is below the recommended minimums and is not recommended for current Kubernetes versions. |
There was a problem hiding this comment.
This section recommends Firecracker kernel 5.2.21, but this PR’s build lists/matrices appear to publish Firecracker kernels 4.19.215, 5.10.77, and 6.1.102 (no 5.2.21 config/version in kernel/Makefile or the kernel workflows). Either add 5.2.21 to the build/publish lists, or update this paragraph to only reference versions that are actually produced.
| RUN curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg | ||
| RUN echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list | ||
| ### Add apt repos (pkgs.k8s.io, same as x86 capmvm) | ||
| RUN curl -fsSL https://pkgs.k8s.io/core:/stable:/v${KUBERNETES_MAJOR_MINOR}/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg |
There was a problem hiding this comment.
This writes the keyring to /etc/apt/keyrings/... but the Dockerfile never creates /etc/apt/keyrings. On Ubuntu 20.04 (the default OS_VERSION here), that directory is typically absent, so this step can fail with a “No such file or directory”. Add a mkdir -p /etc/apt/keyrings before the gpg --dearmor -o ... step.
| RUN curl -fsSL https://pkgs.k8s.io/core:/stable:/v${KUBERNETES_MAJOR_MINOR}/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg | |
| RUN mkdir -p /etc/apt/keyrings && curl -fsSL https://pkgs.k8s.io/core:/stable:/v${KUBERNETES_MAJOR_MINOR}/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg |
| ``` | ||
| ghcr.io/weaveworks-liquidmetal/capmvm-k8s-os-arm:1.21.8 | ||
| ghcr.io/weaveworks-liquidmetal/capmvm-k8s-os-arm:1.22.3 | ||
| ghcr.io/weaveworks-liquidmetal/capmvm-k8s-os-arm:1.22.8 | ||
| ghcr.io/weaveworks-liquidmetal/capmvm-k8s-os-arm:1.23.5 | ||
| ghcr.io/liquidmetal-dev/capmvm-k8s-ubuntu-22.04-arm64:1.30.14 | ||
| ``` | ||
|
|
||
| For platforms using versions older than [Flintlock `0.5.0`][fl5] and [CAPMVM `0.8.0`][cap8] | ||
| us the following legacy images: | ||
|
|
||
| ``` | ||
| ghcr.io/weaveworks-liquidmetal/capmvm-kubernetes-arm:X.X.X | ||
| ``` | ||
| (Use the same pattern for other Ubuntu versions, e.g. `capmvm-k8s-ubuntu-20.04-arm64` or `capmvm-k8s-ubuntu-24.04-arm64`; the tag is the full Kubernetes version.) |
There was a problem hiding this comment.
The example here uses Ubuntu 22.04 and Kubernetes 1.30.14, but the intro still says these ARM OS images are for Kubernetes 1.21.x–1.23.x on Ubuntu 20.04. Please update the introductory version/OS statements (or the example) so the README reflects what the Dockerfile/Makefile actually build now (Ubuntu selector + full K8s version tags).
Summary
This PR brings image naming, workflows, and references in line with the current Liquid Metal setup: registry detection for forks, ARM (experimental) workflows, explicit architecture suffixes on image names, and the move from
weaveworks-liquidmetaltoliquidmetal-dev.Successful builds can be found here:
1. Registry detection (fork-friendly)
REGISTRY=ghcr.io/${{ vars.REGISTRY_OWNER || github.repository_owner }}microscaler/image-builder) publish toghcr.io/microscaler/..., and upstream toghcr.io/liquidmetal-dev/....REGISTRY_OWNERoverrides the owner (e.g. for a different org).kernel-images.yml,kernel-images-manual.yml,capmvm-kubernetes-manual.yml.2. Kernel workflows and images
workflow_dispatchadded to Build and release kernel images for manual runs.firecracker-kernel-amd64,firecracker-kernel-bin-amd64,firecracker-kernel-modules-amd64;cloudhypervisor-kernel-amd64,cloudhypervisor-kernel-bin-amd64,cloudhypervisor-kernel-modules-amd64.firecracker-kernel-arm64,firecracker-kernel-bin-arm64,firecracker-kernel-modules-arm64.git://tohttps://for CI..github/workflows/kernel-images-arm.yml(push onexperimental/arm/kernel/**, manual dispatch, QEMU + Buildx forlinux/arm64).3. CAPMVM workflows and images
capmvm-k8s-ubuntu-{20.04|22.04|24.04}-amd64capmvm-k8s-ubuntu-{20.04|22.04|24.04}-arm64.github/workflows/capmvm-kubernetes-arm-manual.yml(manual dispatch, same inputs as x86: k8s version, containerd, Ubuntu version).-1.1, and OS_VERSION for Ubuntu choice.4. References: weaveworks-liquidmetal → liquidmetal-dev
weaveworks-liquidmetaltoliquidmetal-dev(default REGISTRY, README examples, and links to flintlock, cluster-api-provider-microvm, site, microvm-action-runner).REGISTRYis nowghcr.io/liquidmetal-devwhere applicable.Files changed (22)
kernel-images.yml,kernel-images-manual.yml, newkernel-images-arm.yml,capmvm-kubernetes-manual.yml, newcapmvm-kubernetes-arm-manual.yml.