Section 01
KEP-2008 as it actually stands
title: Forensic Container Checkpointing kep-number: 2008 authors: ["@adrianreber"] owning-sig: sig-node status: implementable creation-date: 2020-09-16 last-updated: 2024-02-08 stage: beta latest-milestone: "v1.30" milestone: alpha: "v1.25" beta: "v1.30" stable: "v1.33"
Set that against the gate registry, which is what actually ships:
ContainerCheckpoint: {
{Version: version.MustParse("1.25"), Default: false, PreRelease: featuregate.Alpha},
{Version: version.MustParse("1.30"), Default: true, PreRelease: featuregate.Beta},
}
// no GA entry
The KEP says stable at 1.33. Three releases later there is still no GA entry. A KEP’s milestone block records an intention at the time of writing; it is not updated when the intention slips. For a status question — is this GA? — the feature-gate registry is the authoritative file.
Practically: the checkpoint endpoint is on by default and usable, but it carries Beta guarantees. It can still change.
Section 02
The runtime support matrix
| Runtime | CheckpointContainer | Evidence |
|---|---|---|
| CRI-O | implemented | server/container_checkpoint.go, gated on CheckpointContainerEnabled(); exercised throughout chapters 01–05 |
| containerd | not implemented | internal/cri/server/container_checkpoint.go on main — a 34-line file returning codes.Unimplemented |
That is the whole matrix for the runtimes examined, and it is the single most consequential fact for anyone planning around this feature: the default runtime of most clusters cannot serve it.
containerd tracks this work in its own issues — including one referencing the pod-level KEP below. If the 34-line stub is ever replaced, the feature becomes available to the majority of clusters overnight. Until then, “Kubernetes supports checkpointing” means “Kubernetes-on-CRI-O supports checkpointing”.
Section 03
The working group
authors : Radostin Stoyanov, Viktória Spišaková, Adrian Reber, Peter Hunt
purpose : facilitate integration of checkpoint/restore capabilities into Kubernetes
by promoting discussion between the Kubernetes community and the CRIU ecosystem
use cases listed:
- optimising resource use for interactive workloads (e.g. Jupyter notebooks)
- accelerating startup for applications with long initialisation
- periodic checkpointing for fault tolerance in long-running distributed workloads
- interruption-aware scheduling (preempt while preserving runtime state)
- pod migration across nodes for load balancing and maintenance
- forensic checkpointing for security incident investigation
Only the last of those six is what the shipped feature does. The other five need a restore verb, pod-level consistency, or both — which is exactly what the next section is about.
Section 04
Pod-level checkpoint/restore
title: Pod-level Checkpoint/Restore kep-number: 5823 owning-sig: sig-node status: implementable creation-date: 2026-01-21 ← the same day the working group was announced stage: alpha latest-milestone: "v1.37" milestone: alpha: "v1.37"
The corresponding RPCs are already in cri-api on master, and their proto comments are unusually explicit about the contract:
// CheckpointPod creates a Pod-level checkpoint. ... The runtime must pause every
// selected container before capturing any of them, keep all selected containers
// paused until every selected container has been captured, and resume all of them
// before returning on success, error, or deadline expiry. This produces one
// consistent pod-wide cut while ensuring the runtime never returns a frozen pod.
rpc CheckpointPod(CheckpointPodRequest) returns (CheckpointPodResponse) {}
// RestorePod prepares a pod sandbox and containers from a checkpoint. ... On success,
// every returned container must be in the CREATED state and must not have executed
// the restored process; the caller invokes its pre-start hooks and then calls ...
rpc RestorePod(RestorePodRequest) returns (RestorePodResponse) {}
Section 05
This is not live migration, and the gap is mechanical
It is easy to read “checkpoint and restore” and hear “move a running workload between machines with no downtime”. The upstream path does not do that, and the difference is not polish — it is missing machinery. CRIU documents the mechanisms that live migration needs, and none of them are reachable through the CRI request described in chapter 02.
| Mechanism | What it is for | In the upstream path? |
|---|---|---|
| Pre-dump / iterative migration | Copy memory while the process still runs, repeatedly, so only a small final delta is copied during the freeze | No. The CRI request has no pre-dump concept; one call produces one full dump |
| Dirty page tracking | The kernel feature that makes iterative copying cheap — knowing which pages changed since last pass | No. Not used; and on the lab kernel criu check --all reports “Dirty tracking is OFF. Memory snapshot will not work.” |
| Page server | Stream memory pages directly to another host instead of writing a file | No. The request carries a location — a path on the node |
| TCP connection repair | Freeze and re-establish live connections so peers do not notice (--tcp-established) | No. No field can carry the option — and without it an established connection blocks the checkpoint (chapter 04) |
What upstream provides is a cold, one-shot, node-local dump to a file, plus a userland way to start a container from that file. It is closer to gcore than to a hypervisor’s live migration.
Products that do offer live migration build on the same CRIU primitives, using the published mechanisms in the table above — iterative pre-copy, a page server, connection repair — orchestrated outside Kubernetes. Nothing in this set inspects any such product; the point is only that those mechanisms are documented CRIU features which the Kubernetes CRI path does not expose.
Section 06
What to watch, concretely
- The gate registry, not the KEP. A GA entry appearing for
ContainerCheckpointinkube_features.gois the real status change. - containerd’s 34-line file. If
CheckpointContainerstops returningUnimplemented, the audience for this feature multiplies. - KEP-5823 reaching alpha in a release. That is when pod-level checkpoint and an actual
RestorePodRPC become testable. - Whether a restore verb ever reaches the kubelet. The CRI RPC is runtime-facing; a user-facing restore would be a separate, larger decision.
- Whether CRIU options ever become expressible. Without that, connected workloads stay outside the feature’s reach regardless of everything else.
Section 07
Further reading
Section 08
Closing note — this chapter has no measurements
Everything above is rung 2 or rung 3: files read at a stated commit, or published posts with dates. That is a deliberate limit, because the subject is status and direction, which cannot be measured on a lab.
What could not be verified
- Anything about KEP-5823’s behaviour. It is not in a release. Its proto comments describe intended semantics; no implementation was run.
- Whether containerd’s stub is being worked on. The file’s current content is verifiable; roadmap intent is not, and no claim is made about timing.
- Any vendor’s live-migration implementation. None was examined. §05 describes only mechanisms published by the CRIU project and shows they are not reachable through the CRI request — it does not describe how any product uses them.
- Whether dirty page tracking is off on other kernels. The warning quoted is from this lab’s kernel (6.8.0-139-generic, aarch64) and is not a general claim about Linux.
- The working group’s current activity. Only the announcement post was read; no meeting notes or subsequent output was reviewed.
How to re-check this chapter
Every claim here is a file you can fetch. sources.md beside this chapter lists each one with the exact path and the retrieval date, so a reader can re-walk the whole thing in a few minutes and see what has moved.
Section 09
Spoken drills
Is container checkpointing GA in Kubernetes?
A strong answer hits
- No — Beta since 1.30, no GA entry in the gate registry as of 1.36/master
- KEP-2008’s own file says stable at 1.33, which did not happen
- The gate registry is authoritative; KEP milestone blocks are intentions, not status
- The honest half: it is on by default, so “not GA” does not mean “not usable” — it means the interface can still change under you
check against §01
Someone proposes using checkpoint/restore to migrate pods between nodes for maintenance. Assess it.
A strong answer hits
- Upstream has no restore verb, so the return path is a build-an-image procedure
- No pre-dump, no page server — one full dump to a file on the node, and a freeze for its duration
- Any pod with an established TCP connection cannot be checkpointed at all
- It is on the working group’s stated use-case list, which is a fair signal it is not solved today
- The honest half: for a stateless-but-slow-to-start batch workload with no open sockets, this could genuinely work now — the objection is about connected services, not about the idea
check against §05
What single upstream change would most widen this feature’s reach?
A strong answer hits
- containerd implementing
CheckpointContainer— it is the default runtime nearly everywhere - Today that file returns
Unimplementedin 34 lines - A close second: a way to express CRIU options, which would unblock connected workloads
- The honest half: reach and usefulness are different. Runtime support widens who can call it; option support widens what it can capture. Neither alone gives you migration
check against §02 and §06