One POST, One Tar

Section 01

There is no feature gate to turn on

Most write-ups on this feature begin by telling you to set --feature-gates=ContainerCheckpoint=true. On any currently supported cluster that instruction is obsolete.

// owner: @adrianreber
// kep: https://kep.k8s.io/2008
// Enables container Checkpoint support in the kubelet
ContainerCheckpoint featuregate.Feature = "ContainerCheckpoint"
...
ContainerCheckpoint: {
    {Version: version.MustParse("1.25"), Default: false, PreRelease: featuregate.Alpha},
    {Version: version.MustParse("1.30"), Default: true,  PreRelease: featuregate.Beta},
},

Beta and on by default since 1.30, with no GA entry. The kubelet will tell you its own opinion if you ask it:

kubernetes_feature_enabled{name="ContainerCheckpoint",stage="BETA"} 1

Section 02

The endpoint

The route is registered on the kubelet’s authenticated port, and only when the gate is on.

checkpointPath      = "/checkpoint/"                      // line 117

// Only enable checkpoint API if the feature is enabled         // line ~692
    s.addMetricsBucketMatcher("checkpoint")
    ws.Path(checkpointPath).Produces(restful.MIME_JSON)
        To(s.checkpoint).
        Operation("checkpoint")

func (s *Server) checkpoint(request *restful.Request, response *restful.Response)   // line 1278

Shape: POST /checkpoint/{namespace}/{pod}/{container} on port 10250. There is no body and there are no options — which matters more than it looks, and chapter 04 shows why.

Section 03

The first call: 403, and a precise one

Calling with the API server’s own kubelet client certificate — the most privileged credential on the node — still fails.

Forbidden (user=kube-apiserver-kubelet-client, verb=create,
           resource=nodes, subresource(s)=[checkpoint])

HTTP 403

The kubelet does not decide this itself — it delegates to the API server, which evaluates RBAC against a subresource of nodes that nothing grants by default. The error is unusually helpful: it names the user, the verb and the subresource, which is exactly the ClusterRole you need to write.

Section 04

The successful call

{"items":["/var/lib/kubelet/checkpoints/checkpoint-counter_default-app-2026-09-13T17:38:36Z.tar"]}
HTTP 200

$ sudo ls -lh /var/lib/kubelet/checkpoints/
-rw------- 1 root root 242K Sep 13 17:38 checkpoint-counter_default-app-2026-09-13T17:38:36Z.tar

Three things to read off that. The response is a JSON array, not a file — the archive stays on the node and you fetch it by other means. The filename encodes pod, namespace, container and an RFC 3339 timestamp. And the mode is 0600, root:root, which chapter 05 argues is the only thing standing between that file and everything the process had in memory.

Section 05

Inside the tar

top-level members
stats-dump
dump.log
config.dump
spec.dump
bind.mounts
rootfs-diff.tar
io.kubernetes.cri-o.LogPath

checkpoint/ — 21 CRIU image files
checkpoint/cgroup.img        checkpoint/ids-1.img
checkpoint/core-1.img        checkpoint/inventory.img
checkpoint/descriptors.json  checkpoint/ipcns-var-11.img
checkpoint/fdinfo-2.img      checkpoint/mm-1.img
checkpoint/files.img         checkpoint/mountpoints-13.img
checkpoint/fs-1.img          checkpoint/netns-10.img
                             checkpoint/pagemap-1.img   ...

total entries: 28

Section 06

The container never stopped

The most common assumption about checkpointing is that it stops the thing it captures. On this path it does not, and that is not configuration — it is written into the code.

opts := &lib.ContainerCheckpointOptions{
	TargetFile: req.GetLocation(),
	// For the forensic container checkpointing use case we
	// keep the container running after checkpointing it.
	KeepRunning: true,
}

KeepRunning is hardcoded, not read from the request — and the CRI message has no field that could carry it (chapter 02). Through the kubelet API you cannot ask for a destructive checkpoint.

Running restarts=0 started=2026-09-13T17:38:28Z
counter value now: 21      (still advancing)

Section 07

Further reading

Section 08

Closing note — what varies, and what was not verified

Durable versus run-specific

• Durable: the 403 and its exact wording; that a ClusterRole on nodes/checkpoint is required; that the response is a JSON array of paths; mode 0600 root:root; the archive member layout; KeepRunning being hardcoded.
• Run-specific: the 242 KB size, the 21-file count (CRIU emits per-process and per-namespace images, so it varies with the workload), the timestamp in the filename, and the counter value.

Not verified

• Other authentication paths. Everything here used the API server’s kubelet client certificate from the node. A ServiceAccount token, or a call from off-node, was not tested.
• The exact cache TTL. The denial clearly expired; the kubelet’s authorization-cache flags were not read back from the running process, so no duration is claimed.
• containerd. Not tested here at all — chapter 02 shows from source why it cannot serve this call.
• Multi-container pods and init containers. Only single-container pods were checkpointed.

Section 09

Spoken drills