Kubelet Configuration

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Kubelet configuration is supported for:

  • Control plane nodes via clusterConfig.controlPlane.kubeletConfiguration
  • Worker nodes via workerConfig.kubeletConfiguration

There is no cluster-level default; control plane and worker settings are configured independently.

All fields are optional. When a field is not set, the kubelet default applies and no patch is emitted for that field.

For full upstream documentation on each setting, see the KubeletConfiguration reference.

Supported options

FieldTypeDescription
maxPodsinteger (50–256)Maximum number of pods per node.
automaticReservationsobject with profile (CapacityTiered)Opt in to node-size-aware kubeReserved and hard eviction thresholds computed at boot. Mutually exclusive with systemReserved, kubeReserved, and evictionHard. See Automatic resource reservations.
systemReservedmap of cpu, memory, ephemeral-storage, pid to quantitiesResources reserved for OS system daemons.
kubeReservedmap of cpu, memory, ephemeral-storage, pid to quantitiesResources reserved for Kubernetes components.
evictionHardmap of signal names to thresholdsHard eviction thresholds (immediate pod eviction).
evictionSoftmap of signal names to thresholdsSoft eviction thresholds (eviction after grace period).
evictionSoftGracePeriodmap of signal names to durationsGrace periods for soft eviction signals. Keys must match evictionSoft.
protectKernelDefaultsbooleanCauses the kubelet to error if kernel flags differ from expected values.
topologyManagerPolicynone, best-effort, restricted, single-numa-nodeNUMA-aware resource alignment policy.
cpuManagerPolicynone, staticControls cpuset assignment. static enables exclusive CPU pinning for Guaranteed QoS pods.
memoryManagerPolicyNone, StaticControls memory management. Static enables NUMA-aware memory allocation for Guaranteed QoS pods.
podPidsLimitinteger (1024–16384)Maximum number of PIDs per pod.
containerLogMaxSizequantity (e.g. "10Mi")Maximum size of a container log file before rotation.
containerLogMaxFilesinteger (≥2)Maximum number of rotated log files per container.
imageGCHighThresholdPercentinteger (0–100)Disk usage percent above which image GC always runs. Must be > imageGCLowThresholdPercent.
imageGCLowThresholdPercentinteger (0–100)Disk usage percent below which image GC never runs.
maxParallelImagePullsinteger (≥0)Maximum concurrent image pulls. When > 0, serializeImagePulls is automatically set to false.
shutdownGracePeriodduration (e.g. "30s")Total time the node delays shutdown for pod termination.
shutdownGracePeriodCriticalPodsduration (e.g. "10s")Time reserved for terminating critical pods during shutdown. Must be ≤ shutdownGracePeriod.
seccompDefaultbooleanApply the runtime's default seccomp profile (RuntimeDefault) to pods that do not specify one. See Default seccomp profile.
enforceNodeAllocatablelist of pods, system-reserved, kube-reserved, system-reserved-compressible, kube-reserved-compressibleWhich resource reservations are enforced via cgroups. See Enforce node allocatable.

Automatic resource reservations

Instead of hand-picking systemReserved/kubeReserved per node size, you can opt in to automatic, node-size-aware reservations. Each node computes its kubeReserved (CPU and memory) and a hard eviction threshold at boot from its actual capacity — the same approach GKE and EKS use.

automaticReservations is mutually exclusive with systemReserved, kubeReserved, and evictionHard; setting it alongside any of them is rejected at admission. Other kubelet fields (such as maxPods) can still be set.

The CapacityTiered profile mirrors GKE's node-allocatable formula (EKS uses the same approach) and reserves:

  • CPU: 6% of the first core, 1% of the second, 0.5% of cores three and four, and 0.25% of each core beyond four.
  • Memory: 255Mi below 1Gi total; otherwise 25% of the first 4Gi, 20% of the next 4Gi, 10% of the next 8Gi, 6% of the next 112Gi, and 2% of memory above 128Gi.
  • A hard eviction threshold of memory.available: 100Mi.
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    workers:
      machineDeployments:
      - class: default-worker
        name: md-0
        variables:
          overrides:
          - name: workerConfig
            value:
              kubeletConfiguration:
                automaticReservations:
                  profile: CapacityTiered

Default seccomp profile

seccompDefault instructs the kubelet to apply the container runtime's RuntimeDefault seccomp profile to every pod that does not explicitly set spec.securityContext.seccompProfile (or the equivalent on a container). This provides a baseline syscall filter for unhardened workloads without requiring per-pod changes.

Enabling seccompDefault: true on both control plane and worker kubeletConfiguration mitigates Linux kernel local-privilege-escalation issues that depend on syscalls excluded from RuntimeDefault (for example, the Dirty Frag exploit chain CVE-2026-43284 / CVE-2026-43500, which relies on unshare, add_key, and keyctl).

Caveats:

  • Pods that opt out with seccompProfile.type: Unconfined are not constrained.
  • Pods running with privileged: true or CAP_SYS_ADMIN are not constrained by seccomp.
  • Workloads that legitimately require syscalls outside RuntimeDefault (for example, sandboxed runtimes, profiling agents, or some networking tools) may need a custom seccomp profile or Unconfined.
  • Changing this value rolls the affected machines, since it is rendered into the KubeadmConfig and triggers a node template change.

Examples

Control plane

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          controlPlane:
            kubeletConfiguration:
              maxPods: 200
              protectKernelDefaults: true
              seccompDefault: true

Worker nodes

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    workers:
      machineDeployments:
      - class: default-worker
        name: md-0
        variables:
          overrides:
          - name: workerConfig
            value:
              kubeletConfiguration:
                maxPods: 250
                podPidsLimit: 4096
                seccompDefault: true

Resource reservations

systemReserved and kubeReserved accept a map with keys cpu, memory, ephemeral-storage, and pid. Values are Kubernetes resource quantities.

kubeletConfiguration:
  systemReserved:
    cpu: "500m"
    memory: "1Gi"
    ephemeral-storage: "10Gi"
  kubeReserved:
    cpu: "200m"
    memory: "512Mi"

Eviction thresholds

evictionHard and evictionSoft accept a map with signal names as keys and thresholds (absolute quantities or percentages) as values. Valid signal names are memory.available, nodefs.available, nodefs.inodesFree, imagefs.available, imagefs.inodesFree, and pid.available.

When using evictionSoft, you must also set evictionSoftGracePeriod with matching keys.

kubeletConfiguration:
  evictionHard:
    memory.available: "100Mi"
    nodefs.available: "10%"
    imagefs.available: "15%"
  evictionSoft:
    memory.available: "200Mi"
    nodefs.available: "15%"
  evictionSoftGracePeriod:
    memory.available: "30s"
    nodefs.available: "1m0s"

Graceful node shutdown

shutdownGracePeriod sets the total time the node delays shutdown for pod termination. shutdownGracePeriodCriticalPods sets the portion of that time reserved for critical pods and must be less than or equal to shutdownGracePeriod.

kubeletConfiguration:
  shutdownGracePeriod: "60s"
  shutdownGracePeriodCriticalPods: "15s"

Image garbage collection

imageGCHighThresholdPercent must be greater than imageGCLowThresholdPercent when both are set.

kubeletConfiguration:
  imageGCHighThresholdPercent: 85
  imageGCLowThresholdPercent: 70

Container log rotation

kubeletConfiguration:
  containerLogMaxSize: "50Mi"
  containerLogMaxFiles: 10

NUMA-aware topology management

For workloads sensitive to hardware topology (GPU, HPC, telco), you can combine topologyManagerPolicy, cpuManagerPolicy, and memoryManagerPolicy.

kubeletConfiguration:
  topologyManagerPolicy: single-numa-node
  cpuManagerPolicy: static
  memoryManagerPolicy: Static

Enforce node allocatable

By default, systemReserved and kubeReserved only affect scheduling: the kubelet subtracts them from the node's capacity to calculate the Allocatable value that the scheduler sees. However, nothing prevents system daemons or the kubelet itself from consuming more than the declared reservation. If a system process spikes beyond its reservation, it can starve pods of resources.

The enforceNodeAllocatable field adds runtime enforcement by creating cgroups that cap the reserved processes to their declared limits. Accepted values are:

ValueEnforcesSince K8s
podsPod resource limitsv1.0
system-reservedAll system-reserved resources (CPU + memory)v1.6
kube-reservedAll kube-reserved resources (CPU + memory)v1.6
system-reserved-compressibleOnly compressible (CPU) system-reserved resourcesv1.32
kube-reserved-compressibleOnly compressible (CPU) kube-reserved resourcesv1.32

The -compressible variants are the recommended starting point for enabling enforcement. They enforce only CPU (which is throttlable) and skip memory (which requires OOM-killing). This matches the upstream Kubernetes recommendation and is the default in OpenShift 4.22+.

system-reserved and system-reserved-compressible are mutually exclusive, as are kube-reserved and kube-reserved-compressible. The maximum number of items is 3 (one system variant, one kube variant, and pods).

When any system-reserved variant is included, CAREN automatically configures the well-known systemd cgroup path /system.slice for enforcement. When any kube-reserved variant is included, CAREN configures /system.slice/kubelet.service. You do not need to specify cgroup paths.

This field is optional. When not set, the kubelet default behaviour (pods only) applies and no changes are made to existing clusters.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          controlPlane:
            kubeletConfiguration:
              systemReserved:
                cpu: "500m"
                memory: "1Gi"
              kubeReserved:
                cpu: "200m"
                memory: "512Mi"
              enforceNodeAllocatable:
                - pods
                - system-reserved-compressible
                - kube-reserved-compressible

Example: full enforcement (CPU + memory)

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          controlPlane:
            kubeletConfiguration:
              systemReserved:
                cpu: "500m"
                memory: "1Gi"
              kubeReserved:
                cpu: "200m"
                memory: "512Mi"
              enforceNodeAllocatable:
                - pods
                - system-reserved
                - kube-reserved