Kubeadm

• 1: Audit policy
• 2: Auto-renewal of control plane certificates
• 3: Containerd metrics
• 4: DNS
• 5: Encryption At Rest
• 6: etcd
• 7: Extra API Server Certificate SANs
• 8: kube-proxy mode
• 9: Kubelet Configuration
• 10: Kubernetes Image Repository
• 11: Node registration configuration
• 12: Parallel Image Pulls
• 13: Pod Security Admission

1 - Audit policy

Kubernetes auditing provides a security-relevant, chronological set of records documenting the sequence of actions in a cluster. The cluster audits the activities generated by users, by applications that use the Kubernetes API, and by the control plane itself.

There are currently no configuration options for the Audit Policy customization and this customization will be automatically applied when the provider-specific cluster configuration patch is included in the ClusterClass.

2 - Auto-renewal of control plane certificates

autoRenewCertificates variable enables automatic renewal of control plane certificates by triggering a rollout of the control plane nodes when the certificates on the control plane machines are about to expire.

More information about certificate renewal: Automatically rotating certificates using Kubeadm Control Plane provider.

Example

To enable automatic certificate renewal use the following configuration, applicable to all CAPI providers supported by CAREN:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          controlPlane:
            autoRenewCertificates:
              daysBeforeExpiry: 30

Applying this configuration will result in the following configuration being applied:

• KubeadmControlPlaneTemplate:

  • spec:
      template:
        spec:
          rolloutBefore:
            certificatesExpiryDays: 30
    

3 - Containerd metrics

Containerd exports metrics to a Prometheus endpoint. The metrics cover containerd itself, its plugins, e.g. CRI, and information about the containers managed by containerd.

There are currently no configuration options for metrics, and this customization will be automatically applied when the provider-specific cluster configuration patch is included in the ClusterClass.

4 - DNS

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

The DNS configuration can then be manipulated via the cluster variables. If the dns property is not specified, then the customization will be skipped.

CoreDNS

The CoreDNS configuration can then be manipulated via the cluster variables. If the dns.coreDNS property is not specified, then the customization will be skipped.

Example

The CoreDNS version can be updated automatically. To do this, set coreDNS to an empty object:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          dns:
            coreDNS: {}

Applying this configuration will result in the following value being set, with the version of the CoreDNS image being set based on the cluster's Kubernetes version:

• KubeadmControlPlaneTemplate:

  • spec:
      kubeadmConfigSpec:
        clusterConfiguration:
          dns:
            imageTag: "v1.11.3"
    

To change the repository and tag for the container image for the CoreDNS pod, specify the following configuration:

Note do not include "coredns" in the repository, kubeadm already appends it.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          dns:
            coreDNS:
              image:
                repository: my-registry.io/my-org/my-repo
                tag: "v1.11.3_custom.0"

Applying this configuration will result in the following value being set:

• KubeadmControlPlaneTemplate:

  • spec:
      kubeadmConfigSpec:
        clusterConfiguration:
          dns:
            imageRepository: "my-registry.io/my-org/my-repo"
            imageTag: "v1.11.3_custom.0"
    

5 - Encryption At Rest

encryptionAtRest variable enables encrypting kubernetes resources at rest using provided encryption provider. When this variable is set, kuberntetes secrets and configmaps are encrypted before writing them at etcd.

If the encryptionAtRest property is not specified, then the customization will be skipped. The secrets and configmaps will not be stored as encrypted in etcd.

We support following encryption providers

• aescbc
• secretbox

More information about encryption at-rest: Encrypting Confidential Data at Rest

Example

To encrypt configmaps and secrets kubernetes resources using aescbc encryption provider:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          encryptionAtRest:
            providers:
              - aescbc: {}

Applying this configuration will result in

1. <CLUSTER_NAME>-encryption-config secret generated.

A secret key for the encryption provider is generated and stored in <CLUSTER_NAME>-encryption-config secret. The APIServer will be configured to use the secret key to encrypt secrets and configmaps kubernetes resources before writing them to etcd. When reading resources from etcd, encryption provider that matches the stored data attempts in order to decrypt the data. CAREN currently does not rotate the key once it generated.

1. Configure APIServer with encryption configuration:

• KubeadmControlPlaneTemplate:

  • spec:
      kubeadmConfigSpec:
        clusterConfiguration:
          apiServer:
            extraArgs:
              encryption-provider-config: /etc/kubernetes/pki/encryptionconfig.yaml
      files:
        - contentFrom:
            secret:
              key: config
              name: <CLUSTER_NAME>-encryption-config
          path: /etc/kubernetes/pki/encryptionconfig.yaml
          permissions: "0640"
    

6 - etcd

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

The etcd configuration can then be manipulated via the cluster variables. If the etcd property is not specified, then the customization will be skipped.

Example

To change the repository and tag for the container image for the etcd pod, specify the following configuration:

Note do not include "etcd" in the repository, kubeadm already appends it.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          etcd:
            image:
              repository: my-registry.io/my-org/my-repo
              tag: "v3.5.99_custom.0"

Applying this configuration will result in the following value being set:

• KubeadmControlPlaneTemplate:

  • spec:
      kubeadmConfigSpec:
        clusterConfiguration:
          etcd:
            local:
              imageRepository: "my-registry.io/my-org/my-repo"
              imageTag: "v3.5.99_custom.0"
    

7 - Extra API Server Certificate SANs

If the API server can be accessed by alternative DNS addresses then setting additional SANs on the API server certificate is necessary in order for clients to successfully validate the API server certificate.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

To add extra SANs to the API server certificate, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          extraAPIServerCertSANs:
            - a.b.c.example.com
            - d.e.f.example.com

Applying this configuration will result in the following value being set:

• KubeadmControlPlaneTemplate:

  • spec:
      kubeadmConfigSpec:
        clusterConfiguration:
          apiServer:
            certSANs:
              - a.b.c.example.com
              - d.e.f.example.com
    

8 - kube-proxy mode

This customization allows configuration of the kube-proxy proxy mode. Currently, only iptables and nftables modes are supported. By default, kube-proxy is enabled in iptables mode by kubeadm.

Examples

Enabling nftables kube-proxy mode

Enabling nftables is done via the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          kubeProxy:
            mode: nftables

Applying this configuration will result in the following configuration being applied to create a KubeProxyConfiguration and append it to the kubeadm configuration that is used when kubeadm init is executed:

• KubeadmControlPlaneTemplate:

  • spec:
      template:
        spec:
          kubeadmConfigSpec:
            files:
              - path: "/etc/kubernetes/kubeproxy-config.yaml"
                owner: "root:root"
                permissions: "0644"
                content: |-
                  ---
                  apiVersion: kubeproxy.config.k8s.io/v1alpha1
                  kind: KubeProxyConfiguration
                  mode: nftables
          preKubeadmCommands:
            - /bin/sh -ec 'cat /etc/kubernetes/kubeproxy-config.yaml >> /run/kubeadm/kubeadm.yaml'
    

Skipping kube-proxy installation

To disable the deployment and upgrade of kube-proxy, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          kubeProxy:
            mode: disabled

Applying this configuration will result in the following configuration being applied:

• KubeadmControlPlaneTemplate:

  • spec:
      template:
        spec:
          kubeadmConfigSpec:
            initConfiguration:
              skipPhases:
                - addon/kube-proxy
    

9 - Kubelet Configuration

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Kubelet configuration is supported for:

• Control plane nodes via clusterConfig.controlPlane.kubeletConfiguration
• Worker nodes via workerConfig.kubeletConfiguration

There is no cluster-level default; control plane and worker settings are configured independently.

All fields are optional. When a field is not set, the kubelet default applies and no patch is emitted for that field.

For full upstream documentation on each setting, see the KubeletConfiguration reference.

Supported options

Automatic resource reservations

Instead of hand-picking systemReserved/kubeReserved per node size, you can opt in to automatic, node-size-aware reservations. Each node computes its kubeReserved (CPU and memory) and a hard eviction threshold at boot from its actual capacity — the same approach GKE and EKS use.

automaticReservations is mutually exclusive with systemReserved, kubeReserved, and evictionHard; setting it alongside any of them is rejected at admission. Other kubelet fields (such as maxPods) can still be set.

The CapacityTiered profile mirrors GKE's node-allocatable formula (EKS uses the same approach) and reserves:

• CPU: 6% of the first core, 1% of the second, 0.5% of cores three and four, and 0.25% of each core beyond four.
• Memory: 255Mi below 1Gi total; otherwise 25% of the first 4Gi, 20% of the next 4Gi, 10% of the next 8Gi, 6% of the next 112Gi, and 2% of memory above 128Gi.
• A hard eviction threshold of memory.available: 100Mi.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    workers:
      machineDeployments:
      - class: default-worker
        name: md-0
        variables:
          overrides:
          - name: workerConfig
            value:
              kubeletConfiguration:
                automaticReservations:
                  profile: CapacityTiered

Default seccomp profile

seccompDefault instructs the kubelet to apply the container runtime's RuntimeDefault seccomp profile to every pod that does not explicitly set spec.securityContext.seccompProfile (or the equivalent on a container). This provides a baseline syscall filter for unhardened workloads without requiring per-pod changes.

Enabling seccompDefault: true on both control plane and worker kubeletConfiguration mitigates Linux kernel local-privilege-escalation issues that depend on syscalls excluded from RuntimeDefault (for example, the Dirty Frag exploit chain CVE-2026-43284 / CVE-2026-43500, which relies on unshare, add_key, and keyctl).

Caveats:

• Pods that opt out with seccompProfile.type: Unconfined are not constrained.
• Pods running with privileged: true or CAP_SYS_ADMIN are not constrained by seccomp.
• Workloads that legitimately require syscalls outside RuntimeDefault (for example, sandboxed runtimes, profiling agents, or some networking tools) may need a custom seccomp profile or Unconfined.
• Changing this value rolls the affected machines, since it is rendered into the KubeadmConfig and triggers a node template change.

Examples

Control plane

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          controlPlane:
            kubeletConfiguration:
              maxPods: 200
              protectKernelDefaults: true
              seccompDefault: true

Worker nodes

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    workers:
      machineDeployments:
      - class: default-worker
        name: md-0
        variables:
          overrides:
          - name: workerConfig
            value:
              kubeletConfiguration:
                maxPods: 250
                podPidsLimit: 4096
                seccompDefault: true

Resource reservations

systemReserved and kubeReserved accept a map with keys cpu, memory, ephemeral-storage, and pid. Values are Kubernetes resource quantities.

kubeletConfiguration:
  systemReserved:
    cpu: "500m"
    memory: "1Gi"
    ephemeral-storage: "10Gi"
  kubeReserved:
    cpu: "200m"
    memory: "512Mi"

Eviction thresholds

evictionHard and evictionSoft accept a map with signal names as keys and thresholds (absolute quantities or percentages) as values. Valid signal names are memory.available, nodefs.available, nodefs.inodesFree, imagefs.available, imagefs.inodesFree, and pid.available.

When using evictionSoft, you must also set evictionSoftGracePeriod with matching keys.

kubeletConfiguration:
  evictionHard:
    memory.available: "100Mi"
    nodefs.available: "10%"
    imagefs.available: "15%"
  evictionSoft:
    memory.available: "200Mi"
    nodefs.available: "15%"
  evictionSoftGracePeriod:
    memory.available: "30s"
    nodefs.available: "1m0s"

Graceful node shutdown

shutdownGracePeriod sets the total time the node delays shutdown for pod termination. shutdownGracePeriodCriticalPods sets the portion of that time reserved for critical pods and must be less than or equal to shutdownGracePeriod.

kubeletConfiguration:
  shutdownGracePeriod: "60s"
  shutdownGracePeriodCriticalPods: "15s"

Image garbage collection

imageGCHighThresholdPercent must be greater than imageGCLowThresholdPercent when both are set.

kubeletConfiguration:
  imageGCHighThresholdPercent: 85
  imageGCLowThresholdPercent: 70

Container log rotation

kubeletConfiguration:
  containerLogMaxSize: "50Mi"
  containerLogMaxFiles: 10

NUMA-aware topology management

For workloads sensitive to hardware topology (GPU, HPC, telco), you can combine topologyManagerPolicy, cpuManagerPolicy, and memoryManagerPolicy.

kubeletConfiguration:
  topologyManagerPolicy: single-numa-node
  cpuManagerPolicy: static
  memoryManagerPolicy: Static

Enforce node allocatable

By default, systemReserved and kubeReserved only affect scheduling: the kubelet subtracts them from the node's capacity to calculate the Allocatable value that the scheduler sees. However, nothing prevents system daemons or the kubelet itself from consuming more than the declared reservation. If a system process spikes beyond its reservation, it can starve pods of resources.

The enforceNodeAllocatable field adds runtime enforcement by creating cgroups that cap the reserved processes to their declared limits. Accepted values are:

The -compressible variants are the recommended starting point for enabling enforcement. They enforce only CPU (which is throttlable) and skip memory (which requires OOM-killing). This matches the upstream Kubernetes recommendation and is the default in OpenShift 4.22+.

system-reserved and system-reserved-compressible are mutually exclusive, as are kube-reserved and kube-reserved-compressible. The maximum number of items is 3 (one system variant, one kube variant, and pods).

When any system-reserved variant is included, CAREN automatically configures the well-known systemd cgroup path /system.slice for enforcement. When any kube-reserved variant is included, CAREN configures /system.slice/kubelet.service. You do not need to specify cgroup paths.

This field is optional. When not set, the kubelet default behaviour (pods only) applies and no changes are made to existing clusters.

Example: compressible-only enforcement (recommended)

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          controlPlane:
            kubeletConfiguration:
              systemReserved:
                cpu: "500m"
                memory: "1Gi"
              kubeReserved:
                cpu: "200m"
                memory: "512Mi"
              enforceNodeAllocatable:
                - pods
                - system-reserved-compressible
                - kube-reserved-compressible

Example: full enforcement (CPU + memory)

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          controlPlane:
            kubeletConfiguration:
              systemReserved:
                cpu: "500m"
                memory: "1Gi"
              kubeReserved:
                cpu: "200m"
                memory: "512Mi"
              enforceNodeAllocatable:
                - pods
                - system-reserved
                - kube-reserved

10 - Kubernetes Image Repository

Override the container image repository used when pulling Kubernetes images.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

To configure HTTP proxy values, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          kubernetesImageRepository: "my-registry.io/my-org/my-repo"

Applying this configuration will result in the following value being set:

• KubeadmControlPlaneTemplate:
  • /spec/template/spec/kubeadmConfigSpec/clusterConfiguration/imageRepository: my-registry.io/my-org/my-repo

11 - Node registration configuration

Below is a list of node registration configuration options that can be set for kubeadm init and kubeadm join.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

ignorePreflightErrors

Kubeadm runs preflight checks to ensure the machine is compatible with Kubernetes and its dependencies. The SystemVerification check is known to result in false positives. For example, it fails when the Linux Kernel version is not supported by kubeadm, even if the kernel has all the required features. For this reason, we skip the check by default.

Control plane

To configure ignorePreflightErrors for the control plane nodes, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          controlPlane:
            nodeRegistration:
              ignorePreflightErrors:
                - SystemVerification

Applying this configuration will result in the following value being set:

• KubeadmControlPlaneTemplate:

  • spec:
      kubeadmConfigSpec:
        initConfiguration:
          nodeRegistration:
            nodeRegistration:
              ignorePreflightErrors:
                - SystemVerification
        joinConfiguration:
            nodeRegistration:
              ignorePreflightErrors:
                - SystemVerification
    

Worker node

ignorePreflightErrors for individual nodepools can be configured similarly:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    workers:
      machineDeployments:
      - class: default-worker
        name: md-0
        variables:
          overrides:
          - name: workerConfig
            value:
              nodeRegistration:
                ignorePreflightErrors:
                  - SystemVerification

Applying this configuration will result in the following value being set:

• KubeadmConfigTemplate:

  • spec:
      joinConfiguration:
        nodeRegistration:
          ignorePreflightErrors:
            - SystemVerification
    

By default, the following value will be set for both control plane and worker nodes:

    variables:
      - name: clusterConfig
        value:
          controlPlane:
            nodeRegistration:
              ignorePreflightErrors:
                - SystemVerification
      - name: workerConfig
        value:
          nodeRegistration:
            ignorePreflightErrors:
              - SystemVerification

This can be enabled by setting ignorePreflightErrors to an empty list:

    variables:
      - name: clusterConfig
        value:
          controlPlane:
            nodeRegistration:
              ignorePreflightErrors: []
      - name: workerConfig
        value:
          nodeRegistration:
            ignorePreflightErrors: []

12 - Parallel Image Pulls

Deprecated: This customization is deprecated. Use kubeletConfiguration.maxParallelImagePulls under control-plane or worker config instead. See the Kubelet Configuration documentation.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

The parallel image pull configuration can then be manipulated via the cluster variables. If the maxParallelImagePullsPerNode property is not specified, then the default value of 1 will be used which is equivalent to serialized image pulls.

Setting this value to 0 results in unlimited parallel image pulls.

Example

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          maxParallelImagePullsPerNode: 10

Applying this configuration will result in a KubeletConfiguration patch being added which will be applied by kubeadm on init and join:

• KubeadmControlPlaneTemplate:

  • spec:
      template:
        spec:
          kubeadmConfigSpec:
            files:
              - path: "/etc/kubernetes/patches/kubeletconfigurationmaxparallelimagepulls+strategic.json"
                owner: "root:root"
                permissions: "0644"
                content: |-
                  ---
                  apiVersion: kubelet.config.k8s.io/v1beta1
                  kind: KubeletConfiguration
                  serializeImagePulls: false
                  maxParallelImagePulls: 10
    

• KubeadmConfigTemplate

  • spec:
      kubeadmConfigSpec:
        files:
          - path: "/etc/kubernetes/patches/kubeletconfigurationmaxparallelimagepulls+strategic.json"
            owner: "root:root"
            permissions: "0644"
            content: |-
              ---
              apiVersion: kubelet.config.k8s.io/v1beta1
              kind: KubeletConfiguration
              serializeImagePulls: false
              maxParallelImagePulls: 10
    

13 - Pod Security Admission

This customization allows configuration of the Pod Security Admission plugin with cluster-wide defaults. When specified, CAREN configures the API server to use the PodSecurity admission plugin with the provided settings.

This is an opt-in feature. When podSecurityAdmission is not specified in the cluster configuration, no Pod Security Admission configuration is applied and existing clusters are unaffected.

Configuration

Version is always set to latest and is not configurable.

Examples

Enforce restricted Pod Security Standard

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          podSecurityAdmission:
            enforce: restricted
            audit: restricted
            warn: restricted

Applying this configuration will result in the following being applied to the KubeadmControlPlaneTemplate:

• A PodSecurityConfiguration file at /etc/kubernetes/pod-security-admission.yaml
• An AdmissionConfiguration file at /etc/kubernetes/admission.yaml referencing the plugin
• The --admission-control-config-file and --enable-admission-plugins API server extra args
• Volume mounts for both configuration files

Audit and warn only (no enforcement)

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          podSecurityAdmission:
            enforce: privileged
            audit: restricted
            warn: restricted

Custom exemptions

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          podSecurityAdmission:
            enforce: restricted
            audit: restricted
            warn: restricted
            exemptions:
              namespaces:
                - kube-system
                - my-privileged-namespace
              usernames:
                - system:serviceaccount:kube-system:some-sa
              runtimeClassNames:
                - kata