This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Generic

1: Global Image Registry Mirror
2: HTTP proxy
3: Image registries
4: Network Time Protocol (NTP)
5: Tainting nodes
6: Users

The customizations in this section are applicable to all providers.

1 - Global Image Registry Mirror

Add containerd image registry mirror configuration to all Nodes in the cluster.

When the globalImageRegistryMirror variable is set, files with configurations for Containerd default mirror.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

To provide an image registry mirror with a CA certificate, specify the following configuration:

If the registry mirror requires a private or self-signed CA certificate, create a Kubernetes Secret with the ca.crt key populated with the CA certificate in PEM format:

kubectl create secret generic my-mirror-ca-cert \
  --from-file=ca.crt=registry-ca.crt

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          globalImageRegistryMirror:
            url: https://example.com
            credentials:
              secretRef:
                name: my-mirror-ca-cert

Applying this configuration will result in following new files on the KubeadmControlPlaneTemplate and KubeadmConfigTemplate resources:

/etc/containerd/certs.d/_default/hosts.toml
/etc/certs/mirror.pem

To use a public hosted image registry (e.g. ECR) as a registry mirror, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          globalImageRegistryMirror:
            url: https://123456789.dkr.ecr.us-east-1.amazonaws.com

Applying this configuration will result in following new files on the KubeadmControlPlaneTemplate and KubeadmConfigTemplate resources:

/etc/containerd/certs.d/_default/hosts.toml

2 - HTTP proxy

In some network environments it is necessary to use HTTP proxy to successfuly execute HTTP requests. This customization will configure Kubernetes components (containerd, kubelet) with appropriate configuration for control plane and worker nodes, utilising systemd drop-ins to configure the necessary environment variables.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

To configure HTTP proxy values, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          proxy:
            http: http://example.com
            https: http://example.com
            additionalNo:
              - no-proxy-1.example.com
              - no-proxy-2.example.com

The additionalNo list will be added to default pre-calculated values that apply on k8s networking localhost,127.0.0.1,<POD CIDRS>,<SERVICE CIDRS>,kubernetes,kubernetes.default,.svc,.svc.cluster.local, plus provider-specific addresses as required.

Applying this configuration will result in the following value being set:

KubeadmControlPlaneTemplate:

spec:
  kubeadmConfigSpec:
    clusterConfiguration:
      files:
        - path: "/etc/systemd/system/containerd.service.d/http-proxy.conf"
          content: <generated>
        - path: "/etc/systemd/system/kubelet.service.d/http-proxy.conf"
          content: <generated>

KubeadmConfigTemplate:

spec:
  files:
    - path: "/etc/systemd/system/containerd.service.d/http-proxy.conf"
      content: <generated>
    - path: "/etc/systemd/system/kubelet.service.d/http-proxy.conf"
      content: <generated>

Applying this configuration will result in new bootstrap files on the KubeadmControlPlaneTemplate and KubeadmConfigTemplate.

3 - Image registries

Add image registry configuration to all Nodes in the cluster.

When the credentials variable is set, files and preKubeadmnCommands with configurations for Kubelet image credential provider and dynamic credential provider will be added.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

If your registry requires static credentials, create a Kubernetes Secret with keys for username and password:

kubectl create secret generic my-registry-credentials \
  --from-literal username=${REGISTRY_USERNAME} --from-literal password=${REGISTRY_PASSWORD}

If your registry requires a private or self-signed CA certificate, create a Kubernetes Secret with the ca.crt key populated with the CA certificate in PEM format:

kubectl create secret generic my-mirror-ca-cert \
  --from-file=ca.crt=registry-ca.crt

To set both image registry credentials and CA certificate, create a Kubernetes Secret with keys for username, password, and ca.crt:

kubectl create secret generic my-registry-credentials \
  --from-literal username=${REGISTRY_USERNAME} --from-literal password=${REGISTRY_PASSWORD} \
  --from-file=ca.crt=registry-ca.crt

To add image registry credentials and/or CA certificate, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          imageRegistries:
            - url: https://my-registry.io
              credentials:
                secretRef:
                  name: my-registry-credentials

Applying this configuration will result in new files and preKubeadmCommands on the KubeadmControlPlaneTemplate and KubeadmConfigTemplate.

4 - Network Time Protocol (NTP)

You can configure cluster nodes (control plane, and workers) to update their system clock from a specific set of NTP servers.

Keep in mind that each node's operating system is almost certainly configured with a default set of NTP servers. In some cases, the node must use different NTP servers. For example, if the node cannot reach the default servers.

Example

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
            ntp
              servers:
                - time1.example.com
                - time2.example.com

Applying this configuration will result in the following values being set

for the control plane, on the KubeadmControlPlaneTemplate

spec:
  kubeadmConfigSpec:
    ntp:
      enabled: true
      servers:
        - time1.example.com
        - time2.example.com

and for every worker, on its respective KubeadmConfigTemplate

spec:
  ntp:
    enabled: true
    servers:
      - time1.example.com
      - time2.example.com

5 - Tainting nodes

Tainting nodes prevents pods from being scheduled on them unless they explicitly tolerate the taints applied to the nodes. See the Kubernetes Taints and Tolerations documentation for more details.

This customization will be available when the provider-specific cluster configuration patch is included in the ClusterClass.

Example

Control plane taints

To configure taints for the control plane nodes, specify the following configuration:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          controlPlane:
            taints:
              - key: some-key
                effect: NoSchedule
                value: some-value

Applying this configuration will result in the following value being set:

KubeadmControlPlaneTemplate:

spec:
  kubeadmConfigSpec:
    initConfiguration:
      nodeRegistration:
        taints:
          - key: some-key
            effect: NoSchedule
            value: some-value
    joinConfiguration:
      nodeRegistration:
        taints:
          - key: some-key
            effect: NoSchedule
            value: some-value

Default control-plane taint applied by kubeadm

When using this customization, the default taint added by kubeadm to the control plane nodes will not be added unless explicitly specified as well.

To add the default taint back to the control-plane, add the following taint along with any custom taints you wish to add to the control-plane taints:

- key: node-role.kubernetes.io/control-plane
  effect: NoSchedule

Removing all taints from control-plane nodes

To remove the default control plane taints set by kubeadm (and therefore allow scheduling to control plane nodes without adding explicit tolerations to your pod manifests), set controlPlane.taints to an empty array:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          controlPlane:
            taints: []

Worker node taints

Taints for individual nodepools can be configured similarly:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    workers:
      machineDeployments:
      - class: default-worker
        name: md-0
        variables:
          overrides:
          - name: workerConfig
            value:
              taints:
                - key: some-key
                  effect: NoSchedule
                  value: some-value

Applying this configuration will result in the following value being set:

KubeadmConfigTemplate:

spec:
  joinConfiguration:
    nodeRegistration:
      taints:
        - key: some-key
          effect: NoSchedule
          value: some-value

6 - Users

Configure users for all machines in the cluster, the user's superuser capabilities using sudo user specifications, and the login authentication mechanism.

SSH authorized keys are just public SSH keys that are used to authenticate a login. See the SSH man page for more information.
For information on sudo user specifications, see the sudo documentation.
Local password authentication is disabled for the user by default. It is enabled only when a hashed password is provided.

Examples

Creates a user with the name admin, grants the user the ability to run any command as the superuser, and allows you to login via SSH using the username and private key corresponding to the authorized public key.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          users:
            - name: username
              sshAuthorizedKeys:
                - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAua0lo8BiGWgvIiDCKnQDKL5uERHfnehm0ns5CEJpJw optionalcomment"
              sudo: "ALL=(ALL) NOPASSWD:ALL"

Creates a user with the name admin, grants the user the ability to run any command as the superuser, and allows you to login via serial console using the username and password.

Note that this does not allow you to login via SSH using the username and password; in most cases, you must also configure the SSH server to allow password authentication.

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: <NAME>
spec:
  topology:
    variables:
      - name: clusterConfig
        value:
          users:
            - name: admin
              hashedPassword: "$y$j9T$UraH8eN4XvapXBmmSaUrP0$Nyxdf1cJDGZcp0WDKu.CFHprrkPG4ubirqSqiD43Ix3"
              sudo: "ALL=(ALL) NOPASSWD:ALL"

Generic

1 - Global Image Registry Mirror

Example

2 - HTTP proxy

Example

3 - Image registries

Example

4 - Network Time Protocol (NTP)

Example

5 - Tainting nodes

Example

Control plane taints

Default control-plane taint applied by kubeadm

Removing all taints from control-plane nodes

Worker node taints

6 - Users

Examples

Admin user with SSH public key login

Admin user with serial console password login