kubeadm Bootstrap Kubernetes

Version 1.37 · source

Start here

kubeadm init
Create a control plane
kubeadm init --pod-network-cidr=<cidr>
Init, sized for a CNI
kubeadm join <host>:6443 --token <token>
Add a node
kubeadm token create --print-join-command
Get a fresh join command
kubeadm certs check-expiration
When do certs expire?
kubeadm upgrade plan
What can I upgrade to?
kubeadm upgrade apply v<version>
Upgrade first control plane
kubeadm reset
Undo init or join

Concepts

  1. Install runtime, kubelet, kubeadm
  2. Control plane static pods, admin.conf
  3. Network nodes become Ready
  4. Workers kubelet joins

Bootstrap a cluster

kubeadm init
Initialise a control-plane node
kubeadm init --pod-network-cidr=<cidr>
Reserve the pod IP range for a CNI
kubeadm init --apiserver-advertise-address=<ip>
Choose the API server IP
kubeadm init --control-plane-endpoint=<dns>:6443
Stable endpoint for HA control plane
kubeadm init --upload-certs
Share certs for more control planes
kubeadm init --config <file>
Init from a config file
kubeadm init --dry-run
Show what init would do
kubeadm init --kubernetes-version <version>
Pin the control-plane version
kubeadm init --cri-socket <socket>
Pick the container runtime socket
kubeadm init --service-cidr <cidr>
Change the Service IP range
kubeadm init --apiserver-cert-extra-sans <name>
Extra names on the API cert
kubeadm init --ignore-preflight-errors=<check>
Downgrade a preflight error
kubeadm init --skip-phases=<phase>
Skip a phase, e.g. addon/kube-proxy
kubeadm init phase <phase>
Run a single init phase
mkdir -p $HOME/.kube && sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
Let kubectl talk to the cluster

Join nodes

kubeadm join <host>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>
Join a worker node
kubeadm join <host>:6443 ... --control-plane --certificate-key <key>
Join as a control-plane node
kubeadm join --config <file>
Join from a config file
kubeadm join ... --node-name <name>
Override the node name
kubeadm join ... --cri-socket <socket>
Pick the container runtime socket
kubeadm join ... --discovery-token-unsafe-skip-ca-verification
Skip CA pinning (insecure)
kubeadm join phase <phase>
Run a single join phase
kubeadm init phase upload-certs --upload-certs
Re-upload control-plane certs

Tokens

kubeadm token create --print-join-command
New token plus full join command
kubeadm token list
List bootstrap tokens
kubeadm token create
Create a token (24h by default)
kubeadm token create --ttl 0
Token that never expires
kubeadm token delete <token>
Revoke a token
kubeadm token generate
Print a random token locally
kubeadm certs certificate-key
Generate a certificate key
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
Compute the CA cert hash

Certificates

kubeadm certs check-expiration
Show expiry of all certs
kubeadm certs renew all
Renew every certificate
kubeadm certs renew <cert>
Renew one cert, e.g. apiserver
kubeadm certs generate-csr
Write CSRs for an external CA
kubeadm kubeconfig user --client-name <user>
Make a kubeconfig for a user
kubeadm kubeconfig user --client-name <user> --org <group>
Put the user in a group
kubeadm kubeconfig user --client-name <user> --validity-period <duration>
Shorter-lived user cert

Upgrade

kubeadm upgrade plan
Show available versions and checks
kubeadm upgrade apply v<version>
Upgrade the first control plane
kubeadm upgrade node
Upgrade other nodes' local config
kubeadm upgrade diff v<version>
Preview static pod manifest changes
kubeadm upgrade apply v<version> --dry-run
Rehearse an upgrade
kubeadm upgrade apply v<version> --yes
Skip the confirmation prompt
kubeadm upgrade apply v<version> --certificate-renewal=false
Do not renew certs on upgrade
kubectl drain <node> --ignore-daemonsets
Evict pods before upgrading kubelet
kubectl uncordon <node>
Make the node schedulable again
kubeadm version -o short
Print the kubeadm version

Config and images

kubeadm config print init-defaults
Print the default init config
kubeadm config print join-defaults
Print the default join config
kubeadm config print init-defaults --component-configs KubeletConfiguration
Include kubelet config too
kubeadm config validate --config <file>
Check a config file
kubeadm config migrate --old-config <file> --new-config <file>
Convert to the newest API version
kubeadm config images list
List images kubeadm needs
kubeadm config images pull
Pre-pull the control-plane images
kubectl -n kube-system get cm kubeadm-config -o yaml
Read the stored cluster config

Reset

kubeadm reset
Revert init or join on this node
kubeadm reset --force
Reset without the prompt
kubeadm reset --cleanup-tmp-dir
Also clean /etc/kubernetes/tmp
sudo rm -rf /etc/cni/net.d
Remove leftover CNI config
rm -rf $HOME/.kube
Remove the stale kubeconfig

Recipes

Single-node cluster
sudo kubeadm init --pod-network-cidr=<cidr>mkdir -p $HOME/.kubesudo cp /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/config# install a CNI plugin, then let pods run on the control planekubectl taint nodes --all node-role.kubernetes.io/control-plane-
Add a worker later
# on the control planekubeadm token create --print-join-command# on the new node, as rootkubeadm join <host>:6443 --token <token> \  --discovery-token-ca-cert-hash sha256:<hash>
Upgrade a control plane node
# upgrade the kubeadm package first, thensudo kubeadm upgrade plansudo kubeadm upgrade apply v<version>kubectl drain <node> --ignore-daemonsets# upgrade kubelet and kubectl packagessudo systemctl daemon-reload && sudo systemctl restart kubeletkubectl uncordon <node>
Check and renew certs
sudo kubeadm certs check-expirationsudo kubeadm certs renew all# restart kube-apiserver, controller-manager, scheduler, etcd static podssudo cp /etc/kubernetes/admin.conf $HOME/.kube/config