k8s start pod at specific node

specify nodeSelector or nodeName in Pod or VirtualMachineInstance manifest

these selectors are in the spec field

1
2

kubectl run <pod name> --image=<image name> -it --rm --overrides='{"spec":{"nodeName": "<node name>"}}' -- /bin/sh

to label a node, run:

1
2

kubectl label node <node_name> <key>=<value>

k8s reboot pod and vm

to reboot you need to kill the pod/vmi and recreate it, thereby all its states will be lost.

1
2
3

kubectl delete pod <pod_name>
kubectl delete vmi <vmi_name>

for vmi there is an option called soft-reboot which is absent in pods. however the vm runner pod must not exit.

1
2

virtctl soft-reboot <vmi_name>

for pod you you can scale down the replicas of deployment (recommended), or kill the pod directly.

1
2
3

kubectl scale deployment <deployment_name> --replicas=0
kubectl scale deployment <deployment_name> --replicas=<original replica num>

for vm you are supposed to use virtctl

1
2
3
4

virtctl start <vm_name>
virtctl stop <vm_name>
virtctl restart <vm_name>

k8s worker cluster setup

k3s has different ways to form a cluster than kubeadm join.

https://docs.k3s.io/quick-start

k3s specifies the init command in /etc/systemd/system/k3s.service. (k3s-agent.service if installed as agent) usually it is k3s server.

you need to change it to k3s agent in order to join the master node, or pass additional environment variables K3S_URL=https://<node_ip>:6443 and K3S_TOKEN=<node-token> while running k3s installation script.

the node token is at /var/lib/rancher/k3s/server/node-token

the agent node still needs to configure registry mirrors at /etc/rancher/k3s/registries.yaml for successfully pulling images

k3sup can automatically install k3s cluster using ssh connection

multi server cluster setup:

https://docs.k3s.io/datastore/ha-embedded

k8s python api library

install the official library with pip install kubernetes

the default api config path for k8s is at ~/.kube/config

k3s is at /etc/rancher/k3s/k3s.yaml

microk8s is at /var/snap/microk8s/current/credentials/kubelet.config

you can also set KUBECONFIG environment variable to let kubernetes python library know where the config is

to use it:

https://github.com/kubernetes-client/python/blob/master/kubernetes/README.md

1
2
3
4
5
6
7
8
9
10

from kubernetes import client, config
# Configs can be set in Configuration class directly or using helper utility
config_path = ...
config.load_kube_config(config_path)
v1 = client.CoreV1Api()
print("Listing pods with their IPs:")
ret = v1.list_pod_for_all_namespaces(watch=False)
for i in ret.items:
print("%s\t%s\t%s" % (i.status.pod_ip, i.metadata.namespace, i.metadata.name))

network policy:

https://minikube.sigs.k8s.io/docs/handbook/network_policy/

https://docs.tigera.io/calico/latest/network-policy/get-started/calico-policy/calico-network-policy

persistent volume:

https://minikube.sigs.k8s.io/docs/handbook/persistent_volumes/

1
2
3

sudo snap install --classic microk8s
sudo microk8s enable dns:<dns_ip>

config files are at /var/snap/microk8s/current, and you need to replace all docker.io with some docker mirror to prevent init errors.

run microk8s inspect to get errors like hostname casing, and missing file like /var/snap/microk8s/current/var/kubernetes/backend/localnode.yaml

you need to configure multiple registries for docker.io and registry.k8s.io under /var/snap/microk8s/current/args/certs.d

in order to use some mirror site which does not support /v2 url, you have to add override_path = true in config

mirror sites:

https://github.com/docker-mirrors/website

https://github.com/cmliu/CF-Workers-docker.io/issues/8

https://github.com/kubesre/docker-registry-mirrors

https://github.com/lawrenceching/gitbook/blob/master/docker-repositories-in-china.md

reference:

https://github.com/containerd/containerd/blob/main/docs/hosts.md

https://microk8s.io/docs/registry-private

install k3s

1
2
3
4

curl -sfL https://get.k3s.io > k3s_setup.sh
# replace the line if GITHUB_URL to some github mirror instead
bash k3s_setup.sh

k3s mirror

1
2

curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -​

registry config:

https://docs.k3s.io/installation/private-registry

k0s install:

https://docs.k0sproject.io/stable/install/

1
2
3
4

curl -sSLf https://get.k0s.sh | sudo sh
sudo k0s install controller --single
sudo k0s start

config:

https://docs.k0sproject.io/stable/runtime/