Skip to main content

Posts

Showing posts from January, 2022

Kubernetes Logging

Kubernetes Logging /var/log/containers ==> default log location on nodes. Last two log files are retained on the node. In order to aggregate logs from varies sources like pods, containers , nodes, control plane and events we use fluentd and elastic search for searching and grafana for visualization. Logging Architecture - Nodes: Kubectl logs $POD_NAME ==> accessing pod logs Kubectl logs $POD_NAME -c $CONTAINER_NAME ==> specific container inside a pod. Kubectl logs $POD_NAME --all-containers Kubectl logs $POD_NAME --all-containers --follow ==> real time logs Kubectl logs --selector app=backend --all-containers When an api server is down then in that case we cannot use kubectl get logs instead we can log in to a node and directly run  docker logs $CONTAINER_NAME What if docker is not available?  tail -f /var/log/containers/$CONTAINER_NAME$CONTAINER_ID Logging Architecture - Control Plane: Get a list of control plane pods using a selector kubectl get pods --namespace kube-syst

Cluster Upgrade

 Upgrade Control Plane Steps: Update kubeadm package using package manager. Drain the master node Run kubeadm upgrade plan which runs some checks Run kubeadm upgrade apply uncordon the master node update kubelet and kubectl  Example with commands: sudo apt-mark unhold kubeadm sudo apt-get update sudo apt-cache policy kubeadm sudo apt-get install kubeadm=$TARGET_VERSION sudo apt-mark hold kubeadm kubectl drain c1-master1 --ignore-daemonsets sudo kubeadm upgrade plan sudo kubeadm upgrade apply v$TARGET_VERSION kubectl uncordon c1-master1 sudo apt-mark unhold kubectl kubelet sudo apt-get update sudo apt-get install -y kubelet=$TARGET_VERSION kubectl=$TARGET_VERSION sudo apt-mark hold kubectl kubelet DEMO: SSH to Master node Run apt-get update  Run apt-cache kubeadm        ==> available kubeadm versions Run kubectl version --short   ==> what version we are on Run kubectl get nodes           ==> lists all the nodes with kubectl version Run sudo apt-mark unhold kubeadm Run sudo apt-

Kubernetes ETCDCTL

ETCD is a key-value data store for storing kubernetes state and objects. ETCDCTL is the tool used to back up with snapshot. ETCDCTL a command line tool for interacting with the etcd server  Key Features:  Keep the event history until compaction. access to old version of keys. user controlled key compaction. Support range query. pagination support with limit argument. support consistency guarantee across multiple queries. Replace TTL key with lease. Support watch feature. ETCDTOOL Installation: Build binary from source code  Download tool directly  Build binary from source code: Checkout the code repository git clone -b v3.4.16 https://github.com/etcd-io/etcd.git Change directory to etcd cd etcd Run build script ./build The binaries are under the bin directory. exportPATH="$PATH:`pwd`/bin" Check etcd version etcd --version Download ETCD tool directly: kubectl exec -it etcd-docker-desktop -n kube-system -- /bin/bash -c 'ETCDCTL_API=3 /usr/local/bin/etcd --version' | h

Kubernetes Helm

Helm Helm is a package manager for deploying applicatons. Adding stable repository helm repo add <alias> <helm repo> helm repo add stable https://charts.helm.sh/stable Search chart in a repo helm search repo stable/mysql Deploy application using helm helm install <Name of the application> <repo/chart-name> Eg: helm install mysql testing/mysql Confirm current context kubectl config current-context Chart definition helm show chart <repo/chartname> helm show readme <repo/chartname> helm show values <chart-name> dry run helm install <name-of -the-application> <chart name>  --dry-run --debug Confirm deployment helm list List pods, services, deployments and replica sets kubectl get all  status of the release helm status <app-deployed> eg: helm status mysql helm get manifest <chart-deployed> View release hhistory helm history mysql Unistall a release helm uninstall mysql --keep-history helm delete mysql Show all the version of

PODS, SERVICES & DEPLOYMENTS

PODS Pod is a wrapper to container and it is the smallest deployable unit in Kubernetes. pod yaml configuration   Pod Useful Commands: $kubectl version --short Client Version: v1.22.4 Server Version: v1.22.4 $kubectl get svc NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE kubernetes   ClusterIP   10.96.0.1                    <none>           443/TCP   5d20h $kubectl get nodes NAME             STATUS   ROLES                       AGE     VERSION docker-desktop   Ready    control-plane,master   3d23h   v1.22.4 $kubectl cluster-info Kubernetes control plane is running at https://kubernetes.docker.internal:6443 CoreDNS is running at https://kubernetes.docker.internal:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. $kubectl cluster-info dump $kubectl apply -f pod.yml pod/hello-pod created $kubectl get pods NAME        READY   STATUS    RESTARTS   AGE hello-pod