Kubernetes Basics - Part III

Info

This workshop segment expect you to be inside a cloned repository of k8s-specs.

1
2
git clone https://github.com/vfarcic/k8s-specs.git
cd k8s-specs

Points to cover

  • Persisting State
  • Deploying Stateful Applications At Scale

Persisting State

Without state preservation

1
cat pv/jenkins-no-pv.yml
1
kubectl create -f pv/jenkins-no-pv.yml --record --save-config
1
kubectl -n jenkins get events
1
2
3
kubectl -n jenkins create secret generic jenkins-creds \
    --from-literal=jenkins-user=jdoe \
    --from-literal=jenkins-pass=incognito
1
kubectl -n jenkins rollout status deployment jenkins

Retrieve Jenkins Address

1
2
JENKINS_ADDR=$(kubectl -n jenkins get ing jenkins \
    -o jsonpath="{.status.loadBalancer.ingress[0].ip}")
1
2
JENKINS_ADDR=$(kubectl -n jenkins get ing jenkins \
    -o jsonpath="{.status.loadBalancer.ingress[0].hostname}")
1
JENKINS_ADDR=$(minikube ip)

Log into Jenkins

Warning

If you're on windows, open probably doesn't work. Copy paste the url printed and open that in a browser.

1
2
echo $JENKINS_ADDR
open "http://$JENKINS_ADDR/jenkins"
  • use jdoe as username and incognito as password
  • create a job, doesn't matter what

Kill the pod

1
kubectl -n jenkins get pods --selector=app=jenkins -o json
1
2
POD_NAME=$(kubectl -n jenkins get pods --selector=app=jenkins \
    -o jsonpath="{.items[*].metadata.name}")
1
echo $POD_NAME
1
kubectl -n jenkins exec -it $POD_NAME pkill java
1
2
echo "http://$JENKINS_ADDR/jenkins"
open "http://$JENKINS_ADDR/jenkins"

Create Volume

GKE

1
CLUSTER_NAME=
1
2
gcloud compute instances list --filter="name:('${CLUSTER_NAME}')" \
    --format 'csv[no-heading](zone)' | tee zones
1
AZ_1=$(cat zones | head -n 1)
1
AZ_2=$(cat zones | tail -n 2 | head -n 1)
1
AZ_3=$(cat zones | tail -n 1)

Replace ??? with your name to make sure the disk name is unique.

1
PREFIX=???
1
gcloud compute disks create ${PREFIX}-disk1 --zone $AZ_1
1
gcloud compute disks create ${PREFIX}-disk2 --zone $AZ_2
1
gcloud compute disks create ${PREFIX}-disk3 --zone $AZ_3

Warning

Later commands will depend on these variables. So stay in the same console session or make sure you recreate these!

1
2
3
VOLUME_ID_1=${PREFIX}-disk1
VOLUME_ID_2=${PREFIX}-disk2
VOLUME_ID_3=${PREFIX}-disk3
1
gcloud compute disks describe VOLUME_ID_1

EKS

...

Create Persistent Volume

GKE

1
2
YAML=pv/pv-gke.yml
cat $YAML
1
2
3
4
5
cat $YAML \
    | sed -e "s@REPLACE_ME_1@$VOLUME_ID_1@g" \
    | sed -e "s@REPLACE_ME_2@$VOLUME_ID_2@g" \
    | sed -e "s@REPLACE_ME_3@$VOLUME_ID_3@g" \
    | kubectl create -f - --save-config --record
1
kubectl get pv

Claim Persistent Volume

1
cat pv/pvc.yml
1
kubectl create -f pv/pvc.yml --save-config --record
1
kubectl -n jenkins get pvc
1
kubectl get pv

Attach PVC

1
cat pv/jenkins-pv.yml
1
kubectl apply -f pv/jenkins-pv.yml --record
1
kubectl -n jenkins rollout status deployment jenkins

Demonstrate the persistent part

  • open Jenkins open "http://$JENKINS_ADDR/jenkins"
  • create a job
1
2
POD_NAME=$(kubectl -n jenkins get pod --selector=app=jenkins \
    -o jsonpath="{.items[*].metadata.name}")
1
kubectl -n jenkins exec -it $POD_NAME pkill java

Let's delete Jenkins' deployment.

1
kubectl -n jenkins delete deploy jenkins

Confirm it is gone.

1
kubectl get all -n jenkins

Now, let's recreate Jenkins.

1
kubectl apply -f pv/jenkins-pv.yml --record
1
kubectl -n jenkins rollout status deployment jenkins
  • confirm job is still there
  • open Jenkins open "http://$JENKINS_ADDR/jenkins"
1
kubectl -n jenkins get pvc
1
kubectl get pv

Cleanup

1
2
3
kubectl -n jenkins delete pvc jenkins
kubectl get pv
kubectl delete -f pv/pv.yml
1
2
3
gcloud compute disks delete $VOLUME_ID_1 --zone $AZ_1 --quiet
gcloud compute disks delete $VOLUME_ID_2 --zone $AZ_2 --quiet
gcloud compute disks delete $VOLUME_ID_3 --zone $AZ_3 --quiet
1
2
3
aws ec2 delete-volume --volume-id $VOLUME_ID_1
aws ec2 delete-volume --volume-id $VOLUME_ID_2
aws ec2 delete-volume --volume-id $VOLUME_ID_3

Storage Classes

View

1
kubectl get sc
1
cat pv/jenkins-dynamic-gke.yml
1
cat pv/jenkins-dynamic.yml

Create

1
kubectl apply -f pv/jenkins-dynamic-gke.yml --record
1
kubectl apply -f pv/jenkins-dynamic.yml --record
1
kubectl -n jenkins rollout status deployment jenkins

Use

1
2
3
4
5
kubectl -n jenkins get events

kubectl -n jenkins get pvc

kubectl get pv
1
2
PV_NAME=$(kubectl get pv -o jsonpath="{.items[0].metadata.name}")
gcloud compute disks list --filter="name:('$PV_NAME')"
1
2
aws ec2 describe-volumes \
    --filters 'Name=tag-key,Values="kubernetes.io/created-for/pvc/name"'

Use 2

1
2
3
kubectl -n jenkins delete deploy,pvc jenkins

kubectl get pv
1
gcloud compute disks list --filter="name:('$PV_NAME')"
1
2
aws ec2 describe-volumes \
    --filters 'Name=tag-key,Values="kubernetes.io/created-for/pvc/name"'

Use Default

1
kubectl get sc
1
kubectl describe sc
1
cat pv/jenkins-default.yml
1
diff pv/jenkins-dynamic.yml pv/jenkins-default.yml
1
kubectl apply -f pv/jenkins-default.yml --record
1
kubectl get pv

Prepare alternative SC

1
kubectl -n jenkins delete deploy,pvc jenkins
1
YAML=sc-gke.yml
1
YAML=sc.yml

Create alternative SC

1
cat pv/$YAML
1
kubectl create -f pv/$YAML
1
kubectl get sc

Use alternative SC

1
2
cat pv/jenkins-sc.yml
kubectl apply -f pv/jenkins-sc.yml --record
1
2
aws ec2 describe-volumes \
    --filters 'Name=tag-key,Values="kubernetes.io/created-for/pvc/name"'
1
2
PV_NAME=$(kubectl get pv -o jsonpath="{.items[0].metadata.name}")
gcloud compute disks list --filter="name:('$PV_NAME')"

Cleanup

1
kubectl delete ns jenkins
1
kubectl delete sc fast

StatefulSet

Create StatefulSwet

1
cat sts/jenkins.yml
1
kubectl apply -f sts/jenkins.yml --record
1
kubectl -n jenkins rollout status sts jenkins
1
kubectl -n jenkins get pvc
1
kubectl -n jenkins get pv

Get Jenkins Address

1
2
JENKINS_ADDR=$(kubectl -n jenkins get ing jenkins \
    -o jsonpath="{.status.loadBalancer.ingress[0].ip}")
1
2
JENKINS_ADDR=$(kubectl -n jenkins get ing jenkins \
    -o jsonpath="{.status.loadBalancer.ingress[0].hostname}")

Use Jenkins

1
open "http://$JENKINS_ADDR/jenkins"
1
kubectl delete ns jenkins

Use DB without STS

1
cat sts/go-demo-3-deploy.yml
1
kubectl apply -f sts/go-demo-3-deploy.yml --record
1
kubectl -n go-demo-3 rollout status deployment api
1
kubectl -n go-demo-3 get pods
1
2
DB_1=$(kubectl -n go-demo-3 get pods -l app=db \
    -o jsonpath="{.items[0].metadata.name}")
1
2
DB_2=$(kubectl -n go-demo-3 get pods -l app=db \
    -o jsonpath="{.items[1].metadata.name}")

Investigate problems

1
kubectl -n go-demo-3 logs $DB_1
1
kubectl -n go-demo-3 logs $DB_2
1
kubectl get pv
1
kubectl delete ns go-demo-3

Use DB with STS

1
cat sts/go-demo-3-sts.yml
1
kubectl apply -f sts/go-demo-3-sts.yml --record
1
kubectl -n go-demo-3 get pods
1
kubectl get pv

Configure Mongo

If we want MongoDB to use the three instances as single dataplane, we have to tell it the dataplane members.

First, we shell into one of the db containers via exec.

1
kubectl -n go-demo-3 exec -it db-0 -- sh

Then, make sure we're talking with MongoDB, via its REPL, with mongo.

1
mongo

And finally we explain MongoDB what we want to do.

1
2
3
4
5
6
7
8
rs.initiate( {
   _id : "rs0",
   members: [
      {_id: 0, host: "db-0.db:27017"},
      {_id: 1, host: "db-1.db:27017"},
      {_id: 2, host: "db-2.db:27017"}
   ]
})

Let's confirm with MongoDB before we exit the container.

1
rs.status()

And now you're free to exit the MongoDB REPL and the container via ctrl+d (so twice).

1
kubectl -n go-demo-3 get pods

Observe update process

1
diff sts/go-demo-3-sts.yml sts/go-demo-3-sts-upd.yml
1
kubectl apply -f sts/go-demo-3-sts-upd.yml --record
1
kubectl -n go-demo-3 get pods

Cleanup

1
kubectl delete ns go-demo-3