Monday 5 December 2022

First three months in Denver

Landed in the United States on the 03rd of September 2022 and in 3 months, there are a lot of cultural learning, getting used to the people, finding new friends, and much more. And in this post, am sharing some of my experiences and they might help other newcomers like me.

Leaving from India

Some of the pre-plan that we should have before arriving are where we stay, how to carry money from India for expenses, and being connected with the family. If looking for a budget hotel with an option for self-cooking then Extended Stay America, is an excellent choice. It provides an excellent deal of up to 67$ per night if the booking is done for more than 7 nights. Next is having a safe and flexible way to carry US dollars and a good option is to buy a Forex card in India with an easy option to reload it online. The last and most important preparation is always being connected with the family and we can get the US prepaid SIM when we are in India itself at a very affordable plan that includes data as well; actually, I have seen the plan cost is more competitive than the US mobile network providers.

Arriving in the US really means, the immigration officer in the port of entry should be satisfied with all the documents we produce and then issue an electronic i94 document. The i94 document will contain important information like when we arrived and how long we are authorized to stay etc and it can be retrieved from https://i94.cbp.dhs.gov/I94/#/recent-search. In some rare cases, the details in i94 may be entered wrongly and it can be corrected later by filing a ticket at https://help.cbp.gov/s/questions?language=en_US. After this, you are officially in the US and welcome. 

Denver Tech Centre

Beautiful landscape on the side of the road

Public park and waterfronts

After a good night sleep after a long hauling journey, be prepared for the next set of tasks that need to be finished sooner, like vising the SSN office, taking an appointment with a bank, and opening a bank account. One thing that I noticed is everything goes by appointment only and there is no drop in service (including the hair salon! which I will give details about below), so where ever we go, it is advised to take an appointment over the phone and then visit.

When in India, look at the city map and get used to the county and area names. This will help in starting conversations with new people in the city and also will help in shortlisting the location for a long stay either in an apartment or an individual home. Some of the shortlisting criteria are what is the ratings to the school in the county and can be found by looking at https://www.greatschools.org/ other criteria would be the neighborhood like whether it is accessible by the light rail, what supermarkets are nearby, whether it has hospitals closeby etc.  Some of the websites that can help in finding the right home is https://www.apartments.com/ and https://www.zillow.com/. Again, it is advisable to take an appointment with the leasing office for an apartment tour and then visit. One good thing with housing in the US is that as soon as you are moved in, at the same moment you can start cooking the food because by default it comes up with a stove including the electric or a gas connection, and refrigerator.

Moving into the apartment

Coming to the commutations in Denver is a little average if depending only on public transport, however uber and Lyft will be available round the clock. The public transport is managed by RTD and it operated both light rail and the bus, actually if we are well prepared and planned then public transport works in ease and very very cost-effective. During summer, bicycle ride would make the ride pleasant and also saves fossil fuel.

Lots of Indian restaurants and eateries are available and can be easily ordered and get it delivered to our door via https://grubhub.comhttps://www.doordash.com/https://www.ubereats.com/. If wanted to try something outside of Indian cuisine, then Mexican and Mediterranean would be a good try and some of the well-known food chains are https://www.chipotle.com/https://eatfelfel.com/

After moving into an apartment or home, then the next thing would be to fill up the home, and obviously  https://amazon.com/https://www.walmart.com/ etc are starters to get a fair idea and price ranges. But you can go out to showrooms and get the real feel of it like visiting the mattress showroom and trying it out, visiting a furniture shop etc. Also, there are malls where we can peek into multiple different showrooms and get the experience https://www.parkmeadows.com/. In the US, there is no concept of MRP, so the same product might have different prices in a different showroom, so we have to be careful and compare and buy the right deal one.  If you can wait and the time for the BlackFriday sale is not too far, then it is really worth a wait than buying immediately. As heard all the products segment will go on an offer sale and definitely will get the best price of the year. One big difference you would see with India is that all products need to be self-assembled by ourselves if want to save money, the human labor is very expensive and it might cost more than the product cost itself sometimes!!

DIY projects


Assembling furniture


In my first three months, I witnessed three different seasons of the year like the summer, the fall, and the winter. When I came it was so sunny, and the days are lengthy gradually the fall came and enjoyed the view of what it really means a fall color by looking at the different colors of the leaf over the period of 2 weeks. And then the winter started in November with the year's first snow.

Fall Color


Road, Car covered with snow


Definitely, after a few weeks, we need to go to a hair salon and one option is https://www.greatclips.com/ which provides services to men, women, and kids. As mentioned earlier, take an appointment and then visit. Sometimes during the Black Friday sale, they sell coupons at a discounted price which can be used during the rest of the year. 

If you are an avid reader and a book lover then you are covered, each county has a library with a very good collection of books for all ages, the subscription is free and we can borrow books and return them. Also, it houses beautiful reading rooms with great ambiance etc. More details about the library where I have a subscription https://www.dcl.org/

Tuesday 13 September 2022

K8S Service Account verification using REST API

Service Account in K8S is used to invoke all the k8s admin server APIs within the POD provided the service account is granted with necessary RBAC permissions via role-binding or cluster-role-binding

The recommended way to invoke the K8S API from the POD is to use the official client libraries. However if we want to make a quick testing of service account configuration then we can use the K8S REST APIs to invoke and check whether the POD is having sufficient privilage to access the resources or not.

Before going to steps of invoking the REST API, few lines about how the POD gets that privilage. When a deployment/pod is binded with a service account, the POD will get the certficatetoken and namespace in the location /var/run/secrets/kubernetes.io/serviceaccount

$ kubectl exec backend-api-b874f697f-fgqk7 -c backend-api -n vemohanr -- cat /var/run/secrets/kubernetes.io/serviceaccount/token

On executing the above command we would get the token I we can use to invoke the REST API, and now we can use the above token and form the curl command

$ kubectl exec backend-api-b874f697f-fgqk7 -c backend-api -n vemohanr -- curl -k -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6Ikd6VFI0R3g2RW9tckNwbnhMOE5oWDlLc2ZEMVRUZU1qVExXbktkVlh2YW8ifQ.eyJhdWQiOlsiYXBpIl0sImV4cCI6MTY5NDU2MjE3MiwiaWF0IjoxNjYzMDI2MTcyLCJpc3MiOiJodHRwczovL2t1YmVybmV0ZXMuZGVmYXVsdC5zdmMuY2x1c3Rlci5sb2NhbCIsImt1YmVybmV0ZXMuaW8iOnsibmFtZXNwYWNlIjoidmVtb2hhbnIiLCJwb2QiOnsibmFtZSI6ImljLWFwaS1pbnRnLWR0LWI4NzRmNjk3Zi1mZ3FrNyIsInVpZCI6ImI4OWE4MGJlLWYwZGUtNGFlZi1iODBiLWRlYWQ3ZDc2YzVhMSJ9LCJzZXJ2aWNlYWNjb3VudCI6eyJuYW1lIjoiYXBwcy1zYSIsInVpZCI6ImFmZjRiNGU5LTY4ZTEtNDY0ZS1iZGJhLTQxMTYzOTIxMWM1NyJ9LCJ3YXJuYWZ0ZXIiOjE2NjMwMjk3Nzl9LCJuYmYiOjE2NjMwMjYxNzIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDp2ZW1vaGFucjphcHBzLXNhIn0.Lp_FOzcCz19KLwTm4qU_VHOjqpM9M6wSpPAfGWZQQyjFz276xLBEYU22dwaFcuHkOXw83S1xy0rWAAhgvkaWFmpLwuC9GxlMh583XJo1b1GC-BIei_EgzdTrD3TFOtQ9CCTC4Jf0FWmmY5Uz5ng5xglLbw7220YsRIG9NIj1PkfBfVJCVrezE-wXyNb4jkr86wlNz3uKhYw8FdIffUuOyXpNfTt1IyOkGnQGtow_E3F5asqO7ZlaB6DjUJBZhwgP90SqAmpVyu10hTELNLchV-NeTtyQJEHIbLqZj64wJb3SrBqenHft_g_2SRivoMzDoDQUFtk_N3HHNdpHDfEC7A" https://147.154.106.173:6443/apis/apps/v1/namespaces/vemohanr/cronjobs

If want to know what is the K8S admin API server IP etc, execute the below command to get it

$ kubectl cluster-info
Kubernetes master is running at https://147.154.106.173:6443
CoreDNS is running at https://147.154.106.173:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
kubedashboard-kubernetes-dashboard is running at https://147.154.106.173:6443/api/v1/namespaces/kube-system/services/https:kubedashboard-kubernetes-dashboard:https/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

References

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.25/#-strong-api-overview-strong-

https://www.ibm.com/docs/en/cloud-paks/cp-management/2.0.0?topic=kubectl-using-service-account-tokens-connect-api-server

Accessing K8S API from POD

Wednesday 27 July 2022

XMing - Alternative to VNC server

Ever wondered how to open the UI/GUI of the applications installed in a remote Linux server in our local development environment. Usecases like codebase is in remote server and we SSH into it to build the code etc and instead of using VNC to get the graphical interface, we can use this technique to open the applications installed in the remote server

Steps

  1. Install XMing server https://sourceforge.net/projects/xming/ in the local machine

  2. Enable X11 forwarding in putty or other SSH tool

Enable X11

  1. SSH into the remote server

  2. Launch the application and we should see the application opening in local machine

Monday 18 April 2022

CKA Kubernetes ( K8S ) Networking

 The preparation for the CKA (Certified Kubernetes Administrator) requires lots of practice and practice. Fortunately, we have lots of online playgrounds to keep practicing, there are lots of free courseware available and lots of paid as well are available. In addition to that, we get two attempts to clear the exam




Switching and Routing

• Switching enables communication between hosts within the network
• Commands to enable
○ ip link
○ ip addr add 192.168.1.10/24 dev eth0
• Router connected two networks together
○ Command to know the routing table
§ route
○ Command to add a routing
§ ip route add 192.168.2.0/24 via 192.168.1.1
§ ip route add default 192.168.1.1
• For forwarding traffic from one eth0 to another eth1
○ cat /proc/sys/net/ipv4/ip_forward
○ modify the file /etc/sysctl.conf

DNS

• /etc/hosts
• Each node can have its own domain-name mapping in /etc/hosts, however soon it will become cumbersome thats why we use DNS host
• The location of DNS host is defined in /etc/resolv.conf
• If the are the two definition of host name available in local /etc/hosts as well as in DNS then the selection will be based /etc/ns-switch.conf
• Within our nameserver, we can point to public nameserver ex: 8.8.8.8 nameserver which is hosted by Google
• There is 'search' property we can set in /etc/resolv.conf so the nameresolver intelligently appends the search text with the user provided name
• coreDNS is one implementation of DNS Server which is opensource

Network Namespaces

• Namespaces are used by containers like docker to create a network isolation
• Command to create namespaces
○ ip netns add green
• To list down all the interfaces in the host
○ ip link
• To list down all the interfaces visible inside the network namspace
○ ip netns exec red ip link

Docker networking

• There are different networking type when we run a docker container
○ none - the container cannot reach the container and no one can reach the container
○ host - uses the host's IP as its own ip
○ bridge - an internal private network is created
• Command
○ docker run --network <type> nginx
○ docker network ls
§ lists down the network
○ docker inspect <container_id>
§ Under the NetworkSettings we can see what is the network namesapce the container is using

Kubernetes networking

• The CNI (Container Network Interface) is a plugin used by k8s to establish a network connectivity 
• As per K8S reqt, CNI should assign each pod with a IP address and each of them should be accessible by other POD and it should reach all other PODs
• There are many flavors of CNI plugin
○ bridge
○ flannel
○ weave-net 
○ ipvlan
○ ...
• kubelet will point to the CNI configuration file while bringing up the kubelet in the pod
• 'ps -aux | grep kubelet' will show the path to the configuration file
• ipam - ip Address Management
○ It is the plugin implementer responsibility to manage the IP range, avoid duplicate assignment of IP to the pod etc
○ Two types
§ dhcp
§ host-local 

Service Networking

• In general we use service for accessing the pod instead of accessing the pod directly
• When a service is created it is accessible cluster wide by default
○ ClusterIP - accessible within the cluster
○ NodePort - Will be accessible by the nodeIPs with the port 
• kube-proxy watches the changes in kube-apiserver for any new service creation and it will take action
• Service is a cluster wide concept and really there is no service or process running that listens to the IP. It is just a virtual object
• kube-proxy creates a forward routing rules and gets the IP within the range what is configured 
• Three ways of configuring the forward routing rules (--proxy-mode param needs to be set while bringing up the kube-proxy)
○ userspace
○ iptables
○ ipvs
• The service ip range are set while bringing up the kube-api-server using the --service-cluster-ip-range parameter
• We can check the iptables in the node
○ iptables -L -t net | grep db-service

DNS in Kubernetes

• DNS runs as a service and pod in k8x under kune-system namespace
• The DNS nameserver ip will be configure to pod's /etc/resolve.conf by the kubelet when the pod is started
• Each wont get a hostname by default but it can be configured in the CoreDNS configuration in the kube-dns pod

Ingress

• Ingress takes care of
○ Loadbalancing
○ Authentication SSL
○ URL based routing configuration
• It acts like a layer 7 load balancer
• Ingress Controller - There are many implementation and by default we will not have one running in the k8s
○ Nginx
○ HA Proxy
○ Contour
○ traefik

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-controller
  namespace: ingress-space
spec:
  replicas: 1
  selector:
    matchLabels:
      name: nginx-ingress
  template:
    metadata:
      labels:
        name: nginx-ingress
    spec:
      serviceAccountName: ingress-serviceaccount
      containers:
        - name: nginx-ingress-controller
          image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.21.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --default-backend-service=app-space/default-http-backend
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
            - name: https
              containerPort: 443
• Ingress Resource - is the set of configuration passed on to the ingress controller to route the traffic appropriately

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
  name: ingress-pay
  namespace: critical-space
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: pay-service
          servicePort: 8282
        path: /pay(/|$)(.*)
        pathType: ImplementationSpecific

CKA Kubernetes ( K8S ) Storage

 The preparation for the CKA (Certified Kubernetes Administrator) requires lots of practice and practice. Fortunately, we have lots of online playgrounds to keep practicing, there are lots of free courseware available and lots of paid as well are available. In addition to that, we get two attempts to clear the exam



Docker Storage Driver

• When we run the docker container, the docker creates another layer called as 'Container Layer' as a writable layer to store content like logs, temp file created by app or to modify the existing file. The files in the container layer will be lost when the container stops
• Commands
○ docker volume create data_volume
§ this will create a directory under /var/lib/docker/data_volume
○ docker run -v data_volume:/var/lib/mysql mysql
§ Here it mounts the volume to the 
○ docker run -v data_volume2:/var/lib/docker/data_volume2
§ It creates a folder /var/lib/docker/data_volume2
○ docker run -v /data/mysql:/var/lib/mysql mysql
§ Local folder is mounted
○ docker run --mount type=bind,source=/data/mysql,target=/var/lib/mysql mysql
• Docker uses storage driver for
○ Creating the writable layered and maintaining the files in it and terminating when the container stops etc
○ There are many drivers like AUFS, ZFS, BTRFS etc
○ Docker itself will choose the best driver based on the native operating system. But we can override it

Docker Volume Driver

• Default driver is 'Local' which will use the host OS filesystem
• There are many other drivers like
○ AzureFileStorage
○ DigitalOcean
○ gce-docker
○ convoy etc

Container Interfaces

• Container Runtime Interfaces
○ Used to abstract the runtime containers like docker, rkt, cri-o etc
○ If any new runtime container support is introduced they simply have to follow the CRI docs and can implement without touching the k8s code
• Container Network Interfaces
○ Used to abstract the networking implementation used to support communication between nodes, pods etc
○ Some examples are flannel, weaveworks, cilium
• Container Storage Interfaces
○ Used to abstract the underlying storage used by using drivers like portworx, Amazon EBS, Dell EMC, Gluster FS
○ CSI is not K8S standard, it is universal standard. So if any storage vendor has the contract for CSI then it can be plugged

Volumes and Mounts

• When we create a POD, under the spec we can define the list of volumes under spec and the mounts under the containers
• There are multiple volume providers like the 'hostPath' which creates a volume in current running node. Apart from that there are many providers for it

PersistentVolume

• It allows administrator to define a different set of storage options using the persistent volume and the POD can use one of them. This gives the advantage that now each pod definition dont have to maintain all the storage configuration within itself

apiVersion: v1
kind: PersistentVolume
metadata:
  name:  pv-log
spec:
  capacity:
    storage: 100Mi
  accessModes:
    - ReadWriteMany
  hostPath:
    path: /pv/log

PersistentVolumeClaim

• PVC is another k8s object created by the user with definition like requires storage size, mode etc
• Once the PVC is created the by the user, the kubernetes binds the PVC with PV
• PVC and PV are 1-1 means like only one claim can be made to a PV. Even if there are free space in PV, it cannot accommodate additional PVC 

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: claim-log-1
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 50Mi
  volumeName: pv-log

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: webapp
  name: webapp
spec:
  containers:
  - image: kodekloud/event-simulator
    name: pod
    resources: {}
    volumeMounts:
    - mountPath: /log
      name: log-pvc
  volumes:
  - name: log-volume
    hostPath:
     path: /var/log/webapp
  - name: log-pvc
    persistentVolumeClaim:
      claimName: claim-log-1
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
# Storage Class
• Creating a PV and creating a storage type like aws, gce are called as static provisioning

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

apiVersion: v1
kind: PersistentVolume
metadata:
  name:  local-pv
spec:
  capacity:
    storage: 500Mi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: /opt/vol1
  storageClassName: local-storage


apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: local-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 500Mi
  storageClassName: local-storage

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx:alpine
    name: nginx
    resources: {}
    volumeMounts:
      - mountPath: "/var/www/html"
        name: volume-html
  volumes:
    - name: volume-html
      persistentVolumeClaim:
        claimName: local-pvc
  dnsPolicy: ClusterFirst
  restartPolicy: Always