02 Kubernetes Service Definition

Kubernetes service is an abstraction above the pods, the pods cannot be accessed/exposed outside the cluster. Inorder to expose the pods outside the cluster, we need to define service.

Service Definition yaml

https://raw.githubusercontent.com/venkatesh-mohanram/microservices/master/01-microservices-starter-kit/add-service/k8s-service.yaml

The selector tells which deployment needs to be abstracted. The port specifies the service port and the targetPort specify the port in which the deployment is exposed. 

Use the below command to execute the service definition

$kubectl apply -f k8s-service.yaml

We can use the below command to see the clusterIP in which the service is made available. The default IP is clusterIP, this IP will be accessible within the cluster.

$kubectl get service

In order to test the service, we can run the CURL command with the service IP

$curl http://10.107.33.6/addservice/application.wadl

We can do a roling update of containers inside the pod, any rolling update on the pod will not affect the service IP and we can still use the same IP.

01 Kubernetes Deployment Definition

Let's begin with writing a 'deployment.yaml' file, we can directly write yaml for a pod but that is not recommended. In the deployment yaml, we can specify the pod template and even the replication factor

Deployment Definition Yaml

https://raw.githubusercontent.com/venkatesh-mohanram/microservices/master/01-microservices-starter-kit/add-service/k8s-deployment.yaml

Here the "Kind:" specifies what kind of k8s resource it is, in this case it is 'Deployment'. It has the metadata and the spec information inside the definition document. 

Applying the deployment

Execute the deployment yaml using the below command

$ kubectl apply -f k8s-deployment.yaml

Upon executing, we should see something like below

Monitoring container logs

$kubectl logs -f add-service-deployment-59996d5779-5q8kj

We can use above log command to see the container logs inside the particular pod.

Rolling update

$ kubectl set image deployments/add-service-deployment add-service=venkateshm/add-service:3

For rolling updates, we can execute the above commands to recreate the pods with new version of the container. The old pods will be gracefully terminated while the new one is gradually created.

Follow on https://venkateshbook.blogspot.com/2019/05/02-kubernetes-service-definition.html for kubernetes service definition

Kubernetes Commands

There are various basic commands which are helpful to begin. Along with that, it will be good, if we know the hierarchy of different resources in Kubernetes

Let's start from the micro level of K8S resources

1. POD

The POD is the logical group of containers. The containers will not directly run inside the node instead, it will be running inside a POD. It is always a good idea to group tightly coupled containers in a single POD along with any shared storage volumes if needed. Each POD in the node will have its unique IP.

Useful commands

$ kubectl get pods 

$ kubectl describe pods [pod-name]

$ kubectl describe pods --help 

$ kubectl logs [pod-name]

$ kubectl delete pod <>pod-name 

$ kubectl delete pods -all

The usual structure of all the K8S command is kubectl <action> <resource>

The very interesting command which I like most is 

$kubectl exec -ti <pod_name> bash

The above command actually takes us inside the POD and we can execute the commands inside it.

Deployment Vs PODS:

We can think of deployment as a template and the PODs are the actual deployments. In java language, the deployment is a class and PODS are objects, the replicas of deployment will create those many numbers of pods.

$ kubectl get deployments 

$ kubectl scale deployments/<deployment-name> --replicas=4 

$ kubectl describe deployment

2. Node

The node could be a VM or a real machine in K8S, the collection of nodes form a cluster. The K8S master monitors PODs and Nodes health and recreates the PODs in another node if the if one of the nodes is not healthy.

$ kubectl get nodes 

$ kubectl describe nodes [nodename]

3. Services

Is an abstraction which defines a logical set of PODs. It enables loose coupling between pods, it is preferred to be defined using YAML or JSON. The PODS are actually accessible only inside the cluster using their unique IP, if we want to expose the PODs outside then we need to define services. Also the service acts like a load balancer and distributes the load across multiple replicas.

There are different ways in which a service can be exposed in different ways by specifying the type

1. Cluster IP - Using the default cluster IP, accessible only inside the cluster
2. NodePort - Can be accessed outside by using the node IP and the port number specifies in NodePort
3. LoadBalancer - In exposed a public IP if the cloud provider supports
4. External name - using a CNAME

$ kubectl get services 

$ kubectl describe services/<service name> 

$ kubectl delete service

Other general commands

To get the cluster information

$ kubectl cluster-info

Rolling updates

$ kubectl rollout

If we have the definitions in yaml file then 

$ kubectl apply -f <loc of yaml>

Below link are good to get started with learning K8S

https://www.katacoda.com/courses/kubernetes/playground

https://kubernetes.io/docs/tutorials/kubernetes-basics/

In the next blog, I will mention sample yaml file all the resources

FONS0004: no namespace found for prefix

Are you seeing the below exception stack while executing the countNodes() function,

FONS0004: no namespace found for prefix...
        at oracle.xml.xpath.JXPathExpression.evaluate(JXPathExpression.java:269)
        at oracle.xml.xpath.JXPath.evaluate(JXPath.java:383)
        at com.collaxa.cube.xml.xpath.BPELXPathUtil.selectNodes(BPELXPathUtil.java:533)
        at com.collaxa.cube.xml.xpath.BPELXPathUtil.selectNodes(BPELXPathUtil.java:517)
        at com.collaxa.cube.xml.xpath.functions.xml.CountNodesFunction.evaluate(CountNodesFunction.java:143)
        at com.collaxa.cube.xml.xpath.functions.xml.CountNodesFunction.call(CountNodesFunction.java:78)
        at com.collaxa.cube.xml.xpath.BPELXPathFunctionWrapper.evaluate(BPELXPathFunctionWrapper.java:80)
        at oracle.xml.xpath.JXPathContext$JXFunction.invoke(JXPathContext.java:213)
        at oracle.xml.xpath.JXPathContext$JXFunction.invoke(JXPathContext.java:182)
        at oracle.xml.xpath.XPathExtFunction.evaluate(XPathExtFunction.java:335)

Then you need to check whether you are using the namespace defined in the root element for your xsi:type extension usage,

eg;

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:typ="http://xmlns.oracle.com/cloud/adapter/erp/FindItem_REQUEST/types" xmlns:typ1="http://xmlns.oracle.com/adf/svc/types/">
   <soapenv:Body>
      <typ:findItem>
         <typ:findCriteria xsi:type="typ1:ChildFindCriteria" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <typ1:fetchStart>10</typ1:fetchStart>
         </typ:findCriteria>
      </typ:findItem>
   </soapenv:Body>
</soapenv:Envelope>

Instead, add another namespace definition in <findCriteria>

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:typ="http://xmlns.oracle.com/cloud/adapter/erp/FindItem_REQUEST/types" xmlns:typ1="http://xmlns.oracle.com/adf/svc/types/">
   <soapenv:Body>
      <typ:findItem>
         <typ:findCriteria xsi:type="typ3:ChildFindCriteria" xmlns:typ3="http://xmlns.oracle.com/adf/svc/types/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            <typ1:fetchStart>10</typ1:fetchStart>
         </typ:findCriteria>
      </typ:findItem>
   </soapenv:Body>
</soapenv:Envelope>


Interesting Material on Authentication

As the ideology is going towards a microservices and making each service stateless; even the session based authentication is fading away and the token-based authentication is getting more popularity.

With token-based authentication, we never have to store the session details of each user in the server that makes the scalability of servers so beautiful.

More details in below link

https://scotch.io/tutorials/authenticate-a-node-js-api-with-json-web-tokens

https://dzone.com/articles/cookies-vs-tokens-the-definitive-guide

Below link talks about the hashing of passwords and how adding a salt to the password before hashing makes hacking difficult

https://www.wordfence.com/learn/how-passwords-work-and-cracking-passwords/

Virtual Private Cloud (VPC) Notes

Below are the notes I captured while watching the below very good video on VPC

https://www.youtube.com/watch?v=LX5lHYGFcnA

Router and Route table:

1. Route table holds the info on how to route
2. Router routes based on the route table

Elastic IP

1. Static public IP
2. Each time we stop and start the EC2 instance, we get dynamic IP
3. Alllows to use the same IP 

Elastic Network Interface

1. The IP is not directly assigned to EC2
2. It is the ENI that is attached to EC2
3. We can have more than one network interface for an EC2
4. Which means we can have more than on1 IP assigned to EC2

Internet Gateway

1. It is like a door to VPC for both inbound and outbound  

Customer Gateway, VPN Connection, and Virtual Private Gateway

1. These 3 together helps in making a connection between on-premise and aws

VPC peering

1. Communication between 2 VPCs
2. One should send a request and the other should accept it

VPC endpoint

1. For eg: S3 endpoints are public endpoints
2. If we want to our EC2 to talk to S3 privately without going via internet gateway then we go for VPC endpoint
3. We have a VPC interface endpoint too, for talking to other applications within a subnet

NAT Gateway

1. Network Address Translation
2. This is for allowing the private subnet to talk to the internet gateway for accessing the internet
3. The NAT gateway will not allow any inbound request to reach the private subnet, it will just allow the private subnet to access the internet
4. Works only for IPV4, for IPV6, we need to use Egress

IP Address and Subnets

• IPV4 is 32 bit
• IPV6 is 128 bit
• IP CIDR Range
• Eg: 10.0.0.0/16 means the first 16 bits of the IP are not going to change
• Eg: 10.0.0.0/8 means the first 8 bits of the IP are not going to change
• The CIDR Range is 10.0.0.0 to 10.255.255.255
• Eg: 10.0.1.15/32 means only one IP
• It is not necessary to be the multiple of 8; Eg: 10.0.0.0/26
• Private IP address range as per RFC1918 standard
• 10.0.0.0 - 10.255.255.255 (10/8)
• 172.16.0.0 - 172.31.255.255 (172.16/12)
• 192.168.0.0 - 192.168.255.255 (192.168/16)

Routing

• Is based on the Route Table definition
• It has information on where the request needs to be routed  
• Need to associate a route table for a particular subnet
• Route table defines whether the subnet is a private subnet or a public subnet
• There will be a default route table created in a VPC and that will allow all the local/private access within the VPC

Security Groups

• Default of all SG is allow all outbound, deny all inbound
• Need to edit the inbound and outbound connections
• Applies at instance or individual resource level like EC2, RDS etc
• This is the first level of defense

Network ACL (Access Control List)

• Specify what IPs and Port are allowed inbound and what are for outbound
• Security Group only have allow rules, only the Network ACL we have both allow and deny rules
• Applied at the network level

Flow

Create VPC > Create Internet Gateway and attach to VPC > Create Subnets > Create RouteTables and attach to VPC > Subnets association > Configure Security Groups

Setting the Proxy details in different tools

Many times when we are switching our work between the home network and office network, we may face this proxy issues if the office network is under proxy. 

And most of the time, just setting the system level proxy won't be enough, we need to set the proxy at each application/tools level. In this blog, I wanted to document setting up of proxies setting for few of the development tools

1. Maven

For maven, the proxy details need to be mentioned in the setting.xml file. There may be two copies of this settings.xml file one at the global level and one per user account. If we prefer to affect only the current user, then we need to edit the user's copy of setting.xml. The maven is a little intelligent and it will automatically detect and adds the proxy details in the setting.xml but the problem is I haven't seen them removing the proxy details when we are the home network, we need to manually remove it.

In Ubuntu, the setting.xml can be found from below path

/opt/etc/maven/conf/settings.xml

https://maven.apache.org/settings.html

2. Docker

The interesting thing with Docker is, we need to mention the proxy setting in two places. The first one is for the 'docker' command to work in the host environment. Another one is for passing the proxy details to the guest containers.

Proxy for the host:

• Create the folder

$ sudo mkdir -p /etc/systemd/system/docker.service.d

• Create the file

/etc/systemd/system/docker.service.d/http-proxy.conf

• Add the following entry in the file

[Service]Environment="HTTP_PROXY=http://proxy.example.com:80/"

https://docs.docker.com/config/daemon/systemd/#httphttps-proxy

Proxy for the guest containers:

One option for this is to set the environment variables in the Dockerfile. But the recommended approach is to set via the config.json

https://docs.docker.com/network/proxy/

3. NPM

The node package manager does not take the system proxy settings. We need to set it via its npm config command like below

npm config set proxy http://proxy.company.com:8080

npm config set https-proxy http://proxy.company.com:8080

Inorder to unset the proxy details, execute below command

npm config rm proxy 

npm config rm https-proxy

Did you notice, even for https, it expects to provide only the HTTP proxy detail. 

https://jjasonclark.com/how-to-setup-node-behind-web-proxy/

4. Linux Terminal

It use to fetch the system proxy setting automatically; but for some reasons, sometimes it may not fetch from the system proxy settings. In such cases, we can simply export the environment variables and that should work. We should note that once we close the terminal these values will be lost

export http_proxy=http://proxy.company.com:8080 

export https_proxy=https://proxy.company.com:8080