Monday 29 October 2018

Virtual Private Cloud (VPC) Notes

Below are the notes I captured while watching the below very good video on VPC


Router and Route table:

  1. Route table holds the info on how to route
  2. Router routes based on the route table

Elastic IP

  1. Static public IP
  2. Each time we stop and start the EC2 instance, we get dynamic IP
  3. Alllows to use the same IP 

Elastic Network Interface

  1. The IP is not directly assigned to EC2
  2. It is the ENI that is attached to EC2
  3. We can have more than one network interface for an EC2
  4. Which means we can have more than on1 IP assigned to EC2

Internet Gateway

  1. It is like a door to VPC for both inbound and outbound  

Customer Gateway, VPN Connection, and Virtual Private Gateway

  1. These 3 together helps in making a connection between on-premise and aws

VPC peering

  1. Communication between 2 VPCs
  2. One should send a request and the other should accept it

VPC endpoint

  1. For eg: S3 endpoints are public endpoints
  2. If we want to our EC2 to talk to S3 privately without going via internet gateway then we go for VPC endpoint
  3. We have a VPC interface endpoint too, for talking to other applications within a subnet

NAT Gateway

  1. Network Address Translation
  2. This is for allowing the private subnet to talk to the internet gateway for accessing the internet
  3. The NAT gateway will not allow any inbound request to reach the private subnet, it will just allow the private subnet to access the internet
  4. Works only for IPV4, for IPV6, we need to use Egress

IP Address and Subnets

  • IPV4 is 32 bit
  • IPV6 is 128 bit
  • IP CIDR Range
    • Eg: 10.0.0.0/16 means the first 16 bits of the IP are not going to change
    • Eg: 10.0.0.0/8 means the first 8 bits of the IP are not going to change
      • The CIDR Range is 10.0.0.0 to 10.255.255.255
    • Eg: 10.0.1.15/32 means only one IP
    • It is not necessary to be the multiple of 8; Eg: 10.0.0.0/26
  • Private IP address range as per RFC1918 standard
    • 10.0.0.0 - 10.255.255.255 (10/8)
    • 172.16.0.0 - 172.31.255.255 (172.16/12)
    • 192.168.0.0 - 192.168.255.255 (192.168/16)

Routing


  • Is based on the Route Table definition
  • It has information on where the request needs to be routed  
  • Need to associate a route table for a particular subnet
  • Route table defines whether the subnet is a private subnet or a public subnet
  • There will be a default route table created in a VPC and that will allow all the local/private access within the VPC

Security Groups

  • Default of all SG is allow all outbound, deny all inbound
  • Need to edit the inbound and outbound connections
  • Applies at instance or individual resource level like EC2, RDS etc
  • This is the first level of defense

Network ACL (Access Control List)

  • Specify what IPs and Port are allowed inbound and what are for outbound
  • Security Group only have allow rules, only the Network ACL we have both allow and deny rules
  • Applied at the network level

Flow

Create VPC > Create Internet Gateway and attach to VPC > Create Subnets > Create RouteTables and attach to VPC > Subnets association > Configure Security Groups

Tuesday 9 October 2018

Setting the Proxy details in different tools

Many times when we are switching our work between the home network and office network, we may face this proxy issues if the office network is under proxy. 

And most of the time, just setting the system level proxy won't be enough, we need to set the proxy at each application/tools level. In this blog, I wanted to document setting up of proxies setting for few of the development tools


1. Maven

For maven, the proxy details need to be mentioned in the setting.xml file. There may be two copies of this settings.xml file one at the global level and one per user account. If we prefer to affect only the current user, then we need to edit the user's copy of setting.xml. The maven is a little intelligent and it will automatically detect and adds the proxy details in the setting.xml but the problem is I haven't seen them removing the proxy details when we are the home network, we need to manually remove it.

In Ubuntu, the setting.xml can be found from below path
/opt/etc/maven/conf/settings.xml
https://maven.apache.org/settings.html

2. Docker

The interesting thing with Docker is, we need to mention the proxy setting in two places. The first one is for the 'docker' command to work in the host environment. Another one is for passing the proxy details to the guest containers.


Proxy for the host:


  • Create the folder

$ sudo mkdir -p /etc/systemd/system/docker.service.d

  • Create the file

/etc/systemd/system/docker.service.d/http-proxy.conf

  • Add the following entry in the file

[Service]Environment="HTTP_PROXY=http://proxy.example.com:80/"

https://docs.docker.com/config/daemon/systemd/#httphttps-proxy


Proxy for the guest containers:

One option for this is to set the environment variables in the Dockerfile. But the recommended approach is to set via the config.json

https://docs.docker.com/network/proxy/


3. NPM

The node package manager does not take the system proxy settings. We need to set it via its npm config command like below


npm config set proxy http://proxy.company.com:8080
npm config set https-proxy http://proxy.company.com:8080
Inorder to unset the proxy details, execute below command
npm config rm proxy 
npm config rm https-proxy
Did you notice, even for https, it expects to provide only the HTTP proxy detail. 

https://jjasonclark.com/how-to-setup-node-behind-web-proxy/

4. Linux Terminal

It use to fetch the system proxy setting automatically; but for some reasons, sometimes it may not fetch from the system proxy settings. In such cases, we can simply export the environment variables and that should work. We should note that once we close the terminal these values will be lost

export http_proxy=http://proxy.company.com:8080 
export https_proxy=https://proxy.company.com:8080