Setting up a isolated development environment

Very often whenever we wanted to do a POC or a new project which depends on many different tools. frameworks or software, we prefer to start it with an isolated development environment without affecting our other tasks. The good option for that is to use a virtual OS running on top of our host OS, below steps helps to get started with setting up such an environment 

Steps to set up a working environment in a Virtual box

1. Install Oracle VM Virtual Box in host machine
https://www.virtualbox.org/wiki/Downloads

2. Download the preferred VDI OS image from the below link (eg: Ubuntu 16.04)
https://www.osboxes.org/virtualbox-images/

3. Create a new Virtual Machine in Virtual Box from the downloaded VDI file

4. Setting the bridged network
https://www.youtube.com/watch?v=cDF4X7RmV4Q&t=865s

5. Installing the java and maven
https://www.vultr.com/docs/how-to-install-apache-maven-on-ubuntu-16-04

6. Installing Eclipse
https://www.youtube.com/watch?time_continue=391&v=EiGl2Ag35AI

7. Installing MySQL
https://www.digitalocean.com/community/tutorials/how-to-install-mysql-on-ubuntu-16-04

8. Installing Docker
https://www.vultr.com/docs/installing-docker-ce-on-ubuntu-16-04

9. Installing docker-compose
https://docs.docker.com/compose/install/

10. Installing AWS CLI from the bundle
https://docs.aws.amazon.com/cli/latest/userguide/awscli-install-bundle.html

11. Installing Php 5.6 or 7.0
https://tecadmin.net/install-php5-on-ubuntu/

Linux commands and shell scripts to play with files

Example file lists

88968-4568-8765_EMPLOYEE_LOADER_02.00.0000_ahiutrfjg.zip
67808-6878-6905_EMAIL_QUEUE_2_01.00.0000_bhgkutgk.zip
90978-5368-7165_GET_FATTUR_TEST_MARK_EXCLAM_01.00.0000_zsdutrfjg.zip
.........
...

1. To rename all the file extensions

# Rename all *.zip to *.iar
for f in *.zip; do
mv -- "$f" "${f%.zip}.iar"
done

# Remove all the unnecessary texts before the IAR file name eg: here the file name is 

# EMPLOYEE_LOADER_02.00.0000

ls | grep '\.iar' | sed 's/^\([^_]*\)_\(.*\)$/mv & \2/' | sh

# Remove unwanted after the file name

ls | grep '\.iar' | sed 's/^\(.*\)_\(.*\)$/mv & \1.iar/' | sh

2. To know the number of files in starting with each alphabet

# Count the number of files and prints it counts

for x in {A..Z}

do

        echo "$x"

        ls $1/${x}*$2 -l | wc -l

done

Need to call the countFiles.sh like below

./countfiles.sh ../json/final_till_m json

3. Others Useful commands

3.1 Count the number of files

3.1.1 Total number of file inside a folder and sub-folders

$ find . -type f | wc -l
950

3.1.2 Total number of zip files

$ find . -iname \*.zip | wc -l

17454

3.2 List the files matching the content

$ find  -name "*.xml" | xargs grep "Start Staging" 2> /dev/null

Upload/Download a file using SFTP with a private key

If we want to copy/transfer the file from Windows to Linux then we can use the popular WinScp software. But if we want to transfer files between Linux machines then we can either use FTP (File Transfer Protocol) or SFTP (SSH File Transfer Protocol). This article tells the steps of transferring files using the SFTP protocol with a private key

STEPS

• Keep the private key of the remote host in a folder in local Linux machine

eg: /home/venkatesh/openssh.pk

• Change your directory

cd /home/venkatesh/

• We can use SFTP(SSH File Transfer Protocol) to connect to the remote Linux server and download/upload the files

Usage of sftp command

sftp [-1Cv] [-B buffer_size] [-b batchfile] [-F ssh_config]
            [-o ssh_option] [-P sftp_server_path] [-R num_requests]
            [-S program] [-s subsystem | sftp_server] host
sftp [user@]host[:file ...]
sftp [user@]host[:dir[/]]
sftp -b batchfile [user@]host 

• Execute the below command to connect to the remote server

-sh$ sftp -o "IdentityFile=openssh.pk" user@remote.domain.com

We need to pass the private key to SSH command, so whatever we need to give to SSH, we need to include it as a part of -o command line argument.

          -sh$  sftp -oPort=10124 -oPreferredAuthentications=password admin@190.35.180.156

• Once we execute the above command, we will be connected and will see the sftp prompt like below

sftp>

• For downloading the file to the local Linux machine, 

sftp> get testFile.txt

This would download the file to your directory from where we have connected to the sftp.

• Similarly, if we want to upload a file, we need to execute

sftp> put uploadMyFile.txt

Different tools to monitor memory utilization in Linux

There are plenty of tools when it comes to monitoring memory usage. The main purpose of using tools like these is to identify any leaks and to know any performance issue that can occur to our program.

Linux natively has tools like 'top', 'pmap' etc. Apart from that Java provides a tool called 'jconsole', I love this tool than any other tool.

1 top

This command is very much similar to what we see in Windows Task Manager. This one shows the real-time memory usage and CPU utilization by each process. We can also monitor for a particular process by passing 'pid' as a command argument. 

$top

Use this command if you would like to monitor all the process

$top -p 29257

Use this one if you would like to monitor for a particular process

$top -p 29257 -b >  /home/venkatesh/memory_footprint.txt

If you would like to record the memory utilization over time, then execute the top command with '-b' option which will output the status ever 'n' seconds

As you can see from the output, it will just tell the percentage of utilization and it will not break the process any further.

2 pmap

pmap will give the snapshot of the process's memory map. It even provides addresses of the each jar that this process has. What I understand is, it is showing only the static memory usage by the processor and does not tells how much it is taking at runtime. 

$pmap -x 29257

We can also get the memory map for multiple processes as well.

3 jconsole

This can be used to monitor only java processes. It gives a clear breakdown of HeapMemory usage, Number of threads, classes, CPU usage, when GC kicked in. All these are real-time information that it shows

$jconsole 29257

4 Oracle Java Mission Control

Zip file is invalid for file zipped using ZipOutputStream

Are you seeing an error like below when you try to unzip a file which was zipped using the java.util.zip.ZipOutputStream

Or are you seeing an error like below when you unzip from Linux


unzip myfile.zip 
Archive:  myfile.zip 
  End-of-central-directory signature not found.  Either this file is not 
  a zipfile, or it constitutes one disk of a multi-part archive.  In the 
  latter case the central directory and zipfile comment will be found on 
  the last disk(s) of this archive. 
unzip:  cannot find zipfile directory in one of 
myfile.zip or 
        myfile.zip.zip, and cannot find 
myfile.ZIP, period.

If the answer is YES, then check how you are closing the ZipOutputStream in your code. The ZipOutputStream should be closed before calling the toByteArray() of your ByteArrayOutputStream

ByteArrayOutputStream baos = new ByteArrayOutputStream();
try(ZipOutputStream zos = new ZipOutputStream(baos);) {
zip(zos, incidentDir, incidentDir);
zos.flush();
baos.flush();
}
catch (Exception e) {                
throw new Exception("Unable to Zip the folder");
}
byte[] result = baos.toByteArray();

The documentation of ByteArrayOutputStream says

The methods in this class can be called after the stream has been closed without generating an IOException.

https://docs.oracle.com/javase/8/docs/api/java/io/ByteArrayOutputStream.html

DIY: Balcony Ideas - Wall Murals

Want to test the creativity side of the brain? then balcony ideas are the best food for thoughts. If we sit back and think then we will get an enormous list of ideas that we can try there. There wouldn't be a single person who hasn't experienced sipping a tea or coffee in Balcony. Even though the balcony is the smallest square area on any apartment building, we all love to spend quality time with our loved ones by sitting, talking, playing etc.

Balcony or sit out is a special space in every home where we would try our hobbies like potting the plants, using upcycled old furniture, novel lighting systems, painting the walls etc., to name few.  When it comes to the balcony, we will be more courageous in attempting new thoughts as it is a small square area and is mostly private to our family members, so even our idea didn't turn up well, we take it easy.

Before the Murals

One such idea we got is wall murals. Instead of a single lite tone color all over the balcony, we wanted to give some life to it by adding more active colors. Wall mural are what we finally decided on rather than just active colors. We were not professional painters, not even a painters; Oh God, the last painting I did was for my high school drawing class!! Based on our proficiency level, we chose to draw a tree with simple branches. No doubt, Google is always the third eye, from so many tree models we searched, we shortlisted to three and started our rough sketch on the wall.

Materials that are required

1. Mural model
2. Chalk pieces
3. Different varieties/widths of paint brushes
4. Paint
5. Sandpaper
6. Hand gloves 
7. Waste clothes

Paints are required brushes

The paint we selected was Asian Royal paint, this is 100% odorless and is stain-free paint. Most of the paint shop has the base paint and instantly prepares the color we chose from the base paint. And the Asian Royal can be easily cleaned from our skin if paint spills on us.

Step 1 of the art

If the wall is dirty then having a coat with emulsion is good. Low-cost emulsions like Tractor can be applied 5 to 6 hours before starting the mural work.

Finished masterpiece

One more snap

Work like this will keep us engaged on the weekends and also cherish us every moment we see it. Also, ideas like this will convert the balcony into a livable area rather a dumping area.

Trip to God's Own Country : Kerala

KERALA, one of the wettest and greenest states in India. The monsoon in this state starts early and from late May or early June we can see showers to heavy rains. The state has lots of lakes, river, and backwaters with the year full of waters in it.

We started our journey from Bangalore to Cochin, which is the business capital of Kerala even though Trivandrum is the official capital. Cochin is famous for its harbor, naval base, ship building yards, and of course fishing. Actually, Cochin and Ernakulam are twin cities. Along the coastal area, it is Cochin and inside it is Ernakulam town. Cochin is also surrounded by many small islands including a manmade artificial island and each island has its own importance in history and its current functions.

Alleppey

The first spot/image that comes to our mind when we say Kerala is its unique houseboat experience. Alleppey town which is also synonymously called has 'Venice of East' has a large number of canals and the town is surrounded by the Arabian Sea on one side and Vembanad lake on the other side. The Vembanad lake is the largest lake in India and this lake is spread across multiple districts of Kerala. Alleppey town and Vembanad lake are also famous for it traditional rowing boat race.

1. Starting point of House Boats
2. Snap of another houseboat inside the lake
3. With the crew member

The houseboats are in operational for the past 25 years and in the recent past, it gained so much of attractions. As of today, there is nearly 1800 houseboat in total it seems. There are so many licensed houseboats vendors and we book one online or we can take one after going there. If it is a peak season like November or December then it is better to book in advance. The advantage of going there and booking is we can bargain and get a better deal than online booking. There are multiple variants of houseboat ranging from one room to maximum of 10 rooms house boat. Available in both air-conditioned and non-air-conditioned rooms. Almost all houseboat comes with a living room, dining hall, entertainment unit, and kitchen.

1. One of the rooms in houseboat
2. Living room
3. Corridor

A range of packages is available to choose from, like a day package, night package or day-night package. Two to three crew member will be accompanying us throughout the journey, they will help in preparing food, cleaning etc. The depth of the lake is huge, 12 feet to 40 feet in some places. The maximum speed of the houseboat is 20 kmph. We can see police personals patrolling on the speed boat for the safety of tourists and to avoid any incidents. On the way, if we need to buy any items, shops are there on the banks of the lake. On the journey, we can enjoy the scenic beauty where we are completely surrounded by water and small islands, we can also witness different species of birds and varieties of plants and trees.

In the evening after 5:30, they will dock the boat in a nearby village and we can go for a nature walk and sight seeing in that village. Apart from the houseboat, we can also see express boats which are like bus service to ferry people from one village to another.

Ernakulam

Ernakulam town is a diversified place with both modern and traditional values. It has Asia's largest mall 'Lulu mall' and there is another mall dedicated only for mobile phone and accessories. International brand shops across marine drive road. Chottanikkara is just a 1-hour drive from Ernakulam and it has the famous Bhagavathi temple. All temples in Kerala has its own unique architectural style, we feel blessed as we enter the sanctum.  

Fort Cochin is also near where we can see the famous Portugal's church, this is the church where Vasco da Gama was buried when he died here during his third visit to India. After few minutes drive from the church, we can reach the spot where fishermen installed Chinese fishing net for fishing, these Chinese fishing nets are shore operated lift nets.

1. Place where Vasco Da Gama was buried
2. Church built by Portugues 

Chinese fishing nets in Cochin

Vypin

Vypin is one the island around Cochin and it hosts lots and lots of resorts and homestays across the coast. To reach Vypin island, we need to cross two more islands Mulavukad and Vallarpadam. Mulavukad island houses Bolgatty Palace built by the Dutch which is now converted into a hotel. Vallarpadam island has the international container shipment terminal. Vypin island consists of many small villages and beaches. Cherai Beach is the famous beach of Vypin and is the nearest cleanest beach from Ernakulam. The other side of the beach is a lake. We stayed near Munambam beach and even here we can see a line of Chinese fishing nets in operation. 

In Munambam beach