Sqoop import and export between Mysql and HDFS

Dear Viewers,

Following script will help you to perform import and export between MySql and HDFS using Sqoop.

# Script Starts here SqoopJobs.sh ########

#!/bin/bash
# Title: Shell script to perform import and export between MySQL to HDFS and vice-versa using Sqoop
# Author: Pavan Jaiswal
#Note: This program can be made using shell script or Java. I have chosen shell script here.

Data analysis using Hive script

Dear Viewers,

This post will help you to run hive script on hive table. Hive script is made for below given title.

Hive script can be saved with the extension '.hql' or '.sql' . Copy paste the below code in a file "HiveScript.hql". In hive script, single line comment is given by '--'.

OSD question bank first three units

Unit 1: Introduction to UNIX OS

1. Draw and explain UNIX architecture.

2. Differentiate monolithic, micro and exo kernel.

3. Explain booting process in detail.

4. Compare and contrast GRUB-1 and GRUB-2.

5. Explain with neat diagram buffer header.

6. Write and explain algorithm getblk().

7. Explain algorithm bread() and bwrite()

8. Explain algorithm iget() and iput()

9. Draw and explain UNIX file system architecture.

10. What do you understand by free space management? Explain its different approaches.

11. What is the importance of swapping in OS?

Distributed Programming - questions and solution set (unit 1)

Sincere Thanks to Ms. Shrida Kalamkar for preparing the solution set.

Click here to download

Distributed Programming - presentation

Click here to download

Hive installation and configuration

Dear Viewers,

This post will help you to setup and configure hive on Hadoop and Fedora 17


Introduction:
Apache Hive is a data warehouse architecture which is built on top of Hadoop. Many people refers Hive as Hadoop ecosystem. Hive is useful to perform data summerization and data analysis. Hive supports SQL like language which is called as Hive Query Language (HQL).

Basic Hive Queries

Dear Viewers,

This post will help you to execute basic queries on Hive.

Operations performed are:
1. Create database
2. Create tables
3. Alter table
4. Drop table
5. Drop database

First Hive script on Hadoop cluster

Dear Viewers,

This post will help you to write and execute your first Hive script from Linux terminal. Before we go ahead, you have to start hadoop. You can start it by calling start-all.sh file available in hadoop directory. To run Hive script, you must have Hive directory available on your system.

In my case I have hadoop available at /usr/local/hadoop-2.6.0 and Hive at /usr/local/hive-0.11.0