Skip to main content

Mapreduce Vs Pig Vs Hive


Apache Hadoop is an open source framework intended to make interaction with big data easier. Hadoop has made its place in the industries and companies that need to work on large data sets which are sensitive and needs efficient handling. 

There are several components that Hadoop Ecosystem has to handle the huge data collectively. MapReduce, Pig and Hive are one of the Key components.



MapReduce is a software framework and programming model used for processing huge amounts of data. MapReduce program works in two phases, namely, Map and Reduce. Map tasks deal with splitting and mapping of data while Reduce tasks shuffle and reduce the data.

Apache Pig is an abstraction over MapReduce. It is a tool/platform which is used to analyze larger sets of data representing them as data flows. Pig is generally used with Hadoop. We can perform all the data manipulation operations in Hadoop using Pig.

Hive is an open-source system that processes structured data in Hadoop, residing on top of the latter for summarizing Big Data, as well as facilitating analysis and queries. Hive data warehouse software facilitates reading, writing, and managing large datasets residing in distributed storage using SQL. The structure can be projected onto data already in storage.

There are differences among three components.

 

MapReduce

Pig

Hive

MapReduce built on top of Hadoop

Pig is open source

Hive open source

It is a data processing paradigm.

It is a data flow language.

Hive uses a language called HiveQL.

MapReduce is low level language.

Pig is a high level language.

HiveQL is a query processing language.

MapReduce jobs have a long compilation process.

In pig there is no need for compilation.

Hive compiler parses the query.

Exposure to Java is must to work with MapReduce.

Basic knowledge of SQL is enough to work with Apache Pig.

Basic knowledge of SQL is enough to work with Hive.

MapReduce was developed by Google

It was originally created at Yahoo.

It was originally created at Facebook.

More lines of code

Comparatively less line of codes than MapReduce

Comparatively less line of codes than MapReduce and Pig

More development involved

Development effort is less code efficiency

 

Development effort is less code efficiency

 

MapReduce can handle structured and unstructured data

Apache Pig can handle structured, unstructured, and semi-structured data.

 

Basically Hive handle only structured data.

 

 

Labels:

Comments

Post a Comment

Popular posts from this blog

Photo : Savitri during her last days

Even after 4 decades of her death, people are very much interested to watched her (Savitri) biopic. Mahanati turned to be stupendous success. On eve of remembering Savitri, here we posted Savitri photo. The photo seems her last days. The man stand behind Savitri is Gemini Ganeshan who married Savitri. It seems that she suffered bad days during her last days.
Post a Comment
Read more

Difference between DELETE and TRUNCATE

This is the common interview question for every SQL Developer and DBA position. DELETE TRUNCATE DML (Data Manipulation Language) Command. DDL (Data Definition Language) Command. It can be rollback. It can't be rollback. Using DELETE we can remove some or all records from the table. We can use WHERE clause with it. Using TRUNCATE we can remove all records from the table at once. We can't use WHERE clause with it. DELETE command is slow in performance. TRUNCATE command is super faster in performance. All deleted Triggers fire on DELETE command. No Trigger can be fired on TRUNCATE command. DELETE can't reset high water mark. TRUNCATE can reset high water mark. DELETE can't release the space from the table. TRUNCATE can release the space from the table.
Post a Comment
Read more

Producers demand 18 crores for Ceded for Pavan and Trivikram film

Director Rajamouli showed the path to all other producers on how to sell product (movie) in different ways across the word. Telugu Cinema has vertically divided in two ways Before and after Bahubali. Bahubali proves that there were no boundaries in selling the movie. Other producers who are doing with star heros like Pavan, Mahesh and Jr. NTR will follow Rajamouli in this regard. But there is slight difference with Bahubali and other heros films. Unfortunately, producers and distributors fail to notice the gap. Bahubali is universal movie which was apt to all sections in the world. That is the reason for getting huge collections across the world. We didn't expect the same situation for Prabas's next movie even though his popularity increased with Bahubali. Radha Krishna who produced movies like Julayi and A. AA.. will plan to produce next film with Pavan and Trivikram. Movie was not yet started but distributors rushed toward producer for getting area-wise rights.Obviously, ...
Post a Comment
Read more