Hadoop Ecosystem

The Core of the Hadoop are:

1. YARN

2. Map-Reduce

3. HDFS.

The following are the elements of  Hadoop Ecosystem:

1. MapReduce.

2. HDFS (Hadoop Distributed File System).

3. YARN (Yet Another Negotiator)

4. Avro

5.Parquet

6. Flume

7. Sqoop

8. Pig

9. Crunch

10. Spark

11. HBase

12. Zookeeper

YARN :

A Resource negotiator, manages the processing resources for Hadoop , schedules jobs, run processing framework.

Map Reduce:

A distributed processing framewok.

Spark: in-memory and streaming processing framework.

HBase: 

NoSQL database  built on HDFS.

Hive: 

SQL processing engine designed for batch processing workloads.

Impala:

 Query engine defined for Business Intelligent workload.

Parquet: 

Very efficient columnar data storage format.

Sqoop:

 Data movement to/from database.

Flume, Kafka : 

Streaming data ingestion.

Solr:

 Powerful text search functionality.

Hue: 

Web based user inteface for the Hadoop.

Sentry:

 Authorization tool provides security for Hadoop.

Need for Hadoop:

Hadoop is used for storage,processing and analyzing the data due to generating more data and retrieving more data by organisations. simple, managing excess or bulk of data.