in Technology by
What is the architecture of Flume?

1 Answer

0 votes
by

In general Apache Flume architecture is composed of the following components:

  1. Flume Source
  2. Flume Channel
  3. Flume Sink
  4. Flume Agent
  5. Flume Event
  1. Flume Source: Flume Source is available on various networking platforms like Facebook or Instagram. It is a Data generator that collects data from the generator, and then the data is transferred to a Flume Channel in the form of a Flume.
  2. Flume Channel: The data from the flume source is sent to an Intermediate Store which buffers the events till they get transferred into Sink. The Intermediate Store is called Flume Channel. Channel is an intermediate source. It is a bridge between Source and a Sink Flume channel. It supports both the Memory channel and File channel. The file channel is non-volatile which means once the data is entered into the channel, the data will never be lost unless you delete it. In contrast, in the Memory channel, events are stored in memory, so it’s volatile, meaning data may be lost, but Memory Channel is very fast in nature.
  3. Flume sink: Data repositories like HDFS, have Flume Sink. Which takes Flume events from the Flume Channel and stores them into the Destination specified like HDFS. It is done in such a way where it should deliver the events to the Store or another agent. Various sinks like Hive Sink, Thrift Sink, etc are supported by the Flume.
  4. Flume Agent: A Java process that works on Source, Channel, Sink combination is called Flume Agent. One or more than one agent is possible in Flume. Connected Flume agents which are distributed in nature can also be collectively called Flume.
  5. Flume Event: An Event is the unit of data transported in Flume. The general representation of the Data Object in Flume is called Event. The event is made up of a payload of a byte array with optional headers.

Related questions

0 votes
    What is Apache Flume in Hadoop ?...
asked Aug 7, 2021 in Technology by JackTerrance
0 votes
    What is the main difference between Kafka and Flume?...
asked Nov 1, 2020 in Technology by JackTerrance
0 votes
    Microservice Architecture adapts which of the following concept(s)? A. OOPS B. SOA C. Web Service/HTTP D. All the options...
asked Jan 9, 2023 in Technology by JackTerrance
0 votes
    There is no server in the serverless architecture. True False...
asked Sep 15, 2021 in Technology by JackTerrance
0 votes
    What are the features of Android architecture?...
asked Aug 13, 2021 in Technology by JackTerrance
0 votes
    What is the Apache Pig architecture?...
asked Aug 3, 2021 in Technology by JackTerrance
0 votes
    Which of the following best defines a microservice architecture? 1. An architecture commonly used in functional ... systems based on a loosely coupled component architecture...
asked Jul 25, 2021 in Technology by JackTerrance
0 votes
    How to implement service discovery in microservices architecture?...
asked Jul 24, 2021 in Technology by JackTerrance
0 votes
    What is the difference between Monolithic, SOA and Microservices Architecture?...
asked Jul 23, 2021 in Technology by JackTerrance
0 votes
    What is the working of Microservice Architecture?...
asked Jul 23, 2021 in Technology by JackTerrance
0 votes
    What is the Robot framework? Provide a brief overview of its architecture....
asked Jul 10, 2021 in Technology by JackTerrance
0 votes
    Django is based on which architecture? 1. MTV 2. MVC 3. MVTT 4. MVP...
asked Jul 4, 2021 in Technology by JackTerrance
0 votes
    What is MVVM architecture in Angular?...
asked Jun 30, 2021 in Technology by JackTerrance
0 votes
    Borg is used in Google BigQuery architecture for _________. (1)Job orchestration (2)Processing data (3)Networking (4)Data Storage...
asked Jun 1, 2021 in Technology by JackTerrance
0 votes
    DAM architecture in which a lightweight sensor is attached to the protected databases and continuously polls the ... (3)Interception-based architecture (4)Network Architecture...
asked May 19, 2021 in Technology by JackTerrance
...