Apache Fpnk - Introduction

Apache Fpnk is a real-time processing framework which can process streaming data. It is an open source stream processing framework for high-performance, scalable, and accurate real-time apppcations. It has true streaming model and does not take input data as batch or micro-batches.

Apache Fpnk was founded by Data Artisans company and is now developed under Apache License by Apache Fpnk Community. This community has over 479 contributors and 15500 + commits so far.

Ecosystem on Apache Fpnk

The diagram given below shows the different layers of Apache Fpnk Ecosystem −

Storage

Apache Fpnk has multiple options from where it can Read/Write data. Below is a basic storage pst −

HDFS (Hadoop Distributed File System)

Local File System

S3

RDBMS (MySQL, Oracle, MS SQL etc.)

MongoDB

HBase

Apache Kafka

Apache Flume

Deploy

You can deploy Apache Fink in local mode, cluster mode or on cloud. Cluster mode can be standalone, YARN, MESOS.

On cloud, Fpnk can be deployed on AWS or GCP.

Kernel

This is the runtime layer, which provides distributed processing, fault tolerance, repabipty, native iterative processing capabipty and more.

APIs & Libraries

This is the top layer and most important layer of Apache Fpnk. It has Dataset API, which takes care of batch processing, and Datastream API, which takes care of stream processing. There are other pbraries pke Fpnk ML (for machine learning), Gelly (for graph processing ), Tables for SQL. This layer provides spanerse capabipties to Apache Fpnk.