Community Blog Best Practices for Flink on Zeppelin Stream Computing Processing

Best Practices for Flink on Zeppelin Stream Computing Processing

This article is an overview of the best practices for Flink on Zeppelin stream computing processing taken from a recent lecture.

Lecturer: Jian Feng, Head of Alibaba Cloud EMR Data Development Platform

Content Framework:

  • Big Data Overview
  • Flink Learning Framework
  • Best Practices for Stream Computing on EMR Studio
  • Presentation

I. Big Data Overview

  • Big Data Processing ETL (Data → Data)
  • Big Data Analysis BI (Data → Dashboard)
  • Machine Learning Platform for AI (Data → Model)


II. Flink Learning Framework

Flink Essentials

  • Stateful
  • Time
  • Flink Architecture
  • Flink API
  • Flink Configuration
  • Flink Log


1.  Why

  • Timeliness of stream computing
  • Unbounded for stream computing

2.  When

  • Window
  • Join
  • Pattern

3.  How

  • Statebackend


  • Event Time
  • Processing Time
  • Watermark

Flink Architecture


Flink API


Flink Configuration

  • Cluster Configuration
  • Job Configuration
  • Statebackend
  • Resource Manager
  • SQL/Python

Reference: https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/config/

Flink Log


III. Best Practices for Stream Computing on EMR Studio

EMR Studio Features

  • Compatible with Open-Source Components

EMR Studio is optimized and enhanced based on the open-source software Apache Zeppelin, Jupyter Notebook, and Apache Airflow.


  • Supports connecting to multiple clusters
  • Adapt to multiple computing engines
  • Interactive development + seamless connection of job scheduling
  • Applicable to a variety of big data application scenarios
  • Separated computing and storage

Flink Clients


Flink on Zeppelin (Phase 1) – Interactive Flink Client


Flink on Zeppelin (Phase 2) – Interactive JobManager


Flink on Zeppelin Main Feature


IV. Demonstration

For specific product introduction and demonstration, you can click the following link to watch the playback: https://developer.aliyun.com/live/247106 (Video in Chinese)

V. Related Product

E-MapReduce: https://www.alibabacloud.com/product/emapreduce

EMR Studio (Beta): https://help.aliyun.com/document_detail/208107.html (Article in Chinese)

Realtime Compute for Apache Flink: https://www.alibabacloud.com/product/realtime-compute

0 0 0
Share on

Alibaba EMR

56 posts | 4 followers

You may also like


Alibaba EMR

56 posts | 4 followers

Related Products