Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Best Practices for Flink on Zeppelin Stream Computing Processing

Best Practices for Flink on Zeppelin Stream Computing Processing

This article is an overview of the best practices for Flink on Zeppelin stream computing processing taken from a recent lecture.

Lecturer: Jian Feng, Head of Alibaba Cloud EMR Data Development Platform

Content Framework:

  • Big Data Overview
  • Flink Learning Framework
  • Best Practices for Stream Computing on EMR Studio
  • Presentation

I. Big Data Overview

  • Big Data Processing ETL (Data → Data)
  • Big Data Analysis BI (Data → Dashboard)
  • Machine Learning Platform for AI (Data → Model)

1

II. Flink Learning Framework

Flink Essentials

  • Stateful
  • Time
  • Flink Architecture
  • Flink API
  • Flink Configuration
  • Flink Log

Stateful

1.  Why

  • Timeliness of stream computing
  • Unbounded for stream computing

2.  When

  • Window
  • Join
  • Pattern

3.  How

  • Statebackend

Time

  • Event Time
  • Processing Time
  • Watermark

Flink Architecture

2

Flink API

3

Flink Configuration

  • Cluster Configuration
  • Job Configuration
  • Statebackend
  • Resource Manager
  • SQL/Python

Reference: https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/config/

Flink Log

4

III. Best Practices for Stream Computing on EMR Studio

EMR Studio Features

  • Compatible with Open-Source Components

EMR Studio is optimized and enhanced based on the open-source software Apache Zeppelin, Jupyter Notebook, and Apache Airflow.

5

  • Supports connecting to multiple clusters
  • Adapt to multiple computing engines
  • Interactive development + seamless connection of job scheduling
  • Applicable to a variety of big data application scenarios
  • Separated computing and storage

Flink Clients

6

Flink on Zeppelin (Phase 1) – Interactive Flink Client

7

Flink on Zeppelin (Phase 2) – Interactive JobManager

8

Flink on Zeppelin Main Feature

9

IV. Demonstration

For specific product introduction and demonstration, you can click the following link to watch the playback: https://developer.aliyun.com/live/247106 (Video in Chinese)

V. Related Product

E-MapReduce: https://www.alibabacloud.com/product/emapreduce

EMR Studio (Beta): https://help.aliyun.com/document_detail/208107.html (Article in Chinese)

Realtime Compute for Apache Flink: https://www.alibabacloud.com/product/realtime-compute

Python Apache BI flink Big Data Data analysis Stream Computing Data Visualization Zeppelin EMR Studio
0 0 0
Share on

Read previous post:

Best Practices for Big Data Processing in Spark

Read next post:

The Open-Source Real-Time Data Warehouse Solution Based on EMR OLAP - ClickHouse Transaction Implementation

Alibaba EMR

62 posts | 7 followers

You may also like

Comments

Alibaba EMR

62 posts | 7 followers

Related Products

More Posts by Alibaba EMR

See All