This topic describes the basic concepts used in Realtime Compute for Apache Flink.
Hierarchy
The following figure shows how the basic concepts in Realtime Compute for Apache Flink are organized. This figure serves as a helpful reference for planning future operations, such as draft development, deployment creation, deployment O&M, and security management.
Terms
Term | Description | References |
Management Console | Management Console is the centralized portal for managing and configuring your workspace and namespace throughout their lifecycle. Key features:
| |
Development Console | Each workspace includes an independent Development Console, offering a one-stop platform for Flink job development and operations (O&M). Within this console, you can switch namespaces under the current workspace. It supports the entire job lifecycle, from development to running. Key features:
| |
workspace | A workspace is the basic management unit for namespaces of Realtime Compute for Apache Flink. Workspaces are independent environments and have their own dedicated computing resources. | |
namespace | A namespace is the basic management unit for drafts and deployments of Realtime Compute for Apache Flink. The configurations, drafts, deployments, and permissions of each namespace are managed individually. You can create multiple namespaces and assign separate resources and permissions to each of them. This effectively isolates the resources and permissions among the namespaces of different tenants. | |
resource | Realtime Compute for Apache Flink bills you based on compute units (CUs), which is a unit of measurement for computing resources. One CU provides computing capabilities equivalent to an hour of 1 CPU core, 4 GiB of memory, and 20 GB of local storage. The local storage stores information, such as logs and checkpoints. The number of CUs consumed reflects the resources used by Realtime Compute for Apache Flink. The number of CUs consumed by a Realtime Compute for Apache Flink deployment varies based on the queries per second (QPS) of input data streams, computing complexity, and input data distribution of the deployment. You can estimate the number of CUs that you need to purchase based on the scale of your business and the required computing capability of Realtime Compute for Apache Flink. | |
draft | SQL or YAML scripts are referred to as drafts and are created in the Development Console. Drafts can only be developed within the Realtime Compute for Apache Flink console, not via SDKs. | |
deployment | Deployments isolate environments such as development and production. You can create a deployment from a draft or a user-uploaded JAR/Python package. Modifying a draft after deployment will not affect the running job instance. Deployments are characterized by deterministic streaming and batch properties and can be developed via the Development Console or SDKs. | |
job | A job is an instance of a workload that runs in a deployment. A job has deterministic streaming and batch properties. | N/A |
connector | Realtime Compute for Apache Flink provides a variety of built-in connectors to read data from or write data to upstream and downstream data stores and perform data synchronization. Realtime Compute for Apache Flink also allows you to upload and use JAR packages of custom connectors. | |
function | Realtime Compute for Apache Flink provides functions, including built-in functions and user-defined functions (UDFs). | |
catalog | Metadata management is one of the most critical aspects of data processing. The catalogs of Realtime Compute for Apache Flink contain metadata, such as databases, tables, fields, partitions, and the data stored in databases or other external systems. | |
role | A role is a collection of users who have the same permissions. This concept is used in the security features of Realtime Compute for Apache Flink. One user can assume multiple roles, and multiple users can assume the same role. After you grant permissions to a role, all users who are assigned this role are granted the same permissions. | |
member | An Alibaba Cloud account or a RAM user can be added as a member of Realtime Compute for Apache Flink. This concept is used in the security features of Realtime Compute for Apache Flink. A member can manage data, drafts, deployments, resources, and functions in a namespace only after it is added to the namespace and is granted the related permissions on the namespace. | |
queue | A queue can be specified for deployments to implement resource isolation and management. |