All Products
Search
Document Center

Realtime Compute for Apache Flink:Activate Realtime Compute for Apache Flink

Last Updated:Sep 03, 2024

Before you use Realtime Compute for Apache Flink, you must create a workspace. The computing resources of each workspace are isolated. The development console is independent of each other. This topic describes how to create a Realtime Compute for Apache Flink workspace and the precautions that you must take note of before you activate Realtime Compute for Apache Flink.

Prerequisites

An Alibaba Cloud account is created. If you do not have an Alibaba Cloud account, create one.

If you want to access Realtime Compute for Apache Flink as a Resource Access Management (RAM) user or by using a RAM role, the RAM user or RAM role must have the related permissions to perform operations in the Realtime Compute for Apache Flink console. For more information, see Permission management.

Precautions

  • Realtime Compute for Apache Flink is deployed in a compute-storage separation architecture. When you associate an Object Storage Service (OSS) bucket with your workspace, take note of the following items:

    • If you have not activated OSS, activate OSS first. For more information, see Activate OSS. The OSS bucket must reside in the same region as the Realtime Compute for Apache Flink workspace. The storage class of the OSS bucket must be Standard.

    • We recommend that you do not configure policies, such as the versioning and retention policies, for the associated OSS bucket.

      If the versioning feature is enabled for your OSS bucket, an excessive number of delete markers causes issues such as slow list processing. Therefore, you must configure lifecycle rules to periodically remove delete markers and historical versions that you no longer use if you enable the versioning feature. This helps reduce storage costs and improves OSS performance. For more information about how to remove delete markers, see Configure lifecycle rules based on the last modified time for versioning-enabled buckets to reduce storage costs.

      Warning

      If you do not enable the versioning feature and you want to configure lifecycle rules, we recommend that you do not configure a deletion policy for the directories of Realtime Compute for Apache Flink, including artifacts, flink-jobs, flink-savepoints, sql-artifacts, plan, and flink-sessionclusters. If you configure a deletion policy for one of the preceding directories, files that are related to Realtime Compute for Apache Flink may be deleted. As a result, Realtime Compute for Apache Flink deployments cannot run as expected.

    • To implement the cross-zone high availability feature for a Realtime Compute for Apache Flink workspace, make sure that the Object Storage Service (OSS) bucket selected for the workspace is of the zone-redundant storage type. For more information about how to configure the storage type of OSS buckets, see Change the storage redundancy type of a bucket. If the OSS bucket is not of the zone-redundant storage type, the state data cannot be accessed when a zone to which the workspace belongs becomes unavailable. As a result, Realtime Compute for Apache Flink deployments may not be started with states.

      Note

      In the scenario where a workspace contains a namespace in which the cross-zone high availability feature is enabled and a namespace that uses single-zone CUs, if you change the type of the OSS bucket associated with the workspace to the zone-redundant storage type, the storage costs may increase. You can create a Realtime Compute for Apache Flink workspace for the namespace in which the cross-zone high availability feature is enabled and associate the workspace with the OSS bucket of the zone-redundant storage type. This helps achieve the highest availability at the lowest cost. For more information about OSS storage fees, see Storage fees.

  • When you create a Realtime Compute for Apache Flink workspace, other cloud services, such as OSS, Virtual Private Cloud (VPC), and services related to the monitoring service, are used. For more information about the billing rules of the involved cloud services, see Billable items.

Procedure

Note

If you want to purchase more than 1,000 compute units (CUs) when you activate Realtime Compute for Apache Flink, submit a ticket.

  1. Log on to the Realtime Compute for Apache Flink console.

  2. In the Fully Managed Flink section, click Purchase.

  3. Grant your Alibaba Cloud account the related permissions to access the services that Realtime Compute for Apache Flink can access if you activate Realtime Compute for Apache Flink for the first time.

    On the Authorization Request page, click Authorize in RAM. On the Cloud Resource Access Authorization page, click Confirm Authorization Policy in the lower part of the page. After automated authorization is complete, your Alibaba Cloud account is granted the related permissions to access the services that Realtime Compute for Apache Flink can access.

  4. On the buy page, configure the parameters. The following table describes the parameters.

    image

    Category

    Parameter

    Description

    Example

    Basic parameters

    Billing Method

    The billing method based on which the Realtime Compute for Apache Flink workspace is charged. The subscription and pay-as-you-go billing methods are supported. For more information about billing details, see Billing items.

    Pay-as-you-go

    Duration

    The subscription duration. You need to configure this parameter only if the subscription billing method is used.

    Select a subscription duration based on your business requirements.

    N/A

    Auto-renewal

    Specifies whether to enable auto renewal. You need to configure this parameter only if the subscription billing method is used.

    After the auto-renewal feature is enabled, you can perform the following operations to modify the renewal policy: Log on to the Alibaba Cloud Management Console. In the top navigation bar, choose Expenses > Renewal Management.

    N/A

    CPU architecture

    The architecture of the workspace. You need to configure this parameter only if the subscription billing method is used.

    You can set this parameter to Intel X86 Architecture or YITIAN ARM Architecture based on your business requirements.

    Note

    You can set this parameter to YITIAN ARM Architecture only for a subscription workspace that resides in the China (Ulanqab), China (Beijing), or China (Hangzhou) region. The information on the buy page of Realtime Compute for Apache Flink prevails.

    Intel X86 Architecture

    Region

    The region in which the workspace is deployed. We recommend that you select the same region as the region in which upstream and downstream storage resides.

    Important

    The supported regions and zones change. The information on the buy page prevails.

    China (Beijing)

    Networking parameters

    VPC

    The VPC in which the Realtime Compute for Apache Flink workspace resides. For more information about how to create a VPC, see Create a VPC and a vSwitch.

    • If the destination upstream and downstream storage resides in a VPC, the Realtime Compute for Apache Flink workspace must be in the same VPC as the destination upstream and downstream storage.

    • Only the following CIDR blocks or their subsets are supported: 192.168.0.0/16, 172.16.0.0/12, and 10.0.0.0/8. The CIDR blocks are standard private CIDR blocks as defined by Request for Comments (RFC) documents.

    flink-test-vpc

    CU Info

    The CU information. Configure the CU Type and Resource Quota parameters. Valid values of the CU Type parameter: Single zone and Cross zone. If you select Whether to enable cross zones, you must configure the Resource Quota parameter.

    Note

    Only subscription workspaces that use the Intel X86 architecture processor support cross-zone high availability and cross-zone CUs. For more information about cross-zone high availability, see Zone-high availability.

    Single zone

    Zone

    The zone in which the workspace is deployed. If cross-zone high availability is enabled for the workspace, you must configure the primary and secondary zones in which the workspace resides.

    Zone G

    vSwitch

    The vSwitch of the workspace. If cross-zone high availability is enabled for the workspace, you must specify vSwitches for the primary and secondary zones in which the workspace resides.

    Each workspace requires two to three IP addresses. At least one vSwitch must be selected for each zone. You must plan CIDR blocks based on the scale of your Realtime Compute for Apache Flink deployments.

    Note

    Only the vSwitches that reside in the specified VPC and the specified zone of the Realtime Compute for Apache Flink workspace are displayed in the Realtime Compute for Apache Flink console. If no available vSwitch is displayed, you can create a vSwitch. For more information, see Create a vSwitch.

    flinktest-vsw-2ze4fyq366itq6xqp****

    Workspace settings

    Workspace Name

    The workspace name. The name must be 1 to 60 characters in length and can contain only lowercase letters, digits, and hyphens (-). The name must start with a lowercase letter and cannot end with a hyphen (-).

    Important

    After the Realtime Compute for Apache Flink service is activated, you cannot change the workspace name.

    flink-test

    Resource Quota

    The resource quota. You need to configure this parameter only if the Subscription billing method is used. Computing resources are billed based on the specified quota.

    N/A

    Quota Ceiling

    The quota limit. You need to configure this parameter only if the Pay-as-you-go billing method is used. This parameter specifies the maximum number of CUs that can be configured for a workspace. Default value: 1000. You can modify the value of this parameter. The maximum value of this parameter is 9999999.

    Storage settings

    OSS Bucket

    The OSS bucket with which the workspace is associated. An OSS bucket is used to store information, such as checkpoints, savepoints, logs, and JAR packages of deployments.

    Important
    • After the Realtime Compute for Apache Flink workspace is created, you cannot change the OSS bucket. For more information about how to associate an OSS bucket with a Realtime Compute for Apache Flink workspace, see the Precautions section of this topic.

    • For more information about the content of each directory of an OSS bucket, see Resource management.

    flink-test-oss

    Tags and tag values

    A tag consists of a tag key and a tag value. Tags are used to identify cloud resources. Tags allow you to categorize, search for, and aggregate cloud resources that have the same characteristics from different dimensions. This helps you manage cloud resources in an efficient manner. You can manage tags on the Tag page of the Resource Management console.

    N/A

    Monitoring service

    Monitoring service

    The monitoring service that you want to use. You can select Free monitoring service or Pay-as-you-go Prometheus monitoring service. For more information about differences in features between CloudMonitor and Application Real-Time Monitoring Service (ARMS), see Feature comparison between CloudMonitor and ARMS.

    • Free monitoring service: After the Realtime Compute for Apache Flink workspace is created, CloudMonitor provides basic monitoring and alerting service. For more information, see Free quotas.

    • Pay-as-you-go Prometheus monitoring service: After the Realtime Compute for Apache Flink workspace is created, ARMS is automatically activated to provide the commercial edition of Managed Service for Prometheus. For more information about the billing of the Prometheus Service, see Billing.

    Free monitoring service

  5. Click Confirm Order and complete the payment to activate Realtime Compute for Apache Flink.

    After you complete the payment, click Console. On the page that appears, you can view the workspace that is being created. In most cases, the workspace can be created in 5 to 10 minutes after you complete the payment.

    Note

    If you set the Storage type parameter to Fully Managed Storage, the order payment is not complete in the Realtime Compute for Apache Flink console and you want to complete the payment in the Expenses and Costs console, make sure that you pay for the order in both consoles. Otherwise, the Realtime Compute for Apache Flink workspace cannot be activated.

  6. Perform operations based on your business requirements. The following table describes the operations that you can perform after Realtime Compute for Apache Flink is activated.

    Operation

    Description

    View the details of a workspace

    In the Realtime Compute for Apache Flink console, find the workspace that you want to manage and choose More > Workspace Details in the Actions column to view the information about the workspace. The information includes the workspace name, workspace ID, OSS bucket, VPC name, VPC ID, and vSwitch. The information about the OSS bucket is displayed only if you set the Storage type parameter to OSS Bucket when you activate the Realtime Compute for Apache Flink workspace.

    Obtain basic concepts

    In the Realtime Compute for Apache Flink console, you can learn about how the basic concepts in Realtime Compute for Apache Flink are organized and obtain a helpful reference for planning future operations, such as draft development, deployment creation, deployment O&M, and security management.

    Manage permissions

    If you want to access the Realtime Compute for Apache Flink console as a RAM user or by using a RAM role and perform operations in a namespace in the development console of Realtime Compute for Apache Flink, the RAM user or RAM role must be granted RAM permissions and the permissions to perform operations in the namespace, such as the permissions to view and purchase a workspace and perform draft development and deployment O&M in a namespace.

    Add a namespace

    A namespace is the basic management unit for drafts and deployments of Realtime Compute for Apache Flink. The configurations, drafts, deployments, and permissions of each namespace are separately managed. After a workspace is created, a namespace named Workspace name-default or Workspace name-default-ha is automatically generated. You can create multiple namespaces and assign separate resources and permissions to each namespace. This isolates the resources and permissions among the namespaces of different tenants in an efficient manner.

    Develop drafts and maintain deployments

    Find the workspace that you want to manage and click Console in the Actions column to go to the development console of Realtime Compute for Apache Flink. You can develop drafts and maintain deployments.

    Realtime Compute for Apache Flink provides various upstream and downstream built-in connectors. For more information, see Supported connectors.

    Reconfigure resources

    You can reconfigure resources for workspaces and namespaces.

    Switch the billing method

    You can switch the billing method of a workspace between pay-as-you-go and subscription in the Realtime Compute for Apache Flink console.

    Change a vSwitch

    You can change the vSwitch to resolve deployment startup failures that are caused by insufficient IP addresses of the vSwitch that you select.

    Release resources

    In the Realtime Compute for Apache Flink console, find the workspace that you want to manage and choose More > Release Resources in the Actions column to release resources. You are not charged for the workspace after it is released. The related ARMS resources are released when you release the workspace.