Cross-zone high availability - Realtime Compute for Apache Flink

The cross-zone high availability feature can effectively prevent service interruptions caused by faults in a single zone and ensure service stability. This feature is used to support high availability in different zones of the same region. Realtime Compute for Apache Flink allows you to implement the cross-zone high availability feature for subscription namespaces that use cross-zone compute units (CUs). This topic describes how to enable cross-zone high availability and stop the billing for cross-zone CUs.

Introduction

You can configure two types of CUs for a workspace: single-zone CUs and cross-zone CUs. Based on the type of CUs configured, a namespace can either be non-highly available or highly available:

Non-highly available namespaces: Realtime Compute for Apache Flink selects an optimal computing zone in the chosen region. Zones in the same region can communicate with each other over an internal network. The computing layer performs transparent scheduling across the zones to allow the resources to be shared within the region. This enhances resource elasticity.
Highly available namespaces: In case the primary zone fails, jobs will automatically fail over to the secondary zone. This helps prevent service interruptions from failures in a single zone, ensuring service continuity and high availability.

Usage notes

End-to-end high availability requires that your upstream and downstream systems are also highly available.
Millisecond latencies may occur when Realtime Compute for Apache Flink accesses external systems across zones. The service-level agreement (SLA) guarantees the network latency across zones to be less than 3 milliseconds. For more information, see Average Latency among Zones. In most cases, the latency for accessing upstream and downstream resources except for Tair (Redis OSS-compatible) is acceptable. We recommend that you evaluate the latency based on your business requirements to determine whether to enable cross-zone high availability.

Regions and billing

You can purchase cross-zone CUs only for subscription workspaces that use Intel x86. The price of cross-zone CUs is higher than single-zone CUs. For information about the supported regions and billing, see Subscription.

If your workspace uses the pay-as-you-go billing method, you can change the billing method to subscription and enable cross-zone high availability by referring to Method 2 and Method 3 in this topic.

Note

The billing method of workspaces with the Deployment model set to Cross zone cannot be changed to pay-as-you-go.

Enable cross-zone high availability

Method 1: Create a highly available namespace at workspace creation

Create a highly available namespace when you purchase a workspace.
1. For Billing Method, select Subscription.
2. For CPU Architecture, select Intel X86 Architecture.
3. For Deployment model, select Cross zone.
4. In the CU Info section, enter a value in the Resource Quota field corresponding to the cross-zone CU type.
5. For vSwitch, select two vSwitches from different zones.
After you create the workspace, a highly available namespace named Workspace name-default-ha is generated.
Click the name of the highly available namespace to enter it.
Create and run a job in the namespace.

Method 2: Create a highly available namespace in an existing workspace

If you have not set Deployment model to Cross zone at workspace creation, you can add cross-zone CUs to the workspace and enable high availability for your namespace.

Add cross-zone CUs.
1. Find the desired workspace and choose More > Reconfigure Workspace Resources in the Actions column.
2. In the dialog, click Add CU Type.
3. Enter a value in the Target CU column for the cross-zone CU type and select a vSwitch.
  Only vSwitches created in your VPC and supported zones are displayed. If no vSwitch is available, create a vSwitch.
  Important
  After you add cross-zone CUs in the workspace, make sure that the CIDR block of the new vSwitch in the zone is added to the whitelists of the upstream and downstream systems. This prevents issues caused by network access failures.
4. Read the terms of service, select the check box, and then click OK to complete the payment.
Create a namespace with cross-zone CUs.
This namespace is highly available.
Enter the namespace, and create and run a job.

Method 3: Enable high availability for an existing namespace

Log on to the Realtime Compute for Apache Flink console.
Ensure the number of unallocated cross-zone CUs in the workspace's resource pool equals or exceeds the number of single-zone CUs allocated the target namespace.
Note
If available cross-zone CUs are insufficient, you can add them. Alternatively, reduce cross-zone CUs in another highly available namespace and re-allocate them to the target namespace. For more information, see Reconfigure resources.
Find the target namespace and click Turn on high availability in the Actions column. In the dialog, click OK.
After you click OK, Realtime Compute for Apache Flink automatically reclaims the single-zone CUs from the namespace and allocates equivalent cross-zone CUs to the namespace. The returned single-zone CUs are available for re-allocation or scaling down. For example, Workspace A initially has 5 single-zone CUs and 5 cross-zone CUs, with 2 single-zone CUs allocated to Namespace A. After high availability is enabled for Namespace A, Realtime Compute for Apache Flink reclaims the single-zone CUs from Namespace A and allocates 2 cross-zone CUs to Namespace A. The table below compares Workspace A's resource availability before and after high availability is enabled for Namespace A:
Status
Single-zone CUs (Allocated/Purchased)
Cross-zone CUs (Allocated/Purchased)
Before
2/5
0/5
After
0/5
2/5
Enter the namespace, and create and run a job.

Disable cross-zone high availability

To disable high availability for a namespace, do the following:

Note

After you disable cross-zone high availability, billing for cross-zone CUs does not stop. To stop billing for cross-zone CUs, see Unsubscribe from cross-zone CUs.

Log on to the Realtime Compute for Apache Flink console.
Check that the number of unallocated single-zone CUs in your workspace exceeds or equals the number of cross-zone CUs allocated to the target namespace.
If the number of unallocated single-zone CUs is insufficient, add single-zone CUs. Alternatively, you can click Reconfigure Resources in the Actions column of another single-zone namespace and reduce the value in the Resource Reconfiguration (CUs) column. For more information, see Reconfigure resources.
Find the target namespace and click Turn off high availability in the Actions column. In the dialog, click OK.
After you disable the cross-zone high availability feature, the resource change policies are the same as those when you enable the cross-zone high availability feature. After the feature is completely disabled, you can use the namespace as expected.

Unsubscribe from cross-zone CUs

If the number of allocated cross-zone CUs in a workspace is 0, you are not charged for cross-zone CUs anymore. To unsubscribe from cross-zone CUs, perform the following steps:

Log on to the Realtime Compute for Apache Flink console.
Disable cross-zone high availability for all namespaces in your workspace.
Reduce the number of cross-zone CUs in the workspace to 0.
1. Find the target workspace and choose More > Reconfigure Workspace Resources in the Actions column.
2. In the Reconfigure Workspace Resources dialog box, change the value in the Target CU column for the cross-zone CU type to 0.
3. Read the terms of service, select the check box, and then click OK to complete the payment.

Realtime Compute for Apache Flink:Manage cross-zone high availability