Realtime Compute for Apache Flink:Hybrid billing

Overview

In a hybrid-billed workspace, resources come in three types: management resources, fixed computing resources, and elastic computing resources. To implement hybrid billing, you must first create a subscription workspace (which is fixed computing resources). Next, switch the pricing model to hybrid billing and set a maximum limit for scaling the elastic computing resources. Hybrid billing effectively meets your long-term, consistent need for resources while accommodating short-term, fluctuating demands in workload peaks. For streaming workloads, hybrid billing can enhance cost efficiency along with the automatic tuning feature. In the case of batch deployments, hybrid billing reduces the need to maintain a large amount of resources for data processing tasks with periodic patterns. Overall, hybrid billing not only reduces the need to allocate excessive resources for unpredictable or temporary workload spikes, but also ensures that a fixed amount of computing resources is reliably at your disposal.

Usage notes

• Hybrid billing applies only to an Intel X86-based subscription workspace with cross-zone high availability disabled.

• To switch a hybrid-billed workspace to a subscription or pay-as-you-go workspace, you need to first disable hybrid billing. This will switch the pricing model of the workspace to subscription. You can then change the pricing model from subscription to pay-as-you-go.

• If a queue has idle fixed computing resources, the system prioritizes using those resources over elastic ones for running Flink deployments on that queue.

Billing rules

The total fee for a hybrid-billed workspace = Management resource fee + Fixed computing resource fee + Elastic computing resource fee

• The fees for management resources and fixed computing resources are connected with your subscription workspace. For more information, see Subscription.

• The elastic computing resource fee is based on the actual resource consumption. At the end of each billing cycle, the system generates a bill for pay-as-you-go resources and automatically deducts the corresponding fees from your account. Elastic computing resources are charged according to the pay-as-you-go billing rule of Realtime Compute for Apache Flink. For more information, see Pay-as-you-go.

Enable hybrid billing

1. Enable hybrid billing for your workspace and configure the workspace's elastic computing resource ceiling.

   1. Log on to the Realtime Compute console.

   2. Find the target subscription workspace. In the Actions column, choose More > Switch on flexible billing.

   3. In the Flexible billing settings pop-up dialog box, enter a desired value in the Elastic resource ceiling field for scaling elastic computing resources.

   4. Click OK.

   5. Proceed to complete the payment.

      Note

      You can set the elastic computing resource ceiling between 1 CU and 1,000 CUs. The maximum available computing resources in your workspace are the sum of fixed computing resources and the elastic resource ceiling that you configured.

2. Configure the elastic computing resource ceiling for a namespace.

   1. Click the icon to the left of the target workspace ID.

   2. In the Actions column of the target namespace, click Reconfigure Resources. In the pop-up dialog box, enter a desired value for the Maximum elastic resources field.

3. Assign elastic computing resources to existing or new queues within your namespace.

   Note

   • Elastic computing resources configured for a namespace are automatically allocated to its default queue. You can also assign the elastic computing resources to other queues. For more information, see Manage queues.

   • You do not have to manually specify the use of fixed or elastic computing resources. The system automatically determines the optimal resource utilization approach according to your workspace's resource setup, and prioritizes available fixed computing resources. This avoids incurring pay-as-you-go charges for elastic computing resources when fixed ones are available.

   • Insufficient computing resources can result in Flink job failures. For smooth job execution, make sure that you allocate sufficient resources to queues.

4. Create deployments on the queue assigned with elastic computing resources, according to your business needs. For more information, see Create a deployment.

Example: Optimize costs with hybrid billing and automatic tuning

Hybrid billing can further reduce costs for a deployment with automatic tuning enabled, whether the tuning mode is Autopilot or scheduled tuning. For example, your business experiences peak hours from 09:00:00 to 19:00:00 (10 hours) daily and off-peak hours from 19:00:00 to 09:00:00 (14 hours) the following day. You need 30 CUs of computing resources for peak hours and 10 CUs for off-peak hours. For information about the automatic tuning configurations in this example, see Configure automatic tuning.

If you use the subscription billing model, you need to purchase 30 CUs in advance for workload spikes during peak hours.

For a hybrid-billed workspace, you can purchase 10 CUs fixed computing resources for off-peak demands, and set the elastic computing resource ceiling to 20 CUs. During peak hours, if the fixed computing resources are insufficient, Flink will use up to 20 CUs of elastic computing resources to meet demands.

The cost optimization of hybrid billing is apparent. Assuming you plan to purchase a workspace in the Singapore (Singapore) region for a duration of one month (30 days) to handle the data processing workloads, the following table compares the three billing methods, including the estimated costs:

Billing methods	Purchased CUs	Description	Estimated costs
Subscription	30 CUs	You must purchase sufficient resources to accommodate peak demands, ensuring Flink jobs run smoothly at all times.	Management resource fee + Computing resource fee = (2 CUs x USD 45.94 per CU-month x 1 month) + (30 CUs x USD 45.94 per CU-month x 1 month) = USD 91.88 + USD 1378.2 = USD 1470.08
Pay-as-you-go	Configure an upper limit for your resources, which are billed based on the actual usage.	When your zone is running out of resources, starting your deployments may fail.	Management resource fee + Computing resource fee during peak hours + Computing resource fees during off-peak hours = (2 CUs × USD 0.09 per CU-hour × 24 hours × 30 days) + (30 CUs × USD 0.09 per CU-hour × 10 hours x 30 days) + (10 CUs × USD 0.09 per CU-hour × 14 hours × 30 days) = USD 129.6 + USD 810 + USD 378 = USD 1317.6 (The estimated cost will not be higher.)
Hybrid billing	10 CUs (fixed) + 20 CUs (max elastic)	You can purchase only 10 CUs of fixed computing resources to address off-peak demands, and supplement those resources with elastic computing resources for peak demands. By setting an elastic resource computing ceiling, you can restrict the usage of elastic computing resources to a maximum of 20 CUs during scaling.	Management resource fee + Computing resource fee = Management resource fee + Fixed computing resource fee + Elastic computing resource fee = (2 CUs x USD 45.94 per CU-month x 1 month) + (10 CUs x USD 45.94 per CU-month * 1 month) + (20 CUs x USD 0.09 per CU-hour x 10 hours * 30 days) = USD 91.88 + USD 459.4 + USD 540 = USD 1091.28 (The estimated cost will not be higher.)

View the usage of elastic computing resources

• View your namespace's elastic computing resource usage

  1. Go to the namespace's development console.

  2. In the left-side navigation pane, click Overview.

  3. You can hover over any of the usage progress bar in the Overview section. Alternatively, you can over a progress bar in the bottom-left corner of the page. Also, you can click Queue Management in the left-side navigation bar to view the assigned fixed and elastic computing resources for your queues within the namespace.

     

• View deployments that are using elastic computing resources

  1. Go to O&M > Deployments.

  2. In the Label field, enter vvpElasticity: true to filter for deployments using elastic resources.

     

Disable hybrid billing

• Proactive disabling of hybrid billing

  1. Cancel running deployments that are using elastic computing resources, if any.

     Alternatively, to minimize job disruption, increase your fixed computing resources to cover the current workload demands.

  2. In the Actions column of the target hybrid-billed workspace, select More > Switch off flexible billing.

  3. In the pop-up dialog box, click OK.

• Forcible release of elastic computing resources

  Elastic computing resources are forcibly released due to two reasons:

  • Your Alibaba Cloud account has overdue payments.

    If your account has overdue payments, your elastic computing resources will be automatically released, regardless of whether your fixed computing resources have expired. Your workspace will stay available, but all Flink jobs using elastic computing resources will be stopped. If you do not settle the overdue payments within 15 days of the due date, your workspace will automatically switch to the subscription billing model. Once you have cleared the overdue payments, you will need to manually re-enable hybrid billing if you wish to use it again. For information about the impact of overdue payments on Flink jobs, see Overdue payments and expiration.

  • Fixed computing resources have expired.

    If you do not renew an expired workspace (the fixed computing resources) within 15 days of the due date, it is automatically released, along with its elastic computing resources.

References

• Switch between billing methods

• View bills