Workspace management and operations

• Workspace and namespace management

  • Can I change the name of a workspace?

  • Can I change the VPC and vSwitch?

  • How do I view workspace information such as the workspace ID?

  • What do I do if resource allocation fails when I scale in a namespace?

  • Why do resource changes not take effect after scaling out Flink computing resources?

  • How do I view my AccessKey ID and AccessKey Secret?

  • How do I upload files to OSS for Flink deployments to use?

  • Why is my activated Flink workspace not displayed in the console?

  • Error: Has not enough ip address: abnormal event detected from kubernetes

  • Error: Unknown Error: Http failure response for xxxxx

• Deployment management

  • How do I view the Flink version of a deployment?

  • How do I change the engine version for a deployment?

  • How do I configure custom running parameters for a deployment?

  • How do I persist common Flink configurations to improve efficiency?

  • How do I prevent leaks of sensitive information such as AccessKey pairs or passwords?

  • How do I optimize resource utilization for Flink deployments?

  • Cron expression syntax

Can I change the name of a workspace?

1. In the Realtime Compute for Apache Flink console, find the target workspace and click Details in the Actions column.

2. Click the workspace name to modify it.

Can I change the VPC and vSwitch?

You cannot change the Virtual Private Cloud (VPC), but you can change the vSwitch.

View workspace information

In the Realtime Compute for Apache Flink console, find the target workspace and click Details in the Actions column to view information such as the workspace ID.

The Workspace Details panel also contains information such as Workspace name, OSS bucket, VPC name and ID, Processor architecture, Resource group, and vSwitch.

Resource allocation fails during namespace scale-in

• Issue description

  Resource allocation fails when you scale in a namespace.

  The error message reads: "Namespace resource allocation failed, please go to the console page to adjust the queue resources and retry."

• Cause

  All available resources are already allocated or in use.

• Solution

  First, reduce the resources allocated to the resource queue, and then scale in the namespace. For more information, see Modify resource quotas.

  On the Namespace Resources page, find the target resource queue in the queue list and click Edit a queue in the Actions column to reduce its allocated CUs.

Resource changes not effective after scale-out

• Subscription

  Resource scaling for Realtime Compute for Apache Flink follows a top-down approach. You allocate resources from a workspace to its namespaces, and then from the namespaces to their resource queues. For more information, see Scale resources.

  

• Pay-as-you-go

  With the pay-as-you-go billing method, you are billed based on your actual resource usage. Resource scaling is not applicable. You can find the target workspace and choose More > Quota Limit in the Actions column. Then, configure a resource queue within the namespace to meet the resource isolation and management needs of batch processing scenarios. For more information, see Manage resource queues.

View AccessKey ID and AccessKey Secret

• AccessKey ID

  • For a root account: Log on to the Alibaba Cloud console. Hover over your profile picture in the upper-right corner and click AccessKey Management to view the ID.

  • For a RAM user: See View the AccessKey information of a RAM user.

• AccessKey Secret

  To reduce the risk of leaks, the AccessKey Secret for both root accounts and RAM users is shown only at the time of creation and cannot be retrieved later. You must securely store your AccessKey Secret when you create it.

If you do not have an AccessKey pair or have lost the AccessKey Secret, see Create an AccessKey pair.

Upload files to OSS for Flink deployments

When activating a Flink workspace, you can specify an Object Storage Service (OSS) bucket for storage. This allows you to upload files to the bucket for use in Flink deployments. You can also upload files directly on the Artifacts page in the development console of Realtime Compute for Apache Flink. For more information, see Artifacts.

1. In the Real-time Compute management console, click More > Workspace Details in the Actions column of the target workspace to view the associated OSS bucket information.

2. Upload the resource file to the target OSS bucket.

   • Log on to the OSS console and upload the resource file to the /artifacts/namespaces directory of the target bucket.

   • Upload the resource file by calling the PutObject API operation. For more information, see PutObject.

3. In the left navigation bar of the Realtime Compute development console, click Artifacts to view the resource files uploaded from the OSS console.

   This page lists the uploaded files with columns for Name, Creation time, and Actions (Download, Delete). An Upload resource button is available at the top of the page.

Flink workspace not displayed in the console

Check the following possible causes. If the issue persists, you can submit a ticket for support.

• Confirm that the Flink workspace has finished initializing. This process typically takes 5 to 10 minutes.

• Make sure you have selected the correct region at the top of the console page. Workspaces are displayed only in the region where they were created.

• If you are logged in as a RAM user, ensure that the user has the necessary permissions for the Realtime Compute for Apache Flink console. For more information about permission configuration, see Permissions.

Error: Not enough IP addresses

• Error details

  Has not enough ip address:abnormal event detected from kubernetes (type:[Warning], reason:[CniError_CodeUnKnownErr], message:[CniAllocateError: allocateIP failed: ipamCreate failed: failed to create ENI: all vSwitches ([*****]) cannot be used: CreateNetworkInterface: RequestId: 67959AE5-EA20-5CB4-8560-5BD6752472FD, ErrorCode: InvalidVSwitchId.IpNotEnough, Message: The specified VSwitch "*****" has not enough IpAddress., elapsedTime: 245.03232ms])

• Cause

  The vSwitch has no available IP addresses.

• Solution

  Change to a vSwitch with sufficient available IP addresses. For more information, see Change a vSwitch.

Error: HTTP failure response

• Error details

  File upload fails with the following error message.

  

• Cause

  The Alibaba Cloud International site console domain has changed. If you use the new alibabacloud.com domain to upload files without first configuring a cross-origin resource sharing (CORS) rule in Object Storage Service (OSS), the request will fail due to cross-origin restrictions.

• Solution

  In the OSS console, configure a CORS rule for the OSS bucket that is bound to your Flink workspace. For specific instructions, see Artifacts.

View the Flink version of a deployment

You can check the version in either of the following ways:

• On the ETL or Data ingestion page, click Configurations on the right side. The version is shown in the Engine Version field.

• On the O&M > Deployments page, click the name of the target deployment. On the Configuration tab, view the version information in the Basic section.

  The Engine Version field shows the Flink version, for example, vvr-8.0.9-flink-1.17.

Change the engine version for a deployment

• For SQL or YAML deployments

  On the ETL or Data ingestion page, click Configurations on the right side and select the target version from the Engine Version drop-down list. If the deployment is already running, you must redeploy and start it for the version change to take effect.

• For JAR or Python deployments

  On the Deployments page, click the name of the target deployment. On the Configuration tab, click Edit in the upper-right corner of the Basic section. Select the target version from the Engine Version drop-down list, click Save, and then click Start.

Configure custom running parameters

1. Log on to the Realtime Compute for Apache Flink console.

2. Find the target workspace and click Console in the Actions column.

3. On the Deployments page, click the name of the target deployment.

4. On the Configuration tab, click Edit in the upper-right corner of the Parameters section.

5. In the Other configuration field, enter the parameter configurations.

   Ensure that there is a space after the colon (:) in each key-value pair. The following is a code sample.

   task.cancellation.timeout: 180s

6. Click Save.

Persist common Flink configurations

Realtime Compute for Apache Flink provides a deployment template feature to save frequently used parameters as defaults. This avoids the need to manually configure each new deployment.

In the left-side navigation pane, click Configuration Management and select the Default deployment configuration tab. On the Streaming deployment or Batch deployment sub-tab, you can set running parameters such as Checkpoint interval, Min pause between checkpoints, Checkpoint timeout, State TTL, and Flink restart strategy (for example, Fixed Delay). After you finish the configuration, click Save changes.

Prevent leaks of sensitive information

Use variables to avoid the security risks of exposing sensitive information like AccessKey pairs and passwords in plaintext. Reusing variables also prevents you from repeatedly writing the same code or values, which simplifies configuration management. You can use variables in various scenarios, including SQL deployments, JAR or Python deployments, logging configurations, and UI settings. For more information, see Manage variables.

Optimize resource utilization for Flink deployments

You can adjust resources in the following ways:

• Manual adjustment: On the Diagnosis tab, click Diagnose to view the resource analysis conclusions and adjust resources manually based on the suggestions.

  After the diagnosis is complete, the page displays the results (for example, All 5 checks passed. No fixable issues found.). You can expand the Resource analysis section to view detailed conclusions.

• Automatic adjustment: Enable the automatic tuning feature, which includes intelligent and scheduled tuning modes, to allow the Flink system to adjust resources automatically. For more information about the use cases and configuration, see Configure automatic tuning.

Cron expression syntax

State management uses the Unix-style Cron expression syntax, whereas workflow orchestration uses the Java-style syntax. Make sure to use the correct Cron expression for each scenario.

Unix-style syntax

1. A Unix-style expression has five fields: minute hour day-of-month month day-of-week.

2. Union of Day-of-Month and Day-of-Week: If you specify values for both the third field (day-of-month) and the fifth field (day-of-week), they are combined with a logical OR .

   The expression 0 12 1 * 5 means the task runs at 12:00 PM on the 1st of every month OR every Friday. It does not mean the task runs only when the 1st of the month is also a Friday.

3. Question mark not supported?: Do not use ? because the system will report an error or fail to recognize it. If you want to ignore a field, you can only use *.

Java-style (Quartz) syntax

1. A Java-style expression typically uses six fields: second minute hour day-of-month month day-of-week.

2. Conflict between Day-of-Month and Day-of-Week:

   • You cannot specify values for both the day-of-month and day-of-week fields simultaneously (or set both to *).

   • To avoid logical conflicts, such as specifying the 1st of the month and also Friday when the 1st is not a Friday, one of these two fields must be ?.

   • Rule: If you specify Day-of-month, Day-of-week must be ?. If you specify Day-of-week, Day-of-month must be ?.

3. Different Day-of-Week values: 1 represents Sunday, 2 represents Monday, and so on. We recommend using English abbreviations (SUN, MON, TUE) to avoid confusion.