When you create a workspace, Realtime Compute for Apache Flink creates several directories in the Object Storage Service (OSS) bucket that you provide. These directories store files used or created by your Flink jobs, including checkpoints, savepoints, logs, and JAR packages. This topic describes these directories and explains how to manage your artifacts.
OSS bucket
Directory contents
If you select an OSS bucket as your storage type, Realtime Compute for Apache Flink creates the following directories in that bucket.
|
Directory |
Contents |
|
artifacts |
Stores uploaded JAR packages, additional dependencies, and Python files and their dependencies. |
|
sql-artifacts |
Stores files required by registered user-defined functions and connectors. |
|
flink-jobs |
Stores high availability (HA) information and checkpoint data for Flink jobs. |
|
flink-savepoints |
Stores savepoint files generated when you click Create Savepoint in the Flink console. |
|
flink-sessionclusters |
Stores high availability (HA) information and checkpoint data for session clusters. |
|
plan |
Stores resource configuration information when you use expert mode. |
|
logs |
Stores job logs if you configure your job to output logs to OSS. |
Manage artifacts
When using an OSS bucket, you can only upload, download, and delete files in the artifacts directory from the Flink console.
-
Log on to the Realtime Compute for Apache Flink console.
-
In the Actions column for the target workspace, click Console.
-
In the Object Storage Service (OSS) console, configure a cross-origin resource sharing (CORS) rule for the OSS bucket associated with your workspace.
If a cross-origin resource sharing (CORS) rule is not configured, file uploads may fail due to cross-origin restrictions. When you configure the CORS rule, set the Origin to https://vvpsg.console.alibabacloud.com, allow all Methods (select all), and set the Allowed Headers to
*. -
In the left-side navigation pane, click Artifacts. You can perform the following operations.
Actions
Description
Upload Artifact
Click Upload Artifact and select a local file, or drag the file directly into the artifact list area.
After the upload, the file path is
oss://<Your bound OSS Bucket name>/artifacts/namespaces/<Project name>/<File name>. You can click the file name to copy the path.For more information about the project name, see View and delete a project.
Download Artifact
In the Actions column, click Download to save the artifact to your local machine.
Delete Artifact
In the Actions column, click Delete to permanently remove the artifact from the OSS bucket.
Transfer acceleration
If you experience slow uploads or downloads of large files over long distances, you can enable transfer acceleration. To enable it, turn on the Transfer Acceleration switch in the upper-right corner of the page.
To use this feature, you must first enable transfer acceleration for your OSS bucket. Enabling this feature incurs additional data transfer costs. For more information about billing, see Transfer acceleration fees.
Related topics
-
For instructions on deploying SQL, JAR, and Python jobs, see Deploy a job.
-
For instructions on viewing detailed logs for running jobs, see View startup and runtime logs.
-
For instructions on viewing job checkpoint and savepoint information, see Manage state sets.