All Products
Search

This Product

    Realtime Compute for Apache Flink:Quick start with PyFlink jobs

    Document Center

    Realtime Compute for Apache Flink:Quick start with PyFlink jobs

    Last Updated:Mar 26, 2026

    This topic describes how to deploy and start Flink Python stream and batch jobs in Realtime Compute for Apache Flink.

    By the end of this guide, you will have a running word-frequency counter that reads Shakespeare text, processes it with a Flink Python job, and writes results to Object Storage Service (OSS) — covering both stream and batch deployment modes.

    Prerequisites

    Before you begin, make sure you have:

    Step 1: Prepare the Python code file

    Realtime Compute for Apache Flink does not provide a Python development environment. Develop your job locally before uploading it.

    Important

    The Flink version you use for local development must match the engine version you select in Step 3. For guidance on local development, debugging, and connectors, see Python job development. For dependencies such as custom Python virtual environments, third-party Python packages, JAR packages, and data files, see Use Python dependencies.

    This guide uses a sample word-frequency counter job. Download the files below — you'll upload them in Step 2.

    Python job files (choose based on your deployment mode):

    Data file: Shakespeare

    Step 2: Upload the Python file and data file

    1. Log on to the Realtime Compute for Apache Flink console.

    2. Click Console in the Actions column for your workspace.

    3. In the left navigation pane, click File Management.

    4. Click Upload Resource and upload both the Python file and the data file you downloaded in Step 1. For details on file storage paths, see File management.

    Step 3: Deploy the Python job

    Stream job

    1. On the Job O&M page of the Operation Center, click Deploy Job > Python Job.

    2. Enter the deployment information. For the full list of configuration parameters, see Deploy a job.

      Important

      Jobs deployed to a session cluster do not support monitoring and alerts (or data curves), monitoring and alert configuration, or auto tuning. Use session clusters for development and testing only. See Job debugging.

      ParameterDescriptionExample
      Deployment ModeSelect Stream.Stream
      Deployment NameA name for the Python job.flink-streaming-test-python
      Engine VersionThe Flink engine version for the job. Use a version tagged Recommended or Stable for better reliability. See Release notes and Engine versions.vvr-8.0.9-flink-1.17
      Python File PathSelect the word_count_streaming.py file you uploaded. If it already exists in File Management, select it directly without uploading again.—
      Entry ModuleThe entry class of the program. Leave blank for .py files. For .zip files, enter the module name, such as word_count.Not required
      Entry Point Main ArgumentsThe input parameters passed to the main method. Enter the OSS path of the Shakespeare data file. Copy the full path from File Management.--input oss://<your-oss-bucket-name>/artifacts/namespaces/<project-name>/Shakespeare

      Log on to the OSS console. In the oss://<your OSS Bucket name>/artifacts/namespaces/<project name>/python-batch-quickstart-test-output folder, click the folder that is named with the start date and time of the job. Then, click the object file name and click Download in the panel that appears.

      下载

      --input oss://<your attached OSS Bucket name>/artifacts/namespaces/<project name>/Shakespeare

      --output oss://<your attached OSS Bucket name>/artifacts/namespaces/<project name>/python-batch-quickstart-test-output

      You can copy the full path of the Shakespeare file from File Management.

      --input oss://<your-oss-bucket-name>/artifacts/namespaces/<project-name>/Shakespeare

      Copy the full path of the Shakespeare file from File Management.

      Deployment TargetSelect a resource queue or a session cluster (not for production). See Manage resource queues and Create a session cluster.default-queue

      py_流_zh.jpg

    3. Click Deploy.

    Batch job

    1. On the Job O&M page of the Operation Center, click Deploy Job > Python Job.

    2. Enter the deployment information. For the full list of configuration parameters, see Deploy a job.

      Important

      Jobs deployed to a session cluster do not support monitoring and alerts, monitoring and alert configuration, or auto tuning. Use session clusters for development and testing only. See Job debugging.

      ParameterDescriptionExample
      Deployment ModeSelect Batch.Batch
      Deployment NameA name for the Python job.flink-batch-test-python
      Engine VersionThe Flink engine version for the job. Use a version tagged Recommended or Stable for better reliability. See Release notes and Engine versions.vvr-8.0.9-flink-1.17
      Python File PathSelect the word_count_batch.py file you uploaded.—
      Entry ModuleThe entry class of the program. Leave blank for .py files. For .zip files, enter the module name, such as word_count.Not required
      Entry Point Main ArgumentsThe input and output paths for the job. Copy the Shakespeare file path from File Management. Specify only the output directory name — the directory is created automatically. The output directory's parent must be the same as the input file's parent.--input oss://<your-oss-bucket-name>/artifacts/namespaces/<project-name>/Shakespeare --output oss://<your-oss-bucket-name>/artifacts/namespaces/<project-name>/python-batch-quickstart-test-output
      Deployment TargetSelect a resource queue or a session cluster (not for production). See Manage resource queues and Create a session cluster.default-queue

      py_批_zh.jpg

    3. Click Deploy.

    Step 4: Start the Python job and view results

    Stream job

    1. On the Operation Center > Job O&M page, find the target job and click Start in the Actions column.

      py_部署_zh.jpg

    2. Select Stateless Start and click Start. For details, see Start a job. After the job starts, its status changes to Running or Finished. For this sample job, the final status is Finished.

    3. View the compute results while the job status is Running. In the TaskManager, search for shakespeare in the log file ending with .out to view the Flink compute results.

      Important

      For this sample stream job, results are deleted when the status changes to Finished. View results only when the status is Running.

      image.png

    Batch job

    1. On the Operation Center > Job O&M page, find the target job and click Start.

      py_部署批_zh.jpg

    2. In the Start Job dialog box, click Start. For details, see Start a job.

    3. After the job status changes to Finished, retrieve the results from OSS. Log on to the OSS console. Navigate to oss://<your-oss-bucket-name>/artifacts/namespaces/<project-name>/python-batch-quickstart-test-output, open the subfolder named with the job's start date and time, click the object file, and then click Download. The result is an .ext file. Open it with a text editor or Microsoft Office Word to view the word-frequency output.

      下载

      result

    (Optional) Step 5: Terminate the job

    Stop and restart a job when any of the following apply:

    • You modified the code or changed the job version.

    • You added or removed WITH parameters.

    • The job cannot reuse its state.

    • You are starting a new job.

    • You updated parameters that do not take effect dynamically.

    After modifying a job, redeploy it before restarting. For details, see Stop a job.

    What's next