All Products
Search

This Product

    Dataphin:Launch a real-time instance

    Document Center

    Dataphin:Launch a real-time instance

    Last Updated:May 28, 2025

    After you complete the development and publishing of a real-time task, you can launch the real-time instance in the Operation Center. This topic describes how to launch a real-time instance.

    Entry points for launching real-time instances

    Entry point 1: Launch a real-time instance from the real-time task entry

    1. In the top navigation bar of the Dataphin homepage, choose Development > Task O&M.

    2. In the left navigation pane, choose Task O&M > Stream Task to open the Stream Task page.

    3. On the Stream Task page, click the start icon in the Actionsimage column of the target real-time task.

    Entry point 2: Launch a real-time instance from the real-time instance entry

    1. In the top navigation bar of the Dataphin homepage, choose Development > Task O&M.

    2. In the left navigation pane, choose Instance O&M > Real-time Instance to open the Real-time Instance page.

    3. On the Real-time Instance page, click the start icon in the Actionsimage column of the target real-time task.

    Instructions for launching real-time instances

    The methods for launching real-time instances vary depending on the compute engine, real-time instance type, and environment (development or production). The instructions are as follows:

    Ververica Flink, Fusioninsight, and open source Flink real-time engines

    Launch a real-time computing instance

    Production environment

    1. In the Launch Real-time Instance dialog box, configure the launch information.

      Parameter

      Description

      Launch Mode

      Supports stateless launch and launch from a valid state.

      • Stateless Launch: Launches without any initial state. When launching statelessly, you can specify the time point from which the source table reads data. Setting a specific start time will make all source tables that support startTime read data from that moment.

        Important

        • Currently, Log Service, Kafka, MySQL, MongoDB, and OceanBase support specifying the time point from which the source table reads data.

        • The parameter configuration in SQL jobs takes precedence over this specified start time.

        • Fusioninsight Flink engine does not support reading data from a specific time point.

      • Launch from Valid State: Launches from an existing valid state. You can select a Checkpoint or Savepoint from the dropdown menu. Only the 8 most recent valid states (Checkpoint or Savepoint) are available for selection. By default, the instance is launched from the latest state.

        Note

        If the real-time instance does not have recent valid states, launching from a valid state is not supported.

      Source Data Time Zone

      The time zone of the source data source. This cannot be modified.

      Data Timestamp

      You can configure the data timestamp to assign values to time variables. Time variables are calculated based on the data timestamp. The default is the current day.

      Note

      Real-time integration tasks do not support data timestamp configuration.

      Note

      If your project's compute source is Ververica Flink, the system will automatically determine whether your Ververica Flink compute source uses the latest interface. If it uses the latest interface, you also need to select either Run from Start Offset or Run from Latest State.

      • Run From Start Offset: Starts the instance based on the configured data reading time point and data timestamp.

      • Run From Latest State: Resumes running from the original state of the real-time instance.

    2. Click Launch.

    Development environment

    1. In the Test Instance Parameter Configuration dialog box, configure specific values for the test instance parameters.

      Parameter

      Description

      Specify Source Table Data Reading Time Point

      Set the data reading time point. Setting a specific start time will make all source tables that support startTime read data from that moment.

      Important

      • Currently, Log Service, Kafka, MySQL, MongoDB, and OceanBase support specifying the time point from which the source table reads data.

      • The parameter configuration in SQL jobs takes precedence over this specified start time.

      • Fusioninsight Flink engine does not support reading data from a specific time point.

      Test Method

      The test methods for Flink real-time tasks include log printing and writing to test tables:

      • If you select log printing, the test results are written to logs and no test table is generated.

      • If you select writing to test tables, the test results are not written to logs, but corresponding test tables are generated.

        Important

        You need to ensure that the test table already exists in the physical database (such as MySQL). Otherwise, the real-time instance will fail to run.

      Input Table

      No configuration required. This displays the input table name of the current real-time instance.

      Data Timestamp

      You can configure the data timestamp to assign values to time variables. Time variables are calculated based on the data timestamp. The default is the current day.

      Whether the system reads from development tables or production test meta tables during test instance execution depends on the meta table configuration. To view meta table configurations, see Create a meta table.

      • If the meta table's development environment test reading parameter is set to development table:

        • If the task uses Project_Name_dev.meta_table_name, it reads from the development environment test meta table.

        • If the task uses Project_Name.meta_table_name, it reads from the production meta table. If you do not have permission to access the production environment test meta table, an error occurs. For information about how to apply for production environment test meta table permissions, see Apply for, renew, and relinquish table permissions.

        • If the task uses ${Project_Name}.meta_table_name or just meta_table_name, it reads from the development environment test meta table.

      • If the meta table's development environment test reading parameter is set to production table:

        • If the task uses Project_Name_dev.meta_table_name, it reads from the development environment test meta table.

        • If the task uses Project_Name.meta_table_name, it reads from the production environment test meta table.

        • If the task uses ${Project_Name}.meta_table_name or just meta_table_name, it reads from the production environment test meta table.

      • If the meta table's development environment test writing parameter is set to other test tables:

        • If the task uses Project_Name_dev.meta_table_name, it writes to the development test meta table.

        • If the task uses Project_Name.meta_table_name, it writes to the production environment test meta table. If you do not have permission to access the production environment test meta table, an error occurs. For information about how to apply for production environment test meta table permissions, see Apply for, renew, and relinquish table permissions.

        • If the task uses ${Project_Name}.meta_table_name or just meta_table_name, it writes to the development environment test meta table.

    2. Click Launch.

    Launch a real-time integration instance in development/production environment

    1. In the Launch Real-time Instance dialog box, configure different parameters based on the Launch Scope.

      Incremental + Full

      Parameter

      Description

      Launch Scope

      Select Incremental + Full.

      Only when the Sync solution of the real-time integration node is set to Real-time incremental + full, you can select Incremental + full.

      Important

      When synchronizing objects in full mode, the data in the corresponding target table will be cleared.

      Launch Mode

      Supports Resume and Initialization.

      • Resume: Launches temporary full synchronization tasks for all incomplete synchronization objects (excluding those that are completed, stopped, or ignored). After all temporary tasks are completed, the real-time incremental synchronization task is launched.

      • Initialization: Launches temporary full synchronization tasks for all synchronization objects. After all temporary tasks are completed, the real-time incremental synchronization task is launched.

      Resource Group

      Sets the schedule resource group for the current real-time instance. The default is the resource group specified for full synchronization when the task was created.

      Memory

      Does not support decimals or negative numbers. The maximum memory cannot exceed 16384MB (16GB). The default value is 4096MB. If empty, the default is 1024MB.

      Rate Limiting

      Limits speed based on data volume size. Does not support decimals or negative numbers. The default value is 10M/s. If empty, there is no rate limit.

      Concurrency

      Sets the expected maximum concurrency for full tasks. Does not support decimals or negative numbers. The default value is 10. If empty, the concurrency is 3.

      Incremental only

      Parameter

      Description

      Launch Scope

      Select Incremental Only.

      Launch Mode

      Supports Stateless Launch, which means launching without any initial state.

      Only the real-time incremental synchronization task is launched. The status of Not Started and Synchronization Failed full synchronization temporary tasks is set to Ignored.

      Source Data Time Zone

      The time zone of the source data source. This cannot be modified.

      Data Reading Time Point

      Setting a specific start time will make all source tables that support startTime (Log Service, Kafka, MySQL, MongoDB, OceanBase) read data from that moment. The default is the current time.

      Important

      The parameter configuration in SQL jobs takes precedence over this specified start time.

      Fusioninsight Flink engine does not support reading data from a specific time point.

    2. Click Launch.

    Alibaba Blink real-time engine

    Launch a real-time instance in development environment

    1. In the Test Instance Parameter Configuration dialog box, configure specific values for the test instance parameters.

      Parameter

      Description

      Data Reading Time Point

      Sets the consumption time for reading source data.

      Test Method

      The test methods for Flink real-time tasks include log printing and writing to test tables:

      • If you select log printing, the test results are written to logs and no test table is generated.

      • If you select writing to test tables, the test results are not written to logs, but corresponding test tables are generated.

        Important

        You need to ensure that the test table already exists in the physical storage engine (such as HBase, Kafka, etc.). Otherwise, the real-time instance will fail to run.

      Input Table

      No configuration required. This displays the input table name of the current real-time instance.

      Data Timestamp

      You can configure the data timestamp to assign values to time variables. Time variables are calculated based on the data timestamp. The default is the current day.

      Whether the system reads from development tables or production environment test meta tables during test instance execution depends on the meta table configuration. To view meta table configurations, see Create a meta table.

      • If the meta table's development environment test reading parameter is set to development table:

        • If the task uses Project_Name_dev.meta_table_name, it reads from the development environment test meta table.

        • If the task uses Project_Name.meta_table_name, it reads from the production meta table. If you do not have permission to access the production environment test meta table, an error occurs. For information about how to apply for production environment test meta table permissions, see Apply for, renew, and relinquish table permissions.

        • If the task uses ${Project_Name}.meta_table_name or just meta_table_name, it reads from the development environment test meta table.

      • If the meta table's development environment test reading parameter is set to production table:

        • If the task uses Project_Name_dev.meta_table_name, it reads from the development environment test meta table.

        • If the task uses Project_Name.meta_table_name, it reads from the production environment test meta table.

        • If the task uses ${Project_Name}.meta_table_name or just meta_table_name, it reads from the production environment test meta table.

      • If the meta table's development environment test writing parameter is set to other test tables:

        • If the task uses Project_Name_dev.meta_table_name, it writes to the development test meta table.

        • If the task uses Project_Name.meta_table_name, it writes to the production environment test meta table. If you do not have permission to access the production environment test meta table, an error occurs. For information about how to apply for production environment test meta table permissions, see Apply for, renew, and relinquish table permissions.

        • If the task uses ${Project_Name}.meta_table_name or just meta_table_name, it writes to the development environment test meta table.

    2. Click Launch.

    Launch a real-time instance in production environment

    1. In the Launch Real-time Instance dialog box, configure the launch information.

      Parameter

      Description

      Data Reading Time Point

      Sets the consumption time for reading source data.

      Data Timestamp

      You can configure the data timestamp to assign values to time variables. Time variables are calculated based on the data timestamp. The default is the current day.

    2. Click Launch.