Dataphin:Create a stream processing template

Stream processing templates facilitate the development of real-time computing tasks, enhancing development efficiency. This topic outlines the steps to create and manage stream processing templates.

Scenarios

In basic scenarios, real-time computing tasks are required to operate continuously to ensure timely data output. These tasks must be stable, exhibit low latency, and have robust disaster recovery capabilities. To meet these requirements, production environments often implement multiple links for primary and backup disaster recovery. However, this can increase maintenance costs and complexity. For instance, task modifications necessitate manual updates, which can lead to inconsistencies. By creating a stream processing template in Dataphin, you can instantiate tasks for real-time computing by referencing the template. This allows for streamlined maintenance and updates of multiple real-time tasks, significantly reducing development costs and mitigating the risk of inconsistencies.

Procedure

1. See Data Development Entry to access the Development page.

2. On the Development page, use the following guide to open the New Template dialog box.

   

3. In the New Template dialog, enter the required parameters.

   Parameter	Description
   Template Name	Specify the template name, which is immutable post-creation. Acceptable characters include English letters, numbers, underscores (_), and hyphens (-).
   Select Directory	Choose the directory where the template will reside. If the appropriate directory does not exist, create a new one as per the guide below.
   Description	Provide a concise description of the template, limited to 1000 characters.

4. Click OK.

5. Develop the stream processing template code on the code development page.

   Important

   The template code must include at least one parameter to be compiled and submitted correctly. Use the following format for parameters: @@{parameter name}.

   

   Note

   To automatically format the SQL code, click the button at the top right of the page.

6. Upon finalizing the template code, click the icon at the upper left of the page. Then, in the Parameter Filling dialog box, select Engine Version and Fill Parameter Value.

   Parameter	Description
   Engine Version	Choose the engine version for precompiling the code.
   Parameter Filling	Input the parameter values for precompilation. If unconfigured, select Fill Empty Value or All Empty Values.

7. Click OK to validate the code task's syntax and permissions.

   Note

   • A successful precompilation triggers a notification at the top of the page.

   • A failed precompilation results in a alert at the top of the page. Click the Console at the bottom to review the precompilation error log.

7. After development and precompilation, configure the parameters' description information and default value for the stream processing template. Use the guide below to set up the parameters in the Attribute pop-up.

   

   Parameter	Description
   Parameter Description	Detail the parameter's purpose within 128 characters.
   Default Value	Specify the default value for the parameter.

   Note

   Dataphin supports the configuration of runtime parameter information in real-time templates. Tasks created from the current template will inherit these parameters. For details, see Configure Real-time Template Runtime Parameters.

8. Submit the code template by clicking the button at the top left of the page, and in the Submit Code Template dialog box, fill in the Version Name and Remarks.

9. To finalize, click OK And Submit.

What to do next

Upon successful publication, you can access and control your stream processing template tasks via the Operation Center. For more information, see View and Manage Real-time Tasks or .