All Products
Search

This Product

    Dataphin:Create a custom real-time data source type

    Document Center

    Dataphin:Create a custom real-time data source type

    Last Updated:Jan 19, 2026

    Dataphin lets you create custom real-time data source types. You can use these types to create data source instances, metatables, and Flink SQL tasks to access your custom data. This topic describes how to create a custom real-time data source type.

    Limits

    Only open source Flink (Apache Flink) and Ververica Flink compute engines are supported.

    Notes on custom Ververica Flink connectors

    After you configure a custom real-time data source type in Dataphin, you must also register the connector in the Ververica Flink real-time computing console. For more information, see Upload and use custom connectors.

    To update the connector code, you must update it in both Dataphin and the Ververica Flink real-time computing platform. The connector JAR file that you upload to the Ververica Flink real-time computing platform is the version that takes effect.

    Procedure

    1. On the Dataphin home page, in the top menu bar, click Management Center > Datasource Management.

    2. To navigate to the Create Offline Custom Source Type page, follow these steps.

      Go to Custom Source Type > Create Custom Source Type > Create Real-time Data Source Type.

      image

    3. On the Create Data Source Type page, configure the parameters.

      Parameter

      Description

      Type Name

      The name of the custom data source type. The name can be up to 128 characters long.

      Type Code

      The code of the custom data source type. The code must follow these rules:

      • Must be globally unique.

      • Can be up to 64 characters long.

      • Can contain only letters, digits, and underscores (_).

      • Cannot start with a digit.

        Note

        This parameter cannot be modified after the type is created.

      JAR Package

      Upload the JAR package for the data source type connector.

      • Only files with the .jar extension are supported. The extension is case-insensitive.

      • The file size cannot exceed 500 MB.

      • The connector cannot be the same as an existing or reserved data source connector.

      • The file can contain only data source-level encryption parameters.

        Note

        For more information about custom real-time connectors, see the Flink official website.

      Configuration File

      Upload the configuration file for the data source type connector. Click Download Configuration File Template to configure the file.

      • Only files with the .yaml extension are supported. The extension is case-insensitive.

      • The connector cannot be the same as an existing or reserved data source connector.

      • The file can contain only data source-level encryption parameters.

      The following section describes the configuration file in detail:

      # Declare the usage type of the connector (source table, sink table, or dimension table).
kind:
  source
  sink
  dim
# Define the connector name. The format is CUSTOM_([A-Z|0-9]+).
connector: CUSTOM_SOCKET
# hostname is a property of the connector. It is displayed in the WITH clause of the DDL statement.
# isSensitive: If set to false, the target property (for example, hostname) is not encrypted on the page.
# defaultValue: Specifies the default value of the target property.
# isRequired: If set to true, the property is required and marked with an asterisk (*) on the page.
# module: If the value is datasource, the property can be configured on the data source page. If the value is table, the property can be configured on the real-time metatable page.
hostname:
  isSensitive: false
  defaultValue: localhost
  isRequired: true
  module: datasource
port:
  isSensitive: false
# format: Defines the serialization format for data input or output. The module for this property is usually set to table.
format:
  name: changelog-csv
  module: table
  format_property:
    line-delimiter:
      isSensitive: false
      defaultValue: '/n'
      isRequired: true
    column-delimiter:
      isSensitive: false
      defaultValue: ','

      Description

      Enter a brief description of the custom real-time source type. The description can be up to 128 characters long.

    4. Click OK to create the custom real-time data source type.

    What to do next

    After you create a custom real-time data source type, its configuration items are automatically generated. You can then add a data source of this type to Dataphin. For more information, see Data Source Management.