This topic describes how to create a DLA sink connector to synchronize data from Message Queue for Apache Kafka to Data Lake Analytics (DLA) for analytics and computing.

Prerequisites

Create a DLA sink connector

  1. Log on to the Message Queue for Apache Kafka console.
  2. In the Resource Distribution section of the Overview page, select the region where your instance resides.
  3. In the left-side navigation pane, click Connectors.
  4. On the Connectors page, click Create Connector.
  5. Complete the Create Connector wizard.
    1. In the Configure Basic Information step, set the parameters that are described in the following table and click Next.
      Parameter Description Example
      Name The name of the connector. Take note of the following rules when you specify a connector name:
      • The connector name must be 1 to 48 characters in length. It can contain digits, lowercase letters, and hyphens (-), but cannot start with a hyphen (-).
      • Each connector name must be unique within a Message Queue for Apache Kafka instance.

      The data synchronization task of the connector must use a consumer group that is named in the connect-Task name format.Group If you have not created such a consumer group, Message Queue for Apache Kafka automatically creates one for you.Group

      		dla-test
      Instance The information about the Message Queue for Apache Kafka instance. By default, the name and ID of the instance are displayed. demo alikafka_post-cn-st21p8vj****
    2. In the Configure Source Service step, select Message Queue for Apache Kafka as the source service, set the parameters that are described in the following table, and then click Next. For the runtime environment parameters that are not described in the following table, retain the default settings.
      Parameter Description Example
      Data Source Topic The name of the data source topic from which data is to be synchronized. dla-test
      Consumer Offset The offset where consumption starts. Valid values:
      • Earliest Offset: Consumption starts from the earliest offset.
      • Latest Offset: Consumption starts from the latest offset.
      		Earliest Offset
    3. In the Configure Destination Service step, select Data Lake Analytics as the destination service, set the parameters that are described in the following table, and then click Create.
      Parameter Description Example
      Database Name The name of the DLA database to which data is to be synchronized. The name cannot be the same as that of an existing database. The name can contain letters, digits, and underscores (_) and must start with a letter. dla_fromkafka
      Table Name The name of the DLA table to which data is to be synchronized. By default, the name of the data source topic in the Message Queue for Apache Kafka instance is used as the name of the DLA table to which data is to be synchronized. The characters except for letters and digits are replaced by underscores (_). For example, if the name of the data source topic is dla-test, the name of the DLA table to which data is to be synchronized is dla_test. dla_test
    4. In the Note: The configuration of the connector is not complete. Go to the Data Lake Analytics (DLA) console to complete the required configurations. message, read the instructions and click Configure Now to go to the DLA console.
  6. If you have not created a lakehouse, you are redirected to the Lakehouse Lake warehouse integration page, where you can create a lakehouse. If you have created a lakehouse, you are redirected to the New workload page. On the Kafka data source tab, you can create and start a workload for data analytics.
    On the Connectors page of the Message Queue for Apache Kafka console, you can view the created DLA sink connector. You can also perform the following operations in the Actions column:
    • Click Details to view the configurations of the DLA sink connector.
    • Click View Tasks to go to the DLA console and start a workload for data analytics.
    • Click Delete to delete the DLA sink connector.