ApsaraMQ for Kafka:Create a Kafka sink connector

Create a sink connector task

1. Log on to the ApsaraMQ for Kafka console. In the Resource Distribution section of the Overview page, select the region where the ApsaraMQ for Kafka instance that you want to manage resides.

2. In the left-side navigation pane, choose Connector Ecosystem Integration > Tasks.

3. On the Tasks page, click Create Task.

4. In the Create Task panel, set the Task Name and Description, configure the task as described below, and click Save.

   • Task creation

     1. In the Source step, select Message Queue for Apache Kafka as the Data Provider. Configure the following parameters, and then click Next.

        Parameter	Description	Example
        Region	The region of the source Message Queue for Apache Kafka instance.	China (Beijing)
        Kafka instance	The source Message Queue for Apache Kafka instance.	alikafka_post-cn-jte3****
        Topic	The topic from which messages are consumed.	demo-topic
        Group ID	The consumer group of the source instance. Quick Create (recommended): A group ID in the GID_EVENTBRIDGE_xxx format is automatically created. Use Existing: Select an existing group ID. Do not share this group ID with other services to avoid disrupting message consumption.	Quick Create
        Consumer offset	The offset from which message consumption starts. Latest offset (latest) Earliest offset (earliest)	Latest offset (latest)
        Network configuration	The network type for routing messages. Basic Network Self-managed Internet	Basic Network
        VPC	Required only if Network configuration is set to Self-managed Internet.	vpc-bp17fapfdj0dwzjkd****
        vSwitch	Required only if Network configuration is set to Self-managed Internet.	vsw-bp1gbjhj53hdjdkg****
        Security group	Required only if Network configuration is set to Self-managed Internet.	alikafka_pre-cn-7mz2****
        Data Format	The encoding format for the message content. We recommend Json if you do not have specific encoding requirements. Json: Encodes binary data as a JSON object into the payload using UTF-8. Text: Encodes binary data as a UTF-8 string into the payload. This is the default format. Binary: Encodes binary data as a Base64-encoded string into the payload.	Json
        Messages	Advanced configuration parameter. The maximum number of messages per batch. A request is sent when the accumulated message count reaches this value. Valid values: 1 to 10,000.	100
        Interval (Unit: Seconds)	Advanced configuration parameter. The interval, in seconds, at which messages are aggregated and sent to the sink. Valid values: 0 to 15. A value of 0 means messages are delivered immediately.	3

     2. In the Filtering step, set the Pattern Content to filter events. For more information, see event pattern.

     3. In the Transformation step, configure data transformation to perform operations such as splitting, mapping, enrichment, and dynamic routing. For more information, see Use Function Compute to clean message data.

     4. In the Sink step, set Service Type to ApsaraMQ for Kafka and configure the following parameters.

        Parameter	Description	Example
        Instance ID	Select the destination ApsaraMQ for Kafka instance.	test
        Topic	Select a topic in the destination instance.	test
        Acknowledgment Mode	Select the acknowledgement signal that ApsaraMQ for Kafka sends to the client after receiving data. None LeaderOnly All	None
        Message Value	EventBridge uses a JSONPath expression to extract specific content from an event and route it to the destination. Complete Data Data Extraction Fixed Value Template	Extract from Event $.data.value
        Message Key	EventBridge uses a JSONPath expression to extract specific content from an event and route it to the destination. Null Data Extraction Fixed Value Template	Extract from Event $.data.key

   • Task properties

     Configure the retry policy and dead-letter queue for this task. For more information, see Retry and dead-letter queues.

5. On the Tasks page, find your task and click Enable in the Actions column.

6. In the Note dialog box, read the message and then click OK.

   The task takes 30 to 60 seconds to start after being enabled. You can view the startup progress in the Status column on the Tasks page.

Other operations

On the Tasks page, find the target task and perform the following operations in the Actions column.

• View task details: Click Details. On the Task page, you can view the task's basic information, properties, and monitoring metrics.

• Edit task configuration: Click Edit. In the Edit Task panel, you can modify the task details and properties.

• Enable or pause the task: Click Enable or Pause. In the Note dialog box, click OK.

• Delete the task: Click Delete. In the Note dialog box, click OK.