ApsaraMQ for Kafka:Create a Function Compute sink connector

Function Compute

Function Compute (FC) is a fully managed, event-driven, serverless compute service. You write and upload code without managing servers. Function Compute prepares computing resources and runs your code elastically and reliably. For more information, see What is Function Compute?.

Use cases

• Process business messages with functions. Develop and run business logic for message processing, order processing, and task execution on Function Compute.

• Quickly process messages, such as ETL cleansing.

• Forward messages to downstream systems in a specified VPC with flexible scaling.

• Connect messaging systems with other Alibaba Cloud services and route message data to additional cloud services.

Create a sink connector

1. Log on to the ApsaraMQ for Kafka console. On the Overview page, select a region in the Resource Distribution section.

2. In the left-side navigation pane, choose Connector Ecosystem Integration > Tasks.

3. On the Tasks page, click Create Task.

4. In the Create Task panel, set Task Name and Description, configure the following parameters, and then click Save.

   • Task Creation

     1. In the Source step, select Message Queue for Apache Kafka as the Data Provider. Configure the following parameters, and then click Next.

        Parameter	Description	Example
        Region	The region of the source Message Queue for Apache Kafka instance.	China (Beijing)
        Kafka instance	The source Message Queue for Apache Kafka instance.	alikafka_post-cn-jte3****
        Topic	The topic from which messages are consumed.	demo-topic
        Group ID	The consumer group of the source instance. Quick Create (recommended): A group ID in the GID_EVENTBRIDGE_xxx format is automatically created. Use Existing: Select an existing group ID. Do not share this group ID with other services to avoid disrupting message consumption.	Quick Create
        Consumer offset	The offset from which message consumption starts. Latest offset (latest) Earliest offset (earliest)	Latest offset (latest)
        Network configuration	The network type for routing messages. Basic Network Self-managed Internet	Basic Network
        VPC	Required only if Network configuration is set to Self-managed Internet.	vpc-bp17fapfdj0dwzjkd****
        vSwitch	Required only if Network configuration is set to Self-managed Internet.	vsw-bp1gbjhj53hdjdkg****
        Security group	Required only if Network configuration is set to Self-managed Internet.	alikafka_pre-cn-7mz2****
        Data Format	The encoding format for the message content. We recommend Json if you do not have specific encoding requirements. Json: Encodes binary data as a JSON object into the payload using UTF-8. Text: Encodes binary data as a UTF-8 string into the payload. This is the default format. Binary: Encodes binary data as a Base64-encoded string into the payload.	Json
        Messages	Advanced configuration parameter. The maximum number of messages per batch. A request is sent when the accumulated message count reaches this value. Valid values: 1 to 10,000.	100
        Interval (Unit: Seconds)	Advanced configuration parameter. The interval, in seconds, at which messages are aggregated and sent to the sink. Valid values: 0 to 15. A value of 0 means messages are delivered immediately.	3

     2. In the Filtering step, set the Pattern Content to filter events. For more information, see event pattern.

     3. In the Transformation step, configure data transformation to perform operations such as splitting, mapping, enrichment, and dynamic routing. For more information, see Use Function Compute to clean message data.

     4. In the Sink step, set Service Type to Function Compute and configure the following parameters.

        Parameter	Description	Example
        Function	Select an existing Function Compute function.	test
        Version and Alias	Select whether to use a specific version or an alias. Specified Version Specified Alias	Specified Version
        Version	The function version. The latest version is recommended. This parameter is required if you set Version and Alias to Specified Version.	LATEST
        Alias	The alias of the function. This parameter is required if you set Version and Alias to Specified Alias.	test
        Invocation Mode	The invocation mode. Valid values: Synchronous and Asynchronous.	Asynchronous
        Event Format	Object: Events are delivered to the downstream function as an object. ObjectList: Events are delivered to the downstream function as an array of objects.	Object format
        Event	EventBridge uses JSONPath to extract and route specific message content to the target. Complete Data Data Extraction Fixed Value Template	Complete Data

   • Task Properties

     Configure the retry policy and dead-letter queue for the task. For more information, see Retries and dead-letter queues.

5. Go back to the Tasks page. Find the task that you created and click Enable in the Actions column.

6. In the Note dialog box, read the message and click OK.

   After you enable the task, it may take 30 to 60 seconds for the task to start. Monitor the progress in the Status column on the Tasks page.

Other operations

On the Tasks page, find the target task and perform the following operations in the Actions column.

• View task details: Click Details. On the Task page, you can view the task's basic information, properties, and monitoring metrics.

• Edit task configuration: Click Edit. In the Edit Task panel, you can modify the task details and properties.

• Enable or pause the task: Click Enable or Pause. In the Note dialog box, click OK.

• Delete the task: Click Delete. In the Note dialog box, click OK.