This topic describes how to view connector task configuration information, view connector logs, enable pausing connectors, modify connector configurations, test connectors, view consumption details, and delete connectors in the Message Queue for Apache Kafka console.

Prerequisites

One of the connectors that are described in the following topics is created:
Note You can modify connector configurations only for FC Sink Connector. If you need to modify the connector configuration, you only need to create the FC Sink Connector in advance.

Precautions

  • You can modify connector configurations only for FC Sink Connector.
  • For more information about DLA Sink Connector operations, see Create a DLA sink connector.
  • If the connector is suspended, the connector must be in the Running state. If the connector is enabled, the connector must be in the Suspended state.

View task configurations of a connector

You can view the basic information, source instance configuration, and destination instance configuration of the created connector.

  1. Log on to the Message Queue for Apache Kafka console.
  2. In the Resource Distribution section of the Overview page, select the region where your instance resides.
  3. In the left-side navigation pane, click Connectors.
  4. On the Connectors page, select the instance to which the connector belongs from the drop-down list of the Select Instance, find the target connector in the Connector list under the instance, and click Details in the Actions column.
    On the Connector Details page, the Basic Information,Source Information, and Destination Information of the connector are displayed.

View connector logs

This topic describes how to view connector logs in the Message Queue for Apache Kafka console to troubleshoot problems.

  1. Log on to the Message Queue for Apache Kafka console.
  2. In the Resource Distribution section of the Overview page, select the region where your instance resides.
  3. In the left-side navigation pane, click Connectors.
  4. On the Connectors page, select the instance to which the connector belongs from the drop-down list of the Select Instance. In the Connector list under the instance, find the target connector and select More > View Logs in the Actions column.
    You can also click the name of the target connector or click Details in the Actions column. Then, click View Logs in the upper-right corner of the connector details page.
    Note The View Logs position of the Actions column is related to the task status. Please refer to the actual interface.
    On the displayed page, view the connector logs.

Modify connector configurations

After you create a Function Compute sink connector, you can modify its configurations in the Message Queue for Apache Kafka console.

  1. Log on to the Message Queue for Apache Kafka console.
  2. In the Resource Distribution section of the Overview page, select the region where your instance resides.
  3. In the left-side navigation pane, click Connectors.
  4. On the Connectors page, select the instance to which the connector belongs from the drop-down list of the Select Instance. In the Connector list under the instance, find the target connector and select More > Modify Configuration in the Actions column.
    You can also click the name of the target connector or click Details in the Actions column. Then, click Modify Configuration in the upper-right corner of the connector details page.
  5. In the Modify Configuration panel, change the values of the parameters that are described in the following table and click OK.
    Operation Description
    Consumer Thread Concurrency The number of concurrent consumer threads used to synchronize data from the source topic. By default, six concurrent consumer threads are used. Valid values:
    • 1
    • 2
    • 3
    • 6
    • 12
    Failure Handling Policy Specifies whether to retain the subscription to the partition in which an error occurs after the relevant message fails to be sent. Valid values:
    • Continue Subscription: retains the subscription to the partition in which an error occurs and returns the logs.
    • Stop Subscription: stops the subscription to the partition where an error occurs and returns the logs.
    Note
    • For more information, see Connector-related operations.
    • For more information about how to troubleshoot errors based on error codes, see Error codes.
    • To resume the subscription to the partition where an error occurs, submit a ticket to the technical support of Message Queue for Apache Kafka.
    Transmission Mode The mode in which messages are sent. Valid values:
    • Synchronous: The producer and consumer must be running at the same time to be ready for data exchanges.
    • Asynchronous: After the producer sends a message, the producer does not need to wait for the response from the broker and can continue processing other tasks.
    Data Size The maximum number of messages that can be sent at a time.

    Valid values: 1 to 5000. Default value: 20.

    Retries The maximum number of retries allowed after a message fails to be sent. Default value: 2. Valid values: 1 to 3. In specific cases when a message fails to be sent, retries are not supported. For more information about error codes, see Error codes. The following list describes the types of error codes and whether they support retries:
    • 4xx: Retries are not supported except for 429 error codes.
    • 5xx: Retries are supported.
    Note The connector calls the InvokeFunction operation to send messages to Function Compute.
    After the values of the parameters are changed, go to the Connectors page and find the connector. Click Details in the Actions column. On the Connector Details page, check the updated configurations.

Start and stop the connector

To temporarily suspend a running connector or resume a suspended connector, you can pause or enable the connector in the Message Queue for Apache Kafka console.

  1. Log on to the Message Queue for Apache Kafka console.
  2. In the Resource Distribution section of the Overview page, select the region where your instance resides.
  3. In the left-side navigation pane, click Connectors.
  4. On the Connectors page, select the instance to which the connector belongs from the Select Instance drop-down list, find the target connector in the Connector list under the instance, and pause or enable the connector as needed.
    • Pause Connector: In the Actions column, choose More > Suspend. In the Tip dialog box, confirm that the connector is suspended and click OK.
    • Enable Connector: In the Actions column, click Enable. In the Prompt dialog box, confirm that the connector is started and click OK.

    You can also click in the name of the connector or in the Actions column of the connector, click Details, and then click Suspend or Enable in the upper-right corner of the connector details page.

    On the Connectors page, find the Pause or Enabled Connector. The Status has changed to Suspended or is running.

Test a connector

This topic describes how to test a connector by sending a test message to the connector in the Message Queue for Apache Kafka console.

  1. Log on to the Message Queue for Apache Kafka console.
  2. In the Resource Distribution section of the Overview page, select the region where your instance resides.
  3. In the left-side navigation pane, click Connectors.
  4. On the Connectors page, select the instance to which the connector belongs from the drop-down list of the Select Instance, find the connector in the Connector list under the instance, and click Test in the Actions column.
    You can also click the name of the target connector or click Details in the Actions column. Then, click Test in the upper-right corner of the connector details page.
    Note The Test position of the Actions column is related to the task status. Please refer to the actual interface.
  5. In the Send Message panel, set the parameters or use the method as prompted to send a test message.
    1. In the Message Key field, enter the key of the test message, such as demo.
    2. In the Message Content field, enter the content of the test message, such as {"key": "test"}.
    3. Set the Send to Specified Partition parameter to specify whether to send the test message to a specific partition.
      • If you want to send the test message to a specific partition, click Yes and enter the partition ID, such as 0, in the Partition ID field. For more information about how to query partition IDs, see View partition status.
      • If you do not want to send the test message to a specific partition, click No.

View consumption progress

After the connector task is created and running, you can view the consumption status of the Group used by the connector data synchronization task in the data source topic. The name of the Group is connect-{Task name}.

  1. Log on to the Message Queue for Apache Kafka console.
  2. In the Resource Distribution section of the Overview page, select the region where your instance resides.
  3. In the left-side navigation pane, click Connectors.
  4. On the Connectors page, select the instance to which the connector belongs from the Select Instance drop-down list. In the Connector list under the instance, find the target connector and click Consumption Progress in the Actions column.
    You can also click the name of the target connector task or click Details in the Actions column. Then, click Consumption Progress in the upper-right corner of the connector details page.
    In the Group Consumption Details panel, you can view the consumption details of the Group used by the connector task in the data source topic.

Delete a connector

Message Queue for Apache Kafka limits the number of connectors for each instance. If you no longer need a connector, you can delete it in the Message Queue for Apache Kafka console.

Notice
  • If the MySQL Source Connector task is running and cannot be deleted in the Message Queue for Apache Kafka console, you must log on to the DataWorks console to stop and unpublish the connector task. Then, a ticket to clear the metadata of the Message Queue for Apache Kafka connector task. You can delete other FC Sink Connector, MaxCompute Sink Connector, OSS Sink Connector, and Elasticsearch sink connector tasks in the Message Queue for Apache Kafka console.
  • When you delete a connector, Message Queue for Apache Kafka deletes the five topics and two consumer groups that are required by the connector, regardless of whether the topics and consumer groups were automatically or manually created.Group
  1. Log on to the Message Queue for Apache Kafka console.
  2. In the Resource Distribution section of the Overview page, select the region where your instance resides.
  3. In the left-side navigation pane, click Connectors.
  4. On the Connectors page, select the instance to which the connector belongs from the drop-down list of the Select Instance. In the Connector list under the instance, find the target connector and select More > Delete in the Actions column.
    You can also click the name of the target connector task or click Details in the Actions column. Then, click Delete in the upper-right corner of the connector details page.
  5. In the Note message, click OK to delete the connector.