DataWorks:MetaQ

Supported versions

MetaQ Reader uses the following Java SDK versions:

<dependency>
    <groupId>com.taobao.metaq.final</groupId>
    <artifactId>metaq-client</artifactId>
    <version>4.0.1</version>
</dependency>
<dependency>
    <groupId>com.aliyun.openservices</groupId>
    <artifactId>ons-sdk</artifactId>
    <version>1.3.1</version>
</dependency>

Limits

• MetaQ Reader is only available in the code editor — the GUI mode is not supported.

• Supported resource groups: Serverless resource groups (recommended) and exclusive resource groups for Data Integration.

Data types

MetaQ Reader supports the STRING field type for offline reads.

Type mapping between Data Integration and Message Queue:

Configure a synchronization task

Configure the synchronization task using the code editor. For the step-by-step procedure, see Configure a task in the code editor.

For the complete parameter reference and a ready-to-use code sample, see Appendix: Code and parameters.

Appendix: Code and parameters

All parameters go under reader.parameter in the job script.

{
  "job": {
    "content": [
      {
        "reader": {
          "name": "metaqreader",
          "parameter": {
            "accessId": "<your-access-key-id>",
            "accessKey": "<your-access-key-secret>",
            "consumerId": "Test01",
            "topicName": "test",
            "subExpression": "*",
            "onsChannel": "ALIYUN",
            "domainName": "<your-message-queue-endpoint>",
            "contentType": "singlestringcolumn",
            "beginOffset": "lastRead",
            "nullCurrentOffset": "begin",
            "fieldDelimiter": ",",
            "column": ["col0"]
          }
        },
        "writer": {
          "name": "streamwriter",
          "parameter": {
            "print": false
          }
        }
      }
    ]
  }
}