This topic describes how to use a Kafka data table.

Syntax

CREATE TABLE tbName[(columnName dataType [,columnName dataType]*)]
USING kafka
OPTIONS(propertyName=propertyValue[,propertyName=propertyValue]*);

Configuration parameters

Parameter Description Required
kafka.schema.registry.url The URL of Kafka Schema Registry. Yes
subscribe The name of the associated Kafka topic. Yes
kafka.bootstrap.servers The connection address of the Kafka cluster. Yes
kafka.schema.record.name The definition of the RecordName parameter in Avro. No
kafka.schema.record.namespace The definition of the RecordNameSpace parameter in Avro. No

Table schema

When creating a Kafka data table, you can choose whether to define the fields in the data table or not as required .
Schema defined for a topic in Kafka Schema Registry Schema not defined for a topic in Kafka Schema Registry
Table fields not defined The system defines the table schema based on the topic schema. The system fails to create the table.
Table fields defined The system checks whether the table schema is compatible with the topic schema. If they are incompatible, the system returns an error. If they are compatible, the system defines the table schema based on the custom field type. The system automatically registers the Avro data schema of Kafka data with Kafka Schema Registry. In this case, the kafka.schema.record.name and kafka.schema.record.namespace parameters must be specified.
When using Kafka Schema Registry, note the following:
  • Schema compatibility check: When you write data to Kafka, the system checks the compatibility between the schema of the output data and the schema of the Kafka topic. By default, the schema created by Kafka Schema Registry is backward compatible. If you do not have the schema compatibility requirement, you can set the compatibility policy to NONE.
  • If you encounter a schema compatibility error during usage, check the error log.