This topic describes how to use a Kafka data table.
Syntax
CREATE TABLE tbName[(columnName dataType [,columnName dataType]*)]
USING kafka
OPTIONS(propertyName=propertyValue[,propertyName=propertyValue]*);Configuration parameters
| Parameter | Description | Required |
|---|---|---|
| kafka.schema.registry.url | The URL of Kafka Schema Registry. | Yes |
| subsribe | The name of the associated Kafka topic. | Yes |
| kafka.bootstrap.servers | The connection address of the Kafka cluster. | Yes |
| kafka.schema.record.name | The definition of the RecordName parameter in Avro. | No |
| kafka.schema.record.namespace | The definition of the RecordNameSpace parameter in Avro. | No |
Table schema
When creating a Kafka data table,
you can choose whether to define the fields in the data table or not as required.
| System processing when the schema of a topic has been defined in Kafka Schema Registry | System processing when no schema has been defined for a topic in Kafka Schema Registry | |
|---|---|---|
| You do not define the fields in the data table. | The system defines the table schema based on the topic schema. | The system fails to create the table. |
| You define the fields in the data table. | The system checks whether the table schema is compatible with the topic schema. If they are incompatible, the system returns an error. If they are compatible, the system defines the table schema based on the custom field type. | The system automatically registers the Avro data schema of Kafka data with Kafka Schema Registry. To achieve this, the kafka.schema.record.name and kafka.schema.record.namespace fields must be provided. |
The following constraints apply when you define the schema of a topic in Kafka Schema
Registry:
- All fields must be of the nullable type. For example, the type of the f1 field is
null in the following code:
{"type":"record","name":"myrecord","namespace":"test","fields":[{"name":"f1","type":["string","null"]}]}