This topic describes how to configure a Streaming SQL job.
Background information
For more information about Streaming SQL, see Spark Streaming SQL.
When you configure a Streaming SQL job, you must specify dependency libraries. The following table describes the recent versions and other details about the dependency library provided by Spark Streaming SQL. We recommend that you use the latest version of the dependency library.
Dependency library | Supported version | Release date | Reference string | Description |
---|---|---|---|---|
datasources-bundle | 2.0.0 (recommended) | 2020/02/26 | sharedlibs:streamingsql:datasources-bundle:2.0.0 | Supported data sources include Kafka, LogHub, Druid, Tablestore, HBase, JDBC, DataHub, Redis, Kudu, and DTS. |
1.9.0 | 2019/11/20 | sharedlibs:streamingsql:datasources-bundle:1.9.0 | Supported data sources include Kafka, LogHub, Druid, Tablestore, HBase, JDBC, DataHub, Redis, and Kudu. | |
1.8.0 | 2019/10/17 | sharedlibs:streamingsql:datasources-bundle:1.8.0 | Supported data sources include Kafka, LogHub, Druid, Tablestore, HBase, JDBC, DataHub, and Redis. | |
1.7.0 | 2019/07/29 | sharedlibs:streamingsql:datasources-bundle:1.7.0 | Supported data sources include Kafka, LogHub, Druid, Tablestore, HBase, and JDBC. |
For more information, see Overview.
Prerequisites
- A project is created. For more information, see Manage projects.
- Resources and data files required for a job are obtained, such as JAR packages, names of the data files, and storage paths of both the JAR packages and data files.