| Component | Description |
| Druid | - Upgraded Druid to version 0.15.1.
- Added the Router component.
- The Fastjson version is updated.
|
| Spark | - Updated the code for Spark SQL Thrift Server to fix the issue where IsolatedClassLoader cannot load classes in certain cases.
- Refactored the code related to Spark transactions to improve stability.
- Fixed the issue where optimized row columnar (ORC) files cannot be read or written after the built-in Hive is upgraded to version 2.3.
- Supports the MERGE INTO syntax.
- Supports the SCAN and STREAM syntaxes.
- Supports exactly-once semantics (EOS) for the Structured Streaming Kafka sink.
- Upgraded Delta to version 0.4.0.
|
| Hive | - Removed Hive hooks configured in earlier versions of Hive.
- Supports using multiple COUNT(DISTINCT) for hive.groupby.skew in data optimization.
- Fixed the issue of data loss when joining tables with different bucket versions.
|
| Flink | Upgraded Flink to version 1.8.2. |
| Bigboot | - Upgraded the HDFS FsImage analysis tool.
- Updated the JAR package for Object Storage Service (OSS) to fix the issue where the non-Daemon thread keeps alive after the main thread is terminated.
|
| Kafka | - Supports the DepolymentSet-Aware replication distribution strategy in Kafka.
- Removed the Fastjson dependency.
|
| HDFS | - Optimized the deployment logic of the JAR package for SmartData OSS.
- Updated the JAR package for SmartData OSS.
|
| Flume | The Fastjson version is updated. |
| Tensorflow on Spark | This is a new component. |
| Has | The Fastjson version is updated. |
| Livy | The Fastjson version is updated. |