Use SQL to read and write data in LindormTable wide tables from Lindorm Distributed Processing System (LDPS).
Prerequisites
Before you begin, ensure that you have:
Read the Precautions for using SQL to access LindormTable
Initialized the required resources for your job type:
Java job: Create a job in Java
Python job: Create a job in Python
Access data in a wide table
LindormTable wide tables are accessible through the lindorm_table catalog in LDPS. LDPS supports data manipulation language (DML) operations on wide tables. Data definition language (DDL) operations and partitioning are not supported.
The typical workflow is: switch to the lindorm_table catalog, verify the table schema, then run your DML statements.
Switch to the catalog
USE lindorm_table;Query the table schema
SHOW CREATE TABLE test;The output looks like:
+----------------------------------------------------+
| CREATE TABLE default.test ( |
| `id` INT, |
| `name` STRING) |
| |
+----------------------------------------------------+Insert data
INSERT INTO test VALUES (0, 'Jay');Query data
SELECT * FROM test;For the full list of supported SQL syntax, see DML statements.
Import data in bulk (public preview)
The SQL syntax is the same as a standard INSERT statement. When bulkload is enabled, LDPS generates and registers wide table files directly from the imported data, which increases write throughput.
Configure the following parameters before running the import:
| Parameter | Description | Default |
|---|---|---|
spark.sql.catalog.lindorm_table.bulkLoad.enabled | Enables the bulkload feature. Set to true to activate. | false |
spark.sql.catalog.lindorm_table.bulkLoad.parallelFactor | Number of concurrent sessions used to write data to a single region of a wide table. | 4 |
Secondary and search indexes are not automatically created for data imported by bulkload jobs.
What's next
Explore the full DML syntax reference: DML statements