Lindorm Distributed Processing System (LDPS) is the compute engine service provided by Lindorm. LDPS allows you to execute SQL statements to access data in databases. This topic describes the precautions when you use LDPS.

Apache Spark SQL

You can use Apache Spark SQL to access database data that is stored in the following levels:
  • Catalog: Catalogs are used to identify different data sources for users.
  • Namespace: A namespace corresponds to a database or a schema.
  • Table: A table in a namespace corresponds to a table in a database.

LDPS SQL

LDPS provides a catalog data source based on the engine services that you activated for your Lindorm instance. The engine services include LindormTable, LindormTSDB, LindormSearch, and LindormDFS. You can execute the SHOW CATALOGS statement to query the available data sources. The following table describes the data source names.
Data source nameDescriptionReference
spark_catalogThe default data source provided by LDPS. We recommend that you use Hive Metastore to manage the metadata of this data source. For more information, see Access data in Hive. Spark SQL
lindorm_tableThe data source that is used to read data from or write data to LindormTable. Access data in wide tables
lindorm_cdcThe data source that is used to read and write Lindorm Change Data Capture (CDC) data. Access data in a Lindorm CDC data source
Examples: A database named DB1 exists in the lindorm_table data source, and a table named tableX is created in DB1. You can execute one of the following statements to query data in tableX:
  • Example 1
    SELECT fieldA FROM lindorm_table.DB1.tableX;
  • Example 2
    USE lindorm_table.DB1;
    SELECT fieldA FROM tableX;