This guide walks you through developing a JAR job for the Lindorm compute engine—from configuring dependencies to submitting the job. By the end, you will be able to:
Configure the correct Spark dependencies.
Set up LindormTable credentials in SparkConf.
Write and run a Spark JAR job that queries LindormTable data.
Submit the job using the Lindorm console or Data Management Service (DMS).
Prerequisites
Before you begin, ensure that you have:
A Lindorm instance with LindormTable enabled. See Create an instance.
The compute engine service enabled. See Enable, upgrade, or downgrade the service.
JDK 1.8 or later installed.
Step 1: Configure dependencies
The Lindorm compute engine runs on the community edition of Spark 3.3.1. Set scope to provided so the Spark libraries are not bundled into your JAR. The cluster provides them at runtime.
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.12</artifactId>
<version>3.3.1</version>
<scope>provided</scope>
</dependency>Step 2: Configure permissions
To access LindormTable data from Spark SQL, configure a username and password in SparkConf. The default credentials are root/root.
SparkConf conf = new SparkConf();
conf.set("spark.sql.catalog.lindorm_table.username", "root");
conf.set("spark.sql.catalog.lindorm_table.password", "root");| Parameter | Default value | Description |
|---|---|---|
spark.sql.catalog.lindorm_table.username | root | Username for LindormTable access |
spark.sql.catalog.lindorm_table.password | root | Password for LindormTable access |
For more details on accessing LindormTable data, see Access data in LindormTable.
Step 3: Configure parameters
For the full list of supported configuration items and how to set them, see Job configuration.
Step 4: Code example
JAR job development is fully compatible with the community edition of Spark 3.3.1. For a code example, see Spark job examples.
Step 5: Submit the job
The Lindorm compute engine supports two ways to submit jobs. Choose the one that matches your workflow:
| Method | Best for | Reference |
|---|---|---|
| Lindorm console | Interactive submission and job monitoring through a web UI | Manage jobs in the console |
| DMS | Scheduled and automated job management | Manage jobs using DMS |