This topic describes how to start and stop a Thrift server for job resource groups in Data Lakehouse Edition (V3.0).

Background information

AnalyticDB for MySQL Data Lakehouse Edition (V3.0) provides two modes to perform Spark SQL job development: batch and interactive. By default, job resource groups perform job development in batch mode. If you want to use job resource groups to perform interactive queries, you must start a Thrift server.

Job resource groups that have a Thrift server started perform Spark SQL job development in interactive mode.

Precautions

  • You can start only a single Thrift server for each job resource group.
  • If you stop a Thrift server, all the executing and queuing SQL queries are interrupted. Before you stop a Thrift server, make sure that no SQL queries are being executed and queued or that your business is not affected by such interruption.

Start a Thrift server

  1. Log on to the AnalyticDB for MySQL console.
  2. In the upper-left corner of the page, select the region where the cluster resides.
  3. On the Data Lakehouse Edition (V3.0) tab, find the cluster and click its ID.
  4. In the left-side navigation pane, choose Clusters > Resource Groups.
  5. Find a job resource group and click Advanced Settings in the Actions column.
  6. In the Advanced Settings (Spark Thrift Server Settings) panel, configure the parameters described in the following table.
    Parameter Required Description Corresponding configuration parameter
    Minimum Executors Yes The minimum number of executors allowed for elastic Spark settings. The default value is 0, which indicates that all executors are released if no Spark SQL queries are executed.

    Each executor consumes two AnalyticDB Compute Units (ACUs).

    		spark.dynamicAllocation.minExecutors
    Maximum Executors Yes The maximum number of executors allowed for elastic Spark settings. It indicates the maximum executors that are allowed when Spark SQL queries are executed. The minimum value is 1.

    Each executor consumes two ACUs. If you specify a large value for this parameter and computing resources of this job resource group are insufficient, an error may occur. For example, the maximum computing resources of a job resource group are 16 ACUs, you can set the maximum number of executors to 8.

    		spark.dynamicAllocation.maxExecutors
    JARS No The Object Storage Service (OSS) URLs of third-party JAR packages required for the execution environment.

    If SQL execution requires JAR packages, you must upload the JAR packages to an OSS bucket first. For more information about how to upload objects, see Upload objects.

    		spark.jars
    Maximum Connections No The maximum number of sessions allowed for Spark maintenance. Each session corresponds to an independent thread. A greater value of this parameter may cause out-of-memory (OOM) errors. spark.sql.retainedSessions
    Other Settings No The other Spark community settings. For more information, see Spark Configuration. None
  7. Click Start.

Stop a Thrift server

If you want to perform Spark SQL job development in batch mode for a job resource group that has a Thrift server started, you must stop the Thrift server first.

  1. Log on to the AnalyticDB for MySQL console.
  2. In the upper-left corner of the page, select the region where the cluster resides.
  3. On the Data Lakehouse Edition (V3.0) tab, find the cluster and click its ID.
  4. In the left-side navigation pane, choose Clusters > Resource Groups.
  5. On the Resource Groups page, find a job resource group and click Advanced Settings in the Actions column. In the panel that appears, click Stop.