All Products
Search

This Product

    Dataphin:Create a single-tenant, multi-engine cluster

    Document Center

    Dataphin:Create a single-tenant, multi-engine cluster

    Last Updated:Jul 31, 2026

    Permissions

    • Users with the super administrator or system administrator role, and custom global roles with the Cluster Management - Manage permission, can create and manage clusters. These users can also specify which users can use the cluster when creating a compute source and can assign cluster administrators for it.

    • A cluster administrator can manage their assigned clusters.

    • Users with the Compute Source Management - Create global permission can select authorized clusters when creating a compute source.

    Create a cluster

    1. In the top navigation bar of the Dataphin homepage, choose Plan > Cluster Management.

    2. On the Cluster Management page, click Create Cluster.

    3. On the Create Cluster page, configure the following parameters.

      • Basic information

        Parameter

        Description

        Cluster name

        Enter a name for the cluster. The name can contain Chinese characters, letters, digits, spaces, and the following special characters: -_.@~(). The name can be up to 128 characters long.

        Engine type

        The following engine types are supported:

        • MaxCompute

        • AnalyticDB for PostgreSQL

        • Alibaba Cloud E-MapReduce 3.x

        • Alibaba Cloud E-MapReduce 5.x

        • CDH 5.x

        • CDH 6.x

        • Cloudera Data Platform 7.x

        • Huawei FusionInsight 8.x

        • AsiaInfo DP 5.3

        • StarRocks

        • Databricks

        • Amazon EMR

        • SelectDB

        • Doris

        • GaussDB(DWS)

        • Transwarp TDH 6.x

        • Transwarp TDH 9.3.x

        • Transwarp ArgoDB

        • Lindorm

        • Hologres

        • OushuDB

        • Alibaba Cloud EMR Serverless Spark

        Cluster administrator

        Select one or more members from the current tenant as cluster administrators. A cluster administrator can manage the cluster, which includes editing its settings, viewing historical versions, and deleting it.

        Description (Optional)

        Enter a brief description for the cluster. The description can be up to 128 characters long.

      • Cluster security control

        Available members: Specify which users can use this cluster when creating a compute source. You can select Roles with "Create Compute Source" permission or Specified users.

        • Roles with "Create Compute Source" permission: This option is selected by default.

        • Specified users: Allows you to select one or more individual accounts and user groups.

      • Cluster configuration

        MaxCompute

        Parameter

        Description

        Endpoint

        Enter the endpoint of the compute engine. Example: http://service.odps.aliyun.com/api.

        AccessKey ID

        Enter the AccessKey ID and AccessKey Secret for an account with permission to access the MaxCompute project data.

        You can obtain the AccessKey ID and AccessKey Secret from the User Information Management page.

        Important

        • To ensure a successful connection between your Dataphin and MaxCompute projects, we recommend using the AccessKey of a MaxCompute project administrator.

        • To ensure correct metadata collection, avoid changing the AccessKey of the MaxCompute project.

        AccessKey Secret

        Hadoop

        Hadoop includes the CDH 5.x, CDH 6.x, Cloudera Data Platform 7.x, Alibaba Cloud E-MapReduce 3.x, Alibaba Cloud E-MapReduce 5.x, AsiaInfo DP 5.3, and Huawei FusionInsight 8.x engines.

        In multi-engine mode, the configurations for Hadoop clusters, HDFS compute engines, Hive metadata, Spark JAR service, Spark SQL service, and Impala tasks are the same as in single-engine mode. For details, see Hadoop Cluster Configuration.

        AnalyticDB for PostgreSQL

        In multi-engine mode, the cluster configuration for AnalyticDB for PostgreSQL is the same as in single-engine mode. For details, see ADB PG Cluster Configuration.

        Transwarp TDH 6.x/9.3.x

        In multi-engine mode, the configurations for Transwarp TDH 6.x and 9.3.x clusters, HDFS, Inceptor, and Inceptor metadata connection are the same as in single-engine mode. For details, see Transwarp TDH Cluster Configuration.

        Transwarp ArgoDB

        In multi-engine mode, the configurations for Transwarp ArgoDB clusters, HDFS, ArgoDB, and ArgoDB metadata connection are the same as in single-engine mode. For details, see Transwarp ArgoDB Cluster Configuration.

        SelectDB, Doris, and StarRocks

        In multi-engine mode, the cluster configurations for SelectDB, Doris, and StarRocks are the same as in single-engine mode. For details, see SelectDB, Doris Cluster Configuration and StarRocks Cluster Configuration.

        Databricks

        In multi-engine mode, the cluster configuration for Databricks is the same as in single-engine mode. For details, see Databricks Cluster Configuration.

        Amazon EMR

        In multi-engine mode, the cluster configuration for Amazon EMR is the same as in single-engine mode. For details, see Amazon EMR Cluster Configuration.

        Lindorm

        Parameter

        Description

        core-site.xml

        Upload the core-site.xml, hdfs-site.xml, and hive-site.xml configuration files for Lindorm. For the configuration files, see Connect to and use an instance.

        hdfs-site.xml

        hive-site.xml (Optional)

        JDBC URL

        Configure the JDBC URL for Lindorm. To obtain the URL, see View connection strings.

        Username, Password

        Enter the username and password to access the Lindorm instance.

        GaussDB (DWS)

        Parameter

        Description

        Version

        Only version 9.1.0 is currently supported.

        JDBC URL

        Enter the JDBC connection URL. Example: jdbc:postgresql://{host};{port}/{database name}.

        Connection account

        • Same account for metadata retrieval and execution: Use a single username and password for both metadata retrieval and SQL execution.

        • Separate accounts: Use separate usernames and passwords for metadata retrieval and SQL execution.

        Username, Password

        Enter the username and password to log in to the GaussDB (DWS) database.

        Note

        This parameter is available only when you select Same account for metadata retrieval and execution for Connection account.

        Metadata account username, Metadata account password

        Enter the usernames and passwords for the metadata account and the execution account respectively.

        Note

        These parameters are available only when you select Separate accounts for Connection account.

        Execution account username, Execution account password

        Hologres

        Parameter

        Description

        JDBC URL

        The connection URL for the Hologres compute source. The format is jdbc:postgresql://host:port/dbname.

        Username, Password

        Enter the username and password to connect to the compute source.

        If you use an Alibaba Cloud RAM user, enter the user's AccessKey ID and AccessKey Secret. If you use a database-native account, enter the account's username and password.

        OushuDB

        Parameter

        Description

        Version

        Only version 6.4.0 is currently supported.

        JDBC URL

        Enter the JDBC URL in the format jdbc:oushudb://{host}:{port}/.

        Default execution user, Password

        Enter the username and password. To ensure that tasks run properly, make sure the user has the required data permissions.

        Alibaba Cloud EMR Serverless Spark

        Parameter

        Description

        Endpoint

        Enter the OpenAPI (SDK) endpoint for Alibaba Cloud EMR Serverless Spark.

        AccessKey ID, AccessKey Secret

        Enter the AccessKey ID and AccessKey Secret.

        Workspace

        Select a workspace that the RAM user associated with the AccessKey has joined (ListWorkspaces).

      • Other configurations

        MaxCompute

        Parameter

        Description

        Default storage format for external tables

        The default storage format for new external tables. The following formats are supported:

        • parquet

        • avro

        • rcfile

        • orc

        • textfile

        • sequencefile

        Ad-hoc query MCQA acceleration

        If enabled, you can use MCQA to accelerate ad-hoc queries for MaxCompute projects.

        Logview URL in logs

        Specifies how the logview URL is displayed in logs. You can select Display in plain text or Hide when execution statement contains global variables for account and password.

        Default lifecycle

        The default lifecycle for physical and logical tables. You can enter a value from 1 to 36500 days, or quickly select 7, 14, 30, or 360 days.

        Global compute engine parameters

        Set custom parameters to control runtime behavior, such as memory allocation, task priority, or enabling MapJoin. When enabled, you can configure separate parameters for Modeling Task and SQL Compute Task. SQL compute task parameters also apply when running tasks in the development environment.

        Note

        • Parameters must follow the syntax rules of the current engine type.

        • Parameters set at the task level or in task code override the global parameters.

        Hadoop

        Hadoop includes the CDH 5.x, CDH 6.x, Cloudera Data Platform 7.x, Alibaba Cloud E-MapReduce 3.x, Alibaba Cloud E-MapReduce 5.x, AsiaInfo DP 5.3, and Huawei FusionInsight 8.x engines.

        Parameter

        Description

        Default storage format

        The default storage format for new tables created in Table Management. You can select one of the following formats:

        • Engine default (can be specified in the table creation statement)

        • hudi

        • delta (Delta Lake)

        • paimon

        • iceberg

        • kudu

        • parquet

        • avro

        • rcfile

        • orc

        • textfile

        • sequencefile

        Note

        The hudi, delta (Delta Lake), paimon, or iceberg formats are available only if you enable Spark SQL Service Configuration. The kudu format is available only if you enable Impala Task Configuration.

        Default compute engine for standard modeling

        You can select Hive, Spark, or Impala.

        Note

        Spark is available only if you enable Spark SQL Service Configuration. Impala is available only if you enable Impala Task Configuration.

        Global compute engine parameters

        Set custom parameters to control runtime behavior, such as memory allocation, task priority, or enabling MapJoin. When enabled, you can configure separate parameters for Modeling Task and SQL Compute Task. SQL compute task parameters also apply when running tasks in the development environment.

        Note

        • Parameters must follow the syntax rules of the current engine type.

        • Parameters set at the task level or in task code override the global parameters.

        AnalyticDB for PostgreSQL and OushuDB

        Global compute engine parameters: Set custom parameters to control runtime behavior, such as memory allocation, task priority, or enabling MapJoin. When enabled, you can configure separate parameters for Modeling Task and SQL Compute Task. SQL compute task parameters also apply when running tasks in the development environment.

        Note

        • Parameters must follow the syntax rules of the current engine type.

        • Parameters set at the task level or in task code override the global parameters.

        Transwarp TDH 6.x/9.3.x, Lindorm, and Alibaba Cloud EMR Serverless Spark

        Parameter

        Description

        Default storage format

        The default storage format for new tables created in Table Management. You can select one of the following formats:

        • Engine default (can be specified in the table creation statement)

        • parquet

        • avro

        • rcfile

        • orc

        • textfile

        • sequencefile

        Global compute engine parameters

        Set custom parameters to control runtime behavior, such as memory allocation, task priority, or enabling MapJoin. When enabled, you can configure separate parameters for Modeling Task and SQL Compute Task. SQL compute task parameters also apply when running tasks in the development environment.

        Note

        • Parameters must follow the syntax rules of the current engine type.

        • Parameters set at the task level or in task code override the global parameters.

        Transwarp ArgoDB, SelectDB, StarRocks, and Doris

        Global compute engine parameters: Set custom parameters to control runtime behavior, such as memory allocation, task priority, or enabling MapJoin. When enabled, you can configure separate parameters for Modeling Task and SQL Compute Task. SQL compute task parameters also apply when running tasks in the development environment.

        Databricks

        Parameter

        Description

        Default storage format

        The default storage format for new tables created in Table Management. You can select one of the following formats:

        • Engine default (can be specified in the table creation statement)

        • parquet

        • avro

        • orc

        • binaryfile

        • csv

        • json

        • text

        Global compute engine parameters

        Set custom parameters to control runtime behavior, such as memory allocation, task priority, or enabling MapJoin. When enabled, you can configure separate parameters for Modeling Task and SQL Compute Task. SQL compute task parameters also apply when running tasks in the development environment.

        Note

        • Parameters must follow the syntax rules of the current engine type.

        • Parameters set at the task level or in task code override the global parameters.

        Amazon EMR

        Parameter

        Description

        Default storage format

        The default storage format for new tables created in Table Management. You can select one of the following formats:

        • Engine default (can be specified in the table creation statement)

        • hudi

        • delta (Delta Lake)

        • paimon

        • iceberg

        • parquet

        • avro

        • rcfile

        • orc

        • textfile

        • sequencefile

        Note

        You can select the hudi, delta (Delta Lake), paimon, or iceberg formats only after you enable Spark SQL Service Configuration.

        Default compute engine for standard modeling

        You can select Hive or Spark.

        Note

        You can select Spark only after you enable Spark SQL Service Configuration.

        Global compute engine parameters

        Set custom parameters to control runtime behavior, such as memory allocation, task priority, or enabling MapJoin. When enabled, you can configure separate parameters for Modeling Task and SQL Compute Task. SQL compute task parameters also apply when running tasks in the development environment.

        Note

        • Parameters must follow the syntax rules of the current engine type.

        • Parameters set at the task level or in task code override the global parameters.

    4. Click Test Connection. The system automatically tests the connection to each service.

      If the connection test is successful, you can save the configuration. If the test fails, a Connection Test Failed dialog box appears, listing the failed services and their error details.

    5. After a successful connection test, click Save to create the cluster.