Configure a Trino cluster with connectors and a metadata storage center - E-MapReduce - Alibaba Cloud - E-MapReduce

You must configure a standalone Trino cluster before use. This topic describes how to configure connectors and data lake metadata for a standalone Trino cluster.

Background information

In the E-MapReduce console, you can add the Trino service when creating a DataLake, custom, or Hadoop cluster. Alternatively, you can create a standalone Trino cluster.

Dedicated cluster resources with minimal interference from other components.
Supports auto scaling.
Supports data lake analytics and real-time data warehousing.
Does not store data.

Note

Hudi and Iceberg are not actual processes and do not consume cluster resources.
If you do not use the Hue and JindoData (or SmartData) services, you can stop them.

To use a standalone Trino cluster, you must first create or use an existing DataLake, custom, or Hadoop cluster to serve as the data cluster.

After you create a standalone Trino cluster, perform the following configuration steps:

Configure a connector
Optional:Configure data lake metadata

You can skip this step if your data cluster does not use DLF Unified Metadata.

Configure a connector

Configure the query objects in the connector you want to use. This section uses the Hive connector as an example.

Go to the cluster Services page.
1. Log on to the EMR on ECS console.
2. In the top navigation bar, select the required region and resource group.
3. On the EMR on ECS page, find the target cluster and click Services in the Actions column.
On the Services page, click Configure in the Trino service section.
Modify configuration items.
1. On the Configure page, click the hive.properties tab.
2. Set the value of the hive.metastore.uri parameter to match the hive.metastore.uri value from your data cluster's Trino service.
Save the configuration.
1. In the upper-right corner, click Save.
2. In the Confirm dialog box, configure the parameters and click OK.
Deploy the client configuration.
1. In the upper-right corner, click Deploy Client Configuration.
2. In the Execute Cluster Operation dialog box, configure the parameters and click OK.
3. In the Confirm dialog box, click OK.
Restart the Trino service. For more information, see Procedure.
Configure hosts.

Important
You can skip this step if all the data to be queried is stored in Object Storage Service (OSS), or if a location is specified in the table creation statement.

To allow Trino to read data from these tables on the data cluster, configure host information on each host in the Trino cluster.
- Method 1 (Recommended): In the EMR console, add a cluster script or a bootstrap action to configure the host. For more information, see Manually run a script or Manage bootstrap actions.
- Method 2: Modify the hosts file directly by performing the following steps:
  1. Obtain the internal IP address of the master node in the data cluster. In the E-MapReduce console, go to the Nodes page. In the row of the master node group, click the icon to view the internal IP address of the node.
  2. Log on to the data cluster. For more information, see Log on to a cluster.
  3. Run the hostname command to obtain the hostname.
    Example hostname formats:
    - Hadoop cluster: emr-header-1.cluster-26****.
    - Other cluster types: master-1-1.c-f613970e8c****.
  4. Log on to the Trino cluster. For more information, see Log on to a cluster.
  5. Run the following command to edit the hosts file:
```
vim /etc/hosts
```
  6. Add the following content to the last line of the hosts file.
    Add the internal IP address and hostname of the data cluster's master node to the /etc/hosts file on each host in the Trino cluster.
    - Hadoop cluster
      192.168.**.** emr-header-1.cluster-26****
    - Other cluster types
      192.168.**.** master-1-1.c-f613970e8c****

Configure data lake metadata

Note

EMR versions 3.45.0 and later and 5.11.0 and later support automatic configuration during cluster creation.

If the table metadata uses DLF Unified Metadata, you must perform additional configuration for connectors such as Hive, Iceberg, and Hudi. In this scenario, queries no longer depend on the data cluster. The value for hive.metastore.uri is ignored because Trino directly accesses the DLF metadata under the same account.

The following table describes the data lake metadata configuration parameters.

Parameter	Description	Remarks
hive.metastore	The metastore type.	The value must be `DLF`.
dlf.catalog.id	The data catalog namespace.	The ID of the DLF catalog to bind. This parameter defaults to your Alibaba Cloud account ID.
dlf.catalog.region	The region of the DLF service.	For more information, see Supported regions and endpoints. Note Ensure this region is the same as the region specified for dlf.catalog.endpoint.
dlf.catalog.endpoint	The endpoint of the DLF service.	For more information, see Supported regions and endpoints. Set the dlf.catalog.endpoint parameter to the VPC endpoint for DLF. For example, if your selected region is China (Hangzhou), set the dlf.catalog.endpoint parameter to dlf-vpc.cn-hangzhou.aliyuncs.com. Note You can also use a public endpoint for DLF. For example, if your selected region is China (Hangzhou), set the dlf.catalog.endpoint parameter to dlf.cn-hangzhou.aliyuncs.com.
dlf.catalog.akMode	The AccessKey mode for the DLF service.	Set this parameter to `EMR_AUTO`.
dlf.catalog.proxyMode	The proxy mode for the DLF service.	Set this parameter to `DLF_ONLY`.
dlf.catalog.uid	Your Alibaba Cloud account ID.	Find your account information on the User Information page, or log on to the Alibaba Cloud console to view your Account ID on the Account Management page.

Example: Querying table data

Connect to Trino by using commands. For more information, see Access Trino by using commands.
Run the following command to query the test_hive table:
```
select * from hive.default.test_hive;
```