Frequently asked questions about using Trino on EMR.
Trino vs. Presto
Trino, formerly known as PrestoSQL, is an open-source project from Starburst, a company founded by the original creators of Presto after they left Facebook. The project was officially renamed Trino in 2021. Facebook continues to maintain its own fork, known as PrestoDB. Although the two projects have diverged, they share similar syntax and usage.
In EMR 3.44.0 and EMR 5.10.0 and later, the service is officially named Trino. In earlier versions, the console displays Presto, but the underlying engine is Trino.
Trino versions
For changes in each Trino version, see the official Trino release notes. Newer versions include performance optimizations, so we recommend that you select the latest version available.
Trino deployment and high availability
Trino uses a standard coordinator-worker architecture. The coordinator is deployed on the master-1-1 node, and workers are deployed on all core or task nodes.
Trino does not currently support high availability. On a high availability cluster, the Trino coordinator is deployed only on the master-1-1 node. If you do not need to co-deploy Hadoop components, select only the Trino service when you create the cluster.
If you do not need to co-deploy Hadoop, do not enable high availability to avoid unnecessary costs.
Trino, DLF, and custom connectors
For data lake clusters in EMR versions earlier than 3.45.0 and 5.11.0, if you select the Hive service and choose DLF Unified Metadata when creating the cluster, connectors such as Hive, Iceberg, Hudi, and Delta Lake connect directly to Data Lake Formation (DLF). If the cluster does not include the Hive service, see Configure a metadata storage center for data in data lakes to configure the connection manually. For data lake clusters in EMR 3.45.0 and later and 5.11.0 and later, you can select DLF Unified Metadata when you create a Trino cluster.
After DLF is enabled, default connectors like the Hive connector can no longer access the metastore. If you need to configure multiple Hive metastores, multiple MySQL instances, or use services not available in the default connectors, you can use the five placeholder connectors (connector1 to connector5) provided by EMR Trino. You can set the connector.name property for any of these placeholders to hive or another connector type you require. Then, add other configuration properties based on the specific connector's documentation that point to the desired service address.
Trino access to OSS and OSS-HDFS
EMR includes JindoSDK by default with password-free access. You can access OSS directly or query Hive tables stored in OSS.
EMR Trino supports OSS-HDFS.
Accessing the Trino web UI
If your cluster has Knox installed, you can use the Knox URL to access the Trino web UI. For more information, see Knox.
If your cluster does not include Knox, you can access the UI with a public URL: {public IP address:HTTP port}. The HTTP port can be found in the http-server.http.port configuration, which defaults to 9090. If you cannot access the UI, ensure that port 9090 is open in your security group.
If your cluster runs in high-security mode, the default HTTP port is unavailable. If you have the necessary network permissions, you can access the web UI by adding the following configuration to the config.properties file of the master node group. Then, access the UI using the standard method.
web-ui.authentication.type=fixed
web-ui.user=trino
The Trino web UI displays recently executed queries, including SQL statements and execution plans. By default, Trino stores up to 100 recent queries. Completed queries are quickly overwritten by new entries, while failed queries are retained longer. You can adjust the query.max-history configuration (default: 100) to increase the number of stored queries.
Adding Trino to an existing cluster
-
For a data lake cluster, you can add the Trino service directly if the cluster has sufficient resources.
-
For a Hadoop cluster, check if any services have been manually upgraded. If so, Trino may fail to start after installation. In this case, follow these steps:
-
If JindoSDK was upgraded separately, especially after a major version update, you must rerun the relevant Presto or Trino upgrade script and manually copy the upgraded JindoSDK to the corresponding connector directory in the Trino installation path.
-
For EMR-3.39.1, check the service log. It usually reports that a Delta-related class cannot be found. In this case, on each node, manually copy the
/opt/apps/ecm/service/deltalake/0.6.1-3.3/package/deltalake-0.6.1-3.3/presto-delta/delta-standalone-assembly-0.2.0.jarfile to the/usr/lib/presto-current/plugin/deltapath.
-
Configuration changes not taking effect
Trino configuration files are located in /etc/emr/trino-conf. Check whether the configuration files on the nodes correctly reflect the changes you made in the console and contain your previous additions or modifications:
-
If the configuration file does not contain the new or modified content: Verify that you saved the configuration, deployed the changes, and that the modification scope is correct.
ImportantIf you modify a configuration property for a specific node group or a single node, the default cluster configuration for that property no longer applies.
-
If the configuration file contains the new or modified content: Verify that all Trino nodes have been restarted. Trino applies the new configuration only after a restart.
Stuck queries or worker node failures
If you see an error message such as Could not communicate with the remote task. The node may have crashed or be under too much load. This is probably a transient issue, so please retry your query in a few minutes. or No handle resolver for connector: hive ... Unrecognized token 'io': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false'), it indicates that a worker node is overloaded or has automatically restarted. The system may have terminated a process on the worker. Adjust the configuration based on the workload, especially memory-related settings, or limit the number of concurrent requests.
Viewing Trino logs
Trino log files are stored in /mnt/disk1/log/trino/var/log/ by default. Output and exception stack traces can be found in the server.log file.
To view error details for a specific query, add the --debug flag when you start the client. This prints the exception stack trace.
The Trino coordinator and workers communicate over HTTP. If an HTTP exception occurs on the coordinator node, the error may have originated on a worker node. If there is no other obvious error information, check each worker node individually.
Query failures or empty results
Troubleshoot the issue by following these steps:
-
Access or query the data using other engines, such as Hive or Spark. If they also fail, check whether the data source is reachable and the data is intact.
-
If only Trino cannot access or execute the query, check if the configured metadata is correct.
-
If the metadata is valid, but a query on a table with data returns an empty result, check if you have the necessary data access permissions.
-
If proxyuser is enabled for the HDFS where the data is stored, you must also enable the hive.hdfs.impersonation.enabled property in Trino.
-
If Ranger is enabled, verify that the Ranger permissions are configured correctly.
-
If the cluster has been scaled out, check if the new node group or node has the necessary permissions to access the corresponding files.
-
Incorrect column order for Hudi and Delta Lake queries
Check whether the hive.parquet.use-columns-names property is set to true in the hive.properties file for Trino.
Trino fails to restart after configuration changes
If the Server.log file contains the error Error: Configuration property 'xxxxx' was not used, it means the configuration is in the wrong location or a prerequisite configuration is missing. Trino strictly validates its configuration properties. If a property does not exist, is misspelled, or is placed in the wrong file, the service fails to start. Verify that the new configuration is correct, or roll back the change.
Cannot query xxx table error with the Hive connector
Trino provides dedicated connectors for Iceberg, Hudi, and Delta Lake. Use the dedicated connector for each table format. If you must use the Hive connector, use the Table Redirection feature to forward the query to the corresponding dedicated connector.
For example, to configure the Hive connector to redirect queries for Iceberg, Delta Lake, and Hudi tables, set the following parameters:
hive.iceberg-catalog-name=iceberg
hive.delta-lake-catalog-name=delta-lake
hive.hudi-catalog-name=hudi