For an E-MapReduce (EMR) cluster in which the Trino component is installed, if the query speed of the cluster does not meet the data processing requirements or the amount of memory required for some large queries exceeds the total amount of memory, you can modify the related configuration parameters of the cluster. If the modifications cannot resolve the issues, you can scale out the cluster by increasing the number of worker nodes. During off-peak hours or when few resources are required for your business, you can scale in the cluster to reduce costs. This topic describes how to scale out or scale in an EMR Trino cluster.
Usage notes
When you scale out an EMR cluster, you cannot increase the number of nodes only for a specific component. Instead, you must increase the number of nodes and initialize the new nodes for all components that are installed. If different components such as Trino and Hadoop are installed in your EMR cluster, you need to determine whether to scale out the EMR cluster based on full considerations.
Preparations for scale-out
When you scale out an EMR Trino cluster, you can synchronize the Trino component and most configuration items to the new nodes in the cluster. However, the newly added user-defined functions (UDFs), the manually replaced JAR files, the manually upgraded components such as JindoSDK, and the added event listeners cannot be synchronized to the new nodes during scale-out. To prevent issues such as query failures or unexpected query results from occurring after scale-out, we recommend that you write scripts for the preceding operations. The scripts are automatically executed when nodes are added. For more information, see Manage bootstrap actions.
Scaling operations
- In EMR clusters, automatic scale-in can be performed only on task node groups. If you need to scale in a cluster after scale-out, we recommend that you create a task node group and add nodes to the node group for scale-out. This way, you can configure automatic scale-in rules for the task node group.
- EMR Trino clusters support graceful shutdown. However, you can use only RESTful APIs to trigger graceful shutdown. For more information, see Graceful shutdown in Trino documentation.
- If you scale in a cluster in which a query task is running, the task may fail or become suspended. We recommend that you scale in the cluster when no query task is running. If this issue has occurred, run the query task again.
Operations after scale-out
After cluster scale-out is complete, new nodes are added to the cluster. You can execute the select * from system.runtime.nodes statement to view the status of the nodes in the cluster.
If you modify the configurations of a node group or nodes in a group before scale-out, you must modify the default configurations of the new nodes or the new node group after the scale-out. For example, if you modify the memory of a core node group before scale-out, you must reconfigure the memory of the new task node group after the scale-out.
If you cannot write scripts for some operations, you must manually perform the operations again after the scale-out.For example, if you add a UDF by dynamically loading the UDF before scale-out and it is difficult to use a script to perform this operation, you must execute the DROP JAR statement to delete the JAR file of the UDF and then add the UDF again after the scale-out. For more information about how to dynamically load a UDF, see Dynamically load UDFs. In this case, an error may be reported when you execute the DROP JAR statement. This is because the required JAR file is not found on the newly added nodes. Ignore this error. The DROP JAR statement can be successfully executed for the worker nodes that contain the required JAR file.