This test is performed to provide a comparative performance analysis of queries on Object Storage Service (OSS) between an open source self-managed Presto cluster and a virtual cluster (VC) in the serverless Presto engine of Data Lake Analytics (DLA). This topic describes the configuration requirements of the test environment.
Configuration requirements
- The Elastic Compute Service (ECS) instance, open source self-managed Presto cluster, and serverless Presto cluster of DLA are deployed in the same availability zone (AZ) of the same region. In this test, they are deployed in Hangzhou Zone I of the China (Hangzhou) region.
- The network types of the ECS instance, open source self-managed Presto cluster, and VC of DLA are all VPC.
- The open source self-managed Presto cluster uses Apache Presto 0.228, which does not support access to OSS. You must modify configurations to support access to OSS by performing the following steps:
- Download the Jar package of OSS. Decompress the package and copy the decompressed files to the lib and plugin/hive-hadoop2 directories.
- Add the following configurations to the core-site.xml file in the etc folder of the open source self-managed Presto cluster:
<property> <name>fs.oss.accessKeyId</name> <value>your ak</value> </property> <property> <name>fs.oss.accessKeySecret</name> <value>your sk</value> </property> <property> <name>fs.oss.credentials.provider</name> <value></value> </property> <property> <name>fs.oss.endpoint</name> <value>oss-cn-hangzhou-internal.aliyuncs.com</value> </property> <property> <name>fs.oss.impl</name> <value>org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem</value> </property>
- The following table describes the configuration items of the open source self-managed Presto cluster.
Item Configuration requirement Specifications of worker nodes ecs.hfg6.4xlarge (16 cores, 64 GB) Number of worker nodes 15 Specifications of coordinators ecs.hfg6.4xlarge (16 cores, 64 GB) - DLA uses VCs with the specifications of 256 cores and 1,024 GB.
- The ECS instance uses the specifications of ecs.hfg6.4xlarge (16 cores, 64 GB).