This test is performed to provide a comparative performance analysis of queries on Object Storage Service (OSS) between an open source self-managed Presto cluster and a virtual cluster (VC) in the serverless Presto engine of Data Lake Analytics (DLA). This topic describes the configuration requirements of the test environment.

Configuration requirements

  • The Elastic Compute Service (ECS) instance, open source self-managed Presto cluster, and serverless Presto cluster of DLA are deployed in the same availability zone (AZ) of the same region. In this test, they are deployed in Hangzhou Zone I of the China (Hangzhou) region.
  • The network types of the ECS instance, open source self-managed Presto cluster, and VC of DLA are all VPC.
  • The open source self-managed Presto cluster uses Apache Presto 0.228, which does not support access to OSS. You must modify configurations to support access to OSS by performing the following steps:
    1. Download the Jar package of OSS. Decompress the package and copy the decompressed files to the lib and plugin/hive-hadoop2 directories.
    2. Add the following configurations to the core-site.xml file in the etc folder of the open source self-managed Presto cluster:
          <property>
              <name>fs.oss.accessKeyId</name>
              <value>your ak</value>
          </property>
      
          <property>
              <name>fs.oss.accessKeySecret</name>
              <value>your sk</value>
          </property>
      
          <property>
              <name>fs.oss.credentials.provider</name>
              <value></value>
          </property>
      
          <property>
              <name>fs.oss.endpoint</name>
              <value>oss-cn-hangzhou-internal.aliyuncs.com</value>
          </property>
      
          <property>
              <name>fs.oss.impl</name>
              <value>org.apache.hadoop.fs.aliyun.oss.AliyunOSSFileSystem</value>
          </property>
  • The following table describes the configuration items of the open source self-managed Presto cluster.
    ItemConfiguration requirement
    Specifications of worker nodesecs.hfg6.4xlarge (16 cores, 64 GB)
    Number of worker nodes15
    Specifications of coordinatorsecs.hfg6.4xlarge (16 cores, 64 GB)
  • DLA uses VCs with the specifications of 256 cores and 1,024 GB.
  • The ECS instance uses the specifications of ecs.hfg6.4xlarge (16 cores, 64 GB).