All Products
Search
Document Center

Data Lake Analytics - Deprecated:Limits

Last Updated:Jul 21, 2021

This topic describes the limits of Data Lake Analytics (DLA).

Limits on accounts and metadata

Item

Limit

Maximum number of virtual clusters that can be purchased by each account

10

Maximum number of data source networks that can be configured for each virtual cluster in the serverless Presto engine of DLA

1

Maximum number of databases that can be created by each account

100

Maximum number of tables that can be stored in each database

4,096

Maximum number of columns in each table

4,096

Maximum number of partitions in each table

60,000

Maximum number of Java Database Connectivity (JDBC) connection requests that can be sent by using a client IP address within 1 minute

10

Limits on the serverless Presto engine

Item

Limit

Alibaba Cloud services supported by data sources of the external tables that are created in the serverless Presto engine of DLA

OSS

ApsaraDB RDS

Tablestore

PolarDB

ApsaraDB for Redis

ApsaraDB for MongoDB

AnalyticDB for MySQL

Storage system to which asynchronous query results of the serverless Presto engine are written

OSS

Limits on the serverless Spark engine

Item

Limit

Data sources of external tables whose metadata can be accessed by the serverless Spark engine from the metadata service of DLA

OSS

Spark features that are not supported by the serverless Spark engine of DLA

Spark JDBC (Thrift)

SparkR

Destination system to which resource files of the serverless Spark engine, such as JAR, ZIP, and TAR files, are uploaded

OSS

Accounts and permissions of the serverless Spark engine

RAM users are required. A RAM user must be associated with a DLA sub-account.

Limits on data lake management

Item

Limit

Data sources supported by the metadata discovery feature

OSS

Log Service

Tablestore

File formats of OSS data sources supported by the metadata discovery feature

CSV, JSON, Parquet, and ORC

Directory structure of OSS data sources supported by the metadata discovery feature

Database name/Table name/Partition name

File sampling method of OSS data sources supported by the metadata discovery feature

The latest and oldest files in the directory are sampled, and the first 1,000 rows of each file are read.

Whether the historical log data that is shipped from Log Service can be discovered by using the metadata discovery feature

If the partition format is changed during log shipping, the data before the change cannot be discovered.

Data source network supported by one-click data warehousing or data warehousing based on multi-database merging

VPC

Data sources supported by one-click data warehousing or data warehousing based on multi-database merging

PolarDB for MySQL

ApsaraDB RDS for MySQL

ApsaraDB RDS for SQL Server

ApsaraDB RDS for PostgreSQL

ApsaraDB for MongoDB

Data sources supported by the lakehouse solution

ApsaraDB RDS for MySQL+Data Transmission Service

PolarDB for MySQL+Data Transmission Service