All Products
Search

This Product

    E-MapReduce:Hive connector

    Document Center

    E-MapReduce:Hive connector

    Last Updated:Aug 22, 2026

    The Hive connector lets you query and analyze data stored in a Hive data warehouse.

    Background

    A Hive data warehouse has three components:
    • Data files in various formats, typically stored in the Hadoop Distributed File System (HDFS) or an object storage system, such as Alibaba Cloud OSS.
    • Metadata that maps data files to schemas and tables, stored in a database such as MySQL, and accessed through the Hive Metastore Service (HMS).
    • A query language called HiveQL, which runs on a distributed computing framework such as MapReduce or Tez.

    Prerequisites

    You have created a DataLake cluster or a Hadoop cluster and selected the Presto service. For instructions, see Create cluster.

    Limitations

    • The Hive connector requires the Hive Metastore Service to be configured.
    • The Hive connector supports various distributed storage systems, including HDFS, Alibaba Cloud OSS, and other OSS-compatible systems. The coordinator node and all worker nodes must have network access to both the Hive Metastore and the underlying storage system. By default, the Hive Metastore communicates over the Thrift protocol on port 9083.

    Modify the Hive connector configuration

    To modify the Hive connector configuration, see Configure a connector.

    Default connector configuration

    In the EMR console, go to the Configure page of the Presto service. In the Service Configuration section, click the hive.properties tab. You can view the following parameters and modify them as needed.
    Parameter Description
    hive.recursive-directories Enables reading data from subdirectories of table or partition locations. This is similar to the hive.mapred.supports.subdirectories property in Hive.
    hive.metastore.uri The URI for connecting to the Hive Metastore using the Thrift protocol.

    The default value is in the format thrift://<master-node-name>.cluster-24****:9083.
    hive.config.resources A comma-separated list of HDFS configuration files. These files must exist on all Presto hosts.
    Important Configure this parameter only when access to HDFS is required.
    hive.delta-table-enabled Specifies whether Presto can read Delta Lake tables. The valid values are:
    • true (default): Presto can read Delta Lake tables.
    • false: Presto cannot read Delta Lake tables.
    hive.delta-compatible-mode-enabled Specifies whether to enable compatibility mode for Delta Lake tables. The valid values are:
    • true (default): Compatibility mode is enabled for Delta Lake tables.
    • false: Compatibility mode is not enabled for Delta Lake tables.
    hive.hdfs.impersonation.enabled Specifies whether to enable user impersonation. The valid values are:
    • true: User impersonation is enabled.
    • false (default): User impersonation is not enabled.

    Configuration for multiple Hive clusters

    If you have multiple Hive clusters, you can add multiple catalog files to the etc/catalog directory. Ensure that each file has a .properties extension.

    For example, if the properties file is named sales.properties, Presto uses the connector configuration in that file to create a catalog named sales.

    HDFS configuration

    Presto automatically configures the HDFS client and typically requires no configuration files. However, in some cases, such as when you enable federated HDFS or NameNode high availability, you must specify additional HDFS client options to access your HDFS cluster. To do this, add the hive.config.resources property to reference the required HDFS configuration files.
    Important
    • Include only the minimum set of required properties in the files to prevent incompatibilities.
    • The configuration files must exist on all Presto hosts. If you reference existing Hadoop configuration files, make sure to copy them to any Presto nodes that are not running Hadoop.

    HDFS username and permissions

    Before you run any CREATE TABLE or CREATE TABLE AS statements for Hive tables in Presto, ensure that the user Presto uses to access HDFS has permissions to the Hive warehouse directory. The Hive warehouse directory is specified by the hive.metastore.warehouse.dir configuration variable in the hive-site.xml file. The default value is /user/hive/warehouse.

    Supported file types

    The Hive connector supports the following file types.
    File type Notes
    ORC None
    Parquet None
    Avro None
    RCText RCFile that uses ColumnarSerDe.
    RCBinary RCFile that uses LazyBinaryColumnarSerDe.
    SequenceFile None
    JSON Uses org.apache.hive.hcatalog.data.JsonSerDe.
    CSV Uses org.apache.hadoop.hive.serde2.OpenCSVSerde.
    TextFile None

    Supported table types

    The Hive connector supports the following table types.
    Type Description
    ACID table When connected to a Hive Metastore version 3.x, the Hive connector can read from and write to insert-only and ACID tables. The connector fully supports partitions and buckets.

    It supports row-level DELETE and UPDATE operations for ACID tables. However, it does not support UPDATE operations on partition key columns and bucket columns. The connector does not support querying ACID tables created by Hive Streaming Ingest. For more information, see Streaming Data Ingest.
    materialized view The Hive connector can read data from Hive materialized views. In Presto, these views are presented as regular, read-only tables.

    Hive view

    A Hive view is defined in HiveQL and stored in the Hive Metastore Service.

    The Hive connector supports Hive views in the following three modes.
    Mode Description
    Disabled The business logic and data encoded in the view are not visible in Presto.

    By default, Presto ignores Hive views.
    Legacy A simple implementation that allows Presto to read data from Hive views.

    To enable this mode, set hive.translate-hive-views=true and hive.legacy-hive-view-translation=true.

    To temporarily enable legacy access for a specific catalog, set the catalog session property legacy_hive_view_translation to true.

    Because HiveQL is very similar to SQL, the legacy mode interprets the view's HiveQL query as standard SQL, with no translation.

    This approach works for simple Hive views but can cause issues with complex queries. For example, if a HiveQL function has the same signature as an SQL function but different behavior, the results may differ. In more extreme cases, a query may fail to parse or execute.
    Experimental This mode analyzes, processes, and rewrites Hive views, including their expressions and statements.

    To enable this mode, set hive.translate-hive-views=true.

    This mode does not yet support the following features:
    • HiveQL statements such as current_date, current_timestamp, and the like.
    • Hive function calls such as translate(), window functions, and the like.
    • Common table expressions and simple case expressions.
    • Setting timestamp precision.
    • Correct mapping of all Hive data types to Presto types.
    • Handling custom UDFs.

    Configuration properties

    Hive configuration properties

    The Hive connector supports acceleration with JindoTable. EMR clusters include two built-in Hive connectors: hive.properties and hive-acc.properties. The hive-acc.properties file includes the JindoTable Native Engine to optimize and accelerate files in ORC or Parquet format. See the documentation for your version of SmartData. For details, see Enable native query acceleration.

    The following table lists the configuration properties for the Hive connector.
    Parameter Description
    hive.config.resources A comma-separated list of HDFS configuration files. These files must exist on all Presto hosts.
    Note Configure this property only when access to HDFS is required.
    hive.recursive-directories Allows reading data from subdirectories of table or partition locations, similar to the Hive hive.mapred.supports.subdirectories property.
    hive.ignore-absent-partitions Ignores a partition if its file system location is missing, instead of failing the query. This may cause data to be skipped.

    The default value is false.
    hive.storage-format Specifies the default file format for new tables.

    The default value is ORC.
    hive.compression-codec Specifies the compression codec to use when writing files. Valid values: NONE, SNAPPY, LZ4, ZSTD, or GZIP.

    The default value is GZIP.
    hive.force-local-scheduling Forces splits to be scheduled on the same node as the Hadoop DataNode. This configuration improves efficiency for installations where Presto is co-located with each DataNode.

    The default value is false.
    hive.respect-table-format Controls whether new partitions use the existing table format or the Presto format. Valid values:
    • true (default): Use the existing table format.
    • false: Use the Presto format.
    hive.immutable-partitions Controls whether new data can be inserted into existing partitions.

    If set to true, hive.insert-existing-partitions-behavior cannot be set to APPEND.

    The default value is false.
    hive.insert-existing-partitions-behavior Specifies the behavior when inserting data into an existing partition. Valid values:
    • APPEND (default): Append data to the existing partition.
    • OVERWRITE: Overwrite the existing partition.
    • ERROR: Disallow modifications to existing partitions.
    hive.create-empty-bucket-files Controls whether to create empty files for empty buckets. Valid values:
    • true: Create empty files.
    • false (default): Do not create empty files.
    hive.max-partitions-per-writers The maximum number of partitions per writer.

    The default value is 100.
    hive.max-partitions-per-scan The maximum number of partitions for a single table scan.

    The default value is 100,000.
    hive.hdfs.authentication.type Specifies the HDFS authentication type. Valid values:
    • NONE (default): Standard mode, without Kerberos authentication.
    • KERBEROS: Secure mode, with Kerberos authentication.
    hive.hdfs.impersonation.enabled Enables HDFS user impersonation. Valid values:
    • true: Enable HDFS user impersonation.
    • false (default): Disable HDFS user impersonation.
    hive.hdfs.trino.principal The Kerberos principal used by Presto to connect to HDFS.
    hive.hdfs.trino.keytab The location of the HDFS client keytab file.
    hive.dfs.replication The HDFS replication factor.
    hive.security The default value is legacy. For more information, see Hive connector security configuration.
    security.config-file Specifies the path to the configuration file used when hive.security=file is set.
    hive.non-managed-table-writes-enabled Enables writes to non-managed (external) Hive tables.

    The default value is false.
    hive.non-managed-table-creates-enabled Enables creation of non-managed (external) Hive tables.

    The default value is true.
    hive.collect-column-statistics-on-write Enables automatic collection of column-level statistics on write. For more information, see Table statistics.

    The default value is true.
    hive.file-status-cache-tables A list of tables for which to cache file statuses.

    For example, fruit.apple,fruit.orange caches only the apple and orange tables in the fruit schema. fruit.*,vegetable.* caches all tables in the fruit and vegetable schemas. * caches all tables in all schemas.
    hive.file-status-cache-size The maximum total number of cached file status entries.

    The default value is 1,000,000.
    hive.file-status-cache-expire-time How long cached file statuses remain valid.

    The default value is 1m.
    hive.rcfile.time-zone Adjusts binary-encoded timestamp values to the specified time zone.

    The default is the JVM default time zone.

    Note For Hive 3.1 and later, this value must be set to UTC.
    hive.timestamp-precision Specifies the precision for timestamp columns in Hive. Valid values:
    • MILLISECONDS
    • MICROSECONDS
    • NANOSECONDS

    The default value is MILLISECONDS.

    Note Values with precision higher than the configured value are rounded.
    hive.temporary-staging-directory-enabled Controls whether to use the temporary staging directory configured in hive.temporary-staging-directory-path for write operations. The temporary staging directory is not used when writing to OSS, encrypted HDFS, or unsorted tables in external locations. When writing to sorted tables, this path is used to stage temporary files during the sorting operation. If disabled, the target storage is used for staging when writing to sorted tables, which can be inefficient for object storage.

    The default value is true.
    hive.temporary-staging-directory-path Specifies the location of the temporary staging directory for write operations.
    The default value is /tmp/presto-${USER}.
    Note The ${USER} placeholder can be used to assign a different location for each user.
    hive.translate-hive-views Enables translation of Hive views.

    The default value is false.
    hive.legacy-hive-view-translation Uses the legacy algorithm to translate Hive views. The legacy_hive_view_translation catalog session property can be used for specific catalogs.

    The default value is false.
    hive.parallel-partitioned-bucketed-writes Increases write parallelism for partitioned and bucketed tables.

    The default value is true.

    Note If this property is disabled, the number of writer threads is limited to the number of buckets.

    ORC format configuration properties

    The following properties control how the Hive connector reads and writes ORC files.
    Parameter Description
    hive.orc.time-zone Sets the default time zone for legacy ORC files that do not declare a time zone.

    The default is the JVM default time zone.
    hive.orc.use-columns-names Controls whether to access ORC columns by name.

    By default, columns in ORC files are accessed by their ordinal position in the Hive table definition. This behavior can also be controlled with the orc_use_column_names catalog session property.

    The default value is false.

    Parquet format configuration properties

    The following properties control how the Hive connector reads and writes Parquet files.
    Parameter Description
    hive.parquet.time-zone Adjusts timestamp values to the specified time zone.

    The default is the JVM default time zone.

    Note For Hive 3.1 and later, this value must be set to UTC.
    hive.parquet.use-columns-names Controls how Parquet columns are accessed. Valid values:
    • true (default): Access Parquet columns by name. Column order does not need to match the file.
    • false: Access Parquet columns by their ordinal position in the Hive table definition.
    This behavior can also be controlled with the parquet_use_column_names catalog session property.

    Metastore configuration properties

    The following properties configure the Hive metastore. For properties that further configure Thrift, see Thrift metastore configuration properties.
    Parameter Description
    hive.metastore Specifies the type of Hive metastore to use. Presto supports the default Hive Thrift metastore (thrift) and its derivatives.

    The default value is thrift.
    hive.metastore-cache-ttl Specifies how long metastore data in the Hive metastore cache is considered valid.

    The default value is 0s.
    hive.metastore-cache-maximum-size The maximum number of metastore data objects to cache.

    The default value is 10000.
    hive.metastore-refresh-interval Specifies how often cached metastore data is refreshed asynchronously after access. If the cached data is stale but not expired, subsequent accesses can see refreshed data.
    hive.metastore-refresh-max-threads The maximum number of threads used to refresh the metastore cache.

    The default value is 10.
    hive.metastore-timeout The timeout for Hive metastore requests.

    The default value is 10s.

    Thrift metastore configuration properties

    The following table describes the Thrift metastore configuration properties for the Hive connector.
    Parameter Description
    hive.metastore.uri The URI for connecting to the Hive metastore using the Thrift protocol.

    If you provide multiple URIs, the first is used by default, and the others are used as fallback metastores. This property is required. Example: thrift://192.0.**.**:9083 or thrift://192.0.**.**:9083,thrift://192.0.**.**:9083.
    hive.metastore.username The username that Presto uses to access the Hive metastore.
    hive.metastore.authentication.type Specifies the Hive metastore authentication type. Valid values:
    • NONE (default): Standard mode, without Kerberos authentication.
    • KERBEROS: Secure mode, with Kerberos authentication.
    hive.metastore.thrift.impersonation.enabled Enables Hive metastore user impersonation.
    hive.metastore.thrift.delegation-token.cache-ttl The time-to-live for the metastore delegation token cache.

    The default value is 1h.
    hive.metastore.thrift.delegation-token.cache-maximum-size The maximum size of the delegation token cache.

    The default value is 1000.
    hive.metastore.thrift.client.ssl.enabled Enables or disables SSL when connecting to the metastore. Valid values:
    • true: Use SSL to connect to the metastore.
    • false (default): Do not use SSL to connect to the metastore.
    hive.metastore.thrift.client.ssl.key The path to the private key and client certificate (keystore).
    hive.metastore.thrift.client.ssl.key-password The password for the private key.
    hive.metastore.thrift.client.ssl.trust-certificate The path to the server certificate chain (truststore).
    Note This property is required when SSL is enabled.
    hive.metastore.thrift.client.ssl.trust-certificate-password The password for the server certificate chain.
    hive.metastore.service.principal The Kerberos principal for the Hive metastore service.
    hive.metastore.client.principal The Kerberos principal used by Presto to connect to the Hive metastore service.
    hive.metastore.client.keytab The location of the Hive metastore client keytab file.

    Performance tuning properties

    The following table describes the performance tuning properties for the Hive connector.
    Important Changing these properties from their default values can cause instability and performance degradation. Proceed with caution.
    Parameter Description
    hive.max-outstanding-splits The target number of buffered splits per table scan in a query before the scheduler pauses discovery.

    The default value is 1000.
    hive.max-splits-per-second The maximum number of splits generated per second for each table scan. This setting can reduce the load on the storage system. By default, there is no limit, and Presto maximizes the parallelism of data access.
    hive.max-initial-splits For each table scan, the coordinator allocates an initial number of splits, up to this limit. Initial splits are sized up to max-initial-split-size. Subsequent splits are sized up to max-split-size.

    The default value is 200.
    hive.max-initial-split-size The size of each initial split, used until the split count reaches max-initial-splits. Smaller splits increase parallelism and can speed up small queries.

    The default value is 32MB.
    hive.max-split-size The maximum size of a single split allocated to a worker. Smaller splits result in higher parallelism, which can reduce latency but also increase overhead and system load.

    The default value is 64MB.

    Table statistics

    The Hive connector supports collecting and managing table statistics to improve query performance.

    By default, when you write data, the Hive connector collects basic table statistics (such as file count, row count, and data size) and the column-level statistics shown in the following table.
    Column type Collected statistics
    TINYINT Null count, distinct value count, min/max values
    SMALLINT Null count, distinct value count, min/max values
    INTEGER Null count, distinct value count, min/max values
    BIGINT Null count, distinct value count, min/max values
    DOUBLE Null count, distinct value count, min/max values
    REAL Null count, distinct value count, min/max values
    DECIMAL Null count, distinct value count, min/max values
    DATE Null count, distinct value count, min/max values
    TIMESTAMP Null count, distinct value count, min/max values
    VARCHAR Null count, distinct value count
    CHAR Null count, distinct value count
    VARBINARY Null count
    BOOLEAN Null count, true/false value counts