All Products
Search

This Product

    E-MapReduce:FAQ

    Document Center

    E-MapReduce:FAQ

    Last Updated:Aug 20, 2026

    This topic summarizes common issues when using Hudi.

    Duplicate data when querying Hudi with Spark

    • Cause: This issue occurs because Hudi does not support reading data using the Spark DataSource API.

    • Solution: When querying a Hudi table, set the spark.sql.hive.convertMetastoreParquet=false configuration.

    Duplicate data when querying Hudi with Hive

    • Cause: By default, Hive uses HiveCombineInputFormat, which does not use the table's custom input format.

    • Solution: When querying a Hudi table, run set hive.input.format = org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.

    Partition pruning failure with Spark and Hudi

    • Cause: This issue can occur if a partition field contains a forward slash (/). This causes a mismatch between the number of partition fields and the number of levels in the partition directory, preventing Spark from applying partition pruning.

    • Solution: When writing to a Hudi table using the Spark DataFrame API, set the hoodie.datasource.write.partitionpath.urlencode=true configuration.

    Error using ALTER TABLE in Spark: xxx is only supported with v2 tables

    • Cause: This issue occurs because Hudi's schema evolution feature for Spark requires the hoodie.schema.on.read.enable configuration to be set to true.

    • Solution: Run the set hoodie.schema.on.read.enable=true command when you perform an ALTER TABLE operation on a Hudi table. For more information, see SparkSQL Schema Evolution and Syntax Description in the Apache Hudi documentation.