This topic lists the known issues in JindoData 4.X by version. Each entry describes the affected components, trigger conditions, symptoms, and the workaround or fix.
JindoData 4.6.X
4.6.2
CRC-64 verification impacts OSS-HDFS write performance
Starting from JindoSDK 4.6.0, fs.oss.checksum.crc64.enable defaults to true, enabling CRC-64 checksum verification on write paths to ensure data integrity. This setting reduces OSS-HDFS write performance. If write performance is the priority, disable CRC-64 verification.
Workaround: In the E-MapReduce (EMR) console, go to the Configure tab on the Hadoop-Common service page. Click the core-site.xml tab and add a configuration item with key fs.oss.checksum.crc64.enable and value false. For instructions, see Manage configuration items.
4.6.1
Password-free OSS-HDFS access token not refreshed
Affected components: JindoSDK 4.6.1
When accessing OSS-HDFS in password-free mode, an error occurs indicating that the access token needs to be updated. Multiple jobs are interrupted as a result.
Workaround: Use a fixed AccessKey pair, or upgrade JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.6.2 and later.
JindoUtil permission error in password-free mode
Affected components: JindoSDK 4.6.1
When using JindoUtil in password-free mode, a permission error occurs.
Workaround: Use a fixed AccessKey pair, or upgrade JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.6.2 and later.
CRC-64 verification impacts OSS-HDFS write performance
Affected components: JindoSDK 4.6.1
Starting from JindoSDK 4.6.0, fs.oss.checksum.crc64.enable defaults to true, enabling CRC-64 checksum verification on write paths to ensure data integrity. This setting reduces OSS-HDFS write performance. If write performance is the priority, disable CRC-64 verification.
Workaround: In the EMR console, go to the Configure tab on the Hadoop-Common service page. Click the core-site.xml tab and add a configuration item with key fs.oss.checksum.crc64.enable and value false. For instructions, see Manage configuration items.
4.6.0
Password-free OSS-HDFS access token not refreshed
Affected components: JindoSDK 4.6.0
When accessing OSS-HDFS in password-free mode, an error occurs indicating that the access token needs to be updated. Multiple jobs are interrupted as a result.
Workaround: Use a fixed AccessKey pair, or upgrade JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.6.2 and later.
Memory leak in JindoFSx Namespace Service with Ranger credentials
Affected components: JindoSDK 4.6.0, JindoFSx 4.6.0
When Kerberos authentication is enabled and fs.oss.credentials.provider is set to com.aliyun.jindodata.oss.auth.RangerCredentialsProvider, a memory leak occurs in JindoFSx Namespace Service.
Workaround: Upgrade both JindoFSx and JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoData in EMR clusters in the new EMR console and Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoFSx 4.6.2 and JindoSDK 4.6.2 and later.
JindoUtil permission error in password-free mode
Affected components: JindoSDK 4.6.0
When using JindoUtil in password-free mode, a permission error occurs.
Workaround: Use a fixed AccessKey pair, or upgrade JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.6.2 and later.
CRC-64 verification impacts OSS-HDFS write performance
Affected components: JindoSDK 4.6.0
Starting from JindoSDK 4.6.0, fs.oss.checksum.crc64.enable defaults to true, enabling CRC-64 checksum verification on write paths to ensure data integrity. This setting reduces OSS-HDFS write performance. If write performance is the priority, disable CRC-64 verification.
Workaround: In the EMR console, go to the Configure tab on the Hadoop-Common service page. Click the core-site.xml tab and add a configuration item with key fs.oss.checksum.crc64.enable and value false. For instructions, see Manage configuration items.
JindoData 4.5.X
4.5.2
Memory leak in JindoFSx Namespace Service with Ranger credentials
Affected components: JindoSDK 4.5.2, JindoFSx 4.5.2
When Kerberos authentication is enabled and fs.oss.credentials.provider is set to com.aliyun.jindodata.oss.auth.RangerCredentialsProvider, a memory leak occurs in JindoFSx Namespace Service.
Workaround: Upgrade both JindoFSx and JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoData in EMR clusters in the new EMR console and Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoFSx 4.6.2 and JindoSDK 4.6.2 and later.
JindoUtil permission error in password-free mode
Affected components: JindoSDK 4.5.2
When using JindoUtil in password-free mode, a permission error occurs.
Workaround: Use a fixed AccessKey pair, or upgrade JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.6.2 and later.
4.5.1
Memory leak in JindoFSx Namespace Service with Ranger credentials
Affected components: JindoSDK 4.5.1, JindoFSx 4.5.1
When Kerberos authentication is enabled and fs.oss.credentials.provider is set to com.aliyun.jindodata.oss.auth.RangerCredentialsProvider, a memory leak occurs in JindoFSx Namespace Service.
Workaround: Upgrade both JindoFSx and JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoData in EMR clusters in the new EMR console and Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoFSx 4.6.2 and JindoSDK 4.6.2 and later.
JindoUtil permission error in password-free mode
Affected components: JindoSDK 4.5.1
When using JindoUtil in password-free mode, a permission error occurs.
Workaround: Use a fixed AccessKey pair, or upgrade JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.6.2 and later.
4.5.0
Memory leak in JindoFSx Namespace Service with Ranger credentials
Affected components: JindoSDK 4.5.0, JindoFSx 4.5.0
When Kerberos authentication is enabled and fs.oss.credentials.provider is set to com.aliyun.jindodata.oss.auth.RangerCredentialsProvider, a memory leak occurs in JindoFSx Namespace Service.
Workaround: Upgrade both JindoFSx and JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoData in EMR clusters in the new EMR console and Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoFSx 4.6.2 and JindoSDK 4.6.2 and later.
Password-free OSS/OSS-HDFS access token not refreshed on retry
Affected components: JindoSDK 4.5.0
After a failed attempt to access Object Storage Service (OSS) or OSS-HDFS in password-free mode, retries fail because the access token cannot be refreshed. Multiple jobs are interrupted as a result.
Workaround: Use a fixed AccessKey pair, or upgrade JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.6.2 and later.
JindoUtil permission error in password-free mode
Affected components: JindoSDK 4.5.0
When using JindoUtil in password-free mode, a permission error occurs.
Workaround: Use a fixed AccessKey pair, or upgrade JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.6.2 and later.
JindoData 4.4.X
Memory leak in JindoFSx Namespace Service with Ranger credentials
Affected components: JindoSDK 4.4.0, 4.4.1, or 4.4.2; JindoFSx 4.4.0, 4.4.1, or 4.4.2
When Kerberos authentication is enabled and fs.oss.credentials.provider is set to com.aliyun.jindodata.oss.auth.RangerCredentialsProvider, a memory leak occurs in JindoFSx Namespace Service.
Workaround: Upgrade both JindoFSx and JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoData in EMR clusters in the new EMR console and Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoFSx 4.6.2 and JindoSDK 4.6.2 and later.
Core dumps under high-concurrency password-free access
Affected components: JindoSDK 4.4.0
When initiating high-concurrency access to OSS or OSS-HDFS in password-free mode, core dumps may occur.
Workaround: Use a fixed AccessKey pair, or upgrade JindoSDK to 4.6.2 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.6.2 and later.
JindoData 4.3.X
ls command performance degradation causes missing directory timestamps
Affected components: JindoSDK 4.3.0 (EMR V3.40.0 or V5.6.0)
The ls command performance degrades when displaying directory timestamps. As a result, the time when a directory was last updated cannot be displayed.
Workaround: Upgrade JindoSDK to 4.3.1 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.3.1 and later.
MagicCommitter uploadPart error exceeds part number limit
Affected components: JindoSDK 4.3.0 (EMR V3.40.0 or V5.6.0)
When using MagicCommitter, the uploadPart operation is called excessively. This triggers the following error: Part number must be an integer between 1 and 10000.
Workaround: Upgrade JindoSDK to 4.3.1 or later. For upgrade instructions, see Upgrade JindoSDK in EMR clusters in the new EMR console.
Fixed in: JindoSDK 4.3.1 and later.
Cached data read errors silently return invalid data
Affected components: JindoFSx 4.3.0
When reading cached data in specific paths, an error occurs on the JindoFSx server but is not propagated to the client. The client returns invalid data without indication of failure.
Workaround: Upgrade JindoFSx to 4.3.1 or later. For upgrade instructions, see Upgrade JindoData in EMR clusters in the new EMR console.
Fixed in: JindoFSx 4.3.1 and later.
Preload-to-memory commands not processed
Affected components: JindoFSx 4.3.0
The JindoFSx server cannot process commands that preload data to memory for caching. This may cause load errors and invalid data to be read from the cache.
Workaround: Upgrade JindoFSx to 4.3.1 or later. For upgrade instructions, see Upgrade JindoData in EMR clusters in the new EMR console.
Fixed in: JindoFSx 4.3.1 and later.
File handle leaks cause service unavailability after extended runtime
Affected components: JindoFSx 4.3.0 and 4.3.1
File handle leaks occur on the JindoFSx server. After the server runs for an extended period, the number of file handles opened by operating system processes may reach the system limit. The server can no longer open new file handles, making the service unavailable.
Workaround: Upgrade JindoFSx to 4.3.2 or later. For upgrade instructions, see Upgrade JindoData in EMR clusters in the new EMR console.
Fixed in: JindoFSx 4.3.2 and later.
JindoData 4.2.X
Integer overflow in seek method causes large file read failures
Affected components: JindoSDK 4.2.0
An integer overflow may occur when the seek method is called on large files. Jobs that rely on seek may fail to read large files from OSS.
JindoData 4.1.X
Integer overflow in seek method causes large file read failures
Affected components: JindoSDK 4.1.0
An integer overflow may occur when the seek method is called on large files. Jobs that rely on seek may fail to read large files from OSS.
JindoData 4.0.X
Integer overflow in seek method causes large file read failures
Affected components: JindoSDK 4.0.0 (EMR V3.39.0 or EMR V5.5.0)
An integer overflow may occur when the seek method is called on large files. Jobs that rely on seek may fail to read large files from OSS.
Limitations
The following are permanent product limitations, not version-specific bugs. Upgrading JindoSDK or JindoFSx does not resolve these limitations.
JindoSDK cannot write files larger than 80 GB to OSS.
JindoSDK does not support writing data to OSS in append mode.
JindoSDK does not support client-side encryption of data uploaded to OSS.
JindoSDK does not support earlier versions of JindoFS in block mode or cache mode.
OSS-HDFS does not support upgrading earlier versions of JindoFS in block mode. Use JindoDistCp to migrate data to the latest version.