SmartData is a storage service for the E-MapReduce (EMR) Jindo engine. It provides centralized storage, caching, and computing optimization for EMR computing engines and extends storage features. SmartData consists of JindoFS, JindoTable, and related tools. The following section describes the updates in SmartData 3.5.X.
OSS storage scalability on JindoFS
Improved performance for deleting Object Storage Service (OSS) directories.
JindoSDK
- When JindoSDK for Java is used, JindoSDK logs are output as Java logs to improve diagnostics.
- Memory usage statistics for JindoSDK are logged. You can view the logs to obtain information about the memory used by JindoSDK.
JindoTable-based computing optimization
- Native query acceleration is added to speed up queries when you use Spark, Hive, or Presto to read data from ORC or Parquet files stored in OSS or JindoFS. For more information, see Enable native query acceleration.
- Infrequent-access statistics on Hive tables can be collected with JindoTable. For more information, see Infrequent-access statistics for JindoTable.
Other JindoFS tools
Jindo DistCp enhancements: The CloudMonitor service can be used to monitor and alert on failed tasks, the dependency on Advanced Vector Extensions (AVX) is removed, and Cold Archive is supported when data is written to OSS. For more information, see Use Jindo DistCp.