JindoCache (formerly JindoFSx) is an Alibaba Cloud EMR service that accelerates cloud-native data lakes through data and metadata caching. It offers various read and write policies for different CacheSets to meet access acceleration needs across various data lake scenarios.
Background information
A CacheSet is a cache abstraction in JindoCache. Not all data requires cache acceleration, so JindoCache provides fine-grained access policies to support diverse data lake workloads. You can configure aggressive metadata caching or disable caching for specific data to achieve optimal performance and resource efficiency.
Scenarios
JindoCache applies to the following scenarios:
-
Online Analytical Processing (OLAP) (Presto queries): Improves query performance and reduces query time.
-
Data serving (HBase): Significantly reduces P99 latency and request costs.
-
Big data analytics (Hive/Spark reports): Reduces report generation time and optimizes compute cluster costs.
-
Data lakehouse: Reduces request costs and optimizes the response latency of the data catalog.
-
AI: Accelerates training and other AI workloads, reducing AI cluster costs, and provides comprehensive support.
Cache policies
JindoCache supports data caching and metadata caching. Data caching methods include distributed caching, consistent hashing, and local caching.