All Products
Search

This Product

    Elasticsearch:Use the codec-compression plug-in (Beta)

    Document Center

    Elasticsearch:Use the codec-compression plug-in (Beta)

    Last Updated:Jun 17, 2026

    codec-compression is an index compression plug-in developed by the Alibaba Cloud Elasticsearch team. It supports the brotli and zstd compression algorithms to achieve a higher compression ratio for your indexes, significantly reducing storage costs.

    Prerequisites

    Before you begin, ensure you have completed the following:

    • Create an Alibaba Cloud Elasticsearch V6.7.0 cluster. For more information, see Create an Alibaba Cloud Elasticsearch cluster.

      Important

      The codec-compression plug-in is available only for Alibaba Cloud Elasticsearch V6.7.0 clusters.

    • Verify that the codec-compression plug-in is installed. This plug-in is installed by default on new clusters.

      You can check whether the codec-compression plug-in is installed on the Plug-ins page. If it is not installed, install it manually. For more information, see Install or uninstall a built-in plug-in.

    Background information

    The codec-compression plug-in supports the brotli and zstd compression algorithms. It is suitable for write-heavy workloads with high index storage costs, such as logging and time-series analytics, and can significantly reduce index storage costs.

    • Test environment

      • Configuration: Three data nodes, each with 16 cores, 64 GB of memory, and a 2 TB SSD cloud disk.

      • Dataset: The 74 GB nyc_taxis dataset from the official esrally.

      • Index settings: Default settings were used. A force merge operation was performed after the data was written.

    • Test results

      Compression algorithm

      Index size (GB)

      Write TPS (doc/s)

      Elasticsearch default compression algorithm (LZ4)

      35.5

      202682

      best_compression (DEFLATE)

      26.4

      181686

      brotli

      24.4

      182593

      zstd

      24.6

      181393

    • Conclusion

      Compared to the default compression algorithm (LZ4), the brotli and zstd algorithms reduce write performance by 10% and improve the compression ratio by 45%, with no impact on query performance. Compared to the native best_compression (DEFLATE) setting, these algorithms improve the compression ratio by 8% without affecting write or query performance.

    Procedure

    1. Log on to the Kibana console.

      For more information, see Log on to the Kibana console.

    2. In the left-side navigation pane, click Dev Tools.

    3. In the Console, run the following commands to set a compression algorithm for an index.

      Note

      In Alibaba Cloud Elasticsearch V6.7.0, the codec setting is static and can be set only when an index is created. You cannot modify the codec of an existing index. To change the compression algorithm for an existing index, you must reindex its data:

      1. Create a new index and specify the target compression algorithm in the index settings, such as "codec": "brotli".

      2. Use reindex to migrate data from the old index to the new index.

      3. After verifying that the data is correct, delete the old index.

      • brotli compression algorithm

        PUT index-1
{
    "settings": {
        "index": {
            "codec": "brotli"
        }
    }
}

      • zstd compression algorithm

        PUT index-1
{
    "settings": {
        "index": {
            "codec": "zstd"
        }
    }
}