OpenSearch:Offline cluster configuration

customized_merge_config

Customized_merge_config: This user-customizable configuration allows for the implementation of distinct merge policies tailored to specific scenarios. For instance, a merge policy differing from the default may be employed during the incremental phase. Multiple merge policies can be established and activated concurrently. If several merge policies are triggered simultaneously, they will execute in the sequence defined in the configuration. In addition to merge_config, users can also set the following two parameters:

• period: Specifies the index merging interval in seconds, or schedules merges at a precise time, such as 1 PM, using the format "period": "daytime=13:00".

• need_wait_alter_field: Determines whether to pause merging when a dynamic field addition is in progress. The default is set to true. If false, the current merge policy will automatically transition to the align_version merge policy.

• merge_parallel_num: The count of concurrent merge processes for each partition.

Due to the introduction of dynamic field addition, user-defined merge policies named alter_field are now disallowed.

The default settings for customized_merge_config are:

{
    "full":{
        "merge_config":{
            "keep_version_count":40,
            "merge_strategy":"optimize",
            "merge_strategy_param":"after-merge-max-segment-count=20",
            "merge_thread_count":4
        }
    },
    "large_segment_reclaim":{
        "merge_config":{
            "keep_version_count":40,
            "merge_strategy":"priority_queue",
            "merge_strategy_params":{
                "input_limits":"max-segment-size=20480",
                "output_limits":"max-merged-segment-size=13312;max-total-merged-size=15360",
                "strategy_conditions":"priority-feature=delete-doc-count#desc;conflict-segment-count=2;conflict-delete-percent=8"
            },
            "merge_thread_count":4
        },
        "period":"period=1800"
    },
    "segment_merge":{
        "merge_config":{
            "keep_version_count":40,
            "merge_strategy":"priority_queue",
            "merge_strategy_params":{
                "input_limits":"max-segment-size=12288",
                "output_limits":"max-merged-segment-size=13312;max-total-merged-size=15360",
                "strategy_conditions":"priority-feature=valid-doc-count#asc;conflict-segment-count=2;conflict-delete-percent=10"
            },
            "merge_thread_count":4
        },
        "period":"period=900"
    },
    "large_segment_merge":{
        "merge_config":{
            "keep_version_count":40,
            "merge_strategy":"priority_queue",
            "merge_strategy_params":{
                "input_limits":"max-segment-size=12288",
                "output_limits":"max-merged-segment-size=15360;max-total-merged-size=46080",
                "strategy_conditions":"priority-feature=valid-doc-count#desc;conflict-segment-count=2;conflict-delete-percent=10"
            },
            "merge_thread_count":4
        },
        "period":"period=3600"
    },
    "small_segment_merge":{
        "merge_config":{
            "keep_version_count":40,
            "merge_strategy":"priority_queue",
            "merge_strategy_params":{
                "input_limits":"max-segment-size=1536",
                "output_limits":"max-merged-segment-size=1536;max-total-merged-size=3072",
                "strategy_conditions":"priority-feature=valid-doc-count#asc;conflict-segment-count=2;conflict-delete-percent=20"
            },
            "merge_thread_count":4
        }
    }
}

segment_customize_metrics_updater

Segment_customize_metrics_updater: This tool is designed for creating segment metrics for each segment, which users can tailor to their needs. The indexlib comes with a built-in lifecycle updater. By configuring this updater, each segment will generate minimum and maximum values for the specified fields and assign tags to each segment based on the tagging method outlined in the lifecycle_param configuration. This tagging process, which occurs during the merge and build phases and uses the default tag, facilitates load policies and time_series_merge policies. The default value is an empty array ([]).