:Offline cluster configuration

customized_merge_config

Customized_merge_config: This user-defined configuration allows for the customization of merge policies tailored to specific scenarios. For instance, a merge policy distinct from the default may be employed during the incremental phase. Multiple merge policies can be set up to operate concurrently. If several merge policies are activated simultaneously, they will execute in the sequence defined in the configuration. In addition to merge_config, users can also set the following two parameters:

• Period: The interval for index merging, expressed in seconds, or it can be scheduled to merge at a specific time, such as 1 PM, with the format "period": "daytime=13:00".

• Need_wait_alter_field: Determines whether to pause merging when a dynamic field addition is in progress, with the default set to true. Setting this to false allows the current merge policy to automatically switch to the align_version merge policy.

• Merge_parallel_num: Specifies the number of concurrent merge processes for each partition.

The introduction of the dynamic field addition feature prohibits the use of user-defined merge policies named alter_field.

The default values for customized_merge_config are as follows:

{
    "full":{
        "merge_config":{
            "keep_version_count":40,
            "merge_strategy":"optimize",
            "merge_strategy_param":"after-merge-max-segment-count=20",
            "merge_thread_count":4
        }
    },
    "large_segment_reclaim":{
        "merge_config":{
            "keep_version_count":40,
            "merge_strategy":"priority_queue",
            "merge_strategy_params":{
                "input_limits":"max-segment-size=20480",
                "output_limits":"max-merged-segment-size=13312;max-total-merged-size=15360",
                "strategy_conditions":"priority-feature=delete-doc-count#desc;conflict-segment-count=2;conflict-delete-percent=8"
            },
            "merge_thread_count":4
        },
        "period":"period=1800"
    },
    "segment_merge":{
        "merge_config":{
            "keep_version_count":40,
            "merge_strategy":"priority_queue",
            "merge_strategy_params":{
                "input_limits":"max-segment-size=12288",
                "output_limits":"max-merged-segment-size=13312;max-total-merged-size=15360",
                "strategy_conditions":"priority-feature=valid-doc-count#asc;conflict-segment-count=2;conflict-delete-percent=10"
            },
            "merge_thread_count":4
        },
        "period":"period=900"
    },
    "large_segment_merge":{
        "merge_config":{
            "keep_version_count":40,
            "merge_strategy":"priority_queue",
            "merge_strategy_params":{
                "input_limits":"max-segment-size=12288",
                "output_limits":"max-merged-segment-size=15360;max-total-merged-size=46080",
                "strategy_conditions":"priority-feature=valid-doc-count#desc;conflict-segment-count=2;conflict-delete-percent=10"
            },
            "merge_thread_count":4
        },
        "period":"period=3600"
    },
    "small_segment_merge":{
        "merge_config":{
            "keep_version_count":40,
            "merge_strategy":"priority_queue",
            "merge_strategy_params":{
                "input_limits":"max-segment-size=1536",
                "output_limits":"max-merged-segment-size=1536;max-total-merged-size=3072",
                "strategy_conditions":"priority-feature=valid-doc-count#asc;conflict-segment-count=2;conflict-delete-percent=20"
            },
            "merge_thread_count":4
        }
    }
}

segment_customize_metrics_updater

Segment_customize_metrics_updater: This tool is designed for generating segment metrics for each segment, which users can customize. The indexlib comes with an integrated lifecycle updater. By configuring this updater, each segment will produce minimum and maximum values for the specified fields and apply tags to each segment based on the tagging method outlined in lifecycle_param (tagging occurs during the merge and build phases with the default tag). Tagging is instrumental for load policies and time_series_merge policies. The default value is an empty array: [].