BM25 - OpenSearch - Alibaba Cloud Documentation Center

The BM25 class calculates relevance scores for query terms against a specific field using the BM25 algorithm. Use it inside a custom scorer to rank search results by field-level relevance.

The BM25 score is computed using the following formula:

Variable	Description
`tf(i)`	Term frequency (TF) — the number of times the query term appears in the field
`idf(i)`	Inverse document frequency (IDF) of the query term
`f(ngram)`	`ngram > 1 ? ngram * C : 1`
`K`	Controls term frequency saturation. Higher values reduce the scoring boost from repeated terms.
`C`	Weight of the retrieval unit
`B`	Controls field length normalization. Higher values penalize longer fields more strongly.

Functions

setGroupScoreMergeOp, setParamK, setParamB, setParamC, and setFieldAvgLength can only be called during the initialization of a score calculation object.

Function	Description	Default
`BM25 create(OpsScorerInitParams params, CString indexName, CString fieldName)`	Creates a BM25 object for the specified index and field	—
`void setGroupScoreMergeOp(CString opName)`	Sets the aggregation method (`sum` or `max`) for BM25 scores across multiple query groups	`sum`
`void setParamK(double paramK)`	Sets K — controls term frequency saturation	`2.0`
`void setParamB(double paramB)`	Sets B — controls field length normalization	`0.1`
`void setParamC(double paramC)`	Sets C — the weight of the retrieval unit	`0.7`
`void setFieldAvgLength(double avgFieldLength)`	Sets the average field length used in normalization	`20`
`double evaluate(OpsScoreParams params)`	Calculates the BM25 score for the query term in the specified field. Returns a value in `[0, 1]`.	—

Function details

`BM25 create(OpsScorerInitParams params, CString indexName, CString fieldName)`

Creates a BM25 object bound to a specific index and field.

Parameters:

Parameter	Description
`params`	Initialization parameters. See OpsScorerInitParams.
`indexName`	The index name. Must be a constant.
`fieldName`	The field name within the index. Must be a constant.

The field must be of the TEXT or SHORT_TEXT type. Supported analyzers:

General analyzer for Chinese
Custom analyzer
Single character analyzer for Chinese
Analyzer for English
Analyzer for fuzzy searches

`void setGroupScoreMergeOp(CString opName)`

Sets the aggregation method for BM25 scores across multiple query groups. Query groups are generated when the analyzer processes the original search query. By default, only one query group exists.

Parameter:

Parameter	Description
`opName`	Aggregation method: `sum` (default) or `max`.

`void setParamK(double paramK)`

Sets the K parameter, which controls term frequency saturation. Higher K values reduce the scoring boost from repeated terms.

Parameter:

Parameter	Description
`paramK`	Value for K. Default: `2.0`.

`void setParamB(double paramB)`

Sets the B parameter, which controls field length normalization. Higher B values penalize longer fields more strongly.

Parameter:

Parameter	Description
`paramB`	Value for B. Default: `0.1`.

`void setParamC(double paramC)`

Sets the C parameter, which is the weight of the retrieval unit.

Parameter:

Parameter	Description
`paramC`	Value for C. Default: `0.7`.

`void setFieldAvgLength(double avgFieldLength)`

Sets the average field length used in the length normalization calculation.

Parameter:

Parameter	Description
`avgFieldLength`	Average field length. Default: `20`.

`double evaluate(OpsScoreParams params)`

Calculates the BM25 score for the query term in the configured field and index.

Parameter:

Parameter	Description
`params`	Score calculation parameters. See OpsScoreParams.

Return value: A BM25 score in [0, 1], where 0 indicates no match and 1 indicates maximum relevance.

Example:

package users.scorer;
import com.aliyun.opensearch.cava.framework.OpsScoreParams;
import com.aliyun.opensearch.cava.framework.OpsScorerInitParams;
import com.aliyun.opensearch.cava.features.similarity.fieldmatch.BM25;

class BasicSimilarityScorer {
    BM25 _f1;

    boolean init(OpsScorerInitParams params) {
        // Create a BM25 object for the "text" field in the "text_index" index
        _f1 = BM25.create(params, "text_index", "text");

        // Tune scoring parameters during initialization
        _f1.setParamK(3);              // Increase K to reduce term frequency saturation
        _f1.setParamB(0.5);            // Increase B for stronger field length normalization
        _f1.setParamC(1.2);            // Adjust retrieval unit weight
        _f1.setFieldAvgLength(30);     // Set expected average field length
        return true;
    }

    double score(OpsScoreParams params) {
        return _f1.evaluate(params);   // Returns a score in [0, 1]
    }
};