Configure and use synonyms, update synonyms, manage synonyms, delete synonyms - Elasticsearch

In search scenarios, users often use different words to express the same concept, such as "cell phone" and "smartphone." This can lead to incomplete search results. The synonym feature addresses this by treating these terms as equivalent, expanding the search scope, improving the recall rate, and enhancing the user experience.

Before you begin

Before you change a synonym file, verify the following:

Ensure that critical indexes have at least one replica shard to maintain service availability during a single-node restart. (Deleting a dictionary triggers a cluster restart.)
Monitor the cluster to ensure its load is at a healthy level (recommended: CPU usage < 60%, heap memory usage < 50%).
Connect to your cluster and run GET /_nodes/stats/jvm?filter_path=nodes.*.jvm.mem.heap_* to check CPU and heap memory usage for all nodes.

How it works and decision-making

This guide compares the two methods for configuring synonyms and their trade-offs to help you select the best approach for your needs.

Synonym syntax rules

A synonym file must be a UTF-8 encoded .txt file. Each line in the file defines a synonym rule in one of the following two formats:

Equivalent synonyms (Solr format)
Terms separated by commas are treated as fully equivalent. A search for one term matches documents containing any term in the group.
```
# Example: Searches for "phone", "smartphone", and "mobile phone" yield the same results.
phone,smartphone,mobile phone
ipod,i-pod,i pod
```
Directional mapping (WordNet format)
Use => to map a set of terms to a single, canonical term. This is often used for normalization, where non-standard terms are mapped to a preferred one.
```
# Example: Map "usa" and "us" to "United States".
usa,us => United States
```

Comparison of configuration methods

You can configure synonyms by uploading a file or by defining them inline in the index configuration. The following table compares these two methods.

Item	Method 1: Upload file	Method 2: Define inline
How to configure	Upload a .txt file to the cluster and reference it in the index `settings` by using the `synonyms_path` parameter.	Write synonym rules directly into the `synonym` `filter` within the index settings.
Pros	Easy to manage and reuse large dictionaries. Decouples the dictionary from the index configuration, allowing it to be shared across multiple indexes.	Updates take effect without a cluster restart. Suitable for small, infrequently changing sets of synonyms.
Cons	Existing indexes cannot dynamically load new dictionaries.	Difficult to reuse because rules must be defined separately for each index. Not suitable for managing large or complex dictionaries.
Use cases	Use when the dictionary is stable, changes infrequently, and needs to be shared across multiple indexes.	Use when high availability is critical or when you need to update synonyms frequently and quickly.

Procedure

Method 1: Upload a synonym file (reusable)

This method is best for dictionaries that change infrequently.

Step 1: Upload the synonym file

This example shows how to configure synonyms by using a filter with a test file named aliyun_synonyms.txt, which contains the entry: begin, start.

Log on to the Alibaba Cloud Elasticsearch console. Select the region and resource group where your instance is located, and then click the ID of the target instance.
In the left-side navigation pane, choose Configuration and Management > Cluster Configuration. In the Basic Configuration section, find Synonym Configuration and click Upload.
In the panel that appears, click Configure and select an upload method:
The file must have a .txt extension. The filename can contain uppercase letters, lowercase letters, digits, and underscores (_), and must not exceed 30 characters in length.
- Upload File: Select the synonym .txt file from your local machine.
- Add OSS File: Enter the bucket name and the synonym filename, and then click Add.
  Limitation: The OSS bucket must be in the same region as the Alibaba Cloud Elasticsearch instance.
Click Save and confirm the operation.

Step 2: Create an index and reference the file

Wait for the instance to return to Active. Then, connect to the cluster and create an index that uses the uploaded synonym file.

PUT /aliyun-index-test
{
  "settings": {
    "index":{
      "analysis": {
          "analyzer": {
            "by_smart": {
              "type": "custom",
              "tokenizer": "ik_smart",
              "filter": ["by_tfr","by_sfr"],
              "char_filter": ["by_cfr"]
            },
            "by_max_word": {
              "type": "custom",
              "tokenizer": "ik_max_word",
              "filter": ["by_tfr","by_sfr"],
              "char_filter": ["by_cfr"]
            }
         },
         "filter": {
            "by_tfr": {
              "type": "stop",
              "stopwords": [" "]
              },
           "by_sfr": {
              "type": "synonym",
              "synonyms_path": "analysis/aliyun_synonyms.txt"
              }
          },
          "char_filter": {
            "by_cfr": {
              "type": "mapping",
              "mappings": ["| => |"]
            }
          }
      }
    }
  }
}

The syntax for creating an index varies by cluster version. For more information, see Index operation examples for common Elasticsearch versions.

Step 3: Configure the title field

For Elasticsearch versions earlier than 7.0

PUT /aliyun-index-test/_mapping/doc
{
"properties": {
 "title": {
   "type": "text",
   "analyzer": "by_max_word",
   "search_analyzer": "by_smart"
 }
}
}

For Elasticsearch 7.0 and later

PUT /aliyun-index-test/_mapping/
{
"properties": {
 "title": {
   "type": "text",
   "analyzer": "by_max_word",
   "search_analyzer": "by_smart"
 }
}
}

Step 4: Verify configuration

Use the _analyze API to verify that the analyzer has correctly loaded the synonyms. This example assumes the synonym file contains begin,start.

GET /aliyun-index-test/_analyze
{
"analyzer": "by_smart",
"text":"begin"
}

A successful response includes both the begin and start tokens.

{
  "tokens" : [
    {
      "token" : "begin",
      "start_offset" : 0,
      "end_offset" : 5,
      "type" : "ENGLISH",
      "position" : 0
    },
    {
      "token" : "start",
      "start_offset" : 0,
      "end_offset" : 5,
      "type" : "SYNONYM",
      "position" : 0
    }
  ]
}

Step 5: Test the search results

Index two documents, each containing one of the synonymous terms.

PUT /aliyun-index-test/doc/1
{
"title": "Shall I begin?"
}

PUT /aliyun-index-test/doc/2
{
"title": "I start work at nine."
}

Search for one of the terms, such as begin. The search returns documents that contain both begin and start.

GET /aliyun-index-test/_search
{
 "query" : { "match" : { "title" : "begin" }},
 "highlight" : {
     "pre_tags" : ["<red>", "<blue>"],
     "post_tags" : ["</red>", "</blue>"],
     "fields" : {
         "title" : {}
     }
 }
}

Response:

{
  "took" : 70,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 0.28247005,
    "hits" : [
      {
        "_index" : "aliyun-index-test",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 0.28247005,
        "_source" : {
          "title" : "Shall I begin?"
        },
        "highlight" : {
          "title" : [
            "Shall I <red>begin</red>?"
          ]
        }
      },
      {
        "_index" : "aliyun-index-test",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 0.25069216,
        "_source" : {
          "title" : "I start work at nine."
        },
        "highlight" : {
          "title" : [
            "I <red>start</red> work at nine."
          ]
        }
      }
    ]
  }
}

Method 2: Configure inline (not reusable)

This method involves writing synonym rules directly into the index configuration and is best for small dictionaries that require frequent updates.

Step 1: Create an index and define synonyms

Connect to the cluster and define the synonym rules directly in the synonyms array when you create the index.

PUT /my_index
{
 "settings": {
     "analysis": {
         "analyzer": {
             "my_synonyms": {
                 "filter": [
                     "lowercase",
                     "my_synonym_filter"
                 ],
                 "tokenizer": "ik_smart"
             }
         },
         "filter": {
             "my_synonym_filter": {
                 "synonyms": [
                     "begin,start"
                 ],
                 "type": "synonym"
             }
         }
     }
 }
}

This command creates an index named my_index and configures custom text analysis. Here is how it works:
When a text field uses the my_synonyms analyzer, the ik_smart tokenizer breaks the input text into tokens. The lowercase filter then converts all tokens to lowercase. Finally, the my_synonym_filter applies the synonym rules, treating tokens like begin and start as equivalent.

Step 2: Configure the title field

For Elasticsearch versions earlier than 7.0

PUT /my_index/_mapping/doc
{
"properties": {
 "title": {
   "type": "text",
   "analyzer": "my_synonyms"
 }
}
}

For Elasticsearch 7.0 and later

PUT /my_index/_mapping/
{
"properties": {
 "title": {
   "type": "text",
   "analyzer": "my_synonyms"
 }
}
}

Step 3: Verify configuration

GET /my_index/_analyze
{
 "analyzer":"my_synonyms",
 "text":"Shall I begin?"
}

Response:

{
  "tokens" : [
    {
      "token" : "shall",
      "start_offset" : 0,
      "end_offset" : 5,
      "type" : "ENGLISH",
      "position" : 0
    },
    {
      "token" : "i",
      "start_offset" : 6,
      "end_offset" : 7,
      "type" : "ENGLISH",
      "position" : 1
    },
    {
      "token" : "begin",
      "start_offset" : 8,
      "end_offset" : 13,
      "type" : "ENGLISH",
      "position" : 2
    },
    {
      "token" : "start",
      "start_offset" : 8,
      "end_offset" : 13,
      "type" : "SYNONYM",
      "position" : 2
    }
  ]
}

Impact of synonym operations

Different synonym operations have different impacts on a cluster. Understanding these differences can help you choose the right update method for your business requirements.

Operation	Triggers cluster restart	Description
Incremental update (uploading a file with the same name)	No	Uploading a synonym file with the same name as an existing one is an incremental update (hot update) and does not trigger a cluster restart.
Updating the synonym dictionary from the Update Synonym Dictionary (new filename or file deletion)	Yes	Uploading a synonym file with a new filename or deleting an existing file and then saving the changes triggers a rolling restart of the cluster.

Incremental update (hot update)

When you upload a synonym file with the same name as an existing one, the system performs an incremental update (hot update), which does not trigger a cluster restart. The new file overwrites the original one, and any new indexes you create automatically use the updated dictionary.

Note

After an incremental update, existing indexes do not automatically load the new dictionary. To apply the changes to an existing index, you must close and then reopen the index (Close/Open API), or rebuild the index.

Console updates that trigger a cluster restart

The following operations trigger a rolling restart of the cluster:

Uploading a synonym file with a new name and saving the change.
Deleting an existing synonym file and saving the change.

If your business requires avoiding a cluster restart, we recommend using the elasticsearch-analysis-dynamic-synonym plugin to implement dynamic updates.

A rolling restart can have the following effects:

Service jitter: During a rolling restart where nodes are restarted sequentially, you may experience a temporary increase in query latency, even if you have replica shards.
Risk of service interruption: Under extreme conditions, such as high cluster load or indexes without replica shards, a restart may cause some requests to fail or lead to a brief service interruption.
Restart duration: The total time required for the restart and dictionary distribution depends on the cluster size, data volume, and load. The process can take several minutes or longer.

Read and write availability during changes

After you submit a synonym configuration, the instance status changes to taking effect. During this period:

Read and write operations for the instance are not affected and remain available.
The synonym expansion feature is temporarily unavailable, and search queries that rely on the new synonym rules may return incomplete results.
After the changes take effect, the instance status returns to Active, and newly created indexes automatically use the updated synonym dictionary.

FAQ

Troubleshooting Yellow status or stuck changes

If the cluster status changes to Yellow or subsequent changes are blocked after you configure or update synonyms, check for the following common causes:

Improperly formatted synonym file: The file contains uppercase letters, which causes the analyzer to fail during parsing.
OpenStorePlugin error: Abnormal content in the synonym file triggers an OpenStorePlugin error. This prevents shards from being allocated correctly, causes the cluster status to become abnormal, and blocks subsequent changes.

To troubleshoot and resolve this issue, perform the following steps:

Check and correct the synonym file: Ensure that all words in the file are in lowercase. After you make corrections, upload the file again.

Add a lowercase filter: In the analyzer's filter configuration in the index settings, add a lowercase filter to ensure that tokens are automatically converted to lowercase during analysis. Example configuration:

"filter": {
  "my_synonym_filter": {
    "type": "synonym",
    "synonyms_path": "analysis/your-dict-name.txt"
  }
},
"analyzer": {
  "my_synonyms": {
    "filter": ["lowercase", "my_synonym_filter"],
    "tokenizer": "ik_smart"
  }
}

Recover an index that is not operational: If modifying the analyzer filter requires you to close or rebuild an index in a production environment, or if modifying the analyzer is not feasible in a production environment, try to restore the cluster status by forcing shard reallocation:
```
POST /_cluster/reroute?retry_failed=true
```

Risks of the analysis-dynamic-synonym plugin

The open-source analysis-dynamic-synonym plugin allows you to dynamically load synonyms from a remote or local file, which applies new synonym rules without restarting the cluster. However, this plugin has the following known risks:

Concurrency defect: In high-concurrency read and write scenarios, this plugin may cause a deadlock in the Elasticsearch process, leading to 100% CPU usage and service unavailability.
Use case limitations: Use this plugin with caution on a Serverless instance or in production environments with strict stability requirements. Before enabling it, we recommend fully evaluating its stability under your actual query and write loads in a test environment. If synonyms are not updated frequently, we recommend using the incremental update method (uploading a file with the same name) to avoid the potential risks of third-party plugins.