Elasticsearch:Use the IK analyzer plugin for Chinese text search

Set IK as the default analyzer

This configuration applies IK tokenization to all text fields in the index:

PUT /my_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "default": {
          "type": "ik_max_word"
        }
      }
    }
  }
}

Configure IK for specific fields

This configuration gives you fine-grained control over which fields use IK tokenization:

PUT /my_index
{
  "mappings": {
    "properties": {
      "title": {
        "type": "text",
        "analyzer": "ik_max_word",
        "search_analyzer": "ik_smart"
      },
      "content": {
        "type": "text",
        "analyzer": "ik_max_word",
        "search_analyzer": "ik_smart"
      },
      "tags": {
        "type": "text",
        "analyzer": "ik_smart"
      }
    }
  }
}

Use IK with the Pinyin plugin

You can combine IK analyzer with the Pinyin analysis plugin to support both Chinese characters and Pinyin input:

PUT /my_index
{
  "settings": {
    "analysis": {
      "analyzer": {
        "ik_pinyin_analyzer": {
          "type": "custom",
          "tokenizer": "ik_max_word",
          "filter": ["my_pinyin", "word_delimiter"]
        }
      },
      "filter": {
        "my_pinyin": {
          "type": "pinyin",
          "keep_first_letter": true,
          "keep_separate_first_letter": false,
          "keep_full_pinyin": true,
          "keep_original": true,
          "limit_first_letter_length": 16,
          "lowercase": true,
          "remove_duplicated_term": true
        }
      }
    }
  }
}

Method 1: Dynamic update (no restart needed)

Use this method to update the main dictionary or stop word list without interrupting service.

When to use:

• Adding new product names, technical terms, or domain vocabulary

• Updating stop words based on search analytics

• Frequent dictionary changes in development or testing

Limitations:

• Only main dictionary (main.dic) and stop word list (stopword.dic) supported

• Cannot add or remove dictionary files—only modify existing file content

• Changing file names or file count triggers a cluster restart

Procedure

1. Log on to the Alibaba Cloud Elasticsearch console.

2. Navigate to your Elasticsearch instance:

   • In the left navigation menu, choose Elasticsearch Clusters.

   • Click the name of your target instance.

3. In the left navigation pane, click Plug-ins.

4. On the Built-in Plug-ins tab, find analysis-ik in the plug-in list.

5. Click Rolling Update in the Actions column.

6. In the Configure IK Dictionaries - Rolling Update panel, click Edit for the target dictionary type, and upload the required dictionary file:

   1. Click Edit for the target dictionary type.

   2. Add your custom terms using one of two methods:

      Option A: Upload a local file

      • Select Upload On-premises File.

      • Select your .dic file (UTF-8 encoded, one word per line).

      • Click Save.

      Option B: Upload from OSS

      • Select Upload OSS File.

      • Enter the OSS bucket name.

      • Enter the complete path to your dictionary file, including file name and extension.

      • Click Save.

      Note

      • To update an existing dictionary, download and modify it, remove the original file, and upload the updated one.

      • The OSS bucket and Elasticsearch instance must be in the same region.

      • File name restrictions: Maximum 30 characters (excluding .dic extension), containing only letters, digits, and underscores.

7. Review your changes and click OK.

   The IK configuration takes effect after the dictionary is loaded on each node. The time it takes for a node to load the dictionary file varies. Wait for about one to two minutes and then verify whether the IK dictionary is updated.

Method 2: Update requiring restart

Use this method for initial dictionary setup, adding new dictionary files, or updating preposition/quantifier dictionaries.

When to use:

• First-time dictionary configuration

• Adding new dictionary files (not just modifying existing files)

• Updating preposition or quantifier dictionaries

• Major dictionary overhauls

Procedure

1. Log on to the Alibaba Cloud Elasticsearch console.

2. Navigate to your Elasticsearch instance:

   • In the left navigation menu, choose Elasticsearch Clusters.

   • Click the name of your target instance.

3. In the left navigation pane, click Plug-ins.

4. On the Built-in Plug-ins tab, find analysis-ik in the plug-in list.

5. Click Standard Update in the Actions column.

6. In the Configure IK Dictionaries - Standard Update panel, click Edit for the target dictionary type, and upload the required dictionary file:

   Option A: Upload a local file

   • Select Upload On-premises File.

   • Select your .dic file (UTF-8 encoded, one word per line).

   • Click Save.

   Option B: Upload from OSS

   • Select Upload OSS File.

   • Enter the OSS bucket name.

   • Enter the complete path to your dictionary file, including file name and extension.

   • Click Save.

   Note

   • You can upload only one .dic file per dictionary type. The uploaded file replaces the existing dictionary.

   • The OSS bucket and Elasticsearch instance must be in the same region.

   • File name restrictions: Maximum 30 characters (excluding .dic extension), containing only letters, digits, and underscores.

7. Review the current configuration to verify your upload.

8. Click OK.

9. The system prompts: "After the update, the cluster will be restarted. Continue?"

10. Click OK to confirm. The cluster begins a rolling restart to apply the configuration.

11. Monitor the restart progress on the cluster details page. The cluster status changes to Initializing during the restart, then returns to Normal when complete.

Test that Chinese text is segmented into words, not individual characters:

GET _analyze
{
  "analyzer": "ik_smart",
  "text": "计算机汉字输入方法"
}

Expected output:

{
  "tokens" : [
    {
      "token" : "计算机",
      "start_offset" : 0,
      "end_offset" : 3,
      "type" : "CN_WORD",
      "position" : 0
    },
    {
      "token" : "汉字输入",
      "start_offset" : 3,
      "end_offset" : 7,
      "type" : "CN_WORD",
      "position" : 1
    },
    {
      "token" : "方法",
      "start_offset" : 7,
      "end_offset" : 9,
      "type" : "CN_WORD",
      "position" : 2
    }
  ]
}

What to look for:

• Chinese text is segmented into meaningful words (计算机, 汉字输入...)

• If you see individual characters (计, 算, 机...), the analyzer is not configured correctly

Diagnostic commands

Use these commands to diagnose IK analyzer issues:

Check plugin installation:

GET _cat/plugins?v

Expected output should include analysis-ik in the plugin list.

Verify analyzer configuration for an index:

GET /my_index/_settings

Check the analysis section for IK analyzer configuration.

Check field mapping:

GET /my_index/_mapping/field/content

Verify the analyzer and search_analyzer settings for the field.

Test analyzer directly:

GET _analyze
{
  "analyzer": "ik_max_word",
  "text": "测试文本"
}

This tests the analyzer without requiring an index.

Choosing between ik_smart and ik_max_word

Dictionary maintenance

• Review search analytics: Identify frequently searched terms that are being tokenized incorrectly, then add them to your custom dictionary.

• Use dynamic updates for frequent changes: If you add new product names or technical terms regularly, use dynamic updates to avoid restart overhead.

• Plan updates requiring restart carefully: Schedule these updates during maintenance windows to minimize user impact.

• Version control dictionary files: Store dictionary files in version control (Git) to track changes and enable rollback if needed.

• Test before production: Always test dictionary changes in a development or staging environment before applying them to production.

Performance considerations

Recommendation:

• For cost-sensitive applications with tight storage budgets, consider using ik_smart for both indexing and searching.

• For quality-critical applications (e-commerce, search engines), use the asymmetric configuration: ik_max_word for indexing and ik_smart for searching.

• For large-scale deployments (10+ million documents), test performance impact before choosing a mode. Consider shard sizing and node capacity.

Handling mixed Chinese-English content

The IK analyzer handles mixed Chinese-English content gracefully:

• Chinese tokens: Analyzed by IK dictionary

• English tokens: Passed through to standard tokenization (word boundary detection)

• Numbers and punctuation: Treated as separators

Example:

GET _analyze
{
  "analyzer": "ik_smart",
  "text": "使用Machine Learning技术"
}

Result: 使用, machine, learning, 技术