Time-range queries on large time series indexes can be slow because standard Elasticsearch scans all segments regardless of whether they contain matching data. The pruning feature addresses this by changing how Alibaba Cloud Elasticsearch merges index segments — keeping time-adjacent data together so that queries can skip segments that fall entirely outside the queried range. This reduces query latency by up to 40%.
Run all commands in this topic in the Kibana console. For more information, see Log on to the Kibana console.
Prerequisites
An Alibaba Cloud Elasticsearch V6.7.0 or V7.10.0 cluster is created. The kernel version of the V6.7.0 cluster is 1.2.0 or later. For more information about how to create a cluster, see Create an Alibaba Cloud Elasticsearch cluster.
Supported versions
The pruning feature is available on:
Alibaba Cloud Elasticsearch V6.7.0 with kernel version 1.2.0 or later
Alibaba Cloud Elasticsearch V7.10.0
How the pruning feature works
Standard Elasticsearch merges index segments based on size alone — segments of similar sizes are merged, which means a single segment can span arbitrary time ranges. A time-range query must then scan all segments to find matching data.
With pruning enabled, segment merging considers both size and the time series field. Segments with similar sizes that also contain data from adjacent time periods are merged together. Each resulting segment holds a contiguous time range, so Elasticsearch can skip segments that fall entirely outside the queried time range.
| Standard Elasticsearch | Alibaba Cloud Elasticsearch (pruning enabled) | |
|---|---|---|
| Segment merging policy | Based on size only; segments span arbitrary time ranges | Based on size and the time series field; each segment holds a contiguous time range |
| Query performance | Scans all segments for every query | Skips segments outside the queried time range; improves performance by 40% |
Constraints
Enable pruning at index creation time. Pruning affects how segments are merged during indexing and compaction. If you enable it after the index is created, the expected query performance cannot be achieved.
Do not re-enable pruning after disabling it. After you disable the pruning feature for an index, we recommended that you do not enable the feature again for the index. If you enable it again and the segment merging of the index covers non-time series data, the expected query performance cannot be achieved.
The time series field must be of type DATE or LONG.
Enable the pruning feature
Specify a time series field when creating an index. The following example uses timestamp as the time series field:
PUT index-1/_settings
{
"index" : {
"merge.policy.time_series_field" : "timestamp"
}
}Query time-series data
Use a range filter on the time series field to take advantage of pruning. Elasticsearch skips segments whose entire time range falls outside the filter and searches only the relevant segments.
The following example filters data by the timestamp field and the region field:
POST index-1/_search
{
"query": {
"bool": {
"filter": [
{
"range": {
"timestamp": {
"format": "yyyy-MM-dd HH:mm:ss",
"gte": "2020-06-01 23:00:00",
"lt": "2020-06-06 23:05:00",
"time_zone": "+08:00"
}
}
},
{
"terms": {
"region": [
"sh"
]
}
}
]
}
}
}Disable the pruning feature
Disabling pruning requires closing the index, updating its settings, and reopening it.
After disabling pruning, do not re-enable it on the same index. See Constraints for details.
Close the index.
POST index-1/_closeSet
merge.policy.time_series_fieldtonullto disable pruning.PUT index-1/_settings { "index" : { "merge.policy.time_series_field" : null } }Reopen the index.
POST index-1/_open