You can use various methods to manage your Alibaba Cloud Elasticsearch clusters. This topic provides an overview of the best practices for cluster management to meet your business requirements in various scenarios.
Best practice | References | Description |
Hot and cold data separation and lifecycle management | Time series data increase over time. You can use the index lifecycle management (ILM) feature to periodically roll over the data to new indexes. This ensures high query efficiency and reduces query costs. As indexes age and fewer queries are required, you can migrate the indexes to a less expensive disk and reduce the numbers of primary and replica shards. | |
An Elasticsearch cluster that uses the hot-warm architecture contains hot nodes and warm nodes. This architecture improves the performance and stability of Elasticsearch clusters. When you use an Elasticsearch cluster, you can use the ILM feature to separate hot data from cold data in the cluster. This improves the read and write performance of the cluster, automates the maintenance of hot and cold data, and reduces production costs. | ||
Application of X-Pack advanced features | You can use the cross-cluster replication (CCR) feature to migrate index data between a local Elasticsearch cluster and a remote Elasticsearch cluster. This feature helps implement high availability and disaster recovery for your Elasticsearch cluster. You can also use the feature for cross-region data access from a nearby cluster. | |
When you use an Elasticsearch cluster, you can configure Lightweight Directory Access Protocol (LDAP) authentication for the cluster to allow LDAP users with the required roles to access the cluster. | ||
Use the RBAC mechanism provided by Elasticsearch X-Pack to implement access control | If you want to grant access permissions on items such as clusters, indexes, and fields, you can use the role-based access control (RBAC) mechanism that is provided by the X-Pack plug-in of Elasticsearch. This mechanism allows you to grant permissions to custom roles and assign the roles to users to implement access control. Elasticsearch provides a variety of built-in roles. You can create custom roles based on the built-in roles to meet your business requirements. | |
Elasticsearch allows you to configure Active Directory (AD) user authentication for your Elasticsearch cluster. This way, users in an AD realm that are assigned Elasticsearch roles can be used to access the cluster. | ||
Integrated monitoring | Use Elastic Stack to implement integrated monitoring for containers in Kubernetes | Elastic Stack provides the integrated monitoring feature. This feature allows you to use Kibana to analyze and display the logs, metrics, and application performance monitoring (APM) data of a Container Service for Kubernetes (ACK) cluster in a centralized manner. If you deploy your applications in the pods of an ACK cluster, you can view the logs generated by the pods, event metrics of the hosts and network, and APM data in the Kibana console. This facilitates troubleshooting. |
Data management and visualization | Use Terraform to manage Alibaba Cloud Elasticsearch clusters | Terraform allows you to use code to allocate resources such as physical machines. You can use Terraform to write a configuration file to purchase a cloud server or apply for resources, such as the resources of the Alibaba Cloud Elasticsearch and Object Storage Service (OSS) services. You can use Terraform to manage your Alibaba Cloud Elasticsearch clusters. For example, you can use Terraform to create, update, view, or delete a cluster. |
Curator is an index management tool provided by open source Elasticsearch. This tool allows you to create, delete, and disable indexes. It also allows you to merge index segments. | ||
Time series data increases over time. If you want to store large volumes of data in your Alibaba Cloud Elasticsearch cluster, the storage costs will linearly increase. You can use the rollup mechanism of Elasticsearch to store data at a fraction of the cost. | ||
In addition to Kibana, curl commands, and clients, you can use third-party plug-ins or tools such as Elasticsearch-Head and Cerebro to access an Alibaba Cloud Elasticsearch cluster. | ||
Notification of alerts for clusters | Configure a DingTalk chatbot to receive alert notifications from X-Pack Watcher | X-Pack Watcher is a monitoring and alerting service that is developed for Elasticsearch. If you configure X-Pack Watcher for your Elasticsearch cluster, X-Pack Watcher can trigger actions when specific conditions are met. For example, if the logs index contains errors, X-Pack Watcher triggers the system to send alert notifications by using emails, DingTalk messages, or DingTalk chatbots. X-Pack Watcher is an Elasticsearch-based monitoring and alerting service. |