Alibaba Cloud Logstash is a fully managed, open-source-compatible data pipeline that collects, transforms, and routes data from multiple sources to your preferred destination — without manual server setup or infrastructure management. It is 100% compatible with open-source Logstash, so existing pipeline configurations and plugins work without modification.
How it works
Logstash processes data in three stages:
Collect: Input plugins pull data as a continuous stream from sources such as logs, metrics, web applications, and data stores. Commonly used inputs include
file(reads from the filesystem),beats(receives events from Beats agents), andsyslog(listens for syslog events).Transform: Filter plugins parse each event, identify named fields, build structure, and convert events to a common format. For example,
grokparses unstructured log text into queryable fields, andmutaterenames, replaces, or modifies event fields.Route: Output plugins send processed data to one or more destinations. Send data to Elasticsearch for search and analytics, to OSS for archival, or to Simple Log Service (SLS) for log management.
Key concepts
| Term | Description |
|---|---|
| Pipeline | A running Logstash configuration that connects inputs, filters, and outputs. Manage pipelines centrally using configuration files. |
| Plugin | A modular component that implements an input, filter, or output stage. Alibaba Cloud Logstash provides all official plugins through its plugin center, plus Alibaba Cloud-specific plugins such as logstash-input-sls, logstash-input-oss, and logstash-output-oss. |
| Event | A single unit of data flowing through the pipeline, such as a log line or a metric data point. |
Why use Alibaba Cloud Logstash
As a managed service, Alibaba Cloud Logstash handles the infrastructure so you can focus on your pipeline logic:
Deploy in minutes — spin up an instance without manual server setup or O&M.
Scale on demand — adjust capacity as data volumes grow.
Stay secure — deploy instances in a logically isolated virtual private cloud (VPC). Connect to external endpoints through a NAT Gateway.
Extend with plugins — access all official input, filter, and output plugins, plus cloud-native plugins for SLS and OSS, from the built-in plugin center.
Manage custom files — upload and manage custom extension files directly from the console.
Debug pipelines visually — view pipeline output and use the pipeline configuration debug feature in the console.
Alibaba Cloud Logstash provides a managed service based on open-source Logstash, and its features are consistent with those of the open-source version. If you encounter any service-related issues, you can contact us by submitting a ticket.
We are not responsible for business losses caused by bugs in Logstash or its third-party components, or by improper configuration and use. If you encounter such issues, consult the official Logstash documentation or the community.
Use cases
Alibaba Cloud Logstash is widely used as the data ingestion layer in the ELK stack (Elasticsearch, Logstash, and Kibana):
Real-time log processing — collect logs from servers, containers, or applications and route them to Elasticsearch for indexing.
Full-text search pipelines — transform raw data into a structured format that Elasticsearch can search and rank.
Data analytics — aggregate metrics and events, enrich them with filter plugins, and send results to your analytics destination.
Cloud data migration — move data between Alibaba Cloud services using dedicated plugins for OSS and SLS.
Within this ecosystem, Elasticsearch acts as a real-time distributed search and analytics engine, Logstash provides data collection, transformation, optimization, and output capabilities, and Kibana offers a powerful visualization interface.
What's next
| Task | Reference |
|---|---|
| Create your first Logstash instance | Quick start |
| Create an Alibaba Cloud Logstash instance | Create an Alibaba Cloud Logstash instance |
| Configure input, filter, and output plugins | Plugin configuration |
| Manage pipelines using configuration files | Manage pipelines using configuration files |
| Debug pipeline configurations | Use the Logstash pipeline configuration debug feature |
| Upload and manage custom extension files | Configure extension files |
| Configure Internet access through a NAT Gateway | Configure Internet data transmission with a NAT Gateway |
| Upgrade a cluster | Upgrade a cluster |