Security information and event management (SIEM) performs threat detection, security event management, and compliance detection by collecting and analyzing security events from various data sources. SIEM is developed based on security information management (SIM). It collects, analyzes, and reports log data, and security event management (SEM), which performs real-time analysis of logs and event data to provide threat monitoring, event correlation, and event response.
2021 Gartner's Magic Quadrant for SIEM analysis report gives a new definition of the SIEM market.
SIEM technology focuses on event data generated by security devices, network infrastructure, systems, and applications. Data sources are mainly log data, but SIEM technology can also process other forms of data, such as network telemetry data (streams and data packets.) SIEM also provides real-time event analysis, advanced analysis of users and entity behaviors for security monitoring, historical data analysis for large time spans, event investigation and management, and reports (such as compliance requirements.)
Magic Quadrant is a tool for Gartner to evaluate and compare vendors in the industry. It generates Magic Quadrant diagrams and corresponding analysis reports based on the ability to execute and the completeness of vision of each vendor in the industry market. In the diagram, the vertical axis shows the vendor's ability to execute, which is used to evaluate the vendor's ability to turn its vision into a market reality. The horizontal axis represents the vendor's completeness of vision. Vendors are positioned into four quadrants (LEADERS, CHALLENGERS, VISIONARIES, and Niche Players) based on the vendor's ability to execute and the completeness of vision.
The picture above shows the SIEM Magic Quadrant provided by Gartner. Next, we will focus on several important vendors.
Exabeam is a leader in the Magic Quadrant. Exabeam's SIEM solution can be used as SaaS (Exabeam Fusion SIEM) or used for hybrid and joint deployment. It includes Exabeam data lake, advanced analysis, threat detection, entity analysis, case management, and event response. Users can purchase it flexibly based on the modular architecture of the customized deployment. Exabeam's machine learning (ML)-driven user and entity behavior detection can provide users with risk scoring and automatic context enrichment capabilities.
Securonix is in the leading position in the Magic Quadrant. Its SIEM solution includes next-generation SIEM, secure data lake, UEBA, SOAR, NDR, threat intelligence, competitor behavior analysis, and several applications for specific use cases. It provides good support in data privacy control and threat intelligence.
Splunk is a leader in the Magic Quadrant. Splunk's security product portfolio has been rated as an industry-leading technology by Gartner Market Research for many years. Through the platform of "Data-to-Everything," a complete set of solutions integrating SIEM, UEBA, and SOAR is provided.
Elastic is a niche player in the Magic Quadrant. Based on open-source, Elastic has developed basic data collection, analysis, and visualization capabilities through the combination of Logstash, Elasticsearch, and Kibana. As a log aggregator, Logstash can collect and process data from almost any data source.
Elasticsearch is a storage engine for parsing large amounts of data. Kibana is used as a visualization layer for visualized processing and problem analysis. Elastic 7.14 released the first free and open Limitless XDR, which can provide integrated SIEM and Endpoint Security functions in one platform.
Note: Splunk offers out-of-the-box solutions on a platform basis (such as its complete SIEM, UEBA, and SOAR solutions.)
SLS is more of a platform that provides support and secondary customization and development. This article mainly compares the functions of SLS and Splunk at the platform level.
Splunk provides a powerful "Splunk Data-to-Everything Platform" to help users break through the barriers between data and actions. Moreover, it empowers IT, security, Internet of Things, and business operations teams to understand business in real-time and transform data into business results. The platform mainly includes:
This platform provides data support for decision-making related to IT, security, observability, problems, and actions.
SLS is a log middle office product of Alibaba and Ant Group. It is also a basic product that serves tens of thousands of Alibaba Cloud customers. SLS processes dozens of petabytes of Log/Metric/Trace data every day. It provides support for AIOps, big data analysis, operations services, big data security, and other scenarios and enhances observability for engineers.
The core platform capability of SLS is to provide unified storage and computing capabilities related to various monitoring data of observability.
Powerful analysis engine
Based on this platform, it provides rich scenario support in DevOps, ITOps, SecOps, BusinessOps, and other fields.
In the following table, the sentences highlighted in red indicate better performance:
Based on the comparison above, SLS already has relatively comprehensive capabilities in the functional points in building SIEM solutions. It performs well in some aspects, including service availability, price, data processing and analysis capabilities, and alarm management. Next, we will focus on how to use SLS to build a SIEM scheme.
It can be seen from this figure that SLS already has the basic ability to build SIEM. Next, we focus on the key points of using SLS to build SIEM scenarios from the following aspects:
SLS provides a wide range of collection methods to collect data from multiple sources, such as servers and applications, open-source software, Internet of Things devices, mobile terminals, standard protocols, and Alibaba Cloud products.
The main access methods for application logs are:
Client writes directly using SDK: It allows writing logs directly to SLS in applications or dependent frameworks.
Collected by Logtail as an agent: The application is only responsible for sending logs to the local, and log is used as an agent to collect logs to the SLS side.
It also provides docking of standard protocols (such as HTTP/HTTPS and Syslog) and third-party open-source collection software (such as Logstash and Fluentd.) SLS provides 20+ access points worldwide to meet the demands of quick access of service data and provides a global accelerator scheme for the unstable cross-domain collection network.
SLS is also deeply integrated with multiple cloud products of Alibaba Cloud, providing convenient access to cloud product logs. Users can directly collect the audit data of each cloud product to the SLS side for subsequent analysis and processing.
The list of cloud products that support access continues to grow. For the latest data, please refer to:
Data processing is a managed, highly available, and scalable data processing service launched by SLS. It mainly performs real-time ETL processing on structured or unstructured logs. This function currently includes 200+ operators and is widely used in scenarios, such as data regulation, data aggregation, enrichment, distribution, and aggregation. The data desensitization operators provided by data processing can reduce the exposure of sensitive data when users process, ship, and use the data. Therefore, it can mitigate the risk of data leakage. Common desensitization scenarios include desensitization of sensitive information, such as mobile phone numbers, bank card numbers, email addresses, IP addresses, AccessKey pairs, ID numbers, URLs, order numbers, and strings.
In SIEM scenarios, it is often necessary to collect data from multiple data sources since the format may be messy. At the same time, there are also long-term and massive data analysis requirements. Data processing is structured in advance through the Schema on Write method, which can be convenient for subsequent analysis and processing.
Log Audit is an audit service based on the capabilities of the SLS platform. It supports the automatic and centralized collection of cloud product logs on Alibaba Cloud in multi-account scenarios in real-time. It has the following advantages:
SLS uses SQL as the query and analysis framework and integrates PromQL syntax and machine learning functions into the framework.
SLS SQL = Search + SQL92 (Agg, WIndow, GroupBy...) + PromQL + ...
In the following example:
SLS has a large number of AI-based inspection, prediction, clustering, root cause analysis, and other algorithms, which are provided to users in the form of SQL/DSL functions and can be used in manual analysis and automatic inspection alarms.
The visualization function provided by SLS is based on the unified query and analysis engine of SLS, which presents the query and analysis results in the form of charts, and shows the global situation clearly. At the same time, it can also interface with third-party visualization tools.
SLS alert is an all-in-one intelligent management and maintenance alert system provided on the SLS cloud-native observability platform. It provides alert monitoring for logs, time series, and other data. It can also accept tripartite sides alerts. It performs noise reduction, event management, and notification management for alerts. It adds 40+ functional scenarios, fully considering the management and maintenance of alert monitoring needs for research and development, management and maintenance, security, and operational staff.
Users can configure alarm monitoring rules to check, evaluate, and query the source logs and timing storage periodically and trigger alarms or recovery notifications according to the evaluation results of the monitoring orchestration logic.
SLS alert provides hundreds of built-in alert rules, which are available out of the box and are continuously increasing. These rule bases cover best practices for CIS (including account security and database security) and security scenarios. Users can enjoy all-time security by enabling the corresponding rules.
When an alert rule detects an exception, it is necessary to notify the corresponding developers of the threat event as soon as possible. SLS provides various notification channels to facilitate all-around access to threat events.
When developers receive an alert event notification, they need to take a series of event management actions, such as event confirmation, assignment of handlers, and recording of processing actions.
Gartner views threat intelligence as some evidence-based knowledge, including context, mechanism, label, meaning, and proposals that can be implemented. Threat intelligence describes existing or imminent threats or danger to an asset. It can also be used to inform the entities to take a certain response to the threats or danger. Most of the so-called threat intelligence in the industry is in a narrow sense. Its main contents are the fall identification used to identify and detect threats, such as file HASH, IP, domain name, program running path, registry key, and some related attribution tags.
Threat intelligence service is an intelligence security service provided by Alibaba Cloud. It combines threat intelligence data to collect, analyze, classify, and associate threat sources automatically in real-time to evaluate threats in enterprise assets and provide suggestions for improving security. Alibaba Cloud threat intelligence service can display the threat statistics of all online users and enterprises worldwide over the last 30 days. Currently, it supports providing threat intelligence for IP addresses, domain names, and files.
SLS Log Audit is deeply integrated with threat intelligence service. With the global threat intelligence assessment capability provided by the threat intelligence service, it performs threat intelligence detection on a wide range of cloud product logs (ActionTrail, SLB, OSS, and SAS) that access the SLS, identifying potential threats that exist during the use of cloud products effectively. It also supports timely notification of detected exceptions to relevant security personnel in the form of alerts, improving the efficiency and response speed of threat inspection.
The following figure shows how to use the SLS log audit service to detect and respond to threat intelligence of cloud products.
SLS provides a way to interface with third-party SIEM solutions (such as Splunk) to ensure that all regulations, audits, and other related logs on Alibaba Cloud can be imported into the user's security management and maintenance center (SOC).
wjo1212 - January 3, 2019
Alibaba Cloud Storage - June 10, 2021
Alibaba Cloud Community - February 10, 2022
Alibaba Clouder - October 15, 2020
DavidZhang - January 15, 2021
Alibaba Developer - February 3, 2020
An encrypted and secure cloud storage service which stores, processes and accesses massive amounts of data from anywhere in the worldLearn More
Accelerate and secure the development, deployment, and management of containerized applications cost-effectively.Learn More
Plan and optimize your storage budget with flexible storage servicesLearn More
An all-in-one service for log-type dataLearn More
More Posts by Alibaba Cloud Community