Agentic SOC integrate product logs, logs - Security Center

How it works

Log integration in Agentic SOC runs as a pipeline: logs are read from data sources, parsed by parsing rules into unified fields, and then consumed, stored, and queried according to the log ingestion mode. An access policy binds these components together and drives automatic log ingestion. Four components work together in this pipeline:

Data Source: The storage location for raw logs — typically a Project and Logstore in Simple Log Service (SLS).

Note
Agentic SOC supports three data source types: Agentic SOC Dedicated Collection Channel, User Log Service, and Security Center Log Service. For the log ingestion modes supported by each data source type, see Log ingestion mode comparison.
Standardized Rule:
- A set of SPL-based parsing instructions that extract key fields (such as source IP and destination port) from raw logs and map them to a log schema.
  
  Important
  You can ingest extended fields with the Keep Original setting. Ingesting extended fields increases ingestion traffic volume and results in higher costs.
- Each Standardized Rule is associated with a Dataset — a predefined model of standardized fields that determines what fields are available for log analysis. See Datasets.
Standardization Method: Determines how logs are consumed, stored, and queried after they reach Agentic SOC. Three log ingestion modes are supported. For more information, see Log ingestion mode comparison.
Access Policy: The configuration that binds a Data Source, a Standardization Method, and a Standardized Rule into one integration. It specifies how logs are read, parsed, and processed, and it drives automatic log ingestion.

Integrate Alibaba Cloud products

This section applies to Alibaba Cloud products that deliver logs to your SLS Logstores, such as Web Application Firewall (WAF), Cloud Firewall (CFW), and ActionTrail.

Integration flowchart

Parse and Test

Step 1: Enable product log delivery

Enable the log audit feature for the target product to deliver its logs to SLS.

Important

Alert logs from Alibaba Cloud security products in the central Logstore are automatically delivered to Agentic SOC — you do not need to enable the product's log service for these logs. Examples include WAF alert logs and CFW alert logs.

Central Logstore logs are automatically delivered by the Alibaba Cloud security product business center and require no manual setup. The following table lists the currently supported center-side logs and their contents.

Log name	Log contents
CFW alert logs	Records intrusion alert events detected by the Cloud Firewall intrusion prevention system (IPS) engine. These logs correspond to the alert information shown in the Cloud Firewall console.
CFW VPC alert logs	Records alert events generated when Cloud Firewall performs IPS protection detection on inter-VPC traffic. These logs correspond to the VPC boundary protection alerts shown in the Cloud Firewall console.

Go to the cloud product console (such as WAF or CFW) and find Log Management or Simple Log Service. For reference links, see References for integrating Alibaba Cloud product logs into SLS.
Follow the product documentation to enable log delivery. This step typically creates a Project and one or more Logstores in SLS automatically.

Note
Log on to the Simple Log Service console to view the Project and Logstore created by the product. For example, WAF logs use a project named wafnew-logstore.

Step 2: Enable built-in access policies

Enable manually

Configure the data source. Agentic SOC provides preconfigured data sources for major Alibaba Cloud products. Verify that the configurations match your Logstore information.
1. Find the cloud product.
  1. Log on to the Security Center console > Agentic SOC > Management > Integration Settings. In the upper-left corner, select the region where your assets are located: Chinese Mainland or Outside Chinese Mainland.
  2. On the Service Integration tab, set Access Settings to Alibaba Cloud, select the product (such as Web Application Firewall), and click Access Settings in the Actions column.
2. Check the data source status.
  1. Click the name of the data source associated with the product. The page redirects to the Data Source tab.
  2. Check the connection status:
    - Normal: Agentic SOC can access the Logstore. Proceed to the next step.
    - Abnormal: The configuration is incorrect. Click Edit in the Actions column and verify:
      - Instance information: Confirm the Region, Project, and Logstore match what was created in Step 1.
      - Source log service: Confirm the SLS Logstore is active — the Project is not disabled and new logs are being written.
    - Not connected: The data source is not configured. Click Edit in the Actions column, then click Create Instance and select the Region ID, Project, and Logstore created in Step 1.
Enable the built-in access policy.
1. Return to the Service Integration tab and go to Access Settings for the product.
2. Configure the access policy (optional). For built-in policies, you configure the policy on the Access Binding page. Select a log ingestion mode: Ingest and store (native), Ingest and store (standardized), or Ingest without storage (standardized). This flow does not use the custom access policy panel or its optional Access Test step. For details, see Log ingestion mode comparison.
  Important
  - If the data source type is Security Center Log Service or the log type is an alert log, only Ingest and store modes are supported.
  - If the dataset (StoreView) associated with Standardization Category or Structure already has five access policies that use Ingest without storage (standardized), select an Ingest and store mode for the current policy. Otherwise, the policy cannot be enabled.
3. Access test (optional).
  
  Run a test to verify log delivery and parsing. You can also skip the test and complete the configuration first.
  
  Warning
  The Logstore for the data source must contain data. The test cannot run on an empty Logstore.
  1. Log sample selection: The system automatically fetches log samples from the last 7 days by default. You can also manually select a time range.
  2. SPL syntax: The syntax from the current policy's parsing rule is synchronized by default. To modify the syntax, go to the Parsing rules tab and edit the rule.
  3. After obtaining log samples, click Parse and Test.
  4. After the test completes, click Complete. You can complete the configuration even if the test has not been run or has not passed.
    
    Note
    If the test fails, see What do I do if the access test fails?.
4. Enable the policy.
  
  Turn on the switch in the Enabling Status column.
  
  Important
  If you enabled the recommended access policy when purchasing Agentic SOC, built-in policies for Security Center, WAF, Cloud Firewall, and ActionTrail are already enabled. See Enable access policies for subscription and Enable log access policies for pay-as-you-go.

Enable in batches (recommended)

Batch integration discovers data sources and enables access policies across multiple products at once.

Access the Security Center console - Agentic SOC - Management -Integration Settings. In the upper-left corner of the page, select the region of your assets: Chinese Mainland or Outside Chinese Mainland.
Configure batch integration.

Click Access Settings and complete the configuration:
- Access Settings:
  - Increment Access: Keeps all enabled access policies and adds only the data sources and policies from the current configuration.
  - Full Access: Overwrites and replaces all existing access settings. Policies not selected in the current configuration are disabled.
    
    Warning
    Full Access is an overwrite operation. It may disable running policies and interrupt data integration. Proceed with caution.
- Accessible Account: Select the current account and its member accounts.
- Alibaba Cloud Services: Select the cloud products and their corresponding data sources.
  
  Note
  To quickly enable data analytics, click Use Recommended Policy. The system selects the data sources with the most analytical value for each product based on best practices.
- Auto-Add New Data Sources: When enabled, new Logstores are automatically included in the data source's collection scope — no manual updates needed.
Confirm and start data integration.

Click OK. The system automatically:
- Integrates all Logstores: Ingests all enabled Logstores associated with the selected products and accounts in SLS.
- Activates access policies: Enables the access policies that correspond to the selected data sources.
Important
New access policies may take some time to deploy and initialize. Wait for the process to complete before verifying results.
Confirm the enabled status.
1. Return to the access list, select the product, and click Access Settings in the Actions column.
  
  Note
  For member accounts, go to Multi-account Access Settings, then click Multi-account Access in the Actions column for the target data source.
2. Check the access state in the access policy list. If the state is abnormal, check the data source status. See Check the data source status.

Step 3: Add a custom access policy (optional)

Some products support adding custom access policies. The configuration follows a three-step flow: Access Binding → Access Test → Complete.

Note

Access tests verify log delivery and parsing results. You can skip the test and complete the configuration first.

In the access policy list, click Create Access Policy.

Note
If Create Access Policy is not displayed, the product does not support custom access policies.

Step 1: Access Binding — Bind a data source and a parsing rule.

Select a data source

Field	Interaction	Description
Data source name	Drop-down list	Options are filtered by the current product context.
Data source attributes	Tag display	Shows the data source type and connection health status.
Ingestion account	Read-only	Displays the account, region, Project, and Logstore.
Ingestion instances	Table	Shows the region, Project, Logstore, and status.

Note

When you switch the data source, the data source attributes, ingestion account, instance table, and sample logs are refreshed. The access test status resets to Not run.
If no logs are detected for the data source, a message is displayed: No available logs detected for the current data source. Configure data import first. You can continue the setup after logs arrive in the Logstore. This message does not block you from proceeding to the next step.

Select a log ingestion mode

Choose from the following log ingestion modes:

Log ingestion mode	Tag	Description
Ingest and store (native)	Recommended	Preserves all original fields for forensics and troubleshooting. Cross-source fields are not unified, which increases the cost of reusing detection rules.
Ingest and store (standardized)	Write-time standardization	Unifies field semantics across sources. Detection rules and reports are reusable. Field mapping and transformation require additional governance effort.
Ingest without storage (standardized)	Read-time standardization	Unifies security semantics and controls storage costs. Depends on external pipelines. Query performance and alert closure are limited.

Important

Ingest without storage (native) is not supported in this release.

Important

Multi-account access is not supported for Ingest without storage (standardized). For details, see Log ingestion mode comparison.

Important

When you switch the log ingestion mode, the parsing rule drop-down list is refiltered and the access test status resets to Not run.

Bind a parsing rule

The parsing rule drop-down list is located in the Log ingestion mode section. Available rules are filtered by vendor, product, log ingestion mode, log format (native or standardized), allowed parsing rule IDs for the product package, and SPL scenario compatibility.

If matching rules are available, the first rule is selected by default.
If no rules match, the drop-down list displays No parsing rules available and is disabled. The Next button is also disabled.

After you select a parsing rule, the following read-only attributes are displayed:

Attribute	Description
Vendor/Product	The vendor and product that the rule belongs to.
Source	Custom or Predefined.
Security domain / Log schema	The security domain and log schema, displayed together.
Schema type	Standardized schema or Native schema.
SPL scenario	The SPL syntax scenario for the parsing rule.
Extended field ingestion	Displayed only when the schema type is Standardized schema.

Step 2: Access Test — Verify log delivery and parsing results.

Access tests help confirm that logs are arriving and that the parsing rule can process sample logs. Tests are not required to complete the configuration.

Validation checks:

Check	Description
Log arrival	Checks whether the data source has detected logs.
Parsing validation	Checks whether the parsing rule can produce parsed output from sample logs.

Click Run Validation to execute the checks. Before the checks are executed, the parsing result fields display -.

Passed: The data source has logs and a parsing rule is selected.
Failed: The data source has no logs or no parsing rule is selected.

Note

You can proceed to the Complete step even if the test has not been run or has not passed.

Sample parsing result fields:

Standardized logs: event_time, src_ip, security_domain, log_schema, rule_id
Native logs: __time__, logstore, raw_format, schema, rule_id

Step 3: Complete — Review the configuration summary.

The summary page displays the access policy configuration and test status.

Test status messages:
- Passed: Access test passed.
- Failed: Access test failed. You can continue troubleshooting later.
- Not run: Access test not run. You can run it later.
Summary fields: Data source, data source type, log ingestion mode, parsing rule (displays Not matched if none is selected), vendor/product, security domain, log schema, SPL scenario, and test status.

Click Done to save the access policy and return to the access policy list. The policy is saved in the disabled state. To start log ingestion, turn on the switch in the Enabled status column.

After the policy is enabled, you can:
- View the dashboard.
- View detection rules.
Important
Only data sources from the current account are supported. Data sources from member accounts cannot be used for custom access policies.

Integrate third-party logs

You can integrate logs from third-party clouds — Fortinet, Chaitin, Microsoft, Sangfor, Tencent Cloud, Huawei Cloud, Hillstone Networks, Knownsec, and Microsoft Cloud — as well as logs from custom product applications.

Integrate logs from third-party cloud products

Third-party cloud products with logs not in SLS

For Tencent Cloud and Huawei Cloud WAF and CFW products, Agentic SOC provides a built-in data import feature.

Supported products:

Cloud provider	Product	Log type
Tencent Cloud	Web Application Firewall (WAF)	Attack log, access log
Tencent Cloud	Cloud Firewall (CFW)	Alert log
HUAWEI CLOUD	Web Application Firewall (WAF)	Attack log, access log
HUAWEI CLOUD	Cloud Firewall (CFW)	Alert log

Integration flowchart

Parse and Test

Procedure

Create a data source. Create a dedicated Agentic SOC data source to receive the third-party cloud log data. Skip this step if you already have one.
1. Go to Security Center console > Agentic SOC > Management > Integration Settings. In the upper-left corner, select the region for the assets you want to protect: Chinese Mainland or Outside Chinese Mainland.
2. On the Data Source tab, create a data source. See Create a data source: Logs not in SLS.
  - Source Data Source Type: Select User Log Service or Agentic SOC Dedicated Collection Channel. See Data source type comparison.
  - Add Instances: We recommend that you create a new Logstore to isolate the data.
Import data. Follow Import Huawei Cloud log data or Import Tencent Cloud log data to import the third-party cloud data into the data source created in Step 1.
Configure an access policy.
1. On the Service Integration tab, set Access Settings to the third-party vendor (such as Huawei Cloud).
2. Click Access Settings for the target product, then click Create Access Policy.
3. Configure the data source:
  - Data Source Name: Select the data source created in Step 1.
  - Parsing Rule: Select the built-in parsing rules for Huawei Cloud or Tencent Cloud.
    
    Note
    You can also create custom parsing rules.
  - Log Ingestion Mode: Only Ingest and store modes are supported. Ingest without storage is not available for this scenario.
4. On the Log Parsing page, click Parse and Test.
  
  Warning
  The Logstore corresponding to the data source must contain data. Otherwise, the test cannot proceed.
5. Run the access test if needed. After you click Complete, enable the policy.

Third-party logs already in SLS

If logs from products such as Fortinet, Chaitin, or Sangfor are already collected into an SLS Logstore (via Logtail or Syslog-ng), follow this flow. For data collection methods, see Data collection overview.

Parse and Test

Configure the data source.
1. Find the integration product.
  1. Go to Security Center console > Agentic SOC > Management > Integration Settings. In the upper-left corner, select the region where the assets you want to protect are located: Chinese Mainland or Outside Chinese Mainland.
  2. On the Service Integration tab, set Access Settings to the third-party vendor (such as Huawei Cloud).
2. Update the data source.
  1. Click the name of the data source associated with the product. You are redirected to the Data Source tab. Click Edit in the Actions column.
  2. Click Create Instance and select the Region ID, Project, and Logstore where the third-party logs are stored.
    
    Note
    Log on to the Simple Log Service console to find the Logstore information.
Enable the built-in access policy.
1. Return to the Service Integration tab and go to Access Settings for the product.
2. Click Edit in the Actions column for the built-in access policy. On the Access Test page, click Parse and Test if you want to verify log delivery and parsing. See Access test.
3. Click Complete to save the configuration, then enable the policy.

Integrate logs from custom product applications

Custom product applications

Use this flow to integrate logs from your own applications or products not covered by built-in integrations.

Before you begin, confirm the following:

Whether your logs are already collected in SLS. If not, decide whether to use Agentic SOC Dedicated Collection Channel (Agentic SOC creates and manages the Logstore) or User Log Service (you create the Logstore yourself).
You will need to write a custom SPL-based parsing rule for your log format.

Integration flowchart

Parse and Test

Add a product.
1. Access the Security Center console - Agentic SOC - Management -Integration Settings. In the upper-left corner of the page, select the region of your assets: Chinese Mainland or Outside Chinese Mainland.
2. On the Service Integration tab, in the Multi-cloud Service Access area, click Add Service and enter the Service Provider and Service Name.
Add a data source. On the Data Source tab, click Add Data Source. See Data source.
- If logs are already in SLS: Set Source Data Source Type to User Log Service and select the corresponding Logstore.
  
  Note
  Log on to the Simple Log Service console to find the Logstore information.
- If logs are not yet in SLS:
  - Agentic SOC Dedicated Collection Channel: Agentic SOC creates a dedicated Project (aliyun-cloudsiem-channel-AlibabaCloudUID-cn-RegionID) and Logstore in SLS automatically.
  - User Log Service: Go to the Simple Log Service console and create the Logstore first.
Collect the product logs and send them to the corresponding SLS Logstore. For data collection methods, refer to Data Collection Overview or contact the SLS helpdesk.
Add a parsing rule.

On the Parsing Rules tab, click Create Custom Rule. See Create a custom parsing rule.

Set the vendor and product to match what you added in Step 1.
Add and enable an access policy.
1. Return to the Service Integration tab, find the product added in Step 1, and click Access Settings in the Actions column.
2. On the Ingestion Settings page, click Create Access Policy and configure:
  - Data Source: Select the data source configured in Step 2.
  - Parsing Rule: Select the rule configured in Step 3.
  - Log Ingestion Mode: Select Ingest and store (native), Ingest and store (standardized), or Ingest without storage (standardized).
    
    Important
    If the data source type is Agentic SOC Dedicated Collection Channel or the log type is an alert log, only Ingest and store modes are supported.
3. On the Access Test page, click Parse and Test if you want to verify log delivery and parsing. See Access test.
  
  Warning
  The data source's Logstore must contain data. The test cannot run on an empty Logstore.
4. Click Complete to save the configuration, then enable the policy.

SPL scenario compatibility

Use the following table to look up which SPL scenarios are available under a given log ingestion mode. Not all SPL scenarios are compatible with every log ingestion mode.

Important

The SPL scenario enumeration is subject to change. Confirm the latest supported scenarios with your product team.

SPL scenario	Supported log ingestion modes	Notes
StoreView predefined SPL	Ingest without storage (standardized), Ingest and store (standardized)	StoreView predefined SPL is a restricted subset of real-time consumption SPL.
Real-time consumption SPL	Ingest and store (standardized)	Guaranteed only for write-time standardization.
Scan Query SPL	To be confirmed	Pending confirmation of the SPL scenario enumeration.

Access policy list management

The access policy list displays key information for managing access policies under each product.

Column	Description
Data source	Displays the data source name with a type tag (Center-side, User-side, or Data collection channel).
Parsing rule	The name of the bound parsing rule.
Log ingestion mode	The log ingestion mode for this policy.
Associated security capabilities	Shows the number of security capabilities linked to this policy. Click the number to view details.
Last modified	The time when the policy was last updated.
Log activity time	The time when logs were last detected for this policy. Displays `--` if no logs are available.
Access status	Displays the access status, such as Connected, Abnormal, or Pending verification.
Enabled status	A toggle switch to enable or disable the policy.
Actions	Details, Preview, and Edit.

Edit constraints

Important

Active access policies cannot be edited directly. The Edit action is disabled for enabled policies, and a tooltip displays Disable the access policy before editing.

To edit an access policy, disable it first. The edit flow reuses the same three-step panel (Access Binding → Access Test → Complete) and pre-fills the data source, log ingestion mode, and parsing rule from the existing policy.

Note

The Access status, Enabled status, and Actions columns are pinned to the right side of the table, so they remain visible when scrolling horizontally.

Analyze integrated data

After integrating product logs, configure threat detection rules to analyze logs, generate alerts, and respond to cloud security risks. See Configure threat detection rules.

Limitations

Ingest without storage (standardized) policy limit: A dataset (associated with a parsing rule) supports a maximum of five access policies that use Ingest without storage (standardized).
Multi-account integration:
- Logs from member accounts can only be ingested using Ingest and store modes. Ingest without storage is not supported. See member accounts.
- Custom access policies cannot be added for data sources from member accounts.
Log ingestion mode restrictions: If the data source type is Agentic SOC Dedicated Collection Channel or the log type is an alert log, only Ingest and store modes are supported.

Billing

Log integration fees depend on the data source type.

Note

See billing details for Agentic SOC subscription, Agentic SOC pay-as-you-go, and SLS billing.

Data source type	Agentic SOC billing	SLS billing	Notes
Agentic SOC Dedicated Collection Channel	Log ingestion fees + log storage and write fees (consumes Log Ingestion Traffic)	Internet traffic fees (not storage or writes)	Agentic SOC creates and manages SLS resources, so Logstore storage and write fees are billed by Agentic SOC.
User Log Service	Log ingestion (consumes Log Ingestion Traffic)	All costs: storage, writes, and data transfer	SLS manages all log resources and bills all associated fees.
Security Center Log Service	Consumes Log Ingestion Traffic	None	Internal Security Center logs. No SLS fees apply.

References

Log ingestion mode comparison

Dimension	Ingest and store (native)	Ingest and store (standardized)	Ingest without storage (standardized)
How it works	Agentic SOC consumes logs in real time and stores them in their original format.	Write-time standardization: Agentic SOC consumes logs from the data source, parses them, and stores them in the Agentic SOC log space.	Read-time standardization: Reads logs directly from the data source without storing replicas.
Storage format	Native schema (Logstore)	Standardized schema (dataset / StoreView)	No storage. Query-time parsing.
Core advantages	Preserves all original fields. Ideal for forensics and troubleshooting.	Unified field semantics. Detection rules and reports are reusable.	Controls storage costs while maintaining unified security semantics.
Limitations	Cross-source fields are not unified. Detection rule reuse is more costly.	Field mapping and transformation require governance effort.	Depends on external pipelines. Query performance and alert closure are limited.
Log storage and fees	Stored in Logstore. Storage and write fees are billed by the service that manages the Logstore. See Billing.	If you purchase Log Storage Capacity, parsed logs are automatically delivered to Log Management. This consumes your Agentic SOC Log Storage Capacity and cannot be disabled.	No replicas stored. Raw log storage is billed by the source (SLS).
Multi-account access	Supported	Supported	Not supported

Data source type comparison

Dimension	User Log Service	Agentic SOC Dedicated Collection Channel	Security Center Log Service
Log storage and management	You manage your own SLS projects and Logstores.	Agentic SOC automatically creates and manages dedicated SLS projects and Logstores.	Security Center internal logs; not stored in your SLS.
Supported SPL scenarios	Real-time Consumption, Scan Query (excluding alert logs)	Real-time Consumption	Real-time Consumption
Use cases	Ingest logs delivered to SLS from Alibaba Cloud products (WAF, CFW, etc.) or collected to SLS from third-party products using Logtail.	Collect logs from third-party clouds (Tencent Cloud, Huawei Cloud) not already in SLS; collect custom application logs without an existing delivery method.	Analyze alert logs automatically delivered by the Alibaba Cloud security product business center (such as Cloud Firewall IPS protection alerts and WAF alerts). No manual enablement of the product's log service is required.

Note

In this table, Real-time Consumption and Scan Query refer to SPL syntax scenarios, not to the current log ingestion modes. For the current modes, see Log ingestion mode comparison.

References for integrating Alibaba Cloud product logs into SLS

Web Application Firewall: WAF 3.0 logs
Cloud Firewall: Cloud Firewall logs
Bastionhost: Archive audit logs to Simple Log Service
ActionTrail: Platform operation logs
Anti-DDoS: Get started with full log analysis
CDN: Best practices for delivering CDN real-time logs to SLS
DCDN: Global Accelerator access log
API Gateway: API Gateway access logs
Container Service for Kubernetes: Log management
PolarDB: Log analysis
ApsaraDB for MongoDB: MongoDB logs
RDS: RDS SQL Audit Log, Deliver RDS MySQL logs to Simple Log Service
VPC: VPC flow logs, Elastic IP Address logs
Server Load Balancer: ALB access logs, CLB access logs, Layer-7 access logs for CLB, Layer-4 monitoring metrics for CLB
Object Storage Service: OSS access log
File Storage NAS: Log analysis based on SLS
CloudConfig: Configuration Audit Log

Log Service (SLS) Log Collection Reference

Logstash: Use Logstash to upload logs
OSS: Import OSS data
Log4j: Collect Log4j logs
Alibaba Cloud Log Collection Agent: LoongCollector (formerly Logtail)
Collection Selection Guide: Data collection

FAQ

Why does the data source show "Abnormal" or "Not connected"?
1. The most common cause is that the service-linked role lacks the required permissions. Check that the AliyunServiceRoleForSas service-linked role has at least AliyunLogReadOnlyAccess on the SLS Logstore.
2. If the permissions are correct, go to the data source edit page in Agentic SOC and verify the Region, Project, and Logstore names. Watch for typos and extra spaces.
3. Finally, log on to the SLS console and confirm the Project is not disabled and new data is being written.
Why does the access policy fail to enable?

The policy has most likely hit the limit on Ingest without storage (standardized) policies. Each dataset (StoreView) supports a maximum of five access policies that use this mode. Switch the policy to an Ingest and store mode, and then enable the policy again. See Access policy fails to be enabled.
What do I do if the access test fails or data cannot be parsed?
1. The Logstore is most likely empty. Confirm that the SLS Logstore contains recent data (preferably written within the last hour) — the test cannot run on an empty Logstore.
2. If the Logstore has data but the test still fails, check whether the SPL statement in your parsing rule correctly parses the raw log format. Debug the SPL statement on the query and analysis page in the SLS console.