Ingest cloud service logs from the same account, other accounts, and third-party cloud accounts -

After enabling Agentic SOC, you can ingest cloud service logs from the same account, other Alibaba Cloud accounts, and third-party cloud providers. This allows you to centrally monitor and analyze alerts and logs across all your resources. After you ingest the logs, Agentic SOC monitors and analyzes the collected data to identify and construct complete attack chains, generating detailed security events. This improves alert analysis and response efficiency.

Prerequisites

Agentic SOC is enabled. For more information, see What is Agentic SOC?.
Simple Log Service is enabled for the cloud services whose logs you want to ingest. This does not apply to Security Center. For more information, see the official documentation for the relevant cloud service.
Note
You do not need to enable the log analysis feature for Security Center to ingest its logs.

Ingest logs from Alibaba Cloud services

To ingest logs from a cloud service within your current Alibaba Cloud account, you can directly select the cloud service and log type on the Service Integration page.
To configure a unified log ingestion policy for multiple Alibaba Cloud accounts, you must first set up a multi-account structure. Then, log on to the console as a global account administrator, switch to the Global Account View on the Service Integration page, and follow the steps below. For more information about multi-account management, see Multi-account management.

In the left-side navigation pane, choose Agentic SOC > Manage > Integration Settings. In the upper-left corner of the console, select the region where your assets are located: Chinese Mainland or Outside Chinese Mainland.
In the service integration list, find the cloud service that you want to integrate and click Ingestion Settings in the Actions column.
In the service ingestion settings panel, find the log type that you want to ingest and click the number in the Associated Accounts column.
You can also select multiple log types and click the batch ingestion button at the bottom of the list to configure the accounts for ingestion in bulk.
In the ingestion settings panel, select the accounts from which you want to ingest logs.
Note
If you are using an account with individual real-name verification, the Select Account panel only displays your current account. Only a global account administrator using the Global Account View can select from all accounts managed by Agentic SOC.
- If the cloud service, such as Security Center, supports only a product-defined logstore, you only need to select the account. Security Center automatically ingests logs into the defined logstore.
- If the cloud service uses a custom logstore, you must select the account and then select the corresponding logstore from the logstore (Format: regionId.project.logstore) drop-down list or paste the name of the custom logstore. The logstore name must be in the format regionId.project.logstore.
If needed, enable Automatically Associate New Accounts.
You can choose whether to enable Automatically Associate New Accounts. After you enable this feature, threat analysis and response automatically ingests the cloud service logs from newly managed Alibaba Cloud accounts.
Note
Only a global account administrator using the Global Account View can configure the Automatically Associate New Accounts option.

Ingest logs from third-party cloud services

If your business runs on both Alibaba Cloud and other platforms like Huawei Cloud or Tencent Cloud, you can integrate your third-party cloud accounts with Agentic SOC to manage security alerts across all environments. This provides a unified solution for alert monitoring and security operations.

1. Configure a third-party cloud platform account

Huawei Cloud sub-account

Create two custom policies: siemBasePolicy and siemNormalPolicy. For more information, see the Huawei Cloud documentation on creating a custom policy.

Note

When you create a custom policy in Huawei Cloud, you cannot select global-level and project-level cloud services at the same time. Therefore, you must create two separate policies to apply the principle of least privilege during authorization.

siemBasePolicy: Grants permissions for global-level cloud services. The policy content is as follows:

{
    "Version": "1.1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:roles:listRoles",
                "iam:roles:getRole",
                "iam:groups:listGroupsForUser",
                "iam:groups:listGroups",
                "iam:users:getUser",
                "iam:groups:getGroup"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "rms:resources:list",
                "rms:resources:summarize"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "obs:object:GetObject",
                "obs:bucket:GetBucketLocation",
                "obs:bucket:HeadBucket",
                "obs:object:GetObjectVersionAcl",
                "obs:bucket:ListAllMyBuckets",
                "obs:bucket:ListBucket",
                "obs:object:GetObjectVersion",
                "obs:object:GetObjectAcl"
            ]
        }
    ]
}

siemNormalPolicy: Grants permissions for project-level cloud services. The policy content is as follows:

{
    "Version": "1.1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cfw:ipGroup:list",
                "cfw:acl:list",
                "cfw:ipMember:put",
                "cfw:ipMember:create",
                "cfw:ipGroup:create",
                "cfw:instance:get",
                "cfw:ipGroup:put",
                "cfw:ipMember:list",
                "cfw:ipGroup:get",
                "cfw:ipMember:delete"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "waf:whiteBlackIpRule:list",
                "waf:whiteBlackIpRule:put",
                "waf:ipgroup:get",
                "waf:whiteBlackIpRule:get",
                "waf:ipgroup:list",
                "waf:whiteBlackIpRule:create",
                "waf:whiteBlackIpRule:delete"
            ]
        }
    ]
}

Create two user groups, siemUser and readonlyuser, and grant them the required permissions as shown in the table below. For more information, see the Huawei Cloud documentation on creating a user group and granting permissions.

User group	Required permissions
`siemUser`	Custom policy permissions: `siemBasePolicy`, `siemNormalPolicy`
`readonlyuser`	LTS ReadOnlyAccess: Read-only access to Log Tank Service (LTS). OBS OperateAccess: Basic operational permissions for Object Storage Service (OBS), including viewing bucket lists, getting bucket metadata, listing objects in a bucket, querying bucket locations, uploading objects, getting objects, deleting objects, and getting object ACLs. OBS ReadOnlyAccess: Read-only access to Object Storage Service (OBS), limited to viewing bucket lists, getting bucket metadata, listing objects in a bucket, and querying bucket locations. CFW ReadOnlyAccess: Read-only access to Cloud Firewall. WAF ReadOnlyAccess: Read-only access to Web Application Firewall (WAF).

Create an IAM user and add it to the siemUser user group. For more information, see the Huawei Cloud documentation on creating an IAM user.
Create an access key for the IAM user. For more information, see the Huawei Cloud documentation on creating an IAM user access key.

Tencent Cloud sub-account

Create a custom policy named siemPolicy by using policy syntax.

The policy content is as follows:

{
    "statement": [
        {
            "action": [
                "cfw:DescribeAclApiDispatch",
                "cfw:DescribeBorderACLList",
                "cfw:CreateAcRules"
            ],
            "effect": "allow",
            "resource": [
                "*"
            ]
        },
        {
            "action": [
                "waf:DescribeDomains",
                "waf:DescribeIpAccessControl",
                "waf:DeleteIpAccessControl",
                "waf:UpsertIpAccessControl",
                "waf:PostAttackDownloadTask"
            ],
            "effect": "allow",
            "resource": [
                "*"
            ]
        },
        {
            "action": [
                "ckafka:DescribeDatahubGroupOffsets",
                "ckafka:DescribeGroup",
                "ckafka:DescribeGroupInfo",
                "ckafka:DescribeGroupOffsets",
                "ckafka:CreateDatahubGroup",
                "ckafka:ModifyDatahubGroupOffsets",
                "ckafka:ListConsumerGroup"
            ],
            "effect": "allow",
            "resource": [
                "*"
            ]
        },
        {
            "action": [
                "cam:GetUser",
                "cam:CheckSubAccountName",
                "cam:CheckUserPolicyAttachment",
                "cam:GetAccountSummary",
                "cam:GetPolicy",
                "cam:GetPolicyVersion",
                "cam:ListAllGroupsPolicies",
                "cam:ListAttachedGroupPolicies",
                "cam:ListAttachedRolePolicies",
                "cam:ListAttachedUserAllPolicies",
                "cam:ListAttachedUserPolicies",
                "cam:ListGroupsPolicies",
                "cam:ListPolicies",
                "cam:ListUsers"
            ],
            "effect": "allow",
            "resource": [
                "*"
            ]
        }
    ],
    "version": "2.0"
}

Create a sub-account. For more information, see the Tencent Cloud documentation on creating a sub-account.
Attach the siemPolicy policy to the sub-account you created. For more information, see the Tencent Cloud documentation on authorization management.
Create an access key for the sub-account. For more information, see the Tencent Cloud documentation on managing sub-account access keys.

2. Forward logs to a specified cloud service

To enable threat analysis and response, you must forward logs from your cloud services to a designated storage or messaging service provided by the cloud vendor, such as Object Storage Service (OBS) or Ckafka. Agentic SOC can then efficiently retrieve and analyze these logs from the source. Logs must be routed to the appropriate storage based on their type. Follow the guidelines below.

Cloud provider	Log type	Destination service	Configuration instructions	Data collection latency
Huawei Cloud	Cloud Firewall alert logs Web Application Firewall (WAF) alert logs	OBS	Forward logs stored in Log Tank Service (LTS) to OBS. For details, see Forwarding Logs to OBS. Configure the key parameters as follows: Custom transfer path: Enable this option and configure the path with minute-level granularity. Example: `/LogTanks/cn-north-4/CFW/lts-topic-cfw-0001//%Y/%m/%d/%H/%M`. Transfer period: Set to 2 minutes. Important Configure these parameters exactly as described. Agentic SOC collects data from OBS based on the file directory structure. A mismatch between the collection frequency and the directory structure may result in duplicate data collection. Data in encrypted buckets cannot be ingested. Do not forward logs to an encrypted bucket.	Agentic SOC collects data from OBS offline, which introduces some latency. The system is designed to collect data that is three collection cycles older than the current time. For example, if the collection cycle is 2 minutes and a task starts at 17:58 on September 10, 2024, the system retrieves data from the /2024/09/10/17/52 directory. This data is from 6 minutes (3 cycles) prior. This design ensures data integrity by allowing sufficient time for write operations to complete. This prevents incomplete data collection, especially in high-volume scenarios.
Tencent Cloud	Cloud Firewall alert logs (only intrusion prevention logs are supported)	Ckafka	Ship intrusion prevention logs to a Ckafka topic. For more information, see Log Shipping.	Real-time collection with no latency.
Tencent Cloud	Web Application Firewall (WAF) alert logs	None	Agentic SOC collects these logs by calling the Web Application Firewall (WAF) API every 10 minutes. No forwarding configuration is required.	Data collection has a latency of 10 minutes or more.

3. Add a Third-Party Cloud Account AK to Threat Analysis and Response

Add the access key of your third-party cloud account to Agentic SOC. This allows Agentic SOC to retrieve alert logs from your third-party cloud assets.

Log on to the Security Center console.
Authorize access to the third-party cloud account.
Security Center uses the access key of the third-party account to obtain read permissions and synchronize asset information from the third-party cloud.
1. In the left-side navigation pane, choose Agentic SOC > Manage > Integration Settings. In the upper-left corner of the console, select the region where your assets are located: Chinese Mainland or Outside Chinese Mainland.
2. In the Multi-cloud Service Access section, hover over the icon for the third-party cloud provider you want to integrate and click Grant Permission.
3. In the Edit Multi-cloud Configuration panel, select Manual Configuration. In the Permission Description section, select Agentic SOC, and then click Next.
4. On the Submit AccessKey Pair page, enter the sub-account's access key information and click Next.
5. On the Policy Configuration page, select a period for the AK Service Status Check and click OK.
Bind the third-party cloud account.
1. In the left-side navigation pane, choose Agentic SOC > Manage > Integration Settings. In the upper-left corner of the console, select the region where your assets are located: Chinese Mainland or Outside Chinese Mainland.
2. In the Multi-cloud Service Access section, hover over the icon for the third-party cloud provider you want to bind and click Add Account.
3. In the Add Account panel, click Add.
4. In the Account Association Settings panel, enter the main account name and ID of the third-party cloud provider, select the authorized sub-account's access key, and click Associate Account and Associate Data Source.
5. In the Data Source Settings panel, configure the data sources for the cloud services you want to integrate.
  - Huawei Cloud: A single data source can ingest data from one OBS bucket. If you need to ingest data from multiple OBS buckets, you must create multiple data sources. Otherwise, a single data source is sufficient.
    1. In the Huawei Cloud panel, fill in the Access Method, Data Source Name, Region, and Bucket Name. Click Save Data Source.
    2. Click Add Log Type, select the Log Type to ingest, specify the OBS File Path, and click Save Log Type.
      The custom transfer path in the OBS File Path must be configured with minute-level granularity. Example: /LogTanks/cn-north-4/CFW/lts-topic-cfw-0001//%Y/%m/%d/%H/%M.
      After saving a log type, if both Cloud Firewall and Web Application Firewall (WAF) logs are stored in the same OBS bucket, you must click Add Log Type again to create a configuration for the other log type.
  - Tencent Cloud: Since Cloud Firewall and Web Application Firewall (WAF) alert logs use different ingestion methods, you must create a separate data source for each log type if you want to ingest both. The following steps show an example for Cloud Firewall alert logs. To ingest Web Application Firewall (WAF) logs, follow the instructions in the console.
    1. In the Data Source Settings - Tencent Cloud panel, fill in the Access Method, Data Source Name, Internet URL, Username, and Password. Click Save Data Source.
    2. Click Add Log Type, fill in the Log Topic and Consumer Group Name, select a Log Type, and click Save Log Type.

4. Ingesting logs from third-party cloud services

In the left-side navigation pane, choose Agentic SOC > Manage > Integration Settings. In the upper-left corner of the console, select the region where your assets are located: Chinese Mainland or Outside Chinese Mainland.
In the service integration list, find the third-party cloud service you want to integrate and click Ingestion Settings in the Actions column.
In the ingestion settings panel, find the log type you want to ingest and click the number in the Associated Accounts column.
In the ingestion settings dialog box, select the account to ingest from and click OK.
If needed, enable Automatically Associate New Accounts.
You can choose whether to enable Automatically Associate New Accounts. After you enable this feature, threat analysis and response automatically integrates the cloud service logs for new third-party cloud accounts.
Note
Only a global account administrator using the Global Account View can configure the Automatically Associate New Accounts option.

:Ingest cloud service logs