Before you can use the Cloud Threat Detection and Response (CTDR) feature, you must integrate cloud service logs to it first. This allows CTDR to analyze and process logs to generate alerts and security incidents.
Prerequisites
You have purchased and activated CTDR.
If you want to access Alibaba Cloud services, you must activate the log service for the corresponding service. For more information, see the official documentation of each Alibaba Cloud service.
NoteTo add logs of Security Center, you do not need to purchase the log analysis feature.
Service description
CTDR provides a set of predefined cloud services with default configurations for data sources, standardization rules, and recommended integration policies for quick integration. These services primarily come from Alibaba Cloud and third-party vendors such as Fortinet, Chaitin, Microsoft, Sangfor, Tencent Cloud, Huawei Cloud, Hillstone Networks, Knownsec, and Microsoft Cloud. For more information, see Supported services and logs.
CTDR also supports adding custom cloud services. For custom services, you need to configure data source, standardization rules, standardization method, and access policy based on your needs.
Add predefined services
For services from vendors such as Alibaba Cloud, Fortinet, Chaitin Tech, Microsoft, Sangfor, Tencent Cloud, Huawei Cloud, Hillstone Networks, Dowsure, and Microsoft Cloud, CTDR has configured default data sources, standardization rules, and access policies. Follow these steps to complete the integration.
Workflow
Step 1: Find the service to add
Log on to the Security Center console. In the upper-left corner of the console, select the region where your assets are located: China or Outside China.
In the left-side navigation pane, choose . Find the service you want to add in the service list.
Step 2: Enable built-in integration policy
On the Service Integration tab, find the service you want to add, and click Access Settings to go to the integration policy list.
CTDR initializes some built-in policies for predefined services, with pre-configured data sources, standardization rules, and standardized access methods.
Verify data source
On the access policy list page, you can view the initialized policies and their corresponding data sources.
Verify the validity of the data source based on the Data Source Name using the following methods:
If you have enabled the recommended policy, the default policy is enabled and will synchronize the data source status. You can check the Status to determine if the data source is abnormal. If it is abnormal, disable the enabling status.
If the access policy can be modified, you can click Check Validity on the edit page to complete the verification.
You can also go to the Integration Center and select the Data Source tab to find the corresponding data source by name. As shown in the following figure,
indicates that the data source is abnormal.
If the data source is abnormal, check the data source configuration.
Alibaba Cloud service
Make sure that the log service for the Alibaba Cloud service has been activated.
If it has been activated, check whether the Project and Logstore are configured correctly.
If they are not correct, go to the Integration Center and select the Data Source tab to modify the Logstore information on the data source edit page. For Data Source Type, select Custom Log Capability.
You can log on to the Simple Log Service console to view the Logstore information of the product.
Non-Alibaba Cloud services:
Go to the Integration Center and select the Data Source tab to complete the configuration on the data source edit page.
If the product logs have been collected to SLS, select Custom Log Capability for Data Source Type and select the corresponding Logstore in SLS. You can log on to the Simple Log Service console to view the Logstore information of the product.
If the product logs have not been collected to SLS.
Set Data Source Type to Collection Channel Dedicated for CTDR
CTDR will create a dedicated Project (aliyun-cloudsiem-channel-Alibaba Cloud UID-cn-region ID) and a dedicated Logstore in SLS for you.
Set Data Source Type to Custom Log Capability You need to first go to the Simple Log Service console to create the corresponding Logstore.
ImportantAfter the data source is configured, see Data collection overview and contact technical support to collect product logs to the corresponding Logstore in SLS as soon as possible.
Modify standardization method (Optional)
NoteSome policies can be edited. You can determine this by checking whether there is an Edit button in the Actions column, or refer to the following instructions:
If the data source type of the access policy is Predefined Log Capability, it cannot be edited. If the data source type is Custom Log Capability or Collection Channel Dedicated for CTDR, it can be edited.
If the access policy follows the Security Log-xxx Alert Log standard, the standardization method is "Real-time Consumption" and cannot be modified.
If the access policy supports modification, you can modify the Standardization Method on the edit page. The options are Real-time Consumption and Scan Query. For more information, see Standardization access method.
WarningMake sure that the Logstore corresponding to the Data Source contains data. If the Logstore has no data, the log standardization test cannot be performed, which will cause the standardization method modification to fail.
If the dataset (StoreView) corresponding to the current Standardization Category or Structure already has 5 access policies in Scan Query mode, please select "Real-time Consumption" for the current policy. Otherwise, the access policy will fail to be enabled.
Log standardization test (Optional)
If you have modified the standardization method, you need to perform a log standardization test.
Click the
button, and the system will parse the log data using SPL syntax and return the results.Select one result from the dropdown list as the test data, and click Parse and Test.
WarningIf the current data source has no data, the SPL syntax will not be able to parse the logs and return results. The result dropdown list will be empty, which will prevent the standardization test from being completed.
After the test passes, click Complete.
NoteIf the test fails, return to the previous step and confirm whether the data source and standardization rule match. If the relationship is correct, go to the Standardization Rules tab and modify the relevant configuration on the rule edit page.
Enable policy
Click the switch button in the Enabling Status column of the policy. If you have enabled the recommended access policy, the built-in policies for Security Center, Web Application Firewall, Cloud Firewall, and ActionTrail are enabled by default. For more information about how to enable the recommended access policy, see Enable access policy in subscription mode and Enable log access policy in pay-as-you-go mode.
WarningIf the dataset (StoreView) corresponding to the current Standardization Category or Structure already has 5 access policies in Scan Query mode, the current access policy will fail to be enabled. For solutions, see Access policy enabling failure.
Step 3: Add access policy (Optional)
Some CTDR predefined products support custom access policies. Follow these steps to add a policy:
In the access policy list, click Create Access Policy.
NoteIf there is no Create Access Policy button, the product does not support this feature.
On the Create Access Policy page, select the corresponding Data Source, Standardized Rule, and Standardization Method. If there are no suitable data sources and standardization rules, you can also create a data source and create a custom standardization rule.
ImportantYou can only select data sources under the current account. Member account data sources are not supported.
Test Log Standardization and enable the policy.
Add custom services
Workflow
Step 1: Add service
Log on to the Security Center console. In the upper-left corner of the console, select the region where your assets are located: China or Outside China.
In the left-side navigation pane, choose . On the Service Integration tab, click Add Service in the upper-right corner. In the Add Service dialog box, enter the product information and click OK.
Step 2: Add data source
On the Data Source tab, click Add Data Source in the upper-left corner.
If the product logs have been collected to SLS, select Custom Log Capability for Data Source Type and select the corresponding Logstore in SLS. You can log on to the Simple Log Service console to view the Logstore information of the product.
If the product logs have not been collected to SLS.
Set Data Source Type to Collection Channel Dedicated for CTDR
CTDR will create a dedicated Project (aliyun-cloudsiem-channel-Alibaba Cloud UID-cn-region ID) and a dedicated Logstore in SLS for you.
Set Data Source Type to Custom Log Capability You need to first go to the Simple Log Service console to create the corresponding Logstore.
ImportantAfter the data source is configured, see Data collection overview and contact technical support to collect product logs to the corresponding Logstore in SLS as soon as possible.
Step 3: Add standardization rule
You must first import log data into the Logstore corresponding to the data source in Step 2: Add data source. Otherwise, the log standardization test cannot be completed, which will prevent the following steps.
On the Standardized Rule tab, click Create Custom Rule in the upper-left corner.
The service provider and provider must be consistent with the service added in Step 1: Add service.
Step 4: Add and enable access policy
On the Service Integration tab, find the service you added in Step 1 and click Access Settings in the actions column.
On the Access Settings page, click Create Access Policy. On the Create Access Policy page, configure the Data Source and Standardized Log. Parameter description:
Data Source: Select the data source configured in Step 2.
Standardized Rule: Select the rule configured in Step 3.
Standardization Method: Supports Real-time Consumption and Scan Query. For more information, see Standardization access method.
ImportantIf your data source type is Collection Channel Dedicated for CTDR, select "Real-time Consumption."
Click Next. On the Test Log Standardization page, click Parse and Test. After the test passes, click Complete.
Enable the policy.
WarningIf the dataset (StoreView) corresponding to the current Standardization Category or Structure already has 5 access policies in Scan Query mode, the current access policy will fail to be enabled. For solutions, see Access policy enabling failure.
What to do next
After service logs are accessed, you need to configure threat detection rules to analyze the accessed logs, generate alerts and security incidents, and help you quickly respond to and handle cloud security risks. For more information, see Use threat detection rules.
Standardization access method
CTDR 2.0 provides two log access methods: Real-time Consumption and Scan Query.
Real-time Consumption:
If you have purchased Log Storage Capacity, a standardized log will be generated and delivered to Log Management in CTDR. After delivery, it supports custom log detection rules. For more information about how to purchase log storage capacity, see Purchase log storage capacity.
If you have not purchased Log Storage Capacity, standardized logs will not be stored, and only predefined log detection rules are supported.
Scan Query:
Logs are read directly from the data source instance without storing standardized logs. No log storage capacity purchase is required to support custom detection rules, reducing cost.
However, a dataset supports a maximum of 5 access policies in Scan Query mode. A dataset can correspond to multiple Standardization Category or Structure in Standardized Rule.
Difference | Real-time Consumption | Scan Query |
Standardized log storage | If you have purchased Log Storage Capacity, standardized logs are automatically stored and delivered to Log Management in CTDR, and the delivery switch cannot be enabled or disabled. | Logs are read directly from the data source instance without storing standardized logs. |
Log storage fee | The fee is charged on the CTDR side and consumes CTDR Log Storage Capacity. | The fee is on the Simple Log Service side. |
Applicable data source types |
| Custom Log Capability |
Dataset limit | No limits. | A dataset supports a maximum of 5 access policies in Scan Query mode, with limited query performance. |
Multi-account access | Supported. | Not supported. |
References
For more information about the differences between CTDR 1.0 and 2.0, see CTDR 2.0.
For more information about the products and log types supported by CTDR, see Supported services and logs.
For more information about data source configuration details, see Data sources.
If you encounter issues during operation, see FAQ for solutions.