Security Center:Migrating from Log Analysis to Log Management

Migration process

The core of the migration plan is to establish a parallel log data stream. After you verify that the new stream is stable and reliable, you can switch traffic to it and decommission the old stream.

1. Parallel write: Enable both Log Management and Log Analysis. This writes incremental logs to both services simultaneously.

2. Data validation: Compare the data in Log Management and Log Analysis to verify that the data in Log Management is complete and accurate.

3. Switch to single write: After you confirm data consistency, turn off the Log Analysis delivery switch. All incremental logs are then written only to the Log Management service.

4. Process historical data: Handle the Log Analysis in Log Analysis as needed. You can either archive it to Log Analysis or let it expire and be deleted automatically.

5. Resource cleanup: Unsubscribe from the Log Analysis service to stop billing and complete the migration.

Key risks and compatibility

• Risk of historical data loss: If you clear data and unsubscribe from the Log Analysis service before backing up your historical data, you will permanently lose the data.

• SQL query incompatibility: Due to differences in storage structure, SQL queries for Log Analysis (based on topic) and Log Management (based on logstore) are not compatible. You must adapt your queries for each service.

• Management API incompatibility: Some management API calls, such as those for modifying a delivery switch, and other exclusive features are not compatible between Log Analysis and Log Management. Their API fields and responses differ and must be adapted separately. For more API information, see Log Management API and Log Analysis API.

Cost considerations

• Costs during the parallel write phase: During the parallel write phase, you will incur charges for data writes and storage for both services.

• Costs for processing historical data:

  • If you export data to OSS, you will incur OSS storage fees.

  • If you let the data expire naturally, you must continue to pay storage fees for the Log Analysis service for up to 180 days.

Step 1: Enable and configure Log Management

1. Log on to the console

   Log on to the Security Center console. In the left-side navigation pane, choose Detection and Response > Log Management. In the upper-left corner, select the region where your assets are located: Chinese Mainland or Outside Chinese Mainland.

   Note

   If you have activated Agentic SOC, go to Agentic SOC > Log Management.

2. Activate the service

   On the Log Management page, select a billing method, either Activate Subscription or Activate Pay-as-you-go, and follow the on-screen instructions to activate the service. For details, see Activate or deactivate Log Management.

   • Subscription:

     • Set Purchase or Not for Agentic SOC to Yes.

     • Enter the log storage capacity based on your business needs, click Order Now, and complete the payment. To learn how to estimate capacity, see Agentic SOC Purchase Guide.

   • Pay-as-you-go: Select a storage region and click Activate and Authorize.

3. Enable the delivery switch

   1. On the Log Management page, click Log Settings in the upper-right corner.

   2. In the Log Storage Management section, on the Security Center Logs tab, view and configure the delivery status for the corresponding log types.

      Note

      • By default, Agentic SOC automatically enables the delivery switch for all Security Center log types within 30 minutes.

      • If you have not purchased value-added services such as Application Protection or Malicious File Detection, the delivery switch for those log types is disabled by default.

Step 2: Validate data delivery to Log Management

1. Query logs

   On the Log Management page, select a product log from the Log Type drop-down list under Security Center Logs, and then click Search & Analyze.

   Important

   It may take about one minute before log data can be queried. Please wait.

2. Verify the data

   Verify at least the following aspects of the queried data:

   • Timeliness: Confirm that the timestamp of the latest log is close to the current time.

   • Completeness: Spot-check and compare logs in Log Management and Log Analysis to ensure that all fields are present and no data is missing.

   • Volume: Observe the log volume over a specific period. It should be roughly the same as the incremental volume in the Log Analysis service.

Step 3: Disable data delivery to Log Analysis

1. On the Log Analysis page, find the corresponding log delivery switch and turn it Off.

   

2. After the switch is turned off, new incremental logs are no longer written to the Log Analysis service. They are only written to the Log Management service.

Step 4: Process historical data

For historical data that is still in the Log Analysis service and within its 180-day storage period, use one of the following options:

• Option 1: Export historical data to OSS for archiving

  This option is suitable if you need to retain historical data long-term for compliance audits or future analysis.

  • Procedure:

    1. On the Log Analysis page, click Advanced Management of Simple Log Service to go to the Simple Log Service (SLS) console.

    2. On the details page of the sas-log logstore, in the left-side navigation pane, click Data Processing > Export > Object Storage Service.

    3. Click + to create a data shipping job. For information about how to configure the job, see Create an OSS data shipping job (new version).

    4. After the historical data is saved, go to the Log Analysis page and click Clear.

       Warning

       This action is irreversible. Ensure that data is being written to Log Management correctly and that all historical data has been backed up before you proceed.

  • Note: You can store the data permanently. You can use tools such as MaxCompute and Data Lake Analytics to analyze the data when needed.

• Option 2: Allow historical data to expire naturally

  This option is simpler and is suitable for scenarios where you do not need to archive historical data.

  • Procedure: No action is required. After you migrate the incremental data, keep the Log Analysis service enabled and wait for the system to automatically delete the log data at the end of its 180-day lifecycle.

  • Note: During this period, you will continue to be charged storage fees for the data in the Log Analysis service until it is completely deleted.

Step 5: Unsubscribe from the Log Analysis service

After you process the historical data and clear the storage, you can unsubscribe from the Log Analysis service to stop related billing. For more information, see downgrade.