Use the data transformation feature to replicate log data from one Logstore to another for backup or centralized analysis.
Use case
A company stores access logs in multiple Projects under one Alibaba Cloud account. To centralize query and analysis in project-b, they replicate data from logstore-a in project-a to logstore-b in project-b. Use SLS data transformation to replicate logstore-a to logstore-b.
Prerequisites
-
The source Logstore, logstore-a, is created and contains data. Manage Logstores.
-
The destination Logstore, logstore-b, is created with sufficient shard capacity. Performance guide.
Procedure
Log on to the Simple Log Service console.
-
In the Project list, click project-a.
-
On the tab, click logstore-a.
-
Click Data Transformation to enter data transformation mode.
-
On the right side of the page, click Preview Data. Preview and debug data.
Two modes are available: Quick and Advanced. Select the Quick mode.
-
In the center of the right pane, click Save as Transformation Job.
-
In the Create Data Transformation Job panel, configure the following parameters and click OK.
NoteOther parameters are described in Create a data transformation job.
Parameter
Description
Job name
The name of the data transformation job. Use the default value.
Display name
Enter a display name, such as
test.Authorization method
Grants SLS read access to the source Logstore. Select Default Role.
Destination name
Enter a name for the destination, such as test.
Destination region
The region where the destination Project resides. Select China (Hangzhou).
Destination project
The destination Project that contains logstore-b. Enter project-b.
Target store
The name of the destination Logstore. Enter logstore-b.
Authorization method
Grants SLS write access to the destination Logstore.
Select Default Role.
Time range
The time range of data to process. Data is processed continuously from the earliest log entry until the job is manually stopped. Select All.
Verify results
-
Check the data transformation job status. Manage data transformation jobs.
In the left-side navigation pane, choose Task Management > Data Transformation. Click the job name to view its status in the Data Transformation Overview panel.
-
Go to project-b. On the tab, click logstore-b to go to the query and analysis page.
-
In the upper-right corner, click Enable Index. Create an index.
-
Verify that data is replicated from
logstore-ato logstore-b. The replicated logs should have__source__set toactiontrail_internaland__topic__set toactiontrail_audit_event.