If you want to store multiple copies of data in a Logstore, you can use the data transformation feature to replicate the data. This topic describes the scenarios in which you need to replicate Logstore data and the procedure on how to replicate the data.
Scenarios
The access logs of a company need to be collected and stored in multiple projects within an Alibaba Cloud account. The company can replicate the access logs from the logstore-a Logstore of the project-a project to the logstore-b Logstore of the project-b project. Then, the company can query and analyze the data in the project-b project in a centralized manner. To replicate data from the logstore-a Logstore to the logstore-b Logstore, the company can use the data transformation feature of Simple Log Service.
Prerequisites
A Logstore named logstore-a is created and data is written to the Logstore. For more information, see Manage a Logstore.
A Logstore named logstore-b is created. The performance of the logstore-b Logstore is evaluated. For example, whether the number of shards meets the requirements is evaluated. For more information, see Performance guide.
Procedure
Log on to the Simple Log Service console.
In the Projects section, click the project-a project.
On the tab, click the logstore-a Logstore.
On the page that appears, click Data Transformation.
On the right side of the page that appears, click Preview Data. For more information about how to preview data, see Preview mode overview.
In the middle of the right side of the page, click Save as Transformation Job.
In the Create Data Transformation Job panel, configure the parameters and click OK. The following table describes the parameters.
NoteFor more information about other parameters, see Create a data transformation job.
Parameter
Description
Job Name
The default name of the data transformation job.
Display Name
The custom name of the data transformation job. In this example, enter
test.Authorization Method
The method that is used to authorize Simple Log Service to read data from the logstore-a Logstore. In this example, select Default Role.
Destination Name
The name of the storage destination. In this example, enter test.
Destination Region
The region where the destination project resides. In this example, select China (Hangzhou).
Destination Project
The name of the destination project. In this example, enter project-b.
Target Store
The name of the destination Logstore. In this example, enter logstore-b.
Authorization Method
The method that is used to authorize Simple Log Service to read data from and write data to the logstore-b Logstore.
In this example, select Default Role.
Time Range
The time range for the data transformation job. A value of All specifies that the data transformation job transforms data in the source Logstore from the first log until you manually stop the job. In this example, select All.
View replication results
On the Data Transformation Overview page, view the status of the data transformation job that you created. For more information about the parameters on the Data Transformation Overview page, see Manage a data transformation job.
Open the project-b project. On the tab, select the logstore-b Logstore to go to the query and analysis page.
In the upper-right corner, click Enable. For more information about indexes, see Create indexes.
View the data that is replicated from the
logstore-aLogstore.