After a data synchronization solution is configured, you can manage the solution. You can also view the running details of the solution. This topic describes common O&M operations that can be performed on a data synchronization solution.
Background information
This topic describes only common O&M operations that can be performed on a data synchronization solution. For information about how to perform O&M operations on a real-time synchronization node that is generated by a data synchronization solution, see O&M for real-time synchronization nodes. For information about how to perform O&M operations on a batch synchronization node that is generated by a data synchronization solution, see O&M for batch synchronization nodes.
Manage a data synchronization solution
After a data synchronization solution is configured, you can go to the Tasks page in Data Integration in the DataWorks console to view the data synchronization solution. This page displays all created data synchronization solutions. You can specify filter conditions to search for the desired data synchronization solution. Then, you can perform the operations that are described in the following table on the data synchronization solution.Operation | Description |
---|---|
Start | You can click Submit and Run in the Actions column of the data synchronization solution to start the solution. |
Edit | In a business scenario, your business in the production environment may evolve over time. Your business tables may increase or decrease. In this case, you may need to adjust the number of business tables from which you want to synchronize data. Data Integration allows you to adjust the source tables that are specified in your data synchronization solution. You can click More in the Actions column of the data synchronization solution and select Modify Configuration to go to the configuration page of the solution. On the configuration page, you can add or remove source tables based on your business requirements. After the adjustment is complete, you can go back to the Tasks page, find the data synchronization solution, and then click Submit and Run in the Actions column of the solution to run the solution. When you rerun the solution, the system compares the source tables specified in the solution in this run with the source tables specified in the solution in the previous run. If new tables are detected, the system runs the solution to synchronize data from the tables. For more information, see Add or remove source tables to or from a synchronization solution that is running. If you run a one-click real-time synchronization solution, the solution synchronizes full data from the newly added tables. After the full data is synchronized, the system runs the real-time synchronization node generated by the solution to synchronize incremental data from the newly added tables in real time. Note
|
Forcefully rerun | In some cases, you can click More in the Actions column of the solution and select Force Rerun to rerun the solution. For example, if data in the source is contaminated or errors occur on data links, you can perform the rerun operation. After you forcefully rerun the solution, the system synchronizes full data and incremental data from the source to the destination again. Note
In the following scenarios, a one-click real-time synchronization solution used to synchronize data to MaxCompute needs to be rerun to restore data:
Important
|
Backfill full data | You can perform this operation if you need to synchronize full data from the source again to resolve data accuracy issues, such as data loss, that occur on the data synchronized to MaxCompute tables in the data synchronization solution. Note
To backfill full data for a one-click real-time synchronization solution used to synchronize data to MaxCompute, find the solution on the Tasks page in Data Integration, click More in the Actions column, and then select Backfill Data for All Data.
Important
|
Stop | If the data synchronization solution is running and you want to stop the running of the solution, you can click Stop in the Actions column of the solution. |
View the status overview of data synchronization solutions
You can go to the Running Status Overview page in Data Integration and specify a period of time to view the status overview of data synchronization solutions. The Running Status Overview page contains the following sections:- Solution Status Distribution: displays the total number of data synchronization solutions and displays the status distribution of the solutions in a pie chart. The statistical data about the status distribution shows the number of solutions that are successfully run and the number of solutions that fail to be run. The statistical data is collected in the specified period of time. You can click a sector in the pie chart to go to the solution list page. On this page, you can view the solutions that are successfully run or fail to be run, and the running details of a solution. For more information about the running details of a data synchronization solution, see View the running details of a data synchronization solution.
- Usage of Resources in Resource Groups: displays the specifications and resource usage of the resource groups that are used within the current Alibaba Cloud account. You can click the name of a resource group to go to the details page of the resource group. On the details page, you can view the basic information and resource usage of the resource group. For information about resource groups, see View the resource usage of an exclusive resource group.
- Batch Synchronization Nodes: displays the number of batch synchronization nodes generated by specific data synchronization solutions, the data synchronization speed, the status distribution of the batch synchronization nodes, and the details of the synchronized data. The statistical data is collected in the specified period of time.
- The statistical data about the status distribution shows the number of the batch synchronization nodes that are successfully run and the number of the batch synchronization nodes that fail to be run.
- The Synchronization Data subsection displays the following items:
- Number of synchronization nodes: the number of batch synchronization nodes that are successfully run
- Amount of data synchronized: the amount of data synchronized by batch synchronization nodes that are successfully run or running
- Number of data records synchronized: the number of data records that are synchronized by batch synchronization nodes
Note The statistical data in the Batch Synchronization Nodes section is updated per hour. - Real-time Synchronization Nodes: displays the number of real-time synchronization nodes generated by specific data synchronization solutions, the data synchronization speed, the status distribution of the real-time synchronization nodes, and the top 10 nodes with the highest latency. You can click the name of a node to go to the Real Time DI page and view the details of the node.
View the running details of a data synchronization solution
You can click Data Synchronization Node in the left-side navigation pane of the Data Integration page to go to the Tasks page.
- Upper part in the Running Data section: displays information such as the status of environment preparation, batch synchronization nodes, and the real-time synchronization node. You can check whether the nodes are run as expected based on their status. This way, you can troubleshoot the issues that occur on the data synchronization solution at the earliest opportunity. The following icons are used to indicate different states:
- If the
icon is displayed, the node is successfully run.
- If the
icon is displayed, the node failed to be run.
- If the
icon is displayed, the node is waiting to be run.
- If the
- Lower part of the Running Data section: displays the information about the batch synchronization nodes and the real-time synchronization node generated by the solution. The information includes the source name, data synchronization speed, synchronized data, resource group that is used, and data synchronization latency.
- Steps section: displays all steps that are required to complete the data synchronization solution from node creation to running of batch synchronization nodes and the real-time synchronization node. You can view the start time, end time, and status of each step in this section.
View the running details of a node
- Obtain the name of a node generated by a data synchronization solution
On the Tasks page in Data Integration, find the desired data synchronization solution and click Execution details in the Actions column. In the Steps section of the Execution details page, find the step for generating a node for the solution and click Execution details in the Status column to obtain the name of the node that is generated.
- View the running details of the node
- Batch synchronization of all data in a database
If you run a batch synchronization solution that is used to synchronize all data from a database and configured with scheduling settings, multiple auto triggered nodes are generated by the solution. You can go to the Cycle Instance page in Operation Center to view the running details of the instances generated for the desired auto triggered node.
- One-click real-time synchronization (one-time full synchronization and real-time incremental synchronization)
If you run a one-click real-time synchronization solution to synchronize full data from a source at a time and incremental data from the source in real time, batch and real-time synchronization nodes are generated by the solution. You can go to the Patch Data page in Operation Center to view the running details of a batch synchronization node and go to the Real Time DI page in Operation Center to view the running details of a real-time synchronization node.
- Batch synchronization of all data in a database