Prerequisites

  • Lindorm Tunnel Service (LTS) is purchased.
  • Your client is connected to the network where your LTS service is deployed.
    • Modify the LTS whitelist.
    • telnet {{ BDSMaster }} 12311

HBase historical data migration

Create a task
POST operation: http://{BDSMaster}:12311/hbase/fulldata/create

Parameters:

  • @FormParam ("src") String srcName: the name of the source cluster.
  • @FormParam ("dst") String sinkName: the name of the destination cluster.
  • @FormParam ("tableNames") String tableNames: the names of the tables to be migrated. You can specify multiple table names and separate them with commas (,).
  • @FormParam ("bulkload") boolean bulkload: specifies whether a bulkload is required.
  • @FormParam("createTable") boolean createTable: specifies whether to create destination tables.
  • @FormParam("extraAttrs") String extraAttrs: optional. This parameter specifies advanced settings.
Example:
curl -d "src=hbase1&dst=hbase2&tableNames=t1,t2,t3&bulkload=true&createTable=true" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://{BDSMaster}:12311/hbase/fulldata/create
Return value:
{ 
 "success":"true", // Indicates whether the request is successful. 
 "message":"workflow-36c61d1240f84ecfbb472751a29b6deb" // The ID of the task.
}
Query all tasks

GET operation: http://{BDSMaster}:12311/hbase/fulldata

Example: curl http://{BDSMaster}:12311/hbase/fulldata

Return value:
{
  "tunnels": [ // Information about all tasks.
    {
      "jobName": "workflow-779298e682a44d5f9c226cea0e6f4dfd",
      "tableNames": [
        "t1"
      ],
      "dst": "*. *. *.*",
      "fullJobs": [
        {
          "jobName": "fulldata-104f42d9bbf341099ae02c0b61a70f4c",
          "state": "SUCCEEDED",
          "table": "t1"
        }
      ],
      "bulkloadJobs": [
        {
          "TaskName": "bulkload-73bf97f03f6140da91c3183e3d4fc1d8",
          "state": "SUCCEEDED"
        }
      ],
      "src": "*. *. *.*",
      "mergeTasks": [],
      "state": "SUCCEEDED",
      "inrcJobs": [],
      "schemeTasks": [
        {
          "TaskName": "360338ff-abfb-4f78-9b73-580cc9fd48be",
          "state": "SKIPPED",
          "table": "t1"
        }
      ]
    },
    {
      "jobName": "workflow-36c61d1240f84ecfbb472751a29b6deb",
      "tableNames": [
        "t1",
        "t2"
      ],
      "dst": "*. *. *.*",
      "fullJobs": [
        {
          "jobName": "fulldata-9014e168fccd433dbd56bcc61ba4ccbb",
          "state": "SUCCEEDED",
          "table": "t2"
        },
        {
          "jobName": "fulldata-114cc535693344238572583268ae4c67",
          "state": "SUCCEEDED",
          "table": "t1"
        }
      ],
      "bulkloadJobs": [
        {
          "TaskName": "bulkload-92402af7ef544f6182af892fe4b35c65",
          "state": "SUCCEEDED"
        },
        {
          "TaskName": "bulkload-f7580e9ebf734c999656c1aaad41f67b",
          "state": "SUCCEEDED"
        }
      ],
      "src": "*. *. *.*",
      "mergeTasks": [],
      "state": "SUCCEEDED",
      "inrcJobs": [],
      "schemeTasks": [
        {
          "TaskName": "b31b8f7c-9a74-4840-814a-6cd62d2b08c9",
          "state": "SKIPPED",
          "table": "t1"
        },
        {
          "TaskName": "b1968a67-6684-4dd5-ba07-dbd7a52af24f",
          "state": "SKIPPED",
          "table": "t2"
        }
      ]
    }
  ],
  "success": "true",
  "message": "ok"
}
Query a task with a specified ID
GET operation: http://{BDSMaster}:12311/hbase/fulldata/{jobId}/detail
Abort a task
GET operation: http://{BDSMaster}:12311/hbase/fulldata/{jobId}/abort
Delete a task
GET operation: http://{BDSMaster}:12311/hbase/fulldata/{jobId}/del
Return value:
{
    "success": "true",
    "message": "ok"
}
Rerun a task
GET operation: http://{BDSMaster}:12311/hbase/fulldata/{jobId}/retry
Return value:
{
    "success": "true",
    "message": "ok"
}

HBase incremental data synchronization

Create a task
POST operation: http://{BDSMaster}:12311/hbase/incr/create
Parameters:
  • @FormParam("name") String name: the name of the synchronization channel.
  • @FormParam("src") String srcName: the name of the source cluster.
  • @FormParam("dst") String sinkName: the name of the destination cluster.
  • @FormParam("tableNames") String tableNames: the names of the tables to be migrated. You can specify multiple table names and separate them with commas (,).
  • @FormParam("createTable") boolean createTable: specifies whether to create destination tables.
  • @FormParam("extraAttrs") String extraAttrs: the advanced settings.
curl -d "name=test1&src=hbase1&dst=hbase2&tableNames=t1,t2,t3&createTable=true" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://{BDSMaster}:12311/hbase/incr/create
Return value:
{
 "success":"true",
 "message":"wal-08a087da488349969ff4a4b78efbceba" // The ID of the synchronization channel
}
Update a task
Example:
curl -d "jobId=xxx&tableNames=t1,t2,t3" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://{BDSMaster}:12311/hbase/incr/update
Query all tasks
GET operation: http://{BDSMaster}:12311/hbase/incr
Return value:
{
  "tunnels": [
    {
      "jobId": "wal-2be4c2c89b7140a7b42c050916b3f5bb",
      "tableNames": [
        "default:t1"
      ],
      "dst": "*. *. *.*",
      "offset": 1573560272951,
      "src": "*. *. *.*",
      "name": "wal-2be4c2c89b7140a7b42c050916b3f5bb",
      "syncDate": "2019-11-12 20:04:32",
      "state": "KILLED",
      "synctime": 1573560272951
    },
    {
      "jobId": "wal-08a087da488349969ff4a4b78efbceba",
      "tableNames": [
        "default:t1",
        "default:t2",
        "default:t3"
      ],
      "dst": "*. *. *.*",
      "offset": 1573651478405,
      "src": "*. *. *.*",
      "name": "test1",
      "syncDate": "2019-11-13 21:31:46",
      "state": "RUNNING",
      "synctime": 1573651906518
    }
  ],
  "success": "true",
  "message": "ok"
}
Query a task with a specified ID
GET operation: http://{BDSMaster}:12311/hbase/incr/{jobId}/detai
Abort a task
GET operation: http://{BDSMaster}:12311/hbase/incr/{jobId}/abort
Return value:
{
    "success": "true",
    "message": "ok"
}
Delete a task
GET operation: http://{BDSMaster}:12311/hbase/incr/{jobId}/del
Return value:
{
    "success": "true",
    "message": "ok"
}
Rerun a task
GET operation: http://{BDSMaster}:12311/hbase/incr/{jobId}/retry
Return value:
{
    "success": "true",
    "message": "ok"
}