Tablestore:Get started with Tunnel Service

Step 1: Create a tunnel

1. Navigate to the Tunnels page.

   1. Log on to the Tablestore console.

   2. In the top navigation bar, select a resource group and a region.

   3. On the Overview page, click the name of an instance or click Manage Instance in the Actions column of the instance.

   4. On the Instance Details tab, click the Tables tab. Click the name of the target data table, and then click the Tunnels tab. Alternatively, click in the Actions column of the data table and select Tunnels.

2. On the Tunnels tab, click Create Tunnel.

3. In the Create Tunnel dialog box, configure the following parameters:

   Parameter	Description
   Tunnel name	The name of the tunnel.
   Tunnel type	The type of the tunnel. Valid values: Full: Consumes all existing data in the data table. Incremental: Consumes only new data written to the data table. Differential: Consumes all existing data in the data table, followed by all new data.
   Start time	When you create an Incremental or Full and Incremental tunnel, data written after the tunnel is created is considered incremental data by default. To consume incremental data from a specific point in time, configure the Start Time. You can also configure an End Time as needed. The value for both Start Time and End Time must be a 64-bit timestamp in milliseconds and fall within the range of [current system time - Stream expiration time + 5 minutes , current system time]. The End Time must be later than the Start Time. Important The Stream expiration time is the retention period for the change log, with a maximum of 7 days. You set this duration when you enable the Stream feature for a data table. This setting cannot be changed after it is set.
   End time

4. Click OK.

   After the tunnel is created, click Show Channels on the right side of the tunnel. You can view data content, monitor consumption latency, and check the number of consumed rows for each channel partition.

   After the tunnel is created, its status is Processing Full Data. The list of channel partitions contains one Full partition with a status of Open and one Incremental partition with a status of Waiting. You can click Simulate Consumption in the Actions column to view the consumed data.

Step 2: (Optional) Preview tunnel data format

After creating the tunnel, you can simulate data consumption to preview the format of the data records.

1. Write data to or Delete data from the table.

2. On the table's Tunnels tab, click Show Channels in the Actions column for the tunnel.

3. In the Actions column of a channel partition, click View Simulated Export Records.

4. In the View Simulated Export Records dialog box, click Start.

   The dialog box displays the consumed data records.

   Example of a consumed data record

   The following example shows a consumed data record:

   {
     "sequenceInfo": {
       "epoch": 0,
       "rowIndex": 0,
       "timestamp": 0
     },
     "recordType": "PUT",
     "columns": [
       {
         "actionType": "PUT",
         "name": "create_time",
         "type": "STRING",
         "value": "2024-02-18 22:10:07"
       },
       {
         "actionType": "PUT",
         "name": "modified_time",
         "type": "STRING",
         "value": "2024-02-18 22:10:07"
       },
       {
         "actionType": "PUT",
         "name": "num",
         "type": "INTEGER",
         "value": 29
       },
       {
         "actionType": "PUT",
         "name": "order_status",
         "type": "STRING",
         "value": "00"
       },
       {
         "actionType": "PUT",
         "name": "price",
         "type": "DOUBLE",
         "value": 400
       },
       {
         "actionType": "PUT",
         "name": "sku_id",
         "type": "STRING",
         "value": "9000000007"
       },
       {
         "actionType": "PUT",
         "name": "total_price",
         "type": "DOUBLE",
         "value": 11600
       },
       {
         "actionType": "PUT",
         "name": "user_id",
         "type": "STRING",
         "value": "1000000042"
       }
     ],
     "primaryKey": [
       {
         "name": "order_id",
         "type": "STRING",
         "value": "90fb478c-1360-11f0-a34d-00163e30a2a9"
       }
     ],
     "timestamp": 0
   }

Step 3: Start consuming data from the tunnel

1. On the Tunnels tab, copy the tunnel ID from the list.

2. Use the Tablestore SDK for Java or Tablestore SDK for Go to quickly consume data from the tunnel.

   After the data is consumed, you can view logs that contain consumption statistics and the latest synchronization timestamp for each incremental channel partition. In the Tablestore console, you can also monitor consumption latency and the number of consumed rows for each channel partition.