Create, Query & Drop Tunnel Service via CLI - Tablestore

Tunnel Service is an integrated service built on Tablestore that provides distributed channels for real-time consumption of full and incremental data. You can create incremental, full, or incremental-plus-full data tunnels for your tables. This lets you easily process both historical and new data.

Tunnel type	Mode value	Description
Full tunnel	`base_data_only`	Consumes and processes only full data.
Incremental tunnel	`stream_data_only`	Consumes and processes only incremental data.
Differential tunnel	`base_and_stream`	Consumes and processes full data first, then incremental data.

After you create a tunnel, you can use it to consume historical and incremental data exported from a specified table.

create_tunnel

Creates a tunnel for a data table.

Command syntax

create_tunnel -n name

The following table describes the configuration items.

Parameter	Required	Example	Description
`-t, --table`	No	`mytable`	The name of the data table.
`-n, --name`	Yes	`t1`	The name of the tunnel.
`-m, --mode`	No	`stream_data_only`	The type of the tunnel. Default value: `stream_data_only`. Valid values: `base_data_only`, `stream_data_only`, `base_and_stream`. For details about each mode, see the tunnel type table above.

Example

Create a tunnel named t1 for a data table:

create_tunnel -n t1

Sample response:

New tunnel created, id is '9933470d-8a5e-4972-a5b0-b7ae6f836460'.

describe_tunnel

Queries tunnel information and channel information about a tunnel.

Command syntax

describe_tunnel -n name

The following table describes the configuration items.

Parameter	Required	Example	Description
`-t, --table`	No	`mytable`	The name of the data table.
`-n, --name`	Yes	`t1`	The name of the tunnel.
`-o, --output`	No	`D:\\otstest\\mytunnel.txt`	The local file in which to store the returned result.

Example

Query information about the tunnel named t1:

describe_tunnel -n t1

Sample response:

Tunnel Info:
+--------------------------------------+------------+------------+---------------+---------+
| TunnelId                             | TunnelName | TunnelType | Stage         | Expired |
+--------------------------------------+------------+------------+---------------+---------+
| 9933470d-8a5e-4972-a5b0-b7ae6f836460 | t1         | Stream     | ProcessStream | false   |
+--------------------------------------+------------+------------+---------------+---------+

Channel Info:
+-------------------------------------------------------+-------------+---------------+----------+-------------------------------+
| ChannelId                                             | ChannelType | ChannelStatus | ClientId | ChannelRPO                    |
+-------------------------------------------------------+-------------+---------------+----------+-------------------------------+
| cfd2c05b-54b6-48ec-aa6f-feb427f0ca57_1635771329155688 | Stream      | OPEN          |          | 1970-01-01 08:00:00 +0800 CST |
+-------------------------------------------------------+-------------+---------------+----------+-------------------------------+

consume_tunnel

Simulates or performs the data consumption of a tunnel. After you create a tunnel, you can simulate its data consumption to preview the data format in the tunnel.

Command syntax

consume_tunnel -n name -m mock_consume

The following table describes the configuration items.

Parameter	Required	Example	Description
`-c, --channel`	No	`cfd2c05b-54b6-48ec-aa6f-feb427f0ca57_1635771329155688`	The ID of the channel. If you do not specify this parameter, the data in all channels of the tunnel is consumed.
`-t, --table`	No	`mytable`	The name of the data table.
`-n, --name`	Yes	`t1`	The name of the tunnel.
`-m, --mode`	Yes	`mock_consume`	The consumption mode. Default value: `shadow_copy`. Valid values: `shadow_copy` (copies the consumption traffic of an online tunnel), `mock_consume` (simulates data consumption but does not update checkpoint information), `real_consume` (consumes data and updates checkpoint information; not recommended).

Example

Simulate the data consumption of the tunnel named t1:

consume_tunnel -n t1 -m mock_consume

After you run the command, data consumption records are displayed when data is written to the data table. Sample response:

Starting consume tunnel 't1' of table 'mytable', it may take a few seconds to start, please wait...
null

drop_tunnel

Deletes a tunnel that you no longer need.

Command syntax

drop_tunnel -n name -y

The following table describes the configuration items.

Parameter	Required	Example	Description
`-t, --table`	No	`mytable`	The name of the data table.
`-n, --name`	Yes	`t1`	The name of the tunnel.
`-y, --yes`	Yes	N/A	Specifies that confirmation information appears. You must include this parameter in the command.

Example

Delete the tunnel named t1:

drop_tunnel -n t1 -y