Use Tablestore SDK for Go to consume data in real time - Tablestore

Tablestore SDK for Go encapsulates the automatic data consumption framework and multiple API operations, such as the management operations, of Tunnel Service,

Quick start

Before you use Tunnel Service to consume data in a data table in real time, you must create a tunnel for the table. Then, you can use the tunnel to consume data in the data table. For more information, see Getting started.

Management operations

To create a tunnel for a data table, you must specify the name of the table and the name and type of the tunnel. For more information, see Create a tunnel.

After you create a tunnel, you can manage the tunnel based on your business requirements.

Operation	Description
Query information about all tunnels of a data table	Queries information about all tunnels of a data table.
Query information about a tunnel	Queries the channel information about a tunnel. A channel corresponds to a shard for the Stream operation of Tablestore.
Delete a tunnel	Deletes a tunnel of a data table.

Data consumption framework

The automatic data consumption framework works as follows: Tunnel Client detects active channels through periodic heartbeat detection, updates the status of channels and ChannelConnect, and initializes, runs, and ends data processing tasks.

Tunnel Client can help resolve common issues in full and incremental data processing, such as load balancing, fault recovery, checkpoints, and partition information synchronization to ensure the consumption order of partition information. After using Tunnel Client, developers only need to focus on the processing logic of each record. For more information, see Tunnel clients.

References

For an overview of Tunnel Service, see Overview.
You can use Tunnel Service to migrate data. For more information, see Synchronize data from one table to another table in Tablestore.
Real-time Compute for Apache Flink can use data tunnels of Tunnel Service as input for streaming data to compute and analyze Tablestore data. For more information, see Tutorial for the Wide Column model and Tutorial (TimeSeries model).
Tunnel Service itself does not incur additional fees. When you consume data from Tunnel Service, Tablestore charges fees based on the actual read throughput. For more information, see Billing overview.