Tunnel Service provides distributed channels for real-time consumption of full and incremental data in Tablestore. Create incremental, full, or incremental-plus-full tunnels for your tables to process both historical and new data.
Use cases
Tablestore applications such as metadata management, time series monitoring, and messaging systems use incremental or full-plus-incremental data streams to trigger operations like:
-
Data synchronization: Sync data to a cache, search engine, or data warehouse.
-
Event-driven actions: Trigger Function Compute, notify a consumer client, or call an API.
-
Stream data processing: Connect to a stream or integrated stream-and-batch compute engine.
-
Data migration: Back up data to OSS or migrate to storage-optimized Tablestore instances.
Features
The following table describes Tunnel Service features.
Tunnel Service supports up to 100,000 writes per second per table with millisecond-level latency. Records are returned in update order.
|
Features |
Description |
|
Integrated full and incremental data tunnels |
Consume incremental data, full data in parallel, or full-plus-incremental data. |
|
Ordered incremental data changes |
Data is divided into logical partitions for parallel consumption. Incremental data within each partition is ordered by write time. |
|
Consumption latency monitoring |
Monitor consumption latency through the DescribeTunnel API or the console. |
|
Horizontal scaling of data consumption |
Automatic load balancing across logical partitions. The system scales out by adding consumer clients to increase consumption speed. |
Notes
-
Incremental logs in a tunnel share the stream log retention period of the data table, up to 7 days.
-
When consuming data from an incremental or incremental-plus-full tunnel:
-
During the full data phase, if consumption does not complete within the incremental log retention period (up to 7 days), an
OTSTunnelExpirederror occurs and blocks subsequent consumption.If full data cannot be consumed in time, contact Tablestore technical support for assistance.
-
During the incremental phase, if data is not consumed within the retention period (up to 7 days), consumption may resume from the latest available data point, which can cause data loss.
-
-
Tablestore may disable expired tunnels. Disabled tunnels cannot serve data. If a tunnel remains disabled for more than 30 days, it is permanently deleted and cannot be recovered.
Quick Start
To start consuming data through the console, follow the steps in Quick Start for Tunnel Service.
Billing
Tunnel Service itself is free. You pay only for data reads when consuming tunnel data. Pricing details are in Billing overview.