Configuration items of Tunnel Service in Tablestore SDK for Go - Tablestore

When you use Tunnel Service to consume data, you can configure the tunnel client, worker, and logs based on your business requirements. This topic describes the configuration items of Tunnel Service.

tunnel client

When you initialize the tunnel client, you can use the NewTunnelClientWithConfig operation to specify custom configurations of the client. If you do not specify config to initialize the operation or you set config to nil, DefaultTunnelConfig is used.

The following sample code shows the configurations of DefaultTunnelConfig:

var DefaultTunnelConfig = &TunnelConfig{
      // Set the maximum exponential backoff value to retry a failed request. 
      MaxRetryElapsedTime: 75 * time.Second,
      // Specify the timeout period for HTTP requests. 
      RequestTimeout:      60 * time.Second,
      // Specify http.DefaultTransport. 
      Transport:           http.DefaultTransport,
}

Worker

TunnelWorkerConfig contains the configurations required for a worker. The ProcessorFactory parameter is required. If other parameters are unspecified, their default values are used.

The following sample code shows the configurations of TunnelWorkerConfig:

type TunnelWorkerConfig struct {
   // The heartbeat timeout periods for the worker and Tunnel Service are the same. The default value is used. 
   HeartbeatTimeout  time.Duration
   // Specify the frequency at which worker sends heartbeat information. The default value is used. 
   HeartbeatInterval time.Duration
   // Set the operation that is used to establish connections for consumption of data exported by using Tunnel Service. The default value is used. 
   ChannelDialer     ChannelDialer

   // Generate the specific processor for the connection. The callback function is typically used to initialize the SimpleProcessFactory operation. 
   ProcessorFactory ChannelProcessorFactory

   // Configure ZAP logs. The default value is DefaultLogConfig. 
   LogConfig      *zap.Config
   // Configure log rotation for ZAP. The default value is DefaultSyncer. 
   LogWriteSyncer zapcore.WriteSyncer
}

ProcessorFactory is the operation that is used to configure information such as the callback function information. We recommend that you use the SimpleProcessorFactory operation that is defined in Tablestore SDK for Go:

The following sample code shows the configurations of SimpleProcessorFactory:

type SimpleProcessFactory struct {
   // Define and configure parameters. User-defined information is passed to the ChannelContext parameter contained in ProcessFunc and ShutdownFunc. 
   CustomValue interface{}

   // Set the interval to receive information of two consecutive checkpoints. The worker records the interval. When CpInterval is set to a value smaller than or equal to 0, DefaultCheckpointInterval is used. 
   CpInterval time.Duration

   // Set the synchronous callback function for the worker to process data. If an error is returned from ProcessFunc, the worker sends retry requests to ProcessFunc to obtain the data based on exponential backoff. 
   ProcessFunc  func(channelCtx *ChannelContext, records []*Record) error
   // Set the synchronous callback function that is used when the worker exits. 
   ShutdownFunc func(channelCtx *ChannelContext)

   // Set logs. If the Logger field is set to nil, DefaultLogConfig is used to initialize logger. 
   Logger *zap.Logger
}

Logs

The following examples describe the default log configurations and log rotation configurations:

Default log configurations

// By default, DefaultLogConfig is used in TunnelWorkerConfig and SimpleProcessFactory to configure logs. 
var DefaultLogConfig = zap.Config{
   Level:       zap.NewAtomicLevelAt(zap.InfoLevel),
   Development: false,
   Sampling: &zap.SamplingConfig{
      Initial:    100,
      Thereafter: 100,
   },
   Encoding: "json",
   EncoderConfig: zapcore.EncoderConfig{
      TimeKey:        "ts",
      LevelKey:       "level",
      NameKey:        "logger",
      CallerKey:      "caller",
      MessageKey:     "msg",
      StacktraceKey:  "stacktrace",
      LineEnding:     zapcore.DefaultLineEnding,
      EncodeLevel:    zapcore.LowercaseLevelEncoder,
      EncodeTime:     zapcore.ISO8601TimeEncoder,
      EncodeDuration: zapcore.SecondsDurationEncoder,
      EncodeCaller:   zapcore.ShortCallerEncoder,
   },
}

Log rotation configurations

// By default, DefaultSyncer is used in TunnelWorkerConfig and SimpleProcessFactory to configure log rotation.
var DefaultSyncer = zapcore.AddSync(&lumberjack.Logger{
   // Set the path of logs. 
   Filename:   "tunnelClient.log",
   // Set the maximum size of each log. 
   MaxSize:    512, //MB
   // Set the maximum number of backups that can be compressed for each rotated log. 
   MaxBackups: 5,
   // Set the maximum number of days for which logs can be retained. 
   MaxAge:     30, //days
   // Specify whether to compress the rotated logs. 
   Compress:   true,
})