All Products
Search
Document Center

Time Series Database:Use TSDB for InfluxDB® to collect and analyze data

Last Updated:Dec 11, 2023

You can extract significant value from data. Time Series Database (TSDB) for InfluxDB® is a time series database service that provides high performance in data storage and analysis. TSDB for InfluxDB® is widely used in production and development environments. TSDB for InfluxDB® is released by Alibaba Cloud. This topic describes how to migrate your data to Alibaba Cloud TSDB for InfluxDB®.

Overview

TSDB for InfluxDB® is a time series database service that can process large numbers of write and query requests. This service stores and analyzes large amounts of time series data in real time, including DevOps monitoring data, application metric data, and data that is collected from Internet of Things (IoT) sensors. TSDB for InfluxDB® provides the following features:

  • Provides high-performance storage that is specific to time series data. The Time-Structured Merge Tree (TSM) engine is used to provide features such as high-speed data read or write and high-performance data compression.

  • Provides efficient HTTP-based API operations to write and query data.

  • Allows you to query and aggregate time series data by using an SQL-like query language.

  • Allows you to create indexes for tags to improve query performance.

  • Provides retention policies to clear obsolete data.

After you purchase a TSDB for InfluxDB® instance, you must create a database account and password, and set a storage policy. You can view the endpoints of the TSDB for InfluxDB® instance on the instance details page. The following two HTTP-based endpoints are provided: Virtual Private Cloud (VPC) endpoint and public endpoint. A cloud service instance in a VPC can access other service instances in the VPC. To use the Internet, you must enable a public endpoint when you purchase the instance. HTTPS encryption is enabled. You must use HTTPS channels for connection.

After the instance is created, you can create the test user and the test database. Make sure that the test user has read and write permissions on the test database.

Use Telegraf to collect data

Telegraf is an agent program that is written in the Go programming language. The agent can collect statistical data of systems and services, and then write the data to a TSDB for InfluxDB® database. Telegraf requires only a small size of memory. The plug-in allows developers to add extensions that support other services. After you install Telegraf, you can write data to TSDB for InfluxDB® by using HTTP channels.

Specify a VPC or public endpoint that is provided on the instance details page for the urls parameter. Specify the database and retention_policy parameters based on your business requirements. If you do not specify the retention_policy parameter, the default retention policy is used. Authentication is required to write data to TSDB for InfluxDB®. When you specify the username and password parameters, make sure that the account has write permissions on the database. You can choose Instance Management > Account Management to manage permissions. Then, the data that is collected by Telegraf can be written to TSDB for InfluxDB®.

Start Telegraf

If you use Ubuntu, Debian, RedHat, or CentOS, run the following command:

sudo service telegraf start

If you use Ubuntu version 15.04 or later, Debian version 8 or later, CentOS version 7 or later, or RHEL version 7 or later, run the following command:

sudo systemctl start telegraf

Migrate data to TSDB for InfluxDB®

If you have InfluxDB instances in self-managed environments or other platforms, you can migrate your data to TSDB for InfluxDB® by using a migration tool. The migration tool automatically creates a destination database in TSDB for InfluxDB®. You do not need to manually create a destination database before migration. If you manually create a destination database before migration, write conflicts may occur. TSDB for InfluxDB® provides the O&M feature for the migrated data. This allows you to write, query, and analyze data with ease.

Write data by using different clients

The InfluxDB open source community provides the SDKs for mainstream programming languages. TSDB for InfluxDB® is compatible with various clients. The following sections describe how to write data to TSDB for InfluxDB® by using different clients:

Use TSDB for InfluxDB® SDK for Go

package main

import(
"fmt"
"log"
"math/rand"
"net/url"
"time"

    client "github.com/influxdata/influxdb1-client"
)

func main(){
    host, err := url.Parse(fmt.Sprintf("https://%s:%d","xxx.influxdata.rds.aliyuncs.com",3242))
if err !=nil{
        log.Fatal(err)
}
    config := client.Config{
        URL:*host,
Username:"test",
Password:"test",
}
    con, err := client.NewClient(config)
if err !=nil{
        log.Fatal(err)
}

    _, _, err = con.Ping()
if err !=nil{
        log.Fatal(err)
}

var(
        shapes     =[]string{"circle","rectangle","square","triangle"}
        colors     =[]string{"red","blue","green"}
        sampleSize =1000
        pts        = make([]client.Point, sampleSize)
)

    rand.Seed(42)
for i :=0; i < sampleSize; i++{
        pts[i]= client.Point{
Measurement:"shapes",
Tags: map[string]string{
"color": colors[rand.Intn(len(colors))],
"shape": shapes[rand.Intn(len(shapes))],
},
Fields: map[string]interface{}{
"value": rand.Intn(sampleSize),
},
Time:      time.Now(),
}
}

    bps := client.BatchPoints{
Points:          pts,
Database:"test",
RetentionPolicy:"autogen",
}
    _, err = con.Write(bps)
if err !=nil{
        log.Fatal(err)
}
}

To establish an HTTP connection, you must specify the public endpoint or VPC endpoint of the instance. You must also specify the database user and password. You can view the information on the Instance Management page. Specify the database to which the data is written and a retention policy. If you do not specify a retention policy, the default retention policy is used. Data can be written by row or in batches. To improve the write throughput, we recommend that you write data in batches. The number of rows that can be written in a batch is 100 to 1000 based on the data size. For information about how to install Go, see Documentation. To download TSDB for InfluxDB® SDK for Go, visit GitHub. To compile a program, run the go run main.go command.

Use TSDB for InfluxDB® SDK for Java

package main;

import java.util.concurrent.TimeUnit;
import org.influxdb.InfluxDB;
import org.influxdb.BatchOptions;
import org.influxdb.InfluxDBFactory;
import org.influxdb.dto.Point;


publicclassStartMain{

publicstaticvoid main(String[] args)throwsException{
StartMain startMain =newStartMain();
try{
            startMain.run();
}catch(Exception e){
System.out.println(e);
}
}

publicvoid run()throwsException{

InfluxDB influxDB =InfluxDBFactory.connect("https://xxx.influxdata.rds.aliyuncs.com:3242","test","test");

String dbName ="test";
        influxDB.setDatabase(dbName);
String rpName ="autogen";
        influxDB.setRetentionPolicy(rpName);
        influxDB.enableBatch(BatchOptions.DEFAULTS);

        influxDB.write(Point.measurement("cpu")
.time(System.currentTimeMillis(),TimeUnit.MILLISECONDS)
.addField("idle",90L)
.addField("user",9L)
.addField("system",1L)
.build());

        influxDB.write(Point.measurement("disk")
.time(System.currentTimeMillis(),TimeUnit.MILLISECONDS)
.addField("used",80L)
.addField("free",1L)
.build());

        influxDB.close();
}
}

To download TSDB for InfluxDB® SDK for Java, visit GitHub. You can specify an exception handler in the enableBatch() function.

influxDB.enableBatch(BatchOptions.DEFAULTS.exceptionHandler(
(failedPoints, throwable)->{/* custom error handling here */})
);

Use Influx CLI

TSDB for InfluxDB® provides the Influx CLI command-line interface. When you establish a connection, specify the ssl, host, port, username, and password parameters.

influx -ssl -host xxx.influxdata.rds.aliyuncs.com -port 3242-username admin -password admin

You can write data from a client to TSDB for InfluxDB based on the Line Protocol.

> INSERT cpu,host=serverA,region=us_west value=0.64

Use command-line shells

curl -i -XPOST -u test:test  'https://xxx.influxdata.rds.aliyuncs.com:3242/write?db=test'--data-binary 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000'

Data analysis

TSDB for InfluxDB® supports SQL-like queries.

The following example shows how to query average values after you group data by the color key:

> SELECT MEAN("value") FROM "shapes" GROUP BY "color"
name: shapes
tags: color=blue
time mean
--------
0216.25

name: shapes
tags: color=green
time mean
--------
0434.25

name: shapes
tags: color=red
time mean
--------
0540.25

InfluxDB® is a trademark registered by InfluxData, which is not affiliated with, and does not endorse, TSDB for InfluxDB®.