Simple Log Service provides multiple API operations that you can use to query metrics or write metrics to Metricstores. The API operations are compatible with the open source Prometheus protocol. This topic describes how to call the API operations.
Overview
The API operations provided by Prometheus are stored in the /api/v1 directory. The Metricstore-related API operations provided by Log Service are also stored in the directory. You can use the following URL to call the Metricstore-related API operations provided by Log Service: https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/.
Variable | Required | Description |
{sls-endpoint} | Yes | The Log Service endpoint. For more information, see Endpoints. |
{project} | Yes | The project that is created. For more information, see Create a project. |
{metricstore} | Yes | The Metricstore that is created. For more information, see Create a Metricstore. |
During an API call, basic authentication is required. You must set Username to an AccessKey ID and Password to an AccessKey secret. We recommend that you use the AccessKey pair of a RAM user. You must grant the RAM user the required permissions to query the specified project. For more information, see Configure the permission assistant feature.
API operations also support Security Token Service (STS) authentication. To perform STS authentication, specify Password in the {AccessKey Secret}${STS Token}
format in the basic authentication-related settings. For more information, see What is STS?
API operations for metric queries
The Instant Queries API and Range Queries API operations are used to query metrics.
Instant Queries API
The Instant Queries API operation is used to query metrics at a specified point in time.
GET https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/query
POST https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/query
The following table describes the variables.
Variable | Required | Description |
query | Yes | The PromQL statement. For more information, see PromQL. |
time | No | The point in time to query. The value is a UNIX timestamp that is accurate to the second. The default value is the current time. |
timeout | No | The timeout period for a query. Unit: seconds. You can specify the timeout period in the 1s, 2m, 3h, or 4d format. The value 1s indicates 1 second, the value 2m indicates 2 minutes, the value 3h indicates 3 hours, and the value 4d indicates 4 days. For more information, see Time Durations. |
lookback-delta | No | The variable that you can use to specify a custom value for the query.lookback-delta flag parameter. The value takes effect only for the current query. The value must follow the specification of time durations. For more information, see Time Durations. This variable specifies the maximum backtrack range when PromQL identifies specific points in time during calculation. The default value of 3m is used for the variable in Simple Log Service MetricStores. |
Sample code
curl -X GET 'https://haoqi-sls-metric-test.pub-cn-hangzhou.log.aliyuncs.com/prometheus/haoqi-sls-metric-test/prometheus-metrics/api/v1/query?query=up&time=1676700699' \ -u username:password \ -H 'Content-Type: application/x-www-form-urlencoded' # Specify the AccessKey pair of an Alibaba Cloud account or a RAM user for the username and password.
Query and analysis result
{ "status": "success", "data": { "resultType": "vector", "result": [ { "metric": { "__name__": "up", "instance": "demo.promlabs.com:10001", "job": "demo" }, "value": [ 1676700550.696, "1" ] }, { "metric": { "__name__": "up", "instance": "demo.promlabs.com:10000", "job": "demo" }, "value": [ 1676700550.696, "1" ] } ] } }
Range Queries API
The Range Queries API operation is used to query metrics at multiple points in time within a specified time range.
GET https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/query_range
POST https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/query_range
The following table describes the variables.
Variable | Required | Description |
query | Yes | The PromQL statement. For more information, see PromQL. |
start | NOT | The beginning of the time range to query. The value is a UNIX timestamp that is accurate to the second. |
end | NOT | The end of the time range to query. The value is a UNIX timestamp that is accurate to the second. |
step | NOT | The interval at which queries are performed. Unit: seconds. You can specify the timeout period in the 1s, 2m, 3h, or 4d format. The value 1s indicates 1 second, the value 2m indicates 2 minutes, the value 3h indicates 3 hours, and the value 4d indicates 4 days. For more information, see Time Durations. |
timeout | NOT | The timeout period for a query. Unit: seconds. You can specify the timeout period in the 1s, 2m, 3h, or 4d format. The value 1s indicates 1 second, the value 2m indicates 2 minutes, the value 3h indicates 3 hours, and the value 4d indicates 4 days. For more information, see Time Durations. |
lookback-delta | NOT | The variable that you can use to specify a custom value for the query.lookback-delta flag parameter. The value takes effect only for the current query. The value must follow the specification of time durations. For more information, see Time Durations. This variable specifies the maximum backtrack range when PromQL identifies specific points in time during calculation. The default value of 3m is used for the variable in Simple Log Service MetricStores. |
Sample code
The following sample code provides an example on how to query the metrics from 2023-02-18 14:09:59 to 2023-02-18 14:16:39. The interval is 60 seconds.
curl -X GET 'https://haoqi-sls-metric-test.pub-cn-hangzhou.log.aliyuncs.com/prometheus/haoqi-sls-metric-test/prometheus-metrics/api/v1/query_range?query=up&start=1676700599&end=1676700999&step=60s' \ -u username:password \ -H 'Content-Type: application/x-www-form-urlencoded' # Specify the AccessKey pair of an Alibaba Cloud account or a RAM user for the username and password.
Query and analysis result
{ "status": "success", "data": { "resultType": "matrix", "result": [ { "metric": { "__name__": "up", "instance": "demo.promlabs.com:10000", "job": "demo" }, "values": [ [ 1676700599, "1" ], [ 1676700659, "1" ], [ 1676700719, "0" ], [ 1676700779, "0" ], [ 1676700839, "1" ], [ 1676700899, "0" ], [ 1676700959, "1" ] ] }, { "metric": { "__name__": "up", "instance": "demo.promlabs.com:10001", "job": "demo" }, "values": [ [ 1676700599, "1" ], [ 1676700659, "1" ], [ 1676700719, "0" ], [ 1676700779, "0" ], [ 1676700839, "1" ], [ 1676700899, "1" ], [ 1676700959, "1" ] ] } ] } }
API operations for metadata queries
Log Service allows you to query metadata such as label names and label values. To support metadata queries, Log Service is compatible with the API operations for metadata queries that are provided by Prometheus. For more information about the API operations for metadata queries, see Querying metadata. You can use the API operations for metadata queries to obtain all metrics, label names, and label values. You cannot use the API operations for metadata queries to obtain timestamps or numeric values.
Query Series API
The Query Series API operation is used to query all metric names and label-based numeric pairs that meet specified conditions within a specified period of time.
GET https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/series
POST https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/series
The following table describes the variables.
Variable | Required | Description |
match[] | Yes | The filter condition. Example: up{instance="demo.*"}. You can specify one or more conditions. |
start | NOT | The beginning of the time range to query. The value is a UNIX timestamp that is accurate to the second. The default value is 5 minutes before the current time. |
end | NOT | The end of the time range to query. The value is a UNIX timestamp that is accurate to the second. The default value is the current time. Important If you specify values for the start and end variables, you can call this operation to query only data that is generated within 5 minutes before the time that is specified by the end variable. |
Configuration example
curl -g -X GET 'https://haoqi-sls-metric-test.pub-cn-hangzhou.log.aliyuncs.com/prometheus/haoqi-sls-metric-test/prometheus-metrics/api/v1/series?match[]=up{instance="demo.promlabs.com:10000"}&match[]=go_sched_latencies_seconds_bucket&start=1676700599&end=1676700999' \ -u username:password \ -H 'Content-Type: application/x-www-form-urlencoded' # Specify the AccessKey pair of an Alibaba Cloud account or a RAM user for the username and password.
Query and analysis result
{ "status": "success", "data": [ { "__name__": "go_gc_duration_seconds_count", "instance": "demo.promlabs.com:10000", "job": "demo" }, { "__name__": "go_gc_duration_seconds_count", "instance": "demo.promlabs.com:10001", "job": "demo" }, { "__name__": "up", "instance": "demo.promlabs.com:10000", "job": "demo" } ] }
Query Label Names API
The Query Label Names API operation is used to query all label names that meet specified conditions within a specified period of time.
GET https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/labels
POST https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/labels
The following table describes the variables.
Variable | Required | Description |
match[] | Yes | The filter condition. Example: up{instance="demo.*"}. You can specify one or more conditions. You can also leave the variable empty. |
start | NOT | The beginning of the time range to query. The value is a UNIX timestamp that is accurate to the second. The default value is 5 minutes before the current time. |
end | NOT | The end of the time range to query. The value is a UNIX timestamp that is accurate to the second. The default value is the current time. Important If you specify values for the start and end variables, you can call this operation to query only data that is generated within 5 minutes before the time that is specified by the end variable. |
Configuration example
The following sample code provides an example on how to query the label names of all metrics within a specified period of time:
curl -X GET 'https://haoqi-sls-metric-test.pub-cn-hangzhou.log.aliyuncs.com/prometheus/haoqi-sls-metric-test/prometheus-metrics/api/v1/labels?start=1676700599&end=1676700999' \ -u username:password \ -H 'Content-Type: application/x-www-form-urlencoded' # Specify the AccessKey pair of an Alibaba Cloud account or a RAM user for username and password.
Query and analysis result
{ "status": "success", "data": [ "code", "instance", "job", "le", "method", "mode", "path", "quantile", "status", "type", "version", "__name__" ] }
Query Label Values API
The Query Label Values API operation is used to query all label values for specified label names that meet specified conditions within a specified period of time.
Replace <label_name> in the URL of the Query Label Values API operation with an actual label name.
GET https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/label/<label_name>/values
The following table describes the variables.
Variable | Required | Description |
match[] | Yes | The filter condition. Example: up{instance="demo.*"}. You can specify one or more conditions. |
start | NOT | The beginning of the time range to query. The value is a UNIX timestamp that is accurate to the second. The default value is 5 minutes before the current time. |
end | NOT | The end of the time range to query. The value is a UNIX timestamp that is accurate to the second. The default value is the current time. Important If you specify values for the start and end variables, you can call this operation to query only the data that is generated within 5 minutes before the time that is specified by the end variable. |
Configuration example
The following sample code provides an example on how to query all values of the instance label for the up metric within a specified period of time:
curl -X GET 'https://haoqi-sls-metric-test.pub-cn-hangzhou.log.aliyuncs.com/prometheus/haoqi-sls-metric-test/prometheus-metrics/api/v1/label/instance/values?match[]=up&start=1676700599&end=1676700999' \ -u username:password \ -H 'Content-Type: application/x-www-form-urlencoded' # Specify the AccessKey pair of an Alibaba Cloud account or a RAM user for the username and password.
Query and analysis result
{ "status": "success", "data": [ "demo.promlabs.com:10000", "demo.promlabs.com:10001", "demo.promlabs.com:10002" ] }
API operations for data writes
You can configure the remote_write parameter in the configuration file of a Prometheus process to collect metrics to Metricstores. For more information, see Collect metric data from Prometheus by using the Remote Write Protocol. Metricstores are compatible with the Remote Write Protocol. You can call the remote_write operation over HTTP to write data to Metricstores without the need to use a Prometheus process. For more information about the Remote Write Protocol, see Remote write.
The remote_write API operation is used to parse metrics and write the data that is obtained after parsing to backend storage.
POST https://{project}.{sls-endpoint}/prometheus/{project}/{metricstore}/api/v1/write
Sample code:
import (
"bytes"
"flag"
"fmt"
"github.com/gogo/protobuf/proto"
"github.com/golang/snappy"
"github.com/prometheus/prometheus/prompb"
"io/ioutil"
"net/http"
"time"
)
func MockRemoteWrite() {
project := flag.String("project", "xxxx", "")
metricStore := flag.String("metricstore", "xxxx", "")
endpoint := flag.String("endpoint", "xxxx", "")
akId := flag.String("akid", "xxxx", "") // The AccessKey ID.
akKey := flag.String("aksecret", "xxxx", "")
flag.Parse()
Url := fmt.Sprintf("https://%s.%s/prometheus/%s/%s/api/v1/write", *project, *endpoint, *project, *metricStore)
timestamp := time.Now().UnixNano()
timeSeries := []prompb.TimeSeries{
{
Labels: []prompb.Label{
{Name: "__name__", Value: "test_metric"},
{Name: "app", Value: "HOST"},
{Name: "device", Value: "vda"},
},
Samples: []prompb.Sample{
{Timestamp: timestamp / 1000000, Value: 100},
{Timestamp: timestamp/1000000 + 10000, Value: 200},
{Timestamp: timestamp/1000000 + 20000, Value: 400},
{Timestamp: timestamp/1000000 + 30000, Value: 300},
},
},
{
Labels: []prompb.Label{
{Name: "__name__", Value: "test_metric"},
{Name: "app", Value: "HOST"},
{Name: "device", Value: "vda"},
{Name: "uid", Value: "123456"},
},
Samples: []prompb.Sample{
{Timestamp: timestamp / 1000000, Value: 100},
{Timestamp: timestamp/1000000 + 10000, Value: 200},
{Timestamp: timestamp/1000000 + 20000, Value: 400},
{Timestamp: timestamp/1000000 + 30000, Value: 600},
},
},
}
data, _ := proto.Marshal(&prompb.WriteRequest{Timeseries: timeSeries})
bufBody := snappy.Encode(nil, data)
rwR, err := http.NewRequest("POST", Url, ioutil.NopCloser(bytes.NewReader(bufBody)))
rwR.Header.Add("Content-Encoding", "snappy")
rwR.Header.Set("Content-Type", "application/x-protobuf")
rwR.SetBasicAuth(*akId, *akKey) // The information that is used for basic authentication.
if err != nil {
fmt.Println(err.Error())
return
}
start := time.Now().UnixNano() / 1000000 //ms
do, err := client.Do(rwR)
end := time.Now().UnixNano() / 1000000 // ms
if err != nil {
panic(err)
}
status, result := parseResp(do)
fmt.Println("status:", status, "result:", result, "duration:", end-start)
}
func parseResp(resp *http.Response) (status, data string) {
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body) // Specifies that all content in the request body must be read.
if err != nil {
panic(err)
}
return resp.Status, string(body)
}
SDK sample code
Call a query API operation over HTTP
import (
"flag"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"strconv"
"strings"
"time"
)
const separator = "#"
func http_main() {
project := flag.String("project", "xxxx", "")
metricStore := flag.String("metricstore", "xxxx", "")
endpoint := flag.String("endpoint", "xxxx", "")
akId := flag.String("akid", "xxxx", "")
akKey := flag.String("aksecret", "xxxx", "")
query := flag.String("query", "avg(up)", "")
queryType := flag.String("type", "values", "range or query or labels or values or series")
matches := flag.String("match", "up", "") // Concatenate multiple parameters of the match[] method by using number signs (#).
labelName := flag.String("label", "instance", "")
step := flag.String("step", "1m", "")
fromtime := flag.String("from", "2023-02-15T00:00:00Z", "time 2006-01-02T15:04:05Z07:00")
totime := flag.String("to", "2023-02-15T00:15:00Z", "time 2006-01-02T15:04:05Z07:00")
flag.Parse()
timeFrom, err := time.Parse(time.RFC3339, *fromtime)
if err != nil {
panic(err)
}
timeTo, err := time.Parse(time.RFC3339, *totime)
if err != nil {
panic(err)
}
// URL: https://{project}.{sls-enpoint}/prometheus/{project}/{metricstore}
prometheusEndpoint := fmt.Sprintf("https://%s/prometheus/%s/%s", *project+"."+*endpoint, *project, *metricStore)
var uri string
urlVal := url.Values{}
urlVal.Add("start", strconv.FormatInt(timeFrom.Unix(), 10))
urlVal.Add("end", strconv.FormatInt(timeTo.Unix(), 10))
switch *queryType {
case "range":
urlVal.Add("query", *query)
urlVal.Add("step", *step)
uri = fmt.Sprintf("%s/api/v1/query_range?%v", prometheusEndpoint, urlVal.Encode())
case "query":
urlVal.Add("query", *query)
urlVal.Add("time", strconv.FormatInt(timeTo.Unix(), 10))
uri = fmt.Sprintf("%s/api/v1/query?%v", prometheusEndpoint, urlVal.Encode())
case "labels":
extractAddMatches(*matches, urlVal)
uri = fmt.Sprintf("%s/api/v1/labels?%v", prometheusEndpoint, urlVal.Encode())
case "values":
extractAddMatches(*matches, urlVal)
uri = fmt.Sprintf("%s/api/v1/label/%s/values?%v", prometheusEndpoint, *labelName, urlVal.Encode())
case "series":
extractAddMatches(*matches, urlVal)
uri = fmt.Sprintf("%s/api/v1/series?%v", prometheusEndpoint, urlVal.Encode())
}
req, _ := http.NewRequest(http.MethodGet, uri, nil)
req.SetBasicAuth(*akId, *akKey)
resp, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
buf, err := ioutil.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
panic(err)
}
fmt.Println(string(buf))
}
func extractAddMatches(matches string, uVal url.Values) {
splits := strings.Split(matches, separator)
for _, match := range splits {
uVal.Add("match[]", match)
}
}
Call a query API operation by using an SDK for Prometheus
In this example, the version of the SDK for Prometheus that is used is v1.14.0. For more information, see Prometheus client_golang.
import (
"context"
"flag"
"fmt"
"github.com/prometheus/client_golang/api"
v1 "github.com/prometheus/client_golang/api/prometheus/v1"
"net"
"net/http"
"net/url"
"time"
)
func main() {
project := flag.String("project", "xxxx", "")
metricStore := flag.String("metricstore", "xxxx", "")
endpoint := flag.String("endpoint", "xxxx", "")
akId := flag.String("akid", "xxxx", "")
akKey := flag.String("aksecret", "xxxx", "")
flag.Parse()
// URL: https://{project}.{sls-enpoint}/prometheus/{project}/{metricstore}
prometheusEndpoint := fmt.Sprintf("https://%s.%s/prometheus/%s/%s", *project, *endpoint, *project, *metricStore)
client, err := api.NewClient(api.Config{
Address: prometheusEndpoint,
RoundTripper: &http.Transport{
// set basic auth
Proxy: func(req *http.Request) (*url.URL, error) {
req.SetBasicAuth(*akId, *akKey)
return nil, nil
},
DialContext: (&net.Dialer{
Timeout: 60 * time.Second,
KeepAlive: 60 * time.Second,
}).DialContext,
TLSHandshakeTimeout: 10 * time.Second,
},
})
if err != nil {
panic(err)
}
v1api := v1.NewAPI(client)
ctx, _ := context.WithTimeout(context.Background(), 60*time.Second)
r := v1.Range{
Start: time.Now().Add(-15 * time.Minute),
End: time.Now(),
Step: time.Minute,
}
// query range
result, warnings, err := v1api.QueryRange(ctx, "avg(up)", r)
if err != nil {
panic(err)
}
if len(warnings) > 0 {
fmt.Printf("Warnings: %v %v\n", warnings, result)
}
fmt.Println(result)
// query
result, warnings, err = v1api.Query(ctx, "avg(up)", time.Now())
if err != nil {
panic(err)
}
if len(warnings) > 0 {
fmt.Printf("Warnings: %v %v\n", warnings, result)
}
fmt.Println(result)
// series
series, warnings, err := v1api.Series(ctx, []string{"up"}, time.Now().Add(-15*time.Minute), time.Now())
if err != nil {
panic(err)
}
if len(warnings) > 0 {
fmt.Printf("Warnings: %v %v\n", warnings, result)
}
fmt.Println(series)
// labels
names, warnings, err := v1api.LabelNames(ctx, []string{"up"}, time.Now().Add(-15*time.Minute), time.Now())
if err != nil {
panic(err)
}
if len(warnings) > 0 {
fmt.Printf("Warnings: %v %v\n", warnings, result)
}
fmt.Println(names)
// labelValues
values, warnings, err := v1api.LabelValues(ctx, "instance", []string{"up"}, time.Now().Add(-15*time.Minute), time.Now())
if err != nil {
panic(err)
}
if len(warnings) > 0 {
fmt.Printf("Warnings: %v %v\n", warnings, result)
}
fmt.Println(values)
}
Structure of responses
The following code shows the structure of responses to query API operations and the data write API operation:
{
"status": "success" | "error",
"data": <data>,
// The following code is returned if an error occurs in a query:
"errorType": "<string>",
"error": "<string>",
// The following warning information is returned if the query results are incomplete:
"warnings": ["<string>"]
}
Error handling
You can handle common errors based on the following descriptions:
Failed authentication
If the following information is returned, the authentication fails. We recommend that you specify a valid AccessKey pair.
{
"status": "error",
"errorType": "unauthorized",
"error": "get query instance error: {\n \"httpCode\": 401,\n \"errorCode\": \"Unauthorized\",\n \"errorMessage\": \"AccessKeyId not found: xxxx\",\n \"requestID\": \"xxxx\"\n}"
}
Invalid PromQL statement
If the following information is returned, the PromQL statement is invalid. We recommend that you modify the query statement that is specified in the query parameter.
--> /api/v1/query_range?query=up[2m]&start=1676700599&end=1676700999&step=60s
{
"status": "error",
"errorType": "bad_data",
"error": "invalid expression type \"range vector\" for range query, must be Scalar or instant Vector"
}
Timeout error
If the following information is returned, the query times out. We recommend that you increase the value of the timeout parameter.
{
"status": "error",
"errorType": "timeout",
"error": "query timed out in expression evaluation"
}
Incomplete query results
If the following information is returned, the query results are incomplete. We recommend that you narrow the query time range and perform the query again.
{
"status": "success",
"data": {
"resultType": "matrix",
"result": [
{
"metric": {},
"values": [
[
1673798460,
"11111111"
],
[
1673799060,
"22222222"
],
[
1673799660,
"33333333"
]
]
}
]
},
"warnings": [
"Request to Sls partial incompleted, incomplete task count : 11, total : 108"
]
}