All Products
Search
Document Center

Simple Log Service:Use Prometheus SDK to upload metric data in Function Compute scenarios

Last Updated:Mar 28, 2025

You can use Simple Log Service SDK to upload metric data. However, manually maintaining custom metrics is inefficient. Prometheus automatically generates multi-dimensional metrics and maintains built-in labels. In most cases, Prometheus uses a third-party collector to pull metric data in Pull mode by using an HTTP endpoint. In Function Compute scenarios, Prometheus cannot upload metric data in Pull mode because computing services cannot provide an HTTP endpoint. This topic describes how to use Prometheus SDK to upload metric data in Push mode.

Prerequisites

Integrate Prometheus into Simple Log Service

Prometheus automatically generates multi-dimensional metrics, maintains built-in labels, and uploads metric data to a Metricstore based on the labels. The mode that Prometheus uses to upload metric data to Simple Log Service varies based on the scenario. The following table describes the modes.

Mode

Description

Pull mode

In Pull mode, Prometheus collects and uploads metric data to Simple Log Service based on the following phases:

  1. Import Prometheus SDK to your project and use the SDK to create and maintain metrics.

  2. Expose an HTTP endpoint, such as /metrics and enable a collection process, such as Prometheus, VMAgent, or Logtail to regularly access the endpoint to query metric data for encoding.

  3. The collection process parses the obtained metric data and converts the metric data to encoded data that conforms to the Prometheus Remote Write PB protocol.

  4. The collection process sends a request to the RemoteWrite HTTP endpoint of a Simple Log Service Metricstore to upload the metric data to the Metricstore.

Push mode

In Push mode, metric data is actively pushed to a Metricstore. Metric data can be pushed to a Simple Log Service Metricstore by using one of the following methods:

  1. Call Simple Log Service SDK to write metric data as logs to the Metricstore. For more information, see Simple Log Service SDK.

  2. Call Prometheus SDK to write metric data to the Metricstore in the open source standard format. For more information, see Sample code.

Sample code

You can use Prometheus SDK to integrate the operations that are executed by a collection process to collect metric data in Pull mode into your project process.

Go

Simple Log Service SDK for Go is installed. For more information, see Install Simple Log Service SDK for Go. The Gather() function is used to query all metrics. Then, the Encode or Decode operation is called.

package main

import (
	"bytes"
	"fmt"
	"time"

	"github.com/gogo/protobuf/proto"
	"github.com/golang/snappy"
	"github.com/prometheus/client_golang/prometheus"
	"github.com/prometheus/common/expfmt"
	"github.com/prometheus/prometheus/util/fmtutil"

	"io/ioutil"
	"math/rand"
	"net/http"
)

var durationHistogram = prometheus.NewHistogram(
	prometheus.HistogramOpts{
		Name:    "server_handling_seconds",
		Buckets: prometheus.DefBuckets,
	},
)

var requestCount = prometheus.NewCounter(
	prometheus.CounterOpts{
		Name: "server_requests_count",
	},
)

func main() {
	prometheus.MustRegister(durationHistogram)

	// Replace your-project-name, endpoint, your-project-name, and your-metricstore-name based on your business requirements.
	u := fmt.Sprintf("https://%s.%s/prometheus/%s/%s/api/v1/write", "your-project-name", "endpoint", "your-project-name", "your-metricstore-name")

	for i := 0; i < 300; i++ {
		// Update metric data.
		for j := 0; j < 100; j++ {
			val := rand.Float64() * 10
			durationHistogram.Observe(val)
			requestCount.Inc()
		}

		// Collect all registered metric data.
		mfs, err := prometheus.DefaultGatherer.Gather()
		if err != nil {
			panic(err)
		}

		// Encode the metric data.
		// Note: The metric data must be encoded because the le label +Inf is added to Histogram metrics during data encoding. 
		buf := bytes.NewBuffer(nil)
		encoder := expfmt.NewEncoder(buf, expfmt.NewFormat(expfmt.TypeTextPlain))
		for _, mf := range mfs {
			if err := encoder.Encode(mf); err != nil {
				panic(err)
			}
		}

		// Parse the encoded metric data and convert the parsed data into the data that conforms to the Prometheus Remote Write PB protocol.
		// In this example, an error occurs and the process is directly terminated without handling the error. In actual scenarios, you must select a handling method based on your business requirements. For example, you can skip the error or exit and retry the process.
		request, err := fmtutil.MetricTextToWriteRequest(buf, nil)
		if err != nil {
			panic(err)
		}

		// Serialize and compress the *prompb.WriteRequest object.
		data, _ := proto.Marshal(request)
		bufBody := snappy.Encode(nil, data)
		rwR, err := http.NewRequest("POST", u, ioutil.NopCloser(bytes.NewReader(bufBody)))
		if err != nil {
			panic(err)
		}
		rwR.Header.Add("Content-Encoding", "snappy")
		rwR.Header.Set("Content-Type", "application/x-protobuf")
		// Configure the BasicAuth parameter. You must specify an AccessKey ID and an AccessKey secret. 
		rwR.SetBasicAuth("yourAccessKeyID", "yourAccessKeySecret")

		// Send metric data to the Simple Log Service Metricstore.
		resp, err := http.DefaultClient.Do(rwR)
		if err != nil {
			panic(err)
		}
		d, err := ioutil.ReadAll(resp.Body)
		if err != nil {
			panic(err)
		}
		fmt.Println(string(d))

		time.Sleep(time.Second)
	}

}

Verify the collected data

  1. Log on to the Simple Log Service console. In the Projects section, click the project that you want to manage.

  2. On the Metric Storage > Metricstores tab, click the metricstore that you want to manage.image

  3. In the upper-right corner of the Metricstore page, specify a query time range. Enter a PromQL statement to calculate the 50th percentile of the total duration and click Execute Now. For more information, see PromQL.

    histogram_quantile(0.5, sum by (le) (rate(server_handling_seconds_bucket[1m])))

    image