Prometheus is a time series database that is developed and maintained by SoundCloud. It is the open source implementation of BorgMon, the monitoring system of Google. Prometheus Reader allows you to read data from Prometheus.

Notice

How it works

Prometheus Reader connects to a Prometheus instance by sending an HTTP request. Then, Prometheus Reader obtains raw data points by using the HTTP API operation /api/v1/query_range. A sync node is split to multiple export tasks based on the metric and time range.

Limits

  • The specified start time and end time are automatically converted to on-the-hour time. For example, if you set the time range to [3:35, 4:55) on April 18, 2019, the time range is converted to [3:00, 4:00).
  • Prometheus Reader supports only Prometheus 2.9.x.
  • The default time range for reading data is 10s.

    The /api/v1/query_range operation allows you to query only a limited number of data points. If you specify an excessively large time range, the following error message is returned: exceeded maximum resolution of 11,000 points per timeseries. Therefore, Prometheus Reader obtains data at the time range of 10s by default. Even if the raw data points are stored by millisecond, a maximum of 10,000 data points can be queried by using the/api/v1/query_range operation.

Data types

Category Data Integration data type Prometheus data type
String STRING String to which a data point in Prometheus is serialized, including the timestamp, metric, tags, and value

Parameters

Parameter Description Required Default value
endpoint The HTTP endpoint of Prometheus, in the format of http://IP address:Port. Yes N/A
column The metrics to be synchronized. Yes N/A
beginDateTime The start time of the time range of the data points to be synchronized, in the format of yyyyMMddHHmmss. Specify the beginDateTime and endDateTime parameters to determine the time range of the data points to be synchronized. Yes N/A
Note The start time and end time of the time range are automatically converted to on-the-hour time. For example, if you set the time range to [3:35, 4:55) on April 18, 2019, the time range is converted to [3:00, 4:00).
endDateTime The end time of the time range of the data points to be synchronized, in the format of yyyyMMddHHmmss. Specify the beginDateTime and endDateTime parameters to determine the time range of the data points to be synchronized. Yes N/A
Note The start time and end time of the time range are automatically converted to on-the-hour time. For example, if you set the time range to [3:35, 4:55) on April 18, 2019, the time range is converted to [3:00, 4:00).

Configure Prometheus Reader by using the codeless UI

The codeless user interface (UI) is not supported for Prometheus Reader.

Configure Prometheus Reader by using the code editor

The following example shows how to configure a sync node to read data from a Prometheus database in the code editor. For more information, see Create a sync node by using the code editor.
{
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    },
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "concurrent": 1,
            "throttle": true
        }
    },
    "steps": [
        {
            "category": "reader",
            "name": "Reader",
            "parameter": {
                "endpoint": "http://localhost:9090",
                "column": [
                    "up"
                ],
                "beginDateTime": "20190520150000",
                "endDateTime": "20190520160000"
            },
            "stepType": "prometheus"
        },
        {
            "category": "writer",
            "name": "Writer",
            "parameter": {},
            "stepType": ""
        }
    ],
    "type": "job",
    "version": "2.0"
}

Performance test report

Number of channels Data integration speed (records per second) Data integration bandwidth (Mbit/s)
1 45,000 5.36
2 55,384 6.60
3 60,000 7.15