Load balancing measurement function - Simple Log Service

To implement load balancing in a distributed system, you must first accurately measure the state of its load balancing. This topic describes the basic syntax of the function and provides an example.

Background

The sample logs contain six fields: cluster ID, server ID, time, CPU load, memory load, and network bandwidth load. For more information, see Create an index.

The six fields are configured as follows: cluster_id (text), server_id (text), time_period (text), cpu_load (double), ram_load (double), and network_load (double). All fields are queryable, and a tokenizer is configured for the text fields.

The following are sample logs:

{"cluster_id":"C001","cpu_load":"0.1","network_load":"0.6","ram_load":"0.7","server_id":"S001","time_period":"2024-01-01 00:00:00"}
{"cluster_id":"C001","cpu_load":"0.2","network_load":"0.5","ram_load":"0.8","server_id":"S002","time_period":"2024-01-01 00:01:00"}
{"cluster_id":"C001","cpu_load":"0.1","network_load":"0.6","ram_load":"0.7","server_id":"S001","time_period":"2024-01-01 00:02:00"}
{"cluster_id":"C001","cpu_load":"0.2","network_load":"0.5","ram_load":"0.8","server_id":"S002","time_period":"2024-01-01 00:03:00"}
{"cluster_id":"C001","cpu_load":"0.1","network_load":"0.6","ram_load":"0.7","server_id":"S001","time_period":"2024-01-01 00:04:00"}
{"cluster_id":"C001","cpu_load":"0.2","network_load":"0.5","ram_load":"0.8","server_id":"S002","time_period":"2024-01-01 00:05:00"}
{"cluster_id":"C001","cpu_load":"0.1","network_load":"0.6","ram_load":"0.7","server_id":"S001","time_period":"2024-01-01 00:06:00"}
{"cluster_id":"C001","cpu_load":"0.2","network_load":"0.5","ram_load":"0.8","server_id":"S002","time_period":"2024-01-01 00:07:00"}
{"cluster_id":"C001","cpu_load":"0.1","network_load":"0.6","ram_load":"0.7","server_id":"S001","time_period":"2024-01-01 00:08:00"}
{"cluster_id":"C001","cpu_load":"0.2","network_load":"0.5","ram_load":"0.8","server_id":"S002","time_period":"2024-01-01 00:09:00"}

Load balancing measurement functions

Function name	Syntax	Description	Return type
how_balanced function	how_balanced(array(array(double)) load_matrix)	Measures the degree of load balancing in a distributed system when used with the array_agg function. The return value is in the range `(0,1]`, where a value closer to 1 indicates a more balanced load and 1 represents a perfectly balanced state.	double

how_balanced function

Measures the degree of load balancing in a distributed system with the array_agg function.

double how_balanced(array(array(double)) load_matrix)

Parameter	Description
`load_matrix`	A load matrix where each row represents the time series vector of a single server's load.

Example

Query and analysis statement

* | with server_time_series as
(
    select cluster_id,
        server_id,
        array_agg(to_unixtime(date_parse(time_period, '%Y-%m-%d %H:%i:%s'))) as time_periods,
        array_agg(cpu_load + ram_load + network_load) as metric_values
    from log
    where time_period >= '2024-01-01 00:00:00'
      and time_period < '2024-01-02 00:00:00'
    group by cluster_id, server_id
),
imputed_server_series as
(
    select cluster_id,
        server_id,
        ts_fill_missing(
            time_periods,
            metric_values,
            to_unixtime(date_parse('2024-01-01 00:00:00', '%Y-%m-%d %H:%i:%s')),
            to_unixtime(date_parse('2024-01-02 00:00:00', '%Y-%m-%d %H:%i:%s')),
            '1 minute', 'value=0') as imputed_time_series
    from server_time_series
)
select cluster_id,
    how_balanced(array_agg(imputed_time_series[2])) as balance
from imputed_server_series
group by cluster_id

Results

The returned balance value represents the degree of load balancing on a scale of (0,1]. A value closer to 1 indicates a more balanced load.

Cluster id	Balance
G1	0.5