Correlation analysis functions and their parameter descriptions - Simple Log Service

Quickly identify metrics correlated with a specific metric or time series across multiple observed metrics in your system.

Function list

Function	Description
`ts_association_analysis`	Finds metric names correlated with a specific metric across multiple observed metrics.
`ts_similar`	Finds metric names correlated with a specified time series across multiple observed metrics.

ts_association_analysis

Function format

select ts_association_analysis(stamp, params, names, indexName, threshold)

Parameters:

Parameter	Description	Value
stamp	UNIX timestamp. Data type: long.	-
params	Metric dimensions to analyze. Data type: array(double).	Example: Latency, QPS, and NetFlow.
names	Names of the metrics to analyze. Data type: array(varchar).	Example: Latency, QPS, and NetFlow.
indexName	Name of the target metric to analyze. Data type: varchar.	Example: Latency.
threshold	Correlation threshold between other metrics and the target metric. Data type: double.	Value range: [0, 1].

Outputs

name: Name of the metric.
score: Correlation value between this metric and the target metric. Value range: [0, 1].

Sample code

* | select ts_association_analysis(
              time, 
              array[inflow, outflow, latency, status], 
              array['inflow', 'outflow', 'latency', 'status'], 
              'latency', 
              0.1) from log;

Sample result

| results               |
| --------------------- |
| ['latency', '1.0']    |
| ['outflow', '0.6265'] |
| ['status', '0.2270']  |

ts_similar

Function format 1

select ts_similar(stamp, value, ts, ds)
select ts_similar(stamp, value, ts, ds, metricType)

Parameters for format 1

Parameter	Description	Value
stamp	UNIX timestamp. Data type: long.	-
value	Value of a metric. Data type: double.	-
ts	Time series of the specified curve. Data type: array(double).	-
ds	Value sequence of the specified curve. Data type: array(double).	-
metricType	Correlation type to measure between curves. Data type: varchar.	Valid values: SHAPE, RMSE, PEARSON, SPEARMAN, R2, and KENDALL.

Function format 2

select ts_similar(stamp, value, startStamp, endStamp, step, ds)
select ts_similar(stamp, value, startStamp, endStamp, step, ds, metricType )

Parameters for format 2:

Parameter	Description	Value
stamp	UNIX timestamp. Data type: long.	-
value	Value of a metric. Data type: double.	-
startStamp	Start timestamp of the specified curve. Data type: long.	-
endStamp	End timestamp of the specified curve. Data type: long.	-
step	Time interval between two adjacent points in the time series. Data type: long.	-
ds	Value sequence of the specified curve. Data type: array(double).	-
metricType	Correlation type to measure between curves. Data type: varchar.	Valid values: SHAPE, RMSE, PEARSON, SPEARMAN, R2, and KENDALL.

Outputs

score: Correlation value between this metric and the target metric. Value range: [-1, 1].

Sample code

* | select vhost, metric, ts_similar(time, value, 1560911040, 1560911065, 5, array[5.1,4.0,3.3,5.6,4.0,7.2], 'PEARSON') from log  group by vhost, metric;

Sample result

| vhost  | metric          | score                |
| ------ | --------------- | -------------------- |
| vhost1 | redolog         | -0.3519082537204182  |
| vhost1 | kv_qps          | -0.15922168009772697 |
| vhost1 | file_meta_write | NaN                  |