Generate per-second metrics - ApsaraMQ for RabbitMQ - Alibaba Cloud Documentation Center

This topic describes how to use log management to generate per-second metrics.

Background

The charts from Cloud Monitor display minute-level averages and cannot show per-second TPS data. For ApsaraMQ for RabbitMQ, TPS is the number of AMQP method requests clients initiate per second.

The TPS calculation includes the following AMQP methods:

ConnectionOpen, ChannelOpen
QueueDeclare, QueueDelete, QueueBind, QueueUnbind
ExchangeDeclare, ExchangeDelete
ExchangeBind, ExchangeUnBind
SendMessage, BasicConsume, BasicGet, BasicAck, BasicReject, BasicNack, BasicRecover

For more information about the request methods, see Request methods.

Procedure

Enable log management and configure indexes.
Create a Metricstore to store the cleansed metric data.
1. On the Project details page in the Simple Log Service console, click Create Now. The page then displays The current Project has no Metricstore in an empty state.
2. In the Create Metricstore panel, configure the Metricstore's basic information. Key parameters include Name (the Metricstore name) and data retention period (30 days by default).
Create a data transformation job.
1. In the logstore, enter a query statement. This example queries instance error codes.
```
* | SELECT Code, count(*) as num, microtime / 1000 / 1000 as timeSecond group by Code, timeSecond limit 1000000
```
  The statement format is: Search statement | Analytic statement. The search statement filters the data, and the analytic statement is a standard SQL query. Writing data to a Metricstore requires extracting three components from the query results: the required labels, the metric values for each label, and the timestamp. In this example, Code is the label for the error code, num is the metric value for each code, and timeSecond is the timestamp in seconds.
  
  The query returns a table with three columns: Code (error code), num (count), and timeSecond (per-second timestamp).
2. In the query results, click the Graph tab, and then click Save as Scheduled SQL Job. On the Compute Settings tab, configure the parameters, and then click Next. For SQL Type, select Search & Analysis. For Write Mode, select Import New Data. Configure the Write Column Mapping based on the fields in your SQL query.
  
  Note
  The destination store must be the Metricstore that you created in the previous step.
3. On the Scheduling Settings tab, set the scheduling interval, and then click OK. You can set the scheduling interval to hourly or a custom interval. The default SQL Timeout is 600 seconds, and the default Execution Delay is 60 seconds.
Query the metric value distribution in the Metricstore. In the left-side navigation pane, select the destination Metricstore, go to the query page, and run a query.

A sample query result is as follows:
Optional: Connect the data in the Metricstore to a visualization dashboard. You can use Grafana or the visualization features of Simple Log Service.
- To connect to a Grafana dashboard, see Connect time-series data to Grafana.
- To use the visualization dashboard feature of Simple Log Service, see Visualization.

Note

The preceding tutorial uses instance error code data as an example. You can also cleanse other types of data, such as the message send and receive rate for each channel on each RemoteAddress, the per-second activity of each queue, the total number of messages sent and received per second, and the per-second call count of each API.

Common statements

Instance per-second TPS

* | select microtime/1000/1000 as time, sum(count) as tps 
from 
  (SELECT  microtime, if(Action!='SendMessage', 1, tps) as count 
   from log 
   Where  InstanceId='amqp-xx-xxx' 
     and Action in ('SendMessage', 'ConnectionOpen', 'ChannelOpen', 'ExchangeDeclare', 'QueueBind', 'QueueDeclare', 'QueueDelete', 'ExchangeDelete', 'QueueUnBind', 'ExchangeBind', 'ExchangeUnBind', 'BasicConsume', 'BasicReject', 'BasicRecover', 'BasicAck', 'BasicNAck', 'PullMessage') 
   limit 90000000) 
  
GROUP by time ORDER by time limit 90000000

A sample query result is as follows:

The query returns a table with two columns: time (per-second timestamp) and tps (requests per second). You can switch to the Graph tab to view the trend chart.

Before you run the query, replace the instance ID amqp-xx-xxx with your instance's ID.
For a BasicNack(multiple=false) call, TPS is counted as 1. For a BasicNack(multiple=true) call, TPS is counted as N. Therefore, the TPS value calculated from Simple Log Service logs may be lower than the actual number of requests.
If client traffic is high when you query a TPS chart, limit the time range to 1 hour or less. Also, add limit 90000000, or a limit clause with the largest possible value, to the end of the SQL statement.

Messages sent by exchange

* and Action : SendMessage and Code : 200 | 
select 
  InstanceId as instance_id,
  VHost as virtual_host, 
  split_part(ResourceName,',',2) as exchange_name, 
  split_part(ResourceName,',',3) as routing_key, 
  count(*) as send_total_num 
group by 
  instance_id,
  virtual_host, 
  exchange_name, 
  routing_key 
order by 
  send_total_num 
limit 10000000

A sample query result is as follows:

The result is a table with the following columns: instance_id (instance ID), virtual_host (virtual host), exchange_name (exchange name), routing_key (routing key), and send_total_num (total messages sent).

Send rate by exchange

* and Action : SendMessage and Code : 200 | 
select 
  InstanceId as instance_id,
  VHost as virtual_host, 
  split_part(ResourceName,',',2) as exchange_name, 
  split_part(ResourceName,',',3) as routing_key, 
  microtime / 1000 / 1000 as time_second, 
  count(*) as send_qps 
group by 
  instance_id,
  virtual_host, 
  exchange_name, 
  routing_key,
  time_second 
order by 
  time_second, 
  send_qps 
limit 10000000

A sample query result is as follows:

The result is a table with the following columns: instance_id, virtual_host, exchange_name, routing_key, time_second (per-second timestamp), and send_qps (per-second sending rate).

Consumed messages by queue

* and Action : PushMessage and Code : 200 | 
select 
  InstanceId as instance_id,
  VHost as virtual_host, 
  Queue as queue_name, 
  count(*) as push_total_num 
group by 
  instance_id,
  virtual_host, 
  queue_name 
order by 
  push_total_num 
limit 10000000

A sample query result is as follows:

The result is a table with the following columns: instance_id, virtual_host, queue_name (queue name), and push_total_num (total messages pushed).

Consumption rate by queue

* and Action : PushMessage and Code : 200 | 
select 
  InstanceId as instance_id,
  VHost as virtual_host, 
  Queue as queue_name, 
  microtime / 1000 / 1000 as time_second, 
  count(*) as push_qps 
group by 
  instance_id,
  virtual_host, 
  queue_name, 
  time_second 
order by 
  time_second, 
  push_qps 
limit 10000000

A sample query result is as follows:

The result is a table with the following columns: instance_id, virtual_host, queue_name, time_second (per-second timestamp), and push_qps (per-second consumption rate).

Send rate by client

* and Action : SendMessage and Code : 200 | 
select 
  InstanceId as instance_id,
  VHost as virtual_host, 
  RemoteAddress as client_ip_port, 
  microtime / 1000 / 1000 as time_second, 
  count(*) as send_qps 
group by 
  instance_id,
  virtual_host, 
  client_ip_port, 
  time_second 
order by 
  time_second, 
  send_qps 
limit 10000000

A sample query result is as follows:

The result is a table with the following columns: instance_id, virtual_host, client_ip_port (client IP address and port), time_second, and send_qps (messages sent per second).

Consumption rate by client

* and Action : PushMessage and Code : 200 | 
select 
  InstanceId as instance_id,
  VHost as virtual_host, 
  RemoteAddress as client_ip_port, 
  microtime / 1000 / 1000 as time_second, 
  count(*) as push_qps 
group by 
  instance_id,
  virtual_host, 
  client_ip_port, 
  time_second 
order by 
  time_second, 
  push_qps 
limit 10000000

A sample query result is as follows:

The result is a table with the following columns: instance_id, virtual_host, client_ip_port, time_second, and push_qps (messages consumed per second).

Specific action rate

To query the queries per second (QPS) of a specific action for a client, copy the following statement and replace {action_name} with the action name. Valid Action names include:

ConnectionOpen, ChannelOpen
QueueDeclare, QueueDelete, QueueBind, QueueUnbind
ExchangeDeclare, ExchangeDelete
ExchangeBind, ExchangeUnBind
SendMessage, BasicConsume, BasicGet, BasicAck, BasicReject, BasicNack, BasicRecover

* and Action : {action_name} and Code : 200 | 
select 
  InstanceId as instance_id,
  VHost as virtual_host, 
  RemoteAddress as client_ip_port, 
  microtime / 1000 / 1000 as time_second, 
  count(*) as {action_name}_qps 
group by 
  instance_id,
  virtual_host, 
  client_ip_port, 
  time_second 
order by 
  time_second, 
  {action_name}_qps 
limit 10000000

For example, to query the QPS of ConnectionOpen actions on a client, use the following statement:

* and Action : ConnectionOpen and Code : 200 | 
select 
  InstanceId as instance_id,
  VHost as virtual_host, 
  RemoteAddress as client_ip_port, 
  microtime / 1000 / 1000 as time_second, 
  count(*) as connection_open_qps 
group by 
  instance_id,
  virtual_host, 
  client_ip_port, 
  time_second 
order by 
  time_second, 
  connection_open_qps 
limit 10000000

A sample query result is as follows:

The result is a table with the following columns: instance_id, virtual_host, client_ip_port, time_second, and connection_open_qps (ConnectionOpen requests per second).

QPS by action

This statement gathers the QPS statistics for all actions on each client at once.

* and Code : 200 | 
select 
  InstanceId as instance_id,
  VHost as virtual_host,
  Action as action_type,
  RemoteAddress as client_ip_port, 
  microtime / 1000 / 1000 as time_second, 
  count(*) as action_qps
group by 
  instance_id,
  virtual_host,
  client_ip_port,
  action_type,
  time_second 
order by
  time_second, 
  action_qps
limit 10000000

A sample query result is as follows:

The result is a table with the following columns: instance_id, virtual_host, client_ip_port, action_type (the type of action, such as ExchangeDeclare or ConnectionOpen), time_second, and action_qps.

Error frequency

* and not Code = 200 | 
select 
  Code as error_code,
  VHost as virtual_host,
  split_part(split_part(Info, '[', 1), 'Req', 1) as error_info,
  microtime / 1000 / 1000 as time_second,
  count(*) as error_num
group by 
  virtual_host,
  error_code,
  time_second,
  error_info
order by
  time_second, 
  error_num
limit 10000000

A sample query result is as follows:

The result is a table with the following columns: error_code (error code), virtual_host, error_info (error description), time_second, and error_num (number of errors).

Average message body size

* and Action : SendMessage and Code: 200 | 
select 
  InstanceId as instance_id, 
  VHost as virtual_host, 
  split_part(Queue, ';', 1) as queue_name, 
  microtime / 1000 / 1000 as time_second, 
  avg(cast(split_part(ResourceName, 'bodySize=', 2) as bigint)) as avg_body_size 
group by 
  instance_id, 
  virtual_host, 
  queue_name, 
  time_second 
order by 
  time_second, 
  avg_body_size 
limit 10000000

A sample query result is as follows:

The result is a table with the following columns: instance_id, virtual_host, queue_name, time_second, and avg_body_size (average message body size in bytes).

Push attempts by message

* and Action : PushMessage and Code : 200 | 
select 
  InstanceId as instance_id, 
  VHost as virtual_host, 
  split_part(split_part(ResourceName, ',', 1), '=', 2) as msg_id, 
  count(*) as push_times 
group by 
  instance_id, 
  virtual_host, 
  msg_id 
order by 
  push_times desc 
limit 1000000

A sample query result is as follows:

The result is a table with the following columns: instance_id, virtual_host, msg_id (message ID), and push_times (number of push attempts). The results are sorted in descending order by the number of push attempts.