Message Service:Log management

Scenarios

The log management feature can be applied in the following scenarios:

• Track the messages sent to a message queue if consumer clients cannot receive the messages.
• Identify the clients that have consumed a message and check the number of times that the message is consumed.
• Determine when a message can be consumed again if it fails to be consumed due to consumer client breakdown.
• Identify the cause if endpoints cannot receive messages that are published to a topic.
• View operations logs that are generated for messages a month ago.

In these scenarios, you can solve the problems by using the log management feature of MNS.

• Push logs to Log Service and view operations logs of messages in the Log Service console.
• Specify the parameters to view operations logs of messages by using the log query tool provided by Alibaba Cloud.
• Log on to the OSS console and configure the LifeCycle parameter of the logging bucket. You can even view the operations logs that are generated during the past year.
• In addition to the official query tool, you can also call the GetObject API operation of OSS to download log files.

Push logs to Log Service

• For more information about the configuration method, see Push logs to Log Service.
• For more information about how to view logs, see View MNS logs in the Log Service console.
• When you create a Logstore, you must specify an appropriate data retention period. If you want to modify the data retention period after the Logstore is created, you can specify only a shorter data retention period.
• If you delete the project or Logstore that corresponds to a logging bucket, or revoke the permissions that are granted to MNS to access Log Service, MNS logs cannot be pushed to Log Service.
• Logs can be pushed from MNS to Log Service about five minutes after the logs are generated.

Push logs to OSS

• For more information about configuration methods, see Push logs to OSS.
• For more information about how to view logs, see View MNS logs in the OSS console.
• If you delete the OSS bucket that corresponds to a logging bucket, or cancel the permissions that are granted to MNS to access OSS, MNS logs cannot be pushed to OSS.
• Logs can be pushed from MNS to Log Service about 15 minutes after the logs are generated.

Analysis of details

• Each region has a logging bucket. All operations logs of messages in the queues and topics for which the logging feature is enabled are pushed to the logging bucket.
• You can enable the logging feature for queues or topics based on your needs. The logging feature is disabled by default.

Billing

• You are not charged for using the log management feature of MNS.
• However, you are charged for the memory usage, transferred data, and number of requests when you use MNS to push logs to OSS or Log Service. For more information, see Overview of Log Service pricing and Overview of OSS pricing.
• The size of generated logs depends on queries per second (QPS) of the MNS server and types of operations. For example, if MNS calls the SendMessage API operation 1,000 times per second, the size of generated log entries is about 10 MB (178 bytes × 1000 × 60 / 1024 / 1024 ≈ 10 MB). In this formula, 178 bytes indicates the size of a log that is generated by a single SendMessage operation.

Operation logs of queue messages

Operation logs of queue messages are generated when operations are performed on queue messages, for example, when you send, consume, and delete messages. Each operations log contains multiple fields that indicate different information. The fields contained in a log vary based on different operations. The following tables describe the log fields and the relationships between the operation types and log fields.

• Log fields

  Each operations log contains multiple fields. The following table describes the fields.

  Field	Description
  Time	The time when the operation is performed.
  MessageId	The ID of the message involved in the operation.
  QueueName	The ID of the queue involved in the operation.
  AccountId	The Alibaba Cloud account or RAM user to which the queue belongs.
  RemoteAddress	The endpoint of the client that performs the operation.
  NextVisibleTime	The time when the message is visible after the operation is complete.
  ReceiptHandleInRequest	The value of the ReceiptHandle parameter that is passed into the request when the operation is performed.
  ReceiptHandleInResponse	The value of the ReceiptHandle parameter that is returned after the operation is complete.
  ProcessTime	The duration in which the operation is performed.
  RequestId	The ID of the operation.
  Action	The specific action of the operation. Valid values: Delete and Send.

• Fields of different operations

  Logs that are generated by different operations contain different fields. The following table shows the log fields of different operations.

  Operation	Time	QueueName	AccountId	MessageId	RemoteAddress	NextVisibleTime	ReceiptHandleInResponse	ReceiptHandleInRequest
  SendMessage/BatchSendMessage	Yes	Yes	Yes	Yes	Yes	Yes	No	No
  PeekMessage/BatchPeekMessage	Yes	Yes	Yes	Yes	Yes	No	No	No
  ReceiveMessage/BatchReceiveMessage	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No
  ChangeMessageVisibility	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
  DeleteMessage/BatchDeleteMessage	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes

Operation logs of topic messages

Operation logs of topic messages are generated when operations are performed on topic messages, for example, when messages are published and pushed. The following tables describe the log fields and the relationships between the operation types and log fields.

• Log fields

  Each operations log contains multiple fields. The following table describes the fields.

  Field	Description
  Time	The time when the operation is performed.
  MessageId	The ID of the message involved in the operation.
  TopicName	The ID of the topic involved in the operation.
  SubscriptionName	The ID of the subscription involved in the operation.
  AccountId	The Alibaba Cloud account or RAM user to which the topic belongs.
  RemoteAddress	The endpoint of the client that performs the operation.
  NotifyStatus	The status code or error message returned when MNS pushes a message to a client.
  ProcessTime	The duration in which the operation is performed.
  MessageTag	The tags of the message.
  RequestId	The ID of the operation.
  Action	The specific action of the operation. Valid values: Delete and Send.

• Fields of different operations

  Logs that are generated by different operations contain different fields. The following table shows the log fields of different operations.

  Operation	Time	MessageId	TopicName	SubscriptionName	AccountId	RemoteAddress	NotifyStatus	SubscriptionName
  PublishMessage	Yes	Yes	Yes	No	Yes	Yes	No	No
  Notify	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes

• NotifyStatus

  NotifyStatus is a field that identifies the reason why MNS fails to push messages to endpoints. The following table describes the error codes and the corresponding solutions.

  Error code	Description	Recommended solution
  2xx	The message is pushed.	N/A
  Other HTTP status codes	After the message is pushed to the endpoint, an HTTP status code other than 2xx is returned.	Check the processing logic of the endpoint.
  InvalidHost	The specified endpoint in the subscription is invalid.	Use the curl or telnet command to check whether the specified endpoint in the subscription is valid.
  ConnectTimeout	MNS failed to connect to the specified endpoint in the subscription.	Use the curl or telnet command to check whether the specified endpoint in the subscription is accessible.
  ConnectFailure	MNS failed to connect to the specified endpoint in the subscription.	Use the curl or telnet command to check whether the specified endpoint in the subscription is accessible.
  UnknownError	An unexpected error has occurred.	Contact the MNS technical support.