RDS Custom system events are defined by Alibaba Cloud to record and notify cloud resource information, such as O&M task execution status, resource exceptions, and resource status changes. You can obtain risk and exception information about RDS Custom resources through system events, such as instance migration due to underlying upgrades, or instance restart due to system maintenance. You can then respond to and handle system events promptly to prevent business impact caused by reduced availability or performance of RDS Custom resources.
Use cases
Notification for risks and exceptions
Alibaba Cloud displays system events (such as system maintenance restarts, instance expiration, and other events that may affect resource availability and performance) in the RDS Custom console. Some important system events are also sent through emails, and internal messages. You can respond to these events in the RDS Custom console or through OpenAPI. We recommend that you respond to system events promptly to avoid business impact.
For example, when a subscription instance is about to stop due to expiration, the RDS Custom console prominently reminds you to renew the instance to avoid business impact caused by instance stoppage due to expiration.
Automated O&M
System events displayed in the RDS Custom console have defined event states, making it easy to distinguish the execution status of corresponding system O&M tasks. Changes in the status of system events are synchronized to Cloud Monitor to help you build an automated O&M mechanism. For more information about event states, see System event states of RDS Custom.
NoteEach event state corresponds to a Cloud Monitor event name. For example, the RDS Custom event code InstanceFailure.Reboot supports the event states Executing and Executed, which correspond to the Cloud Monitor event names Instance:InstanceFailure.Reboot:Executing and Instance:InstanceFailure.Reboot:Executed.
Status change events include some system events that are not displayed in the RDS Custom console, such as instance running status changes and preemptible instance interruptions. These system events cannot be directly responded to in the RDS Custom console or through OpenAPI. RDS Custom does not define event states for these system events, but when these system events occur, they are still reported to Cloud Monitor, allowing you to build an event-driven automated O&M system based on your needs.
For example, status change events are triggered when you start or stop ECS instances. The events do not indicate risks or exceptions. If you want to log your operations to your system, you can configure event notifications for status change events and use the alert callback feature to write the startup and stop information of instances to operation logs.
System event types
System events can be classified into the following categories based on their causes.
Category | Description | Displayed in RDS Custom Event Center |
Scheduled O&M events | Alibaba Cloud may need to upgrade host software for security reasons or to predict and handle failure risks that lie in underlying host hardware and software. If the execution of corresponding O&M tasks may affect the availability of RDS Custom resources or cause performance degradation, Alibaba Cloud will inform you in advance about the execution time, objects, impacts, and other information. You can respond to these system events during their scheduled execution time, choosing to handle them during periods of low business activity to avoid impact during peak periods. Note Scheduled O&M events, also known as proactive O&M events, are based on the O&M experience of Alibaba Cloud on millions of servers, the ability to serve tens of thousands of large enterprise customers, and the cutting-edge machine learning algorithms of Alibaba DAMO Academy to predict and handle failure risks that lie in underlying host hardware or software. When host failure risks cannot be avoided, Alibaba Cloud notifies affected RDS Custom users in advance through scheduled O&M events, providing users with a buffer time for business switching. If users do not respond to scheduled O&M events in advance, when failure risks occur, RDS Custom instances may become unavailable or restart. | Yes |
Unexpected O&M events | This category of system events is triggered when ECS instances restart or become unavailable due to unexpected issues such as kernel panic, out-of-memory errors, or hardware or software failures in underlying hosts. Alibaba Cloud sends unexpected O&M events promptly, restores the availability of RDS Custom resources as soon as possible, and notifies you of the execution status of corresponding system O&M tasks. Note Unexpected O&M events generally refer to events where RDS Custom instances suddenly become unavailable or restart due to unpredictable failures in the underlying host or issues such as kernel errors in the operating system of the RDS Custom instance.
| Yes |
System event severities
System events are assigned the following severities based on their impacts on the normal operation of instances:
Critical: Critical system events may result in instance unavailability and must be handled at your earliest opportunity. For example, a critical system event is triggered when resources are released due to an overdue payment or when an instance is redeployed due to an instance error.
Warning: Warning system events have impact on your business. For example, a warning system event is triggered when a burstable instance cannot burst above its performance baseline. You must pay close attention to these events or handle them when appropriate.
Information: You can choose whether to pay attention to these events. For example, a disk snapshot has been created.
States and windows of system events
System events displayed in the console define event states as described in the following table.
State | Property | Description |
Inquiring | Transitory | The system is inquiring and waiting for your authorization. After authorization, the event enters the Executing state. |
Scheduled | Transitory | The O&M task is scheduled for execution but has not yet started. When execution begins, the event enters the Executing state. |
Executing | Transitory | The O&M task related to the system event is being executed. |
Executed | Steady | The O&M task related to the system event is completed. |
Avoided | Steady | The impacts of the system event are prevented because you have migrated the affected instance within the user operation window. |
Failed | Steady | The O&M task related to the system event failed. |
Canceled | Steady | The O&M task related to the system event is automatically canceled. |
The following figure shows the typical transitions between event states.
System events include the following windows.
User operation window
The user operation window of a system event starts when the event is sent and ends at the time when the related O&M task is executed as scheduled. You can manually handle the event within the user operation window or wait for the system to automatically handle the event. Take note of the following items about the lengths of user operation windows:
In most cases, the user operation window of a scheduled O&M event ranges from 24 to 48 hours.
NoteThe lengths of user operation windows are unlimited for system events in the Inquiring state. The O&M tasks related to the events can start only after you authorize the tasks to be executed.
In most cases, unexpected O&M system events caused by failures or unauthorized operations do not have a user operation window.
For system events indicating that subscription instances are about to expire, the window is 3 days.
For system events indicating that pay-as-you-go instances are about to be stopped due to overdue payments, the window is less than 1 hour.
Event execution window
The execution window of a system event starts when the related O&M task is executed and ends when the task is completed. Take note of the following items about the lengths of event execution windows:
For system events such as failure recovery events, the window is within 10 minutes.
Unexpected O&M events caused by failures or unauthorized operations have a short event execution window.
Operations
Operation | Description and reference |
Understand system events | To learn about system events and understand the event names, severities, usage scenarios, limits, states, and name formats, see this topic. |
View system events |
|
Handle system events | For some important system events (such as events that affect the availability of RDS Custom resources or cause performance degradation), we recommend that you promptly respond through the console or OpenAPI and handle them according to recommendations to avoid affecting business operations. |
Monitor system events | To ensure the stability of business operations on RDS Custom instances and implement automated O&M, we recommend that you configure event notifications to monitor changes in the underlying environment. After you configure event notifications, the system uses the notification methods that you specify to send you notifications. To configure alert rules through CloudMonitor and push event notifications, see Use CloudMonitor to subscribe to RDS Custom event notifications. |