This topic describes the Resource Manager feature of PolarDB for PostgreSQL.
Applicability
The following versions of PolarDB for PostgreSQL are supported:
PostgreSQL 18 (minor engine version 2.0.18.0.1.0 or later)
PostgreSQL 17 (minor engine version 2.0.17.2.1.0 or later)
PostgreSQL 16 (minor engine version 2.0.16.3.1.1 or later)
PostgreSQL 15 (minor engine version 2.0.15.7.2.0 or later)
PostgreSQL 14 (minor engine version 2.0.14.5.1.0 or later)
PostgreSQL 11 (minor engine version 2.0.11.2.1.0 or later)
You can check the minor engine version in the console or by running the SHOW polardb_version; statement. If your minor engine version does not meet the requirements, you must upgrade the minor engine version.
Background information
The memory of a PolarDB for PostgreSQL cluster is divided into the following three parts:
Shared memory
Dynamic shared memory
Private memory
Dynamic shared memory and private memory are dynamically allocated. Their usage fluctuates based on the cluster's workload. Excessive usage of dynamic and private memory can cause the total memory usage to exceed the limits of the operating system. This triggers the memory-limiting mechanism of the kernel, which can cause cluster processes to exit unexpectedly, the cluster to restart, or the cluster to become unavailable.
The memory managed by MemoryContext in private memory can be divided into the following two parts:
Work computation area memory is the memory required for the normal operation of your services.
Cache memory: Used by the database process to cache internal metadata. The usage of this memory only affects database performance.
To address these issues, PolarDB for PostgreSQL includes the Resource Manager feature. This feature periodically checks resource usage while the cluster is running. If a process exceeds the resource threshold, Resource Manager enforces a limit on that process to reduce the risk of cluster unavailability.
Resource Manager can limit memory, CPU, and I/O resources. However, it currently supports only memory limits.
How it works
Memory limiting relies on cgroups. If cgroups are not available, resource limits cannot be effectively applied. Resource Manager is a background daemon process for PolarDB for PostgreSQL. It periodically reads memory usage data from cgroups to determine when to apply memory limits. When Resource Manager detects that the cluster's memory usage has exceeded the threshold, it reads the kernel's memory accounting for user processes. It then sorts the processes by memory usage and sends an interrupt signal (SIGTERM) or a cancel signal (SIGINT) to the processes that consume the most memory, starting with the largest consumer.
Memory limit method
The Resource Manager daemon process starts with the cluster and is active on the primary (RW), read-only (RO), and standby nodes. You can change the behavior of Resource Manager by modifying the following parameters.
Resource Manager limits memory as follows: when the cluster's memory usage exceeds the watermark level defined by the Resource Manager parameters, it sends a SIGTERM signal to processes that are consuming a large amount of memory. This terminates the processes and releases memory. The following table describes the relevant parameters.
Parameter | Description |
polar_resource_manager.enable_resource_manager | Specifies whether to enable Resource Manager. Valid values:
|
polar_resource_manager.stat_interva | The interval for periodic checks of resource usage. Unit: milliseconds. Value range: 10 to 10000. Default value: 500. |
polar_resource_manager.total_mem_limit_rate | The percentage of cluster memory usage that acts as a limit. When cluster memory usage exceeds this percentage, memory resources are forcibly limited. Default value: 95. |
polar_resource_manager.total_mem_limit_remain_size | The amount of reserved memory for the cluster. When the available cluster memory drops below this value, memory resources are forcibly limited. Unit: KB. Value range: 0 to MAX_KILOBYTES (the maximum integer value). Default value: 262144. |
polar_resource_manager.mem_release_policy | The policy for limiting memory resources. Valid values:
|
Example
When a user session process receives a SIGTERM signal, the process is terminated. Information about the termination is written to the log. The following is a sample log output:
2022-11-28 14:07:56.929 UTC [18179] LOG: [polar_resource_manager] terminate process 13461 release memory 65434123 bytes
2022-11-28 14:08:17.143 UTC [35472] FATAL: terminating connection due to out of memory
2022-11-28 14:08:17.143 UTC [35472] BACKTRACE:
postgres: primary: postgres postgres [local] idle(ProcessInterrupts+0x34c) [0xae5fda]
postgres: primary: postgres postgres [local] idle(ProcessClientReadInterrupt+0x3a) [0xae1ad6]
postgres: primary: postgres postgres [local] idle(secure_read+0x209) [0x8c9070]
postgres: primary: postgres postgres [local] idle() [0x8d4565]
postgres: primary: postgres postgres [local] idle(pq_getbyte+0x30) [0x8d4613]
postgres: primary: postgres postgres [local] idle() [0xae1861]
postgres: primary: postgres postgres [local] idle() [0xae1a83]
postgres: primary: postgres postgres [local] idle(PostgresMain+0x8df) [0xae7949]
postgres: primary: postgres postgres [local] idle() [0x9f4c4c]
postgres: primary: postgres postgres [local] idle() [0x9f440c]
postgres: primary: postgres postgres [local] idle() [0x9ef963]
postgres: primary: postgres postgres [local] idle(PostmasterMain+0x1321) [0x9ef18a]
postgres: primary: postgres postgres [local] idle() [0x8dc1f6]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f888afff445]
postgres: primary: postgres postgres [local] idle() [0x49d209]