×
Community Blog What Kind of PostgreSQL Do We Need When 80% of Databases Are Automatically Created by AI?

What Kind of PostgreSQL Do We Need When 80% of Databases Are Automatically Created by AI?

This article introduces Alibaba Cloud RDS for PostgreSQL as the core multi-tenant, multi-modal data foundation for AI agents.

By Yu Peng (Zhao Yupeng)

Foreword: What Agents Need Is More Than Just a Vector Database

Agent applications are redefining the role of databases.

In the past, discussions about AI applications tended to focus on model invocations, vector search, RAG, and prompt engineering. But once you move into production, it becomes clear that a vector database is just one piece of an agent's data architecture. An agent also needs to manage task state, permission boundaries, execution results, and exception logs—while confronting new risk vectors such as prompt injection and data poisoning. (Research from arXiv 2503.05445 shows that as little as 0.44% poisoned data can achieve a 79.41% success rate for database-targeted poisoning attacks.)

From an industry perspective, PostgreSQL has become the mainstream choice for AI and agent scenarios. The official announcement of Databricks' acquisition of Neon disclosed that over 80% of databases on the Neon platform are now created automatically by AI agents—a figure that was just 0.1% in October 2023. In less than two years, AI is rewriting how databases are provisioned.

Meanwhile, the Stack Overflow 2025 Developer Survey shows PostgreSQL adoption has reached 55.6%, up 7 percentage points year over year, making it the most widely used database among developers.

1

This brings us back to the opening question—when 80% of databases are automatically created by AI, what kind of PostgreSQL do we actually need? Drawing on practical experience with agent applications at Alibaba Cloud, we identify three new requirements that agent scenarios impose on the data layer: multi-tenant isolation, second-level data branching, and multi-modal data management. These three directions form the core topics of the sections that follow.

As the core product in Alibaba Cloud ApsaraDB's portfolio for transactional agent scenarios, ApsaraDB RDS for PostgreSQL (hereinafter "RDS for PostgreSQL") builds on PostgreSQL's long-standing strengths in transactions, ecosystem, and extensibility, delivering production-grade enhancements tailored to these scenarios.

Database Fundamentals: RDS for PostgreSQL's Comprehensive Advantages

An agent system must first and foremost manage state reliably.

A single agent request is typically decomposed into multiple steps that invoke different tools. The database needs to track task progress, execution results, and retry state to prevent the same operation from being triggered twice. At the same time, agents continuously generate prompts, model responses, tool parameters, trace metadata, and other semi-structured data—data that becomes the first line of evidence for failure recovery, troubleshooting, and daily operations.

PostgreSQL codifies these rules at the database layer: transaction and concurrency control prevent state conflicts; unique constraints, foreign keys, and check constraints block invalid writes; jsonb flexibly stores semi-structured data with index-backed queries; DDL can be wrapped in transactions and rolled back; and WAL, PITR, and replication capabilities provide data recovery and disaster recovery safeguards.

RDS for PostgreSQL is built on a cloud-native high-availability architecture that is fully compatible with the open-source PostgreSQL ecosystem. Through backup and recovery, monitoring and alerting, failover, online major engine version upgrades, and other enterprise-grade capabilities, it delivers a stable and reliable database service. It also provides cross-zone high availability and point-in-time recovery (PITR), helping users build a highly available, observable, and easily manageable data layer for agent applications.

The next three sections focus on three key RDS for PostgreSQL enhancements for agent scenarios: multi-tenant isolation, data branching, and multi-modal retrieval.

Multi-Tenancy: Tenant Isolation for AI Databases

Agent applications will almost inevitably evolve into SaaS architectures.

An agent platform may simultaneously serve multiple enterprises, multiple teams, and multiple users. Take overseas vibe coding platforms as an example: Lovable has disclosed that by mid-2025, its platform creates over 100,000 new user projects per day, each requiring its own database, authentication, and storage. Domestic agent platforms are facing equally rapid growth in tenant scale. Data between different tenants must be isolated, permission boundaries must be clearly defined, and access behavior must be auditable.

PostgreSQL supports multiple multi-tenancy patterns, including shared table + tenant_id, schema per tenant, and database per tenant. On the permission side, isolation can be achieved through roles, grants, default privileges, object-level permissions, and Row-Level Security (RLS)—covering databases, schemas, tables, columns, functions, sequences, and other objects to enable fine-grained access control. Lightweight scenarios can use shared tables or dedicated schemas; enterprise-grade agent platforms are better served by database per tenant—each tenant gets an independent database with clearer isolation boundaries, and backup/recovery, migration/upgrades, resource accounting, and fault isolation all operate at the tenant level.

For the typical SaaS multi-tenant architecture of enterprise-grade agent platforms, RDS for PostgreSQL provides the following enhancements:

• Cache eviction optimization: For scenarios with large numbers of tables and multi-tenant access patterns, RDS for PostgreSQL optimizes system cache management, reducing the performance impact of cache bloat and frequent invalidation in multi-table, multi-database scenarios.

• PgBouncer connection pooling: For scenarios with large numbers of tenants and short-lived connections, PgBouncer reduces backend connection counts and process overhead through connection reuse, improving connection management efficiency under high-concurrency access.

• SQL throttling: For multi-tenant shared instance scenarios, SQL throttling enforces access control on high-concurrency, heavy-load, or resource-intensive SQL statements, reducing the impact of individual tenants or request types on overall instance stability.

• Fast major engine version upgrades: For multi-database scenarios such as database per tenant, this capability reduces the time cost of upgrading databases one by one, improving major version upgrade efficiency for multi-tenant instances.

• Database per tenant isolation enhancements: For the database per tenant model, RDS for PostgreSQL supports restricting regular tenants to see only their own databases and roles, reducing the risk of cross-tenant information exposure and helping users build a secure, stable, and manageable multi-tenant agent platform.

From startup agent applications to enterprise-grade agent platforms, multi-tenancy patterns can be freely combined and upgraded on demand within a single RDS for PostgreSQL instance.

2

Data Branching: Sandbox and Experimentation for AI Databases

In the agent era, data environments must emphasize replicability, experimentation, and rollback capability.

Many agent tasks are not suitable for directly modifying production data. A safer approach is to first create a sandbox environment where the agent can execute SQL or code, compare the before-and-after state, and then decide whether to apply changes to production. The higher the degree of automation, the more critical this "dry-run first, then confirm" capability becomes. Data branching is equally applicable to development testing, canary verification, rollback recovery, and migration drills—and can provide isolated workspaces for different agents.

The traditional dump-restore approach degrades from seconds to minutes or even hours as data volumes grow, unable to keep pace with the high-frequency "create—use—destroy" iteration rhythm that agents demand. PostgreSQL 18's new file_copy_method parameter, combined with copy-on-write file systems such as XFS reflink, Btrfs, and APFS, reduces CREATE DATABASE to a file clone operation—near-instantaneous completion with near-zero initial disk footprint.

RDS for PostgreSQL builds on file system cloning and Alibaba Cloud ESSD snapshot copy-on-write architecture to deliver second-level CREATE DATABASE through its data branching capability. Branch creation time drops from minutes to seconds, initial disk footprint is near zero, and billing is based only on changed data blocks. This capability is well suited for agent sandboxes, canary verification, rollback points, and tenant-level test databases. Combined with database per tenant, platforms can perform finer-grained testing, recovery, and upgrades for individual tenants—making the high-frequency "create—use—destroy" iteration rhythm a reality in production environments.

3

Multi-Modal: A Unified Data Gateway for AI Databases

The data that agents process is no longer just text.

An agent may need to handle business tables, documents, images, audio, video, geolocation data, time series, and vectors—going beyond "retrieving a piece of text" to understanding the relationships among these data types and executing actions within the boundaries of its permissions.

PostgreSQL is well suited to manage the metadata, indexes, permissions, and relationships for all of these: regular tables store business objects and state, jsonb stores model inputs/outputs and tool parameters, full-text search handles document content, pgvector stores embeddings, and the Apache AGE graph extension manages entity relationships.

For vector search scenarios, RDS for PostgreSQL builds on open-source pgvector with deep optimizations along key paths including index compression, query acceleration, and executor architecture, delivering the following enhancements:

• IVFFlat RaBitQ quantization: Applying RaBitQ quantization to IVFFlat indexes, achieving 32× compression while improving query performance by 20×—enabling large-scale vector search that balances lightweight storage with fast response.

• HNSW multi-bit RaBitQ quantization: Addressing the high memory overhead of HNSW indexes, this multi-bit RaBitQ quantization scheme achieves 4× compression with a further 50% query performance improvement, balancing memory efficiency and query speed in high-recall scenarios.

• Vector search kernel optimization: Fine-grained tuning of vector search query execution paths and index build processes, eliminating redundant overhead on critical paths to improve overall throughput and reduce response latency.

• Hybrid vector-scalar search: Optimizing the PostgreSQL executor framework to support hybrid search combining vector similarity with scalar filtering—completing "semantic similarity + condition filtering" compound search in a single query, improving query expressiveness and result accuracy.

Taking a ten-million-scale vector index as an example: compared to a standard HNSW index, index size is compressed from 689 GB to 16 GB (32× compression), index build time drops from 4 days 1 hour to 4 hours 23 minutes (23× speedup), write latency decreases from 469 ms to 6.4 ms (73× improvement), Top-100 query latency drops from 1,789 ms to 359 ms (5× speedup), recall loss is kept within 1%, and vector dimensions up to 16,000 are supported—covering multi-modal embedding scenarios.

4

For graph data and relational analysis scenarios, RDS for PostgreSQL builds on the community Apache AGE project with the following enhancements:

• Graph query and DML optimization: Performance optimization and engineering enhancements for graph query and DML hot paths—reducing redundant projections and execution overhead, and optimizing vertex/edge update/delete paths—to deliver 100× query performance and 10× DML performance improvements while maintaining openCypher compatibility. Active contributions of features back to the community continue.

• Graph RAG and ontology modeling: Supporting Graph RAG and ontology-based agent knowledge modeling—unifying structured data, entity relationships, semantic networks, rule dependencies, and inference chains within RDS for PostgreSQL to provide agents with a searchable, explainable, updatable, and reasonnable enterprise-grade data foundation.

• Industry scenario adoption: Graph capabilities have been validated in production across intelligent driving data mining, financial and operational analysis, retail business diagnostics, real estate data governance, and intelligent O&M fault diagnosis—helping enterprises transform objects and relationships scattered across business systems, data warehouses, log platforms, and configuration centers into agent-understandable, agent-reasonnable business semantic networks.

5

Closing: RDS for PostgreSQL as the Core Data Foundation for Agents

The agent era calls for a new core data foundation.

In summary, agent applications require both reliable transactional consistency and operational capabilities, as well as extended capabilities for multi-tenancy, data branching, and multi-modal data management. On top of PostgreSQL's foundational strengths, RDS for PostgreSQL further enhances multi-tenant isolation, data branching, vector search, and graph relationship analysis—making it a compelling choice for AI agent applications on the cloud.

From data foundation to agent development platform. But a data foundation is not the endpoint. Production-grade agent applications need a complete development platform, not just a database. Alibaba Cloud RDS has launched AI agent development capabilities built on PostgreSQL—using RDS PostgreSQL as the data foundation and providing an open, efficient, one-stop development environment with three core capabilities: Backend as a Service (BaaS), Knowledge Graph, and Agent Runtime. We call these three companion components the "Agent Trio":

• Supabase—the application-layer BaaS backend, providing Auth, Storage, Edge Functions, and Realtime modules so that full-stack applications generated by agents can go live directly without deploying an additional data access layer;

• Knowledge Graph—the semantic-layer knowledge hub, providing Knowledge Base RAG, Ontology modeling, and Skill Hub capability accumulation—enabling agents to understand business relationships, enforce behavioral constraints, and replace probabilistic guessing with deterministic reasoning;

• In-DB Agent Runtime—the execution-layer runtime environment, with Sandbox and Edge Function dual modes covering long-running tasks and event-triggered tasks, giving agents the ability to execute actions close to the data.

The Trio is built entirely on RDS for PostgreSQL—no data migration, no SQL dialect switching, no mental model drift. An agent team can cover vector search, graph relationships, object storage, function compute, authentication, and authorization through a single RDS for PostgreSQL instance, replacing multiple independent components in traditional architectures and fundamentally eliminating cross-system synchronization overhead and consistency burdens.

This is our vision of PostgreSQL for the AI agent era—a cloud-native data foundation that natively handles multi-tenancy, data branching, and multi-modal data.

6

A detailed analysis of the Trio's capabilities and collaboration patterns, along with real-world adoption paths and engineering practices, will be covered in upcoming articles.

Try RDS for PostgreSQL Agent Development

0 1 0
Share on

ApsaraDB

659 posts | 187 followers

You may also like

Comments