Community

Blog Events Webinars Tutorials Forum
  1. Blog
  2. Events
  3. Webinars
  4. Tutorials
  5. Forum
Create Account
×
Community Blog Alibaba Cloud, Ververica, Confluent, and LinkedIn Join Forces on Streaming Innovation for Agentic AI

Alibaba Cloud, Ververica, Confluent, and LinkedIn Join Forces on Streaming Innovation for Agentic AI

Apache Flink Agents: A landmark collaboration to build a scalable, production-grade framework for event-driven streaming agents powered by Apache Flink.

By Xintong Song, Staff Software Engineer, Alibaba Cloud

At Flink Forward Barcelona 2025, we announced a major collaboration between Alibaba Cloud, Ververica, Confluent, and LinkedIn—four influential companies in the $100 billion data streaming industry—to jointly develop and contribute to Apache Flink Agents, a new open-source sub-project from the Apache Flink community designed to bring AI agents into the world of real-time, event-driven systems. This initiative marks a pivotal step toward industrial-scale AI applications that react instantly and autonomously to live data streams.

partenership_design

Why Apache Flink Agents Matters

While AI agents have made rapid progress in interactive applications like chatbots, most still operate outside the high-throughput, low-latency world of real-time data processing. Yet in industrial settings—from e-commerce and finance to IoT and logistics—critical decisions must be made instantly in response to live events: a payment failure, a sensor anomaly, a user click.

These workloads demand more than just intelligence—they require massive scale, millisecond latency, fault tolerance, and stateful coordination, all of which are strengths of Apache Flink. But until now, there’s been no unified framework to bring agentic AI patterns into this proven streaming ecosystem. Apache Flink Agents bridges this gap.

Introducing Apache Flink Agents

Apache Flink Agents, a brand-new sub-project from the Apache Flink community, is an open-source framework for building event-driven agents. Building on Flink's battle-tested streaming engine, Apache Flink Agents inherits distributed, at-scale, fault-tolerant structured data processing and mature state management, and adds first-class abstractions for Agentic AI building blocks and functionalities - large language models (LLMs), prompts, tools memory, dynamic orchestration, observability, and more.

This initiative is the result of a community-based joint effort by developers from Alibaba Cloud, Ververica, Confluent, and LinkedIn, a group of engineers with deep expertise in large-scale stream processing and real-time AI. By combining our experience in production-grade data infrastructure and intelligent systems, we are aligning on a shared vision: bringing agentic AI into the streaming data ecosystem, where it can operate with scalability, reliability, and real-time responsiveness.

The key features of Apache Flink Agents include:

  • Massive Scale and Millisecond Latency: Processes massive-scale event streams in real time, leveraging Flink's distributed processing engine.
  • Seamless Data and AI Integration: Agents interact directly with Flink's DataStream and Table APIs for input and output, enabling a smooth integration of structured data processing and semantic AI capabilities within Flink.
  • Exactly-Once Action Consistency: Ensures exactly-once consistency for agent actions and their side effects by integrating Flink's checkpointing with an external write-ahead log.
  • Familiar Agent Abstractions: Leverages well-known AI agent concepts, making it easy for developers experienced with agent-based systems to quickly adopt and build on Apache Flink Agents without a steep learning curve.
  • Multi-Language Supports: Provides native APIs in both Python and Java, enabling seamless integration into diverse development environments and allowing teams to use their preferred programming language.
  • Rich Ecosystem: Natively integrates mainstream LLMs, vector stores from diverse providers, and tools or prompts hosted on MCP servers into your agents, while enabling customizable extensions.
  • Observability: Adopts an event-centric orchestration approach, where all agent actions are connected and controlled by events, enabling observation and understanding of agent behavior through the event log.

Looking ahead

The first public release of Apache Flink Agents, version 0.1.0 is available today. This initial version provides core agent abstractions, integrates Flink's DataStream and Table APIs, supports Kafka-based action consistency, integrates with selected LLMs and vector stores, includes MCP support, and offers observability through event logs.

This milestone marks the beginning of a powerful new framework for building scalable, event-driven AI agents on top of Apache Flink. For more information, visit the Apache Flink Agents GitHub repository.

Detailed timelines, design discussions, and community input can be found in the GitHub Discussions section—your go-to place to follow and contribute to the project’s evolution.

If you're passionate about building intelligent, autonomous systems that react in real time to streaming events, Flink Agents is a project worth watching—and joining.

We warmly invite developers, contributors, and AI enthusiasts to get involved and help shape the future of event-driven AI with Apache Flink.

Learn more

Read our in-depth blog: Flink Agents: An Event-Driven AI Agent Framework Based on Apache Flink

Watch the presentation from Flink Forward Asia Singapore 2025: Flink Agents – The Agentic AI Framework based on Apache Flink

Explore the original proposal: FLIP-531: Initiate Flink Agents as a new Sub-Project

The future of AI isn’t just smarter models—it’s smarter systems that act continuously, reliably, and at scale.

With Apache Flink Agents, we’re building that future—together.

Apache®, Apache Flink®, Flink®, and the Flink logo are trademarks are Foundation in the United States and/or other countries. No endorsement by the Apache Software Foundation is implied by using these marks. All other trademarks are the property of their respective owners.

Data Processing Real-time Computing Apache flink Realtime Compute for Apache Flink Flink Agents
0 1 0
Share on

Read previous post:

Apache Flink FLIP-18: Accelerating Sorting with Code Generation

Read next post:

Real-Time Lakehouse Solutions: Apache Flink & Apache Paimon Integration

Apache Flink Community

205 posts | 53 followers

You may also like

Comments

Apache Flink Community

205 posts | 53 followers

Related Products

More Posts by Apache Flink Community

See All