How does Alibaba achieve the real-time transmission of nearly trillions of messages every day? The smooth collaboration and transmission of hundreds of thousands of servers and applications are inseparable from RocketMQ, the distributed message middleware. RocketMQ's easy-to-deploy, high-performance, and highly available architecture have supported a large number of business scenarios within and outside the group for decades. In 2016, Alibaba announced it was donating RocketMQ to Apache, and it became an Apache Top-Level Project (TLP). A project initiated by Chinese developers stands on the world stage and attracts the attention of enterprises and developers worldwide. RocketMQ is popular with developers and has won many industry awards. What kind of people can make this project continually vibrant?
It is an open-source project developed on GitHub from birth. It is also China's first non-Hadoop-based Apache Top-Level Project. It unifies the message middleware of all business lines within Alibaba Group and has developed with China's Internet.
The Story Behind Billions of Messages records the little-known founding stories of RocketMQ from birth, open-source, to the present. Let's learn about the people and things behind the distributed message middleware Apache RocketMQ through the The Open Source Folks Talk series!
As early as 2007, the Alibaba middleware team independently developed the message middleware Notify. As the predecessor of RocketMQ and the main message middleware, Notify supported Taobao and Tmall's business for a long time.
Wang Xiaorui, the founder of Apache RocketMQ, joined Taobao middleware in 2011. At that time, the biggest challenge was Notify was always going wrong during every Double 11, which delayed messages for half a day or more than a day. Merchants did not know that buyers had purchased goods, which had a great impact on merchant service and customer experience. After careful research and judgment, the core of the problem was how to achieve high-speed reading and writing. Wang Xiaorui's team believed it was necessary to use a new storage architecture to solve the problem. As a result, RocketMQ was born.
Wang Xiaorui (founder and author of Apache RocketMQ) said, “Alibaba has an open culture. When I was having a meal with Xiao Xie (Head of Alibaba Cloud's Basic Products Division) in the canteen, I said that we hoped the product was called RocketMQ and that it could be open-source from the beginning. Xiao Xie agreed with the idea immediately. Taobao's culture is also “do immediately and wonderfully.” If you have an idea, do it now. Back then, I was even dreaming about how to solve these problems. The boss didn't set a strict deadline, but I thought that since I have figured out the whole structure, I need to implement it quickly.”
RocketMQ has been fully displayed on GitHub since the first day the first line of RocketMQ code was open to users. With the participation of many domestic and foreign developers, RocketMQ quickly iterated and took about three months to complete the first version.
In 2015, RocketMQ won the CJK OSS Award as a representative of China's open-source software.
On the day of Double 11 in 2016, the message volume reached the trillion level, and the peak of TPS was over ten million, creating the largest message flow record in China and worldwide.
At the beginning of Alibaba's establishment, it was because of the open-source software that it could quickly build the prototype of the entire e-commerce platform. Alibaba is grateful for the contribution and value brought by the open-source community. It hopes to share its experience to give back to the community so more people can benefit.
In 2016, Alibaba donated the distributed message middleware RocketMQ to the Apache Open Source Software Foundation. It took only ten months to graduate from Apache and become an Apache Top-Level Project. It also marks China's first non-Hadoop-based Top-Level Project.
(It took about ten months from incubation to graduation. On September 25, 2017, Apache announced that RocketMQ (donated by Alibaba) officially graduated from the community and became Apache's Top-Level Project.)
Wang Ronggang, Head of Apache RocketMQ community operations, said, “We have finally completed a small phased goal. A project initiated by Chinese developers stands on a world stage and attracts the attention of enterprises and developers worldwide. I feel very proud.”
In 2021, after the continuous efforts of many developers in the community, RocketMQ 5.0 appeared and triggered extensive discussions. The core of the new version includes two new highlights. First, the message core scenario is fully expanded. RocketMQ 5.0 is no longer limited to the message decoupling scenario. It will have a new layout of event-driven and message streaming scenarios. Second, the technical architecture continues to evolve, which is more comprehensive and integrated.
In 2022, batch message indexing, logical queue release RocketMQ-MQTT, RocketMQ-Connect, and RocketMQ-Streams completed the upgrade from the business message platform to the message, event, and stream integrated processing platform. Developers can implement one message storage and support multiple scenarios, such as streaming computing, asynchronous delivery, and integrated drivers. A comprehensive solution to technical problems reduces technical complexity and O&M costs and simplifies enterprise application architecture.
Ding Yu, Vice Chairman of the Alibaba Open-Source Committee, said, “In the future, we will increase RocketMQ's open-source investment to build a more prosperous community ecosystem. We hope to work with more developers and contributors to pursue perfection and openness and realize technology inclusiveness.”
People who love open-source and are willing to fight for it are helping China's open-source business have a prosperous future!
Alibaba Cloud Hosts Internet Day 2022 to Bolster Malaysia’s Digitalisation
864 posts | 196 followers
FollowAlibaba Cloud Community - September 5, 2022
Alibaba Cloud Community - March 9, 2023
Alibaba Cloud Community - March 9, 2023
Alibaba Cloud Community - March 9, 2023
Alibaba Cloud Community - December 20, 2022
Alibaba Cloud Community - September 9, 2022
864 posts | 196 followers
Follow
ApsaraMQ for RocketMQ
ApsaraMQ for RocketMQ is a distributed message queue service that supports reliable message-based asynchronous communication among microservices, distributed systems, and serverless applications.
Learn More
ChatAPP
Reach global users more accurately and efficiently via IM Channel
Learn More
Short Message Service
Short Message Service (SMS) helps enterprises worldwide build channels to reach their customers with user-friendly, efficient, and intelligent communication capabilities.
Learn More
AliwareMQ for IoT
A message service designed for IoT and mobile Internet (MI).
Learn MoreMore Posts by Alibaba Cloud Community
