The integration of big data and artificial intelligence is an inevitable trend

Flink open source community "getting better"

From the perspective of an ordinary open source technology enthusiast, Jia Yangqing defined the development of Flink in 2020 as "getting better", which is mainly reflected in three aspects:

technology. This year, Flink has made two new breakthroughs in technology: first, Flink's stream-batch integration technology has been successfully applied on a large scale in Alibaba's double 11 core data business scenarios; second, Flink's real-time computing peak value has exceeded 4 billion records per second for the first time. Compared with last year, the computing performance has doubled without increasing the cluster size.

application. Flink is no longer a high-level technology, but has become the de facto standard of open source stream processing, flying into the "home of ordinary people". For example, China Academy of Information and Communications Technology uses Flink as a technical blueprint to design stream processing standards, and cloud computing companies, including Alibaba Cloud, use Flink as the underlying engine for stream processing.

community. The Flink open source community has grown rapidly, reaching the top of Apache's most active mailing list; the Flink project is one of the top Apache projects with the most submissions on Github. Last year, the number of participants in Flink Forward Asia reached 2,000, and the Flink Geek Challenge attracted 4,000+ developers to participate; More and more developers are passionate about open source technology and have the spirit of daring to challenge.

Open source makes the cloud more standardized
Jia Yangqing believes that open source makes the cloud more standardized. He mentioned that cloud computing is becoming more and more standardized, which is very beneficial to its own development. A large number of development technology systems on the cloud are becoming more and more consistent, which means that it is easier for a large number of cloud computing developers to turn their applications into a part of the cloud after they are familiar with this system.

Open source helps Alibaba Cloud break through its own technology system and form a standard, open, and public technology system on the basis of fully respecting users' original usage habits. It is very difficult for a single company to set standards.

What does Alibaba Cloud mean for open source? Jia Yangqing mentioned the following three points:

From an organizational point of view, Alibaba Group established an Open Source Technology Committee last year to unify and promote the implementation of Alibaba's open source technology strategy at the group level.

From the perspective of participation, Alibaba Group has widely used open source technology to build its own technology system from the beginning. Take the big data products of the Alibaba Cloud Computing Platform Business Unit as an example. For example, real-time computing Flink uses technologies from the Hadoop/K8S ecosystem; machine learning PAI also uses open source deep learning frameworks such as TensorFlow.

From the perspective of contribution, Alibaba Group continues to contribute to the open source community. Take the Flink technical community as an example. Last year alone, Alibaba Cloud contributed millions of lines of code to the Apache Flink community. At present, the group has more than 1,000 open source projects, covering big data, AI, cloud native, database, middleware, hardware and other fields. There are more than 700,000 developers around the world who light up GitHub Star for Ali, and thousands Tens of thousands of enthusiasts participated in the project contribution.

Embracing "convergence" is the new direction of big data evolution
Jia Yangqing said that the three major development trends of the cloud are mainly the cloudification of IT infrastructure, the Internetization of core technologies, and the digitalization and intelligence of applications. This is also the core value of the cloud.

After an enterprise migrates its IT to the cloud and transforms its business to the Internet, its applications will inevitably change from process-driven to data-driven, realizing digitalization and intelligence. Cloud computing and underlying IaaS technologies are competing for infrastructure maturity, stability, and price, while upper-level PaaS technologies are competing for data capabilities and intelligence capabilities.

At present, pan-big data including databases and IOT are still in the most basic way of using big data, and more big data and AI capabilities need to be integrated.

Therefore, embracing "convergence" is a new direction for the evolution of big data, the integration of big data + artificial intelligence is the inevitable direction of big data development, and artificial intelligence is the ultimate outlet for the development of big data business.

Only with AI, big data can exert real business value. The reason why Alibaba Cloud's big data has core competitiveness is because big data is big data combined with AI capabilities, and it is big data with intelligent upgrades. For example, Flink+Alink, Flink can not only provide stream-batch integrated processing capabilities, but also use the open source Alink machine learning algorithm library to realize the integration of Flink's big data + artificial intelligence.

In the future, the social economy will be more integrated with the Internet, and the business of enterprises and organizations will also be fully transformed into Internet business. It is not enough for an enterprise to only have IT infrastructure. Enterprise applications will also be driven from process to data intelligence. Alibaba Cloud Computing Platform will output full-stack big data technology capabilities and provide the world's top AI technology to achieve Digitalization and intelligence.