By Ding Yu (Shutong), Alibaba researcher, director of cloud-native application platform of Alibaba Cloud
This is the best time for developers.
The prosperous development of the digital industry, the maturity of the traditional cloud computing industry, and the emergence of various new technologies, such as big data, IoT, AI, and blockchain, provide developers with many opportunities. At the same time, learning continuously emerging new technologies brings pressure to developers, making them confused about the choice of their future path.
In March 2021, Alibaba Cloud commissioned Forrester to survey Chinese developers. The online questionnaires surveyed 422 people of various posts, including in-house application development, testing, O&M, and many independent developers. Forrester also interviewed nine experts, including enterprise CTO, university professors, open-source community experts, and key opinion leaders (KOL) in the developer community.
There is no denying that cloud-native enables developers to see more possibilities. The evolution of cloud-native technology provides developers with a smarter way to work, which means they can make full use of the components and toolchains of the cloud-native platform, mask the underlying technology, and save time and cost. Thus, they can focus more on business value creation and innovation. If the cloud is a belief, then cloud-native is an attitude. Everyone should be a cloud-native developer in this era.
Throughout thirty years of development, on the one hand, the majority of Chinese developers continue to create huge economic and social value. On the other hand, the development of society and economy is also changing the way society creates value and continuously transforms the delivery mode of software and application R&D. This provides developers with a broader space to unleash their imagination and creativity.
Previously, traditional developers formed the developer group. Later, many traditional developers turned to cloud development. Now, cloud-native developers are gradually growing. The role of developers has also changed from previous executors of enterprise information technology strategies to one that empowers businesses' digital transformation nowadays. In the future, they will become the technology leaders of digital innovation.
Traditional developers are the executive force of the enterprise informatization strategy. During the twenty years from 1990 to 2010, applause and praise often had nothing to do with them, but they implemented the business strategies of corporate decision-makers. Traditional developers help companies build information infrastructure and business platforms and realize the business functions required by internal users and external customers through the development and implementation of various information system applications. They also help companies cope with the rapidly changing market environment and continuously explore new business areas and new business models.
Although the digital transformation of various industries is in full swing, many industries are still dominated by traditional application construction and development modes. For example, the waterfall mode requires functional design, development, and testing. It also requires technical selection, resource preparation, component debugging, continuous construction, and version deployment. The entire process is long and technically complex and requires a large number of development teams.
With the maturity of cloud computing technology, traditional development modes can no longer meet the development needs of the times. Developers have started using the power of cloud platforms. Cloud development can effectively ensure the modernization level of R&D and O&M Teams and promote the continuous improvement of R&D and O&M technical capabilities from the platform tool level. The standardization of technical capabilities brought about by cloud development has greatly improved the work efficiency of developers in a wide range of areas. These areas include the configuration of the development and test environment, the software and hardware development of functional components and business systems, and the release and deployment of services, components, and grayscale.
We have also noticed that most enterprises that have migrated applications to the cloud have not changed much in their application development modes. In these enterprises, developers use cloud computing in isolated scenarios, such as infrastructure level or specific business applications. At the same time, the imperfection of the development toolchain and integrated services of the cloud platform has also prevented the cloud development mode from being accepted by more developers.
Then, the container technology represented by Docker began to rise, and Kubernetes gained momentum through profound accumulation. As developers continue pursuing performance, efficiency, portability, manageability, and convenience, cloud-native technology was created. Cloud-native endows the cloud development mode with more powerful tools and new thinking on application construction and innovation. Cloud-native developers have to master and apply emerging technologies, such as Docker and Kubernetes. Cloud-native technologies can mask the complexity of the underlying technologies and achieve better application integration and delivery, facilitating enterprise innovation. Forrester's research shows that 73% of developers believe cloud-native can make more internal personnel become developers.
Cloud-native is a re-upgrade of cloud computing. Cloud-native contains a large number of new technologies at the PaaS layer and new development concepts. It is the shortest path to unleash the value of cloud computing, and it also promotes the re-upgrade of cloud computing. The entire cloud-native technology stack is based on open-source and open technology standards. The Cloud Native Computing Foundation (CNCF) is also working to standardize cloud-native technologies and provide users with a standard interface to use cloud services to avoid vendor lock-in.
Let's take a deeper look at applications refactored or rewritten based on cloud-native technologies and cloud-native architecture. For example, service mesh- or Serverless-based applications naturally have the capability to scale horizontally and can cope with the rapid growth of business scale in the Internet era at any time. At the same time, they also have built-in high availability capabilities, so applications do not need to focus on high availability solutions in distributed environments.
Cloud-native is a re-upgrade of the application architecture using the cloud and a re-upgrade of the technology and cloud services of the cloud platform. From the perspective of building cloud-native applications, cloud-native refactoring of applications is reflected in the entire lifecycle of application development.
We saw some trends in this process. For example, the combination of containers + Kubernetes has become the technology infrastructure. Containers decouple application runtime from the environment. Kubernetes decouples resource management from infrastructure, encapsulates the infrastructure layer, masks the underlying architecture differences, and connects heterogeneous computing power. It integrates cloud, edge, and devices, building a distributed cloud. They support a variety of heterogeneous workloads. They are distributed, elastic in scaling and provide mesh-based decoupling, driving the modernization and evolution of application architectures. Containers and Kubernetes have become distributed operating systems in the cloud computing era.
In addition to the changes in the development mode, we can see that an era of all-cloud development driven by cloud-native is coming.
By the end of 2021, 68% of institutions used containers in their production environments, an increase of 39% from 2020 and 240% from 2018. Alibaba achieved 100% containerization in 2016. In 2021, 68% of enterprises used containers in production environments. Containers have become an absolute trend.
68% of developers are willing to develop on the cloud. A large number of developers have been aware of the changes brought about by cloud development, including changes in frontend/backend, web pages, mobile terminals, mini programs, logic, models, and components. Cloud development has brought about improvements in productivity and a significant increase in efficiency.
By the end of 2021, 25% of developers will use Serverless products. Today, Alibaba Cloud has begun to invest heavily in Serverless R&D, providing products, such as Function Compute (FC), Serverless App Engine (SAE), Alibaba Serverless Kubernetes (ASK), Alibaba Serverless infrastructure (ASI), and Elastic Container Instance (ECI). At the same time, in the aspect of developer tools, Alibaba has opened up Serverless Devs, the industry's first Serverless developer platform. Developers can experience Serverless products with one click and deploy Serverless projects at an extremely fast speed through this platform.
Concerning the perspective of the majority of developers, developers can feel the mobility of the cloud platform from the aspects of resources, capabilities, and value. This mobility has brought convenience to application development and system O&M. On the other hand, developers have also been plagued by heterogeneous development technologies on and off the cloud, various development tools, and fragmented collaboration processes. They cannot implement technology practices on the cloud.
The development and maturity of cloud-native allows related products and tools to solve the pain points faced by developers. For example, the consistent encapsulation of container images enables developers to realize application mobility in different environments. The separation of code and configuration simplifies the development and testing processes for developers. Serverless computing can help developers quickly develop and test clusters without complex configurations. CI/CD cross-cloud construction and deployment capabilities allow developers and O&M personnel to ensure fast and stable iteration of complex applications in heterogeneous environments.
As an all-in-one DevOps platform, Apsara DevOps integrates the cloud-native development mode summarized by Alibaba for many years into products, providing easy-to-use cloud-native R&D solutions for developers. The out-of-the-box cloud IDE, code management platform, and CI/CD pipeline allow the entire process of development, testing, and deployment to seamlessly integrate with the cloud-native base platform. This lowers the threshold for developers to get started with cloud-native technologies, accelerates cloud-native technology transformation, and improves software production efficiency.
Driven by the market and technology development, the development and construction of enterprise applications and software will turn to cloud-native mode. At the same time, the maturity of the cloud-native development platform will drive more enterprises to choose cloud-native. The cloud-native development mode will become an inevitable choice for enterprise application development.
Standards and open-source have accelerated cloud-native and promoted the better implementation of all-cloud development. Today, Alibaba Cloud has more than 2,600 open-source projects on GitHub, covering big data, cloud computing, AI, middleware, containers, Serverless, and other fields. It has more than 30,000 contributors and more than one million GitHub stars, ranking first in terms of the contribution in the open-source community of Chinese enterprises on GitHub. Among them, some open-source projects have also become factual standards in related fields.
For example, Dubbo has become the most influential and widely used open-source microservice framework in China. RocketMQ is the first Apache top-level project of Internet middleware in China. It is also the most popular open-source middleware project in China. There is KubeVela (an application management engine), OpenYurt (Alibaba's first open-source edge computing project), ChaosBlade (a distributed and highly available chaos engineering tool), Nacos (a platform for service registry and discovery), and Serverless Devs (the first Serverless developer platform).
Alibaba Cloud has contributed its achievements from years of research and development to the world's top foundations, such as the OpenAtom Foundation and the Apache Foundation. Alibaba Cloud is committed to creating an open, standard, healthy, and benign technology ecosystem. If developers want to build an open-source architecture based on cloud-native technology, the tools can be found in the Alibaba Cloud ecosystem. Alibaba Cloud has served a large number of leading enterprise users and is building an open and standard technology system to serve global developers.
Migration to the cloud is a trend for enterprises. Alibaba is the first to make proprietary RocketMQ open-source in China, which has become a top-level open-source project of Apache. RocketMQ provides users with high-performance, high-reliability, and low-latency message services and has become the first choice for financial business messages in the industry. The development of cloud-native technology allows comprehensive cloud transformation to become the general trend. Many enterprises have deeper demands for the evolution of message middleware. We will release the community and commercial versions of RocketMQ 5.0 simultaneously, defining a new concept of integrated processing of messages, events, and streams. RocketMQ 5.0 will bring changes in cloud-native infrastructure, message streaming processing, and event-driven architecture.
First of all, based on the background of Alibaba Cloud's large-scale production practice, RocketMQ 5.0 will provide an extreme elastic architecture that can be adjusted for multiple scenarios to help enterprises balance resource elasticity, O&M complexity, and business ecological innovation. Secondly, RocketMQ 5.0 will provide a new and lightweight streaming processing framework, focusing on high-value message data to help enterprises and developers process message calculation and analysis nearby. Finally, RocketMQ 5.0 will connect cloud services and the open-source ecosystem based on open standards and combine with Serverless development mode to provide users with low-code and serverless next-generation event-driven architecture services.
When cloud-native was first proposed, it focused on resource cluster management and O&M. However, if all workloads run on a set of infrastructure today, there will be a lack of application management. So, we propose taking the application as the center, separating the focus, and defining the development, O&M, infrastructure, cooperation boundaries, protocols, and methods. Thus, KubeVela, an all-in-one application management and delivery platform, was created. In June 2021, KubeVela became a project of CNCF Sandbox. Its image has been downloaded over 300,000 times, and it has more than 20 global enterprise customers. In 2021, OAM was approved by the China Academy of Information and Communications Technology (CAICT) as an industry standard.
Nowadays, more computing power and businesses are beginning to focus on data sources and end users. As the scale and complexity increase day by day, O&M capabilities in edge computing scenarios begin to be overwhelmed. Alibaba Cloud has opened up the core framework OpenYurt of the edge container service ACK@Edge to the industry and donated it to CNCF to promote the collaborative development of edge computing and cloud-native. Adhering to the industry's first non-intrusive concept, OpenYurt covers all edge computing scenarios and has become the preferred platform in ecologically compatible scenarios. It has been applied to dozens of industries, including IoT, audio and video, and logistics, making cloud-native ubiquitous.
In such a rapidly changing era, every developer or individual that wants to be a developer needs to keenly grasp the development direction of the times, the changes in the needs of enterprises' digital transformation, and the trend of technological development. At the same time, it is more important to be able to see various challenges outside your comfort zone and find space for personal growth.
Challenges, such as the career challenge, exist for young people that have just entered the workplace and middle-aged senior engineers. Middle-aged staff cannot compete with young graduates to stay up late and work overtime, and their work efficiency during continuous overtime is reducing. They do not have enough energy to find useful tools from the rapidly developing technologies.
With enterprises migrating to the cloud one after another, the underlying infrastructure and platform tools are constantly standardized, automated, and becoming intelligent. People with traditional technical skill systems find their anxiety of career development increasing day by day with their age when they find out their skill system can hardly meet the demands of the industry.
In one survey, 72% of respondents reported that in the face of the rapidly changing market, they need to master more business and industry knowledge. 70% of respondents believe that as enterprises pay more attention to customer experience, developers need to master relevant knowledge as well. Besides, various internal and external factors impose higher demands on the skills and qualities of developers, which urgently require developers to accelerate self-transformation.
The evolution of cloud-native technology has brought developers hope. The cloud-native development mode will become the mainstream mode for future application development and help developers build their technical competitiveness. From containers/Kubernetes, microservices, and service mesh to event-driven serverless computing applications, from the management of application catalog, service catalog, and image repository to the deployment of public cloud, private cloud, and edge networks, the evolving cloud-native full-stack technology brings developers a new perspective on resources, assets, and architectures. It enables developers to embrace more modern technical methods, tools, and best practices to achieve the vertical improvement and innovation of their technical capabilities.
Developers can enhance their technological adaptability to achieve cross-realm development. Cloud-native technologies, represented by containers, microservices, and Serverless, have allowed database, AI, big data, audio, video, blockchain, and other technical fields to continue to innovate. Based on the platform capabilities of Kubernetes, developers have the opportunity for the first time to coordinate various emerging technology fields that were once separated from each other to realize the horizontal expansion of their technical capabilities. In addition, developers have the opportunity to explore the design, development, testing, and O&M of the distributed application architecture. This broadens their technical horizons and enriches their development experience, effectively enhancing their competitiveness and adaptability.
Developers can improve collaboration effectiveness and explore business innovation. CI/CD and DevOps practices have taken another leap forward with the help of cloud-native technologies, such as containers and Kubernetes. The simplification, automation, and integration of the application development processes enable developers to focus more on the in-depth understanding of business needs and customer value to meet customer needs more quickly and effectively. Developers can also explore business innovation by solving business problems.
Cloud-native offers developers the possibility to realize the vision of an all-cloud practice, but developers should also understand that the mission of leading digital innovation into the next decade will not be achieved overnight. Developers should focus on technical capability requirements for cloud-native developers and value realization from a forward-looking perspective.
Technology can drive transformation and lead innovation, but technology is not an end. It must serve the business. In their daily technical practice, developers should actively understand the specific business requirements behind the implementation of technology, be familiar with the business concepts behind the language used by business personnel, and explore the customer goals behind the business requirements as much as possible. Secondly, developers should abandon the common stereotypical thinking, such as customer experience equates to a beautiful interface, and experience is only related to UI and has nothing to do with technical personnel. Developers need to think differently and integrate the idea of customer value into their technical practice.
Finally, developers need to be diligent in thinking about things. What are the core business problems that technology solves? What are the business causes of these problems? What business pain points have not been taken into account and can be enhanced? This is the only way developers can start from the perspective of business and clients and truly promote the improvement of their technical capabilities. Cloud-native technologies help enterprises quickly implement their business needs and force enterprise developers to have a commercial perspective, pay attention to business logic, and engage with more innovative things.
The times call for cloud-native. This is the best time for cloud-native and developers. Alibaba Cloud has been on the road to making every developer more experienced.
How Can We Monitor Containers as They Become More Widely Used?
481 posts | 48 followers
FollowAlibaba Cloud Native Community - March 8, 2023
DavidZhang - July 5, 2022
淘系技术 - November 4, 2020
Alibaba Clouder - October 21, 2020
Alibaba Cloud Community - March 1, 2022
Alibaba Cloud Community - October 9, 2022
481 posts | 48 followers
Follow
Bastionhost
A unified, efficient, and secure platform that provides cloud-based O&M, access control, and operation audit.
Learn More
Managed Service for Grafana
Managed Service for Grafana displays a large amount of data in real time to provide an overview of business and O&M monitoring.
Learn More
ApsaraMQ for RocketMQ
ApsaraMQ for RocketMQ is a distributed message queue service that supports reliable message-based asynchronous communication among microservices, distributed systems, and serverless applications.
Learn More
Function Compute
Alibaba Cloud Function Compute is a fully-managed event-driven compute service. It allows you to focus on writing and uploading code without the need to manage infrastructure such as servers.
Learn MoreMore Posts by Alibaba Cloud Native Community
Kidd Ip February 19, 2023 at 3:11 am
A good sharing on microservices and open source!