By Zhang Lei, Staff Engineer on Alibaba Cloud Container Platform and Co-maintainer of Kubernetes Project
It's not a secret that Twitter has decided to move Twitter's infrastructure from Mesos to Kubernetes. But behind this major decision, what kind of reason and motivation would be supporting this major change to Twitter's infrastructure?
In fact, in Internet technology scenarios, infrastructures built by engineers on the basis of mature technologies will always be the first choice of leading non-cloud Internet vendors. Therefore, the relatively mature and efficient Mesos project has been widely deployed in production at scale in many cases.
However, with the popularity of cloud computing and the rise of cloud-based containerized infrastructure projects like Kubernetes, this traditional Internet infrastructure starts to show its age—being a much less efficient solution compared with that of Kubernetes:
It is not hard to see that the main technical factor behind the Twitter's switch to Kubernetes Native is to implement a balance between large-scale and customized needs and the community standard by utilizing the core concepts of Kubernetes and its scalability.
This approach is also advocated by Alibaba in the community. In 2018, Alibaba joined a large number of leading Internet companies like Google, Facebook, LinkedIn, Uber, Netflix, and Pinterest to launch the monthly Web-Scale Meetup in Silicon Valley, where Alibaba also shares several events and meetups, too. Twitter also invited engineers from the Alibaba Cloud Container Platform team as featured guests at the small annoucement conference (or meetup), including Li Xiang, Zhang Lei, and He Jian. Jago Macleod, technical engineering manager of the Kubernetes team at Google, was also invited to attend the conference.
At the conference, the Alibaba Cloud Container Platform team revealed that in the next month, Alibaba Cloud would publish the source code of Kruise, a Kubernetes Workloads Advanced project that Alibaba Cloud has worked on for a long time. Kruise makes full use of declarative APIs and controller pattern in Kubernetes to provide the important in-place update feature for containerized applications in Internet scenarios. It also allows users to have more fine-grained business release policies. Many companies like Twitter, Pinterest, and Netflix will collaboratively participate in this innovative cloud-native job management project.
The continuously increasing scale and performance of Kubernetes are another technical reason for the change of Twitter from an on-looker to an adopter. In his speech, Jago Macleod mentioned the ongoing research and cooperation between Google and Alibaba in this field. Recently, engineers from both companies were attempting to add bookmarks for the large number of WATCH operations in k8s. This allows creators of these WATCH operations to only trace a small number of historical changes outside the bookmarks after the restart. In some cases, the Kubernetes API server can even show 40 times better performance.
As the Internet business grows rapidly, the infrastructure software team at the company soon began to surpass its expected scale boundary, but the return on investment ratio did not increase accordingly.
The past in-house infrastructure software development and architecture design are undergoing subtle changes as the cloud computing and Cloud Native become increasingly popular. With the Kubernetes platform project standard, many Internet companies have already had the ability to migrate their infrastructures to the cloud at a relatively low cost. More importantly, with the Kubernetes standard layer, the integration itself does not have to be implemented on a solid binding relationship between Internet companies and cloud service providers, as it is for Netflix and AWS. Instead, Kubernetes retains most advantages of in-house development and also enables the multi-cluster management and the full utilization of the cloud power. This advantage will become more obvious after all AWS engineers in an Internet company become k8s engineers.
It is easy to see that Kubernetes is application-centered and connects cloud, application developers, and infrastructure software teams together. The communication, connection, and delivery capabilities on this "expressway" project serve as meaningful references for fast-changing Internet companies like Twitter deciding the future evolution of their infrastructure and architecture. The change allows a commercial organization like Twitter with rapid business growth to maintain an infrastructure team made up of just a few dozen members.
Twitter's switch from its initial adoption of Mesos to the use of Kubernetes Native today proves again the assertion that Kubernetes has become an industry standard for container orchestration. More importantly, Twitter's embrace of Cloud Native is expected to provide a classic learning model for the large-scale implementation of the cloud-native technology in production.
Alibaba has invested lots of technical resources in the Cloud Native ecosystem and is being recognized as an important leader in the large-scale implementation of the cloud-native technology in production by other global leading Internet companies including Facebook, LinkedIn, Uber, Netflix, and Pinterest besides just Twitter. As cloud computing becomes more popular, traditional Internet technology architectures begin to show many issues with their age. Specifically now, with the rise of cloud-based container infrastructure projects like Kubernetes, more and more international enterprises start to think about how to embrace the open-source ecosystem and open technology standards by using the power of cloud and cloud-native technologies and how to be prepared for a cloud-based digital future featuring strong iteration capability.
Alibaba Clouder - January 8, 2021
Alibaba Clouder - February 14, 2020
Alipay Technology - February 20, 2020
Alibaba Developer - June 30, 2020
Alibaba Developer - January 10, 2020
Alibaba Clouder - March 11, 2019
Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.Learn More
Accelerate and secure the development, deployment, and management of containerized applications cost-effectively.Learn More
Alibaba Cloud DNS PrivateZone is a Virtual Private Cloud-based (VPC) domain name system (DNS) service for Alibaba Cloud users.Learn More
More Posts by Alibaba Container Service