Reading notes: Borg, Omega, and Kubernetes Lessons learned from three container-management systems over a decade-Alibaba Cloud Developer Community

This article introduces three major Google resource scheduling functions: Borg, Omega, and K8S. And a meaningful comparison and Lesson Learn are made. It is very interesting and difficult to read.

About Borg: The first unified container-management system developed at Google was the system we internally call Borg.7? It was built to manage both long-running services and batch jobs, which had previously been handled by two separate systems: Babysitter and the Global Work Queue. The latter's architecture strongly influenced Borg, but was focused on batch jobs; both predated Linux control groups. Borg shares machines between these two types of applications as a way of increasing resource utilization and thereby reducing costs. Such sharing was possible because container support in the Linux kernel was becoming available (indeed, Google contributed much of the container code to the Linux kernel), which enabled better isolation between latency-sensitive user-facing services and CPU-hungry batch processes.

About Omega:Omega, an offspring of Borg, was driven by a desire to improve the software engineering of the Borg ecosystem. It applied many of the patterns that had proved successful in Borg, but was built from the ground up to have a more consistent, principled architecture. Omega stored the state of the cluster in a centralized Paxos-based transaction-oriented store that was accessed by the different parts of the cluster control plane (such as schedulers), using optimistic concurrency control to handle the occasional conflicts. This decoupling allowed the Borgmaster's functionality to be broken into separate components that acted as peers rather than funneling every change through a monolithic, centralized master. Many of Omega's innovations (including multiple schedulers) have since been folded into Borg.

About K8S:The third container-management system developed at Google was Kubernetes.4? It was conceived of and developed in a world where external developers were becoming interested in Linux containers, and Google had developed a growing business selling public-cloud infrastructure.

Key technologies: Containers (container):The resource isolation provided by containers has enabled Google to drive utilization significantly higher than industry norms. For example, Borg uses containers to co-locate batch jobs with latency-sensitive, user-facing jobs on the same physical machines.(Borg always mentioned this point in their various articles, but the result is not open-source yet, just like everyone)

Application-oriented Infrastructure (application-oriented architecture):Over time it became clear that the benefits of containerization go beyond merely enabling higher levels of utilization. Containerization transforms the data center from being machine oriented to being application oriented. This section discusses two examples:Containers encapsulate the Application environment, abstracting away many details of machines and operating systems from the application developer and the deployment infrastructure.Because well-designed containers and container images are scoped to a single application, managing containers means managing applications rather than machines. This shift of management APIs from machine-oriented to application-oriented dramatically improves application deployment and introspection.

Things to Avoid (to Avoid) Don't Make the Container System Manage Port NumbersDon Don't Just Number Containers: Give Them LabelsBe Careful with OwnershipDon Don't Expose Raw State

Some Open, Hard Problems (to be solved) ConfigurationDependency Management

article link: http://queue.acm.org/detail.cfm? id=2898444

Selected, One-Stop Store for Enterprise Applications
Support various scenarios to meet companies' needs at different stages of development

Start Building Today with a Free Trial to 50+ Products

Learn and experience the power of Alibaba Cloud.

Sign Up Now