A Brief History of the Development of Container Technology

Container Technology Introduction:

Container technology gave birth to the cloud-native trend, and cloud-native ecology promoted the development of container technology. The development history of container technology in the past 20 years can be roughly divided into four historical stages.

Container Technology Background


"Cloud-native technologies enable organizations to build and run elastically scalable applications in new and dynamic environments such as public, private, and hybrid clouds . Cloud-native technologies include containers, service meshes, microservices, immutable foundations facilities and declarative APIs .”
Talking about container technology can’t avoid cloud native, and talking about cloud native can’t avoid container technology. Container technology and cloud native are a pair of double helixes. Container technology has spawned the trend of cloud native thinking, and cloud native ecology has promoted the development of container technology . From the birth of docker (container) technology in 2013, to the establishment of CNCF, a heavyweight alliance in the cloud native field, in 2015, this is not a historical coincidence but a historical necessity. As one of the key technologies of cloud native, the container has been one of the focuses of the industry since its birth in 2013. Use an advanced diagram of cloud native container technology widely cited in the industry to understand the historical background of the birth of container technology and cloud native.

Container Technology.Let's take a look at the historical chronology of the development of container technology first, let's intuitively feel this hot land in full swing!
In 1979, Unix v7 systems supported chroot to build an independent view of the virtual filesystem for applications.
In 1999 , FreeBSD 4.0 supported jail, the first commercial OS virtualization technology.
In 2004 , Solaris 10 supported Solaris Zone, the second commercially available OS virtualization technology.
In 2005 , OpenVZ was released, a very important pioneer of Linux OS virtualization technology.
From 2004 to 2007, Google used OS virtualization technologies such as
Cgroups on a large scale. In 2006 , Google open sourced the process container technology used internally, and subsequently changed its name to cgroup .
In 2008 , Cgroups entered the Linux kernel mainline.
In 2008 , the LXC (Linux Container) project had a prototype of Linux containers.
In 2011 , CloudFoundry developed the Warden system, a prototype of a complete container management system.
In 2013 , Google open sourced its internal container system through Let Me Contain That For You (LMCTFY).
In 2013 , the Docker project was officially released, allowing Linux container technology to gradually sweep the world.
In 2014 , the Kubernetes project was officially released, and container technology began to go hand in hand with the orchestration system.
In 2015 , CNCF was co-founded by Google, Redhat , Microsoft and some large cloud vendors, and the cloud native wave was launched. From
2016 to 2017, the container ecology began to be modularized and standardized. CNCF accepts Containerd , rkt projects, OCI releases 1.0, and CRI/CNI is widely supported.
2017 - 2018, the commercialization of container services. AWS ECS, Google EKS, Alibaba ACK/ASK/ECI, CCI, Oracle Container Engine for Kubernetes; VMware, Redhat and Rancher began to provide Kubernetes-based commercial service products. From
2017 to 2019, container engine technology developed rapidly, and new technologies continued to emerge. At the end of 2017, the Kata Containers community was established. In May 2018, Google open sourced gVisor code. In November 2018, AWS open sourced firecracker. Alibaba Cloud released Security Sandbox 1.0. From
2020 to 202x, the container engine technology is upgraded, Kata Containers starts the 2.0 architecture, and Alibaba Cloud releases the sandbox container 2.0. …
The development history of container technology in the past 20 years can be roughly divided into four historical stages, which will be described in detail below.

Container Technology technology infancy


One of the core problems that container technology needs to solve is environment isolation at runtime.

The runtime environment isolation of containers aims to construct an undifferentiated runtime environment for containers to run container images at any time and at any location. Due to the preaching of docker, everyone is accustomed to think that the runtime environment isolation of a container is OS virtualization, or that a container is equal to namespace + cgroup + security protection mechanism. I do not agree with this view. This is just a historical period, a container runtime implementation technology, and there are many other possible technical solutions to implement a container runtime environment. So, back to the origin of the requirements: containers require runtime isolation technology to ensure that the container's operating environment meets expectations . Traditionally, people call this component that implements container isolation technology the container runtime.

Container Technology.From another perspective, container isolation technology solves the problem of resource supply . Why is container isolation technology needed to solve the resource supply problem? Winning or losing does not matter! Moore's Law is so powerful that it gives us more and more computing resources to use. When making minicomputers 10 years ago, the typical specification of minicomputers was 32-way 8-core CPU, and now the computing power of a 4-way PC server exceeds that of minicomputer servers 10 years ago. The typical usage of minicomputer is to divide the whole machine into multiple partitions. Observing the current development trend of cloud service hardware, it becomes more and more familiar that we are transforming the technology related to minicomputers from military to civilian. Now one of our PC servers has a very powerful computing power that is comparable to that of minicomputers ten years ago. Coincidentally, the typical usage of current PC servers is similar to that of minicomputers ten years ago, which is cut into 1-8vCPU. Virtual machine/container usage.

Why are people always accustomed to dividing a large server resource into small partitions instead of developing software that can fully utilize the computing power of a large server? I personally think there are two constraints behind it:

Container Technology.The inherent parallelism of the problem to be solved is limited . With the increasing popularity of multi-core and multi-processor systems, the IT industry has been upgrading from serial programming to parallel programming since 2004. At the beginning, the effect of parallelization transformation for specific industry applications was very obvious, but later it was found that with the increase of parallelism, the transformation cost became larger and the profit became lower and lower. Restricted by Amdahl's law, the benefit will gradually decrease after the parallelism of solving a specific problem exceeds a certain critical point. So blindly improving the parallelism of the system is not an economical approach.

Human intelligence is limited . Restricted by human intelligence, the more complex the system and the higher the degree of parallelism, the easier the software is to fail, and the cost of software maintenance increases exponentially . Therefore, from the perspective of software engineering, everyone also tends to interface, modular, and unitized software architecture design, try to control the complexity of the software and reduce engineering costs.

From experience, the degree of parallelism of 1-8 CPUs is the comfort zone of software engineering, which is also one of the driving factors behind technologies such as containerization and microservices .

A bit off topic. . . In conclusion, isolation-based resource supply is not a pseudo-demand. The isolation requirements for the software operating environment have existed since the beginning of the operating system. The multitasking time-sharing operating system and process virtual address are both to solve the problem of resource sharing among multiple tasks running on the same host, so that each process thinks that it has an exclusive host. Of course, process isolation alone is not enough. Looking at the current resource isolation technologies, we can roughly divide resource isolation technologies into five categories:

Container Technology.Process isolation . The OS uses the process as the abstraction of the Task running process. The process has an independent address space and execution context. In essence, the OS virtualizes the CPU and memory of the process . However, various resources such as file system, network protocol stack , and IPC communication space are also shared between processes, and the interference between processes due to resource contention is very serious. This level of isolation is suitable for running different programs of a single user on different hosts, and the user can ensure resource allocation and security protection through system management means.

Container Technology.OS virtualization . OS isolation, also known as OS virtualization, is a sublimated version of process isolation. Process isolation implements a separate address space and CPU context for each process, while OS isolation uses operating system avatars to construct an independent OS environment for each group of process instances to further virtualize the file system and network protocol stack . , IPC communication space, process ID, user ID and other OS resources. OS isolation needs to address three core issues: independent views, access control, and security. Chroot, Linux namespace mechanism to achieve independent view for process group , cgroup Access control is performed on process groups , and mechanisms such as Capabilities, Apparmor , and seccomp implement security protection. Of course, the OS is a very complex and dynamically changing system. Although the OS clone makes the process group feel that it has an independent OS, the real implementation is still an OS instance, so the overall protection capability is still worrying.

Hardware virtualization . OS virtualization is to realize the avatar of the OS kernel, and hardware virtualization is to realize the avatar of the hardware device. The emergence of hardware virtualization technology allows multiple operating systems to run on the same physical server at the same time, and each operating system thinks that it is managing a complete server. Different operating systems are strictly isolated, and the guest operating system's access to hardware is strictly regulated by VMM or CPU. Hardware virtualization has both good security and isolation, but the disadvantage is that the introduced hardware virtualization layer results in additional performance overhead.

hardware partition . This is the resource separation technology used by the traditional minicomputer system, which completely separates a large server into multiple hardware units from the hardware or firmware layer, so as to obtain the highest level of security and isolation. However, as a gradually declining technology route, the minicomputer has obvious shortcomings: the granularity of resource separation is inflexible, the system cost is high, and the system scalability is limited.

Language runtime isolation . For managed languages such as Java and nodejs that require a language runtime, we also have an option to implement isolation in the language runtime. For cloud-native services such as function computing , it is theoretically optimal to implement an isolation mechanism during language runtime. However, there are still many practical constraints in the implementation of this route, so at present, most function computing still adopts container/VM technology to achieve isolation.

On the technical route of OS virtualization, the biggest technical contribution comes from Google.

From 2003 to 2006, Google successively released the "troika", which laid the framework for big data computing, and then further created the concept of "cloud". It is also from this period that process isolation technology has entered a more advanced stage. Under the cloud computing framework proposed by Google , the isolated process is not only a Jail that is isolated from the outside world but stands still, but also needs to be like a lightweight container. It is controlled and deployed to achieve cross-platform, high availability, and scalability in distributed application scenarios.

In 2006, Google introduced Process Containers to limit, account for, and isolate resources (CPU, memory, disk I/O, network, etc.) for a set of processes. Due to the more mature technology, Process Container entered the Linux kernel trunk in the second year after its official launch in 2006, and was officially renamed Cgroups , marking the beginning of the concept of "container" in the Linux camp to be re-examined and implemented.

In 2008, by combining the resource management capabilities of Cgroups and the view isolation capabilities of Linux Namespace (namespace), a complete container technology LXC (Linux Container) appeared in the Linux kernel, which is now widely used. The implementation basis of container technology.

In general, before the invention of docker in 2013, the Linux operating system has generally solved the operating environment isolation technology, one of the core technologies of containers, or the Linux OS virtualization technology has basically taken shape . Although the container operating environment isolation technology is basically in place, we still need to wait for another key technology to usher in the take-off moment of container technology.

Container Technology.Technology burst


Before 2013, the cloud computing industry has been worrying about the correct opening posture of cloud native. Platform as a Service (PaaS) looks like a promising direction. The Zimi service released by Fotango in 2006 can be said to be the originator of the PaaS industry, with typical cloud -native features such as pay-per - use, serverless, API-based configuration and services ; Google launched GAE in 2008; Pivotal launched in 2011 Release Cloud Foundry. These early PaaS platforms carried out very useful explorations and promoted the healthy development of the cloud computing ecosystem, but these early exploration technologies did not form a major industry trend, but were limited to some specific fields. It was not until Docker was open sourced that everyone woke up like a dream. It turned out that the direction was not right, but the means of application distribution and delivery were not good.
The true core innovation of Docker is the docker image, a new mechanism for packaging, distributing, and running applications. A container image packages the application runtime environment , including code, dependent libraries, tools, resource files, and meta-information , into an unchangeable software package that is independent of the operating system distribution .
•Container images package the environment on which the entire container runs, so as to avoid relying on the operating system of the server running the container, so as to achieve "build once, run anywhere".
•Once a container image is built, it becomes read only and part of an immutable infrastructure.
•The operating system distribution is irrelevant. The core solution is the dependency of the container process on the libraries, tools, and configurations included in the operating system. However, the container image cannot solve the special dependence of the container process on the kernel features. This is often jumped into this big pit in the process of actually using the container:
Docker's slogan is "Build, Ship and Run Any App, Anywhere". We have already understood the mechanism of docker to solve "Run Anywhere" through container image, so how is "Run Any App" implemented? In fact, it also depends on the container image. Users can package the environment that any container process depends on without modifying the application to adapt to the running environment defined by PaaS. Really "Run Any App" broke the dilemma faced by the PaaS industry in one fell swoop, created infinite possibilities, and vigorously promoted the development of cloud native. Let's pay tribute to this great idea together!

Container Technology.far, the container technology system has solved the two core problems: how to release software and how to run software , and the take-off time is coming. Before 2014, the former boss of the company said to me, "Stop working on the Linux kernel all day long, why don't you take a look at docker?" After a short investigation, I gave the former boss a simple and clear answer, "Without it, there is only packaging tool. !" Because of this answer, the door that cloud native opened for me was quietly closed. Looking back at the history, I sometimes regret it because I was too young to see the power of container technology + orchestration system, and I didn't realize the upcoming atmosphere of cloud native!
Docker, as a stand-alone software packaging, publishing, and running system, has great value; however, only limiting docker technology to a single machine cannot give full play to the maximum value of this innovative technology . Cluster system to orchestrate and manage business containers.
When it comes to container orchestration systems, we need to start with Google. In 2008, Google launched the first application hosting platform GAE (Google App Engine) based on LXC, and provided the development platform as a service for the first time.
GAE is a distributed platform service. Google provides users with services such as development environment, server platform, and hardware resources through virtualization technology. Users can customize and develop their own applications based on the platform and distribute them through Google's servers and Internet resources. . Google uses a tool that orchestrates and schedules LXC in GAE - Borg (the predecessor to Kubernetes). Borg is a large-scale cluster management system used internally by Google, which can carry hundreds of thousands of tasks, thousands of different applications, and manage tens of thousands of machines at the same time. Borg achieves high resource utilization through permission management, resource sharing, and performance isolation. It can support high-availability applications, reduce the probability of failure through scheduling strategies, and provide task description languages, real-time task monitoring, and analysis tools. If the isolated containers are containers, then Borg can be said to be the earliest port system, and LXC + Borg is the earliest container orchestration framework.
After the launch of docker in 2013, it quickly swept the world. In 2014, Google created the open source project Kubernetes (K8s for short) based on the Borg system used internally to solve the problems of container deployment, operation, and management of large-scale clusters. Kubernetes adds a new layer of management abstraction Pod on the basis of containers, so as to make better use of containers for functional module segmentation of applications. Thanks to Google's strong accumulation in large-scale cluster infrastructure construction, K8s, which was born out of Borg, has quickly become a standard application in the industry and can be regarded as an essential tool for container orchestration.
In response, Docker also added a container cluster management system Docker swarm to the Docker 1.12 version released in 2015, as well as supporting tools such as Docker machine and Docker Compose, in an effort to build a complete container orchestration system and compete head-to-head with Kubernetes. Since then, the container rivers and lakes have been divided into two camps: the Google faction and the Docker faction; and the container orchestration system is Kubernetes, Docker Swarm and Apache Mesos. The competition between the major factions has intensified, gradually extending to the establishment of industry standards. Let us recall this turbulent history of the rivers and lakes together!

After the Docker company launched docker in 2013, CoreOS came into being. CoreOS is a lightweight operating system based on the Linux kernel, specially designed for the infrastructure construction of computer clusters in the cloud computing era. It had a very prominent label at the time: an operating system designed for containers . With the help of Docker, CoreOS quickly became popular in the field of cloud computing . For a time, Docker + CoreOS became the golden partner of container deployment in the industry.

At the same time, CoreOS has also made great contributions to the promotion and community building of Docker. However, the growing Docker seems to have bigger ambitions. Docker, which is not willing to only be "a simple basic unit", has developed a series of related container components by itself. At the same time, it has acquired some containerization technology companies and started to build its own container ecological platform. Obviously, this is a direct competition for CoreOS. At the end of 2014, CoreOS launched its own container engine Rocket ( rkt for short ), trying to compete with Docker. Similar to Docker, rkt can help developers package applications and dependencies into portable containers, simplifying deployment work such as setting up an environment . The difference between rkt and Docker is that rkt does not have the "friendly functions" that Docker provides for enterprise users, such as cloud service acceleration tools, cluster systems, etc. Conversely, what rkt wants to do is a purer industry standard.

above material leads to "From Virtualization to Cloud Native - The Development History of Container Technology" . The most critical context here is the commercial competition between technology companies, and finding a balance between competition and cooperation leads to the birth of standard specifications, and the birth of standard specifications is the most important cornerstone of the entire cloud native ecosystem .

The result of the competition between container engines (docker vs rocket) and container orchestration (Docker swarm vs Kubernetes vs Apache Mesos) is that people sit down and talk about interface standards. In June 2015, Docker took the lead in establishing OCI, which aims to "formulate and maintain container image formats and container runtime formal specifications (OCI Specifications)", and its core outputs are OCI Runtime Spec (container runtime specification), OCI Image Spec (image format specification), OCI Distribution Spec (image distribution specification). So the OCI organization solves the problem of container construction, distribution and operation .

A month later, Google spearheaded the creation of the Cloud Native Computing Foundation (CNCF), which aims to "build cloud-native computing—an infrastructure-centric architecture that widely used". Therefore , the CNCF organization solves the problems of application management and container orchestration . These two foundations around containers have played a very important role in the development of the cloud-native ecosystem. They are not competition but complement each other, and jointly formulate a series of industry de facto standards. The establishment of these industry de facto standards has injected infinite vitality into various industries, and concrete implementations of interface-based standards are emerging, showing a scene of blooming flowers.

Among them, the most important specifications related to containers include: CRI, CNI, CSI, OCI Distribution Spec, OCI Image Spec, OCI Runtime Spec and Shimv2. The CRI, OCI Image Spec, OCI Runtime and Shimv2 specifications are closely related to the Alibaba Cloud sandbox container.

Therefore, I am very grateful for this golden period of cloud native and container technology. A group of creative people came together to create these key specifications, providing the possibility for various manufacturers to provide technical implementations with their own characteristics and compliance with the specifications. .

Container Technology.Commercial exploration period

After five years of technological development, container technology has basically matured, and the cloud native system has also taken shape. Since 2017, major cloud vendors have begun to test the waters of container services and progressive cloud-native services. From the current business form, container-related public cloud services can be roughly divided into three forms:

1.Universal container orchestration service . Before the results of the container orchestration system Three Kingdoms came out, the container orchestration service system was built based on the multi-party betting strategy. AWS is a self- developed orchestration system, Azure's ACS supports Docker Swarm, DC/OS and Kubernetes at the same time, and Alibaba Cloud ACS supports Docker swarm and Kubernetes. Google and firmly support Kubernetes without launching container services that support other container orchestration systems. As Kubernetes unifies the container orchestration, the container service of this route is declining, and Azure directly terminated the ACS service at the beginning of this year.

2.Kubernetes container orchestration service . As a matter of course, Google was the first major manufacturer to test the waters of Kubernetes container orchestration services, and it also launched K8s container orchestration services earlier. As major factories reached the Kubernetes compatibility certification process on the CNCF negotiating table in 2017, the Kubernetes orchestration service market ushered in a big explosion, and by 2018, the K8s container orchestration services of major cloud vendors will be fully in place .

3.Serverless container instance service . Since 2017, the industry has begun to test the water serverless container instance service, freeing users from the arduous task of maintaining container infrastructure and focusing on the business itself. The core goal of Google Cloud Run is to support Knative , so many constraints are attached to its use form.

As can be seen from the above figure, public cloud container services have been explored since 2014, especially after the two-year preemption period of 2017-2018, the basic business form of container services has become relatively clear. The development trend can be summarized as:
•The industry's acceptance of containerization is already high, and the penetration rate of containerization is increasing year by year.
•The container orchestration system has already settled the battle, and K8s has become the de facto king of container orchestration.
•Serverless container instance services are popular in the market, and the customer base is expanding.
•In the long run, the managed container orchestration service and the serverless container instance service will coexist for a long time to meet customer requirements for service cost and flexibility.

guide and confirm customer needs through trial and error, and build a suitable product form. The idea of constructing the product technical architecture during this period is to use the existing mature technology to quickly build a commercial form, and continue to move forward in the process of trial and error.

Among them, the typical architecture of container orchestration and hosting service node level is to use the IaaS system to generate VM, and then deploy container service components such as kubelet , docker, containerd , runC in the VM, that is , the technical architecture of VM + container . A VM can host multiple Container/Pod instances for the same user. The node-level architecture of the serverless container instance service is more direct, and only one container/pod instance is deployed in a VM to achieve serverless . This kind of short, flat and fast play has quickly promoted the exploration of commercial models and played a very important historical role, but its historical limitations in terms of elasticity, deployment density, and resource costs are still very large.

Commercial expansion period
By 2019, the business form and market trend of container services have become obvious, and the industry as a whole has entered a stage of business expansion, with external publicity to attract more customer groups, and internal hard work to improve product technical competitiveness. There are technical upgrades from "to "excellent" . The industry is going through this historical stage of technological upgrading. There is no conclusion yet. We can only talk about trends and predictions together. The focus of this series of topics is container isolation technology, so let's not talk about business expansion and container orchestration and focus on the development trend of container engine technology. So far, we can roughly divide container engine technology into two generations:

1.Container on VM . That is, according to the layered design idea, the container service is built through the architecture of IaaS + PaaS, which is a typical architecture in the commercial exploration stage. Based on the mature IaaS infrastructure of major cloud vendors, virtual machines are produced, and container service components are deployed in the virtual machines. This architecture adopts the lift and shift strategy, which transfers the operation and maintenance responsibility of container services from users to cloud vendors. Using the same software components as users only transfers the responsibility for operation and maintenance, which is conducive to guiding customers to gradually migrate to the cloud and accept cloud-native thinking. However, the services provided by cloud vendors during this period were purely O&M hosting, which did not have obvious technical advantages over user-built container services, and even some user experience was not as good as user-built container services due to the limitation of multi-tenant isolation .

2.Container with hardware virtualization . If the layered design architecture of Container on VM is followed, it is difficult for cloud vendors to build unique technical advantages. For serverless container instance services, the service delivery plane has been moved from the hardware interface of IaaS to OS Syscall , so do not follow the layered design idea of VM + container. We need to start from the source of demand. Container services need a container engine with high performance, strong isolation, sufficient security and low cost. One of the current industry research and development hotspots is how to build such a container engine. Please pay attention to the follow-up series of articles for specific technical ideas.

Container Technology Summary

To sum up, the container service ecosystem has gone through four stages, respectively solving or trying to solve different problems:
1.The infancy of technology: the isolation problem of the container operating environment is solved
2.Technology burst period: Solve software distribution and container orchestration problems

Related Articles

Explore More Special Offers

  1. Short Message Service(SMS) & Mail Service

    50,000 email package starts as low as USD 1.99, 120 short messages start at only USD 1.00