This article is from Alibaba DevOps Practice Guide written by Alibaba Cloud Yunxiao Team
By He Mian, Senior Technical Expert of the Alibaba Cloud Yunxiao Team
When it comes to DevOps, we have to start with agile and lean development. DevOps develops from them, draws on their methods and concepts, and develops and improves their practice systems.
The practice of agile software development first appeared in the 1990s. At that time, a group of lightweight software engineering methods and frameworks were developed one after another. Their common feature was that they followed evolution and iteration models compared with traditional software engineering, making the processes more lightweight and flexible. Scrum and ExtremeProgramming are the most successful and influential in practice. They are both iterative and incremental software development frameworks. The difference is that Scrum only includes management practices, while ExtremeProgramming covers engineering and management practices.
In the 1990s, the popularity of PC software, the emergence of the fourth-generation programming language, and the rise of object- and design-oriented modes let the development of small projects flourish. In addition, Internet applications and open-source communities emerged. Since they are different from traditional development modes, outstanding individuals have played a prominent role in program development.
All these factors have assisted the practice of non-traditional development methods. As a result, on the one hand, quality problems emerge one after another, which promotes the prosperity and promotion of CMM/CMMI derived from the overall quality management system at this time. On the other hand, many effective practices different from traditional methods have also emerged, opening new possibilities for the industry. The agile movement is already in motion at this time. It is a rebellion against tradition but also a norm for barbaric growth.
In February 2001, 17 representatives of the lightweight software engineering method gathered at the Snowbird Ski Resort in Utah, USA, including several inventors of Scrum and ExtremeProgramming. After the two-day meeting, the Agile Manifesto was released, which stated their shared ideas about software development methods. They also found the word agility to lead these ideas.
The agility concept emerged in 2001, which was the right time. At that time, traditional methods became more bloated and cumbersome but did not solve the software crisis. At the same time, humans entered the Internet era, and demands for responses to changes and innovation in the software industry upgraded rapidly. This is a more fundamental reason. After all, demand is the best booster for the development of the industry.
After the publication of the Agile Manifesto, agility became a movement that was promoted and applied rapidly. However, in the early days, agility focused on the R&D delivery phase. From the business perspective, its goal was to help product and R&D teams improve their agile response capabilities – to deliver value earlier and respond to changes more flexibly. However, in the context of enterprise digital transformation, IT should ensure the development and delivery of products, system deployment, and operation. DevOps inherits the concept of agile development and adds more features to O&M. However, DevOps is not a simple combination of development and O&M.
The leanness idea originated from the production and manufacturing field and was rooted in Toyota's management and engineering practice in product manufacturing. Victory of Lean Production System, a paper on Sloan Management Review from 1988, compared the huge gap in efficiency and quality between the production methods of western nations and Toyota. It challenges the myth that large-scale production brings benefits. Since then, leanness has begun to enter the vision of the western world and gradually become an important part of modern management.
The book Lean Thinking defines lean as a new way of thinking to organize human activities effectively, aiming to eliminate waste and deliver more useful value. The book summarizes the five principles of leanness, which are also the five implementation steps of leanness:
At an abstract level, lean thinking has surpassed the manufacturing industry where it was born and had a profound impact on various industries, such as lean government affairs, lean hospital, lean leadership, lean service industry, lean supply chain, and lean education. Product development is also included. The mainstream agile development methods are directly affected by lean thinking and follow the basic principles of leanness.
At the same time, lean product development is also developing rapidly as an independent practice system. It focuses on two objectives - the value delivery process and the value itself.
First, you need to focus on the value delivery process. The more representative one is the lean kanban, which was developed from David Anderson's practice around 2006. It is summarized in the 2010 book entitled Kanban. Kanban is a specific lean thinking application in software development. It starts with the end-to-end value stream of visualization demand, enhances the flow efficiency of the demand through system practices, and ensures the process quality of flowing to achieve end-to-end system improvement.
The essential change of this kind of lean practice represented by Kanban focuses on the flowing efficiency of values instead of the use efficiency of resources. This drives users to switch from previous local optimization to global end-to-end optimization.
Second, you need to focus on the value itself. Among them, the more representative one is lean startup. The practice of lean startup was initially developed by Steve Blank based on his startup practice and other startup practices in Silicon Valley. Eric Ries wrote about the concept and practice of lean startup in his book entitled The Lean Startup. He made a systematic summary, and the concept of lean startup was popularized rapidly.
Lean startup believes a startup is a huge uncertain process, and its biggest waste is to deliver useless things, such as those that cannot solve user problems or bring business success. Therefore, it integrates the exploration and discovery of value into the product delivery process and puts forward the famous development-measurement-learning cycle. The cycle starts with the concept of pending tests about the market and products. Next, it develops the minimum feasible product (MVP – Minimal Viable Product) to verify this concept. Secondly, it collects feedback from the market and users based on the minimum feasible products and obtains the measurement data. Thirdly, it uses the data to verify the hypothesis to confirm or falsify them and adjust them to generate empirical cognition. Then, it enters the next cycle and continues to explore the business mode and product function design.
The impact of lean startup is far beyond the impact on startups. The Lean Startup defines a startup as creating new businesses and products under an uncertain environment. However, uncertainty seems to have become the commonality of environments in the IT field today. Therefore, the concepts, such as MVP and the development-measurement-learning cycle, have become recognized practices in the IT innovation field. A complete set of innovative practice systems around lean startup has been developed, such as lean data analysis, lean customer development, and lean delivery design.
Lastly, you need to explore and discover effective value and let the value flow smoothly. Lean product development has developed into a systematic practice by focusing on these two goals and following lean thinking. Lean thinking has a fundamental impact on DevOps. The three principles of DevOps follow lean thinking.
At first, the agility and leanness communities only focused on the practices and behaviors from the development side rather than focusing on O&M. However, system development alone is not enough. A developed system must be deployed instantly and smoothly while running continuously and stably to realize its value. Traditionally, this is implemented by O&M.
From the value perspective, development and O&M constitute a relatively complete IT value chain. A more complete one should also include the business. This idea emerges later. This was not the focus of the early DevOps community. When it first started, DevOps still solved the problem between development and O&M, which was the most prominent problem affecting the IT value chain.
In a traditional IT organization, the Development Team (Dev) and the O&M Team (Ops) have different requirements. The Development Team (especially the Agility Team) pursues changes, while the O&M Team pursues stability. There are often conflicts of interest between the two sides. For example, lean and agile teams set continuous delivery as their goal, while O&M teams emphasize change control for online stability. Thus, the department wall is established, which is not conducive to the maximization of IT value.
In 2009, at the second Velocity conference held in the United States, John Allspaw and Pauk Hammond from Flickr delivered a speech entitled 10+ Deploys Per Day: Dev and Ops Cooperation at Flickr. In this speech, Allspaw and Hammond vividly demonstrated various conflicts between development and O&M in a role-playing manner. There were many interesting sentences in the speech, such as "It's not my code, it's your machines!" This speech reflects the current relationship between Dev and Ops. Then, they showed how to eliminate the barriers between the Development Team (Dev) and the O&M Team (Ops) to work together. With the help of tools and cultural changes, the release and O&M of software become continuous and efficient.
This speech was a landmark event in the development of DevOps. It raises the right–topics:
In the same year, Patrick Debois, an independent IT consultant in Belgium, saw this speech and was inspired to organize the first DevOpsDays. DevOps officially appeared on the stage, and the concept of DevOps became popular, while its related tools and practices were also developing rapidly. The emergence of cloud-native technologies, such as containerization and automatic orchestration and scheduling also accelerated this process immensely. Today, DevOps has become one of the core capabilities of enterprise digitalization and is a basic requirement for IT delivery and operation.
In the following two books, The Phoenix Project and The DevOps Handbook, Gene Kim and others summarized the three-step working methods of DevOps implementation:
Kim and others believe this three-step working method is the value and philosophical foundation of all other DevOps processes and practices, and all DevOps modes can be derived from these three principles.
With a few inquiries, you can see that the DevOps three-step working method is a replica of lean principles. More specifically, it is a specific example of lean principles in the context of IT development and operation. The basic part of DevOps reflects the influence and application of lean principles.
Looking back at the development of agility, leanness, and DevOps, we can draw the following two conclusions:
Finally, starting from lean thinking, we can see the inevitable development direction of DevOps, which will extend to the business side. Since the business is the source of product development and O&M, a complete value stream must start from the source. This is not a prediction but a fact that is happening. Most DevOps implementations have included the business side and become BizDevOps. However, the name of DevOps has been widely known, so we will continue to use DevOps. It includes business by default.
While DevOps is developing, digital transformation has become a consensus of the business community. The digital frameworks of most enterprises take DevOps as one of the core capabilities, and the influence scope of DevOps is expanding constantly. It has become an inevitable choice for enterprises striving to improve their digital capabilities. Part 2 of this 27-part series will analyze the fundamental problems to be solved by DevOps in the context of digital transformation.
The Goals of DevOps in the Context of Digital Transformation - Alibaba DevOps Practice Guide Part 2
864 posts | 196 followers
FollowAlibaba Cloud Community - February 3, 2022
Alibaba Cloud Community - February 6, 2022
Alibaba Cloud Community - February 4, 2022
Alibaba Cloud Native Community - July 12, 2022
Alibaba Cloud Community - February 18, 2022
Alibaba Cloud Community - February 6, 2022
864 posts | 196 followers
Follow
Alibaba Cloud Flow
An enterprise-level continuous delivery tool.
Learn More
Bastionhost
A unified, efficient, and secure platform that provides cloud-based O&M, access control, and operation audit.
Learn More
Managed Service for Grafana
Managed Service for Grafana displays a large amount of data in real time to provide an overview of business and O&M monitoring.
Learn More
DevOps Solution
Accelerate software development and delivery by integrating DevOps with the cloud
Learn MoreMore Posts by Alibaba Cloud Community
