Terraform:Practical tips on implementing automation

You can refer to the following transcript:

Hello, welcome back to Auto Talk, the Alibaba Cloud Open Platform Automation Series. In this episode, we will share some tips on automation, using Terraform as an example. I am Tiankai from Alibaba Cloud Open Platform. Let's dive into today's content.

As everyone knows, managing cloud resources and business processes through automation can be divided into several stages. Taking the example of modern car manufacturing: In the early days, cars were hand-assembled. While labor-intensive, this method still produced functional vehicles. Later, production lines introduced partial automation for components while still relying on manual assembly. Today, advanced manufacturers have achieved full-chain automation from painting to final assembly, where most steps are handled by production lines. This progression mirrors the evolution of automation in cloud operations.

I believe each stage has its own rationale and aligns with organizational growth. There's no inherent good or bad distinction between them. However, as businesses face growing market competition and operational demands, they naturally evolve from manual to semi-automated and eventually fully automated workflows. Each phase has trade-offs worth considering.

For example, manual processes offer greater flexibility for responding to market changes. With fewer resources, teams can quickly provision infrastructure via the console without costly platform-building or training investments. Simply leveraging the Alibaba Cloud console suffices.

As the volume of resources continues to grow, simply piling on more manpower will inevitably lead to a gradual decrease in the organization's costs and ROI. The market demands that the operations teams shift toward platform development. On the one hand, this expands the value of the platform. Meanwhile, the value of the operations team will gradually manifest and enhance during the platform development process. The organization will achieve higher standardization. In the process of automation, we will embrace peak productivity and efficiency. At the same time, we will gain additional business value, and employees will receive higher recognition for their contribution.

Today, I will outline the main content through six sections. Based on our client experiences and observations of the market, I have considered key stages and challenges in implementing Terraform automation. First, when starting with automation, the most common question is where to begin and what to focus on. Here, I will provide hierarchical goals. We categorize cloud automation into four layers: infrastructure production, infrastructure management, application production, and application management.

The first two layers focus on efficiently producing cloud resources. Some businesses stop here, prioritizing production efficiency. Others manage their resources afterward, emphasizing standardized procedures. Some tie application deployment directly into the production process for full automation, which yields higher business benefits. Others further manage application versions and lifecycles. These layers grow increasingly complex, but the first layer is where most clients typically start.

Second, technology selection. While today's focus is Terraform, IaC implementations can combine multiple tools depending on specific scenarios and desired outcomes. For example, Terraform excels at baseline management and resource provisioning. It is widely adopted for its active community and talent pool. Alibaba Cloud tools like OpenAPI or Cloud Control API can integrate business logic into automation workflows for custom production pipelines.

Third, team environment. While many might think team readiness is not their biggest limitation, in reality, organizational culture and buy-in play critical roles. Every organization is managed by humans. A single team member's enthusiasm for Terraform or IaC principles does not ensure successful adoption.

Execution requires internal consensus and alignment with the business's current needs. For example, explaining to leadership that Terraform is "declarative and ensures baseline stability" might not resonate as a compelling business value. However, framing it as a solution to reduce misconfigurations by over 50% or maintain stable production baselines directly addresses tangible business outcomes.

Another challenge is learning curves. Not every team member will initially grasp Terraform. Some may find it intuitive after onboarding, but others, especially those resistant to technological change, will require support.

Adopting new tools shakes existing work habits. Many ask, "Why tackle automation if I can achieve tasks via the console?" Without experiencing the efficiency gains firsthand, teams may struggle to see the value. Addressing this requires sharing internal success stories and driving adoption through structured incentives.

Regarding embracing legacy technologies, there's another challenge. Many senior technicians resist shifting to new tools due to the costs of adaptation. How do we cover these costs? How does the team adjust and collaborate? These are all factors requiring comprehensive consideration.

For different-sized enterprises, we also provide tailored suggestions for reference. For large organizations, the cost is higher due to extensive collaboration and internal workflows. For small businesses, if your cloud management needs are basic, such as using the console, there's no need to obsess over which automation technology to choose. For medium-sized businesses, it's time to start experimenting with small pilot projects in specific teams. For large enterprises, automation must be driven by tangible business value to deliver significant benefits.

Fourth: decision-making responsibilities. When implementing automation, we're fundamentally translating business processes into code. This requires a balance of people, technology, and processes. Processes and standards are defined through roles like operators, auditors, and compliance officers. These workflows are often stored informally in team members' minds or systematized in advanced organizations via workflow platforms. These are the things we need to focus on.

For example, when automating workflows like ECS instance creation, you need to consider the following questions: Does approving an ECS instance of type A require a different approval path than type B? Is there any difference between a merged node and an execution node? Does deletion require manager approval? All of these considerations are critical when implementing IaC.

However, we recommend starting with core workflows and addressing small, specific pain points, then gradually expanding rather than attempting a big-bang approach that requires defining every process upfront. For instance, instead of meticulously mapping out every organizational workflow first, begin with the main workflow and deconstruct it incrementally. At a macro level, directional guidelines can help, but micro-processes need not be overcomplicated. Starting with a core workflow and developing reusable frameworks allows gradual rollout and adoption within the organization. Teams need to experience early successes to see tangible business benefits, inspiring iterative improvements and alignment with business growth.

Fifth: workflow development. The minimal requirements for building an IaC workflow include Git-based version control and CI/CD practices. If you only run code locally without versioning — modifying files ad hoc — it offers no true benefit over using the console. The core problem remains unresolved. When designing these workflows, integrate foundational processes like version control, runtime environments, change approvals, merge/release pipelines, and deployment to production. Starting with a framework and piloting in small projects is key.

In production systems, automation can address many scenarios. Below are common high-value use cases: Installation and configuration, such as high-availability (HA) settings for services. When creating a new account, you might need to batch-activate certain services. Resource provisioning requires bulk scaling and server deployment. Operations teams can also focus more on performance and network security, and even FinOps. How can cost metrics be matched with cost strategies, including saving plans, in an automated way and integrated into automated workflows?

However, we advise against attempting to cover all scenarios simultaneously. Half-efforts rarely succeed. Instead, prioritize high-impact, common pain points where automation delivers immediate value with minimal investment. Examples include service provisioning, infrastructure setup for new accounts, bulk resource deployment, and rapid creation/deletion of test environments. These scenarios all deliver significant value to your business with relatively low investment. We also provide a wealth of online examples. You can go to the official website of Terraform to view the examples. By leveraging these examples, you can quickly align teams around the value proposition and accelerate adoption within your organization.

To summarize in four key areas: First, there must be reasonable expectations and thoughtful selection of technology. Every organization and technical team is unique, and different teams may require distinct paths for continuous evolution and improvement. Prioritizing business growth as the primary goal will be the core driving force to sustain IaC initiatives over time. Secondly, team environment and mindset matter as much as technology. Progress cannot be achieved by relying on just one enthusiastic individual. Advanced platforms and tools must align with the organization's broader workflows and management philosophy.

Thirdly, decision-making and accountability must involve more than just technical or operations teams. Business teams must also be involved. Compliance experts and auditors must collaborate to ensure automation aligns with evolving organizational policies. The operations team becomes critical in orchestrating this alignment. Lastly, automation is not a one-time project. Processes require ongoing management and iterative refinement as business needs evolve.

That concludes this episode. If you have any questions or ideas about cloud automation, feel free to scan the QR code at the bottom of the screen to join our DingTalk group and connect with us. We look forward to your feedback and hope to see you in the next episode.