This topic describes Terraform backends, Alibaba Cloud OSS Backend for Terraform, and how to use Alibaba Cloud OSS Backend for Terraform.

Terraform state

Terraform states are used to store the states and properties of your infrastructure resources. Terraform states are divided into local and remote states based on where they are stored.
  • Local state: local storage

    By default, Terraform stores state data locally in a file named terraform.tfstate within the working directory.

  • Remote state: remote storage

    Terraform stores state data in remote services such as Alibaba Cloud OSS, Terraform Cloud, and HashiCorp Consul. This improves state security and management flexibility, allowing state data to be shared between all members of a team regardless of the Terraform working environment and working directory.

The storage of Terraform states is determined by Terraform backends. Local state uses local backends for storage. Except for local backends, you must explicitly define all backends in templates and load and configure these backends by using terraform init before they can be used.

Terraform backends

A backend in Terraform determines how state data is loaded and how state data is updated after a Terraform command is run. The following figure shows how Terraform interacts with a backend during its lifecycle.

terraform backend

Terraform runs the init command to load and configure a backend. Terraform then runs the plan and apply commands to load the resource template and existing state data (if any) by using the backend. After the commands are completed, Terraform updates the data in the responses from the provider or provisioner to the state file through the backend. This way, the state data is consistent with the template definition.

Backends are divided into two types based on what levels of their supported features.
  • Standard

    Standard backends support the standard state management functionality covered in State storage and State locking. Standard backends apply only to scenarios where all operations are performed on the local system. Although Terraform stores state data remotely with standard backends, it still executes its logic locally and makes API requests directly from the system where it was invoked.

    Currently, there are 13 standard backend types. Alibaba Cloud OSS Backend for Terraform is one of them.

  • Enhanced

    Enhanced backends provide all functions that standard backends have, and can perform operations on remote machines. For enhanced backends, Terraform can execute its logic locally, or make API requests or use CLI to execute the logic remotely. Enhanced backends can be divided into local and remote enhanced backends. Remote enhanced backends support only Terraform Cloud.

Alibaba Cloud provides OSS Backend for Terraform, which is a type of standard backend available from Terraform 0.12.2. OSS Backend for Terraform is upgraded in Terraform 0.12.6 and 0.12.8 to support profile configurations and authentication configuration items such as ecs_role_name and assume_role.

Alibaba Cloud OSS Backend for Terraform

OSS Backend for Terraform is a type of standard backend implemented based on Alibaba Cloud Tablestore and OSS. OSS Backend for Terraform uses Tablestore to store locking data generated during the operation of Terraform programs, ensuring the correctness and integrity of states. OSS Backend for Terraform uses OSS to store final state files.

  • Working principles

    The workflow of an OSS backend for Terraform involves three steps: lock, store, and unlock a state file. The following figure shows a simple workflow of an OSS backend for Terraform.

    The workflow includes the following parts:
    • After a Terraform command is run, the OSS backend for Terraform requests the lock ID of the state file from Tablestore. If the lock ID exists in Tablestore, the state file is corrupted or operated and an error message is returned. If the lock ID does not exist in Tablestore, a lock ID is automatically generated, stored in Tablestore, and returned to the OSS backend for Terraform.oss backend
    • If the init command is run for the first time, the OSS backend for Terraform generates a new state file, stores the file in the specified OSS directory, and releases the lock ID.
    • If a command that modifies the state file (such as the plan, apply, or destroy command) is run, the OSS backend for Terraform updates the state file and releases the lock ID after the command is completed.
    • If a command that does not modify the state file (such as the state or show command) is run, the local state data is read and returned.
  • Template definition

    To use an OSS backend for Terraform, you must define the backend template as you would do for a provider or provisioner. Local backends are declared with Keyword backend.

    The following code declares an OSS backend for Terraform whose state data is stored in the prod/terraform.tfstate file in a bucket named terraform-oss-backend-1024. This state file is declared as read-only and encrypted. Lock data is stored in the terraform-oss-backend-1024 table located in a Tablestore instance named tf-oss-backend of the China (Hangzhou) region.
    terraform {
      backend "oss" {
        profile             = "terraform"
        bucket              = "terraform-oss-backend-1024"
        key                 = "prod/terraform.tfstate"
        tablestore_endpoint = "https://tf-oss-backend.cn-hangzhou.Tablestore.aliyuncs.com"
        tablestore_table    = "terraform-oss-backend-1024"
        acl                 = "private"
        encrypt             = true
        ...
      }
    }
    The definition of an OSS backend for Terraform consists of the following parts:
    • terraform: the subject that runs the backend. The backend has its logic implementation stored in the Terraform Registry and serves all providers and provisioners. Terraform, instead of a specific provider, is the subject that runs the backend.
    • oss: the backend type. It is used to identify a specific backend.
    • Parameters are enclosed in the braces to define the properties of the backend such as authentication credentials, OSS bucket name, storage path, and Tablestore configurations. For more information about OSS Backend parameters, see https://www.terraform.io/docs/backends/types/oss.html.
  • Generation of an OSS backend for Terraform template
    Alibaba Cloud provides a Terraform module named remote-backend to help you compile an OSS backend for Terraform template:
    module "oss-backend" {
      source                   = "terraform-alicloud-modules/remote-backend/alicloud"
      create_backend_bucket    = true
      create_Tablestore_lock_instance = true
      create_Tablestore_lock_table    = true
      region                   = "cn-hangzhou"
      state_name               = "new/terraform.tfstate"
      encrypt_state            = true
    }
    After the terraform apply command is run, a configuration file named terraform.tf is generated in the current directory. The file contains the configurations of the OSS backend for Terraform.
    terraform {
      backend "oss" {
        bucket              = "terraform-remote-backend-94a22ee-0714-e8ef-8573-21df8b021f86"
        prefix              = "env:"
        key                 = "new/terraform.tfstate"
        acl                 = "private"
        region              = "cn-hangzhou"
        encrypt             = "true"
        tablestore_endpoint = "https://tf-oss-backend.cn-hangzhou.Tablestore.aliyuncs.com"
        tablestore_table    = "terraform_remote_backend_lock_table_38001042_0714_e8ef_8573_21df8b021f86"
      }
    }

    The generated terraform.tf file can be placed in the same directory as the target template for remote storage of state data.

    To store the state file corresponding to the terraform.tf file in the preceding OSS backend for Terraform, you only need to run the terraform init command again to synchronize the local state file under the local directory to the OSS backend for Terraform.