for_each - Terraform - Alibaba Cloud Documentation Center

Overview

for_each is a powerful meta-argument in Terraform. It allows you to create multiple instances of resources or modules based on maps or sets of strings. Compared with the count meta-argument, for_each provides a more flexible way to manage a collection of similar resources, especially when each instance needs to use different configuration values, and these values cannot be simply derived from a numeric index.

The main benefit of for_each is that it assigns a stable identifier (based on mapping keys or set elements) to each resource instance. This enables Terraform to accurately track a single resource instance even if the sequence of the underlying set changes. This way of identifying resources avoids the "resource update storm" issue that you may encounter when using count, that is, when elements are added or removed from a list, all subsequent indexes change.

By using two special attributes, each.key and each.value, you can reference the key and value of the current iteration in the resource configuration, thus providing a different configuration for each instance. This makes for_each particularly suitable for creating collections of resources that require complex and diverse configurations, such as multiple types of security group rules, buckets with different configurations, or compute instances with different tags and attributes.

This topic describes the syntax and usage of for_each and compares for_each with count. This topic also provides examples in the Alibaba Cloud environment to help you make full use of this powerful feature to simplify and optimize infrastructure management.

Note: A resource or module block cannot use both count and for_each.

Syntax

for_each is a meta-argument defined by Terraform. It can be used with modules and all resource types.

The for_each meta-argument uses a map or a set of strings and creates an instance for each item in the map or set. Each instance has a unique infrastructure object associated with it, and each object is created, updated, or destroyed individually when the apply command is run.

Map example:

resource "alicloud_resource_manager_resource_group" "rg" {
  for_each = {
    group1 = "China (Hangzhou)"
    group2 = "China (Beijing)"
  }
  
  display_name = each.key
  resource_group_name = "${each.key}-${each.value}"
}

String set example:

resource "alicloud_ram_user" "users" {
  for_each = toset(["alice", "bob", "charlie", "dave"])
  
  name = each.key
  display_name = "User ${each.key}"
}

Example of child modules:

# my_buckets.tf
module "bucket" {
  for_each = toset(["assets", "media"])
  source   = "./oss_bucket"
  name     = "${each.key}_bucket"
}

# oss_bucket/bucket.tf
variable "name" {}# This is the input argument of the module

resource "alicloud_oss_bucket" "example" {
  # As var.name contains each.key in the called module block, 
  # its value is different for each instance of this module. 
  bucket = var.name
  
  # ...
}

resource "alicloud_ram_user" "deploy_user" {
  name = "${var.name}_deployer"
  # ...
}

each object

In a block that contains the for_each argument, the expression can use an additional each object. This allows you to modify the code of each instance. This object has two attributes:

each.key - The map key or set element corresponding to this instance.
each.value - The map value corresponding to this instance. (If a set is provided, each.value is the same as each.key.)

Limits

The key of a for_each map or all values of a string set must be known, or you will receive an error message that for_each has a dependency that cannot be determined by the apply command, and you may need to use -target.
The for_each key cannot be the result of or depend on the result of a dynamic function, including uuid, bcrypt, or timestamp because the preview results of these functions are deferred during the execution of the plan command.
Sensitive values (values marked as sensitive, such as sensitive input variables, sensitive outputs, or sensitive resource attributes) cannot be used as arguments of for_each. The values used in for_each are used to identify resource instances and will always be displayed in the UI output, which is why sensitive values are not allowed. If you use sensitive values as arguments of for_each, an error occurs. If you convert a value that contains sensitive data into an argument to be used in for_each, note that most functions in Terraform return sensitive results when given an argument that contains sensitive content. In many cases, you can achieve similar results by using a for expression. For example, if you want to call keys(local.map), where local.map is an object with a sensitive value (but not a sensitive key), you can use toset([for k, v in local.map : k]) to create a value that is passed to for_each.

Use expressions in for_each

The for_each meta-argument supports a map or set expression. However, unlike most arguments, the value of for_each must be known before Terraform performs any remote resource operations. Therefore, for_each cannot reference unknown resource attributes before the application is configured. For example, the unique ID generated by a remote API operation during object creation cannot be referenced.

A for_each value must be a map or set that provides one element for each desired resource instance. To use a sequence as a for_each value, you must use an expression that explicitly returns a set value, such as the toset function. To prevent unnecessary surprises during conversion, the for_each argument does not implicitly convert a list or array to a set. If you need to declare a resource instance based on a nested data structure or a combination of elements of multiple data structures, you can use Terraform expressions and functions to construct appropriate values. Examples:

Use a nested for expression with the flatten function to convert a multi-level nested structure to a flat list.
Use the setproduct function within a for expression to generate an exhaustive list of two or more elements.

Reference for_each between resources

As resources that use for_each appear as object mappings when used in expressions, you can directly use one resource as for_each for another resource in the case of a one-to-one association.

For example, in Alibaba Cloud, a VPC is usually associated with many other resources that depend on the VPC, such as vSwitches and security groups. If you use for_each to declare multiple VPCs, you can link the for_each argument to another resource and declare a vSwitch for each VPC:

variable "vpcs" {
  type = map(object({
    cidr_block = string
    zone_id    = string
  }))
}

resource "alicloud_vpc" "example" {
  # Create a VPC for each element in var.vpcs
  for_each = var.vpcs

  # each.value is a value in var.vpcs
  vpc_name   = each.key
  cidr_block = each.value.cidr_block
}

resource "alicloud_vswitch" "example" {
  # One vSwitch for each VPC
  for_each = alicloud_vpc.example

  # each.value is a complete alicloud_vpc object
  vpc_id      = each.value.id
  cidr_block  = cidrsubnet(each.value.cidr_block, 8, 0)
  zone_id     = var.vpcs[each.key].zone_id
  vswitch_name = "${each.key}-default-vswitch"
}

output "vpc_ids" {
  value = {
    for k, v in alicloud_vpc.example : k => v.id
  }
}

This reference pattern clearly and concisely states the relations between vSwitches and VPCs, which makes it clear for Terraform that the instance keys of the vSwitches and VPCs change together. This also makes the code easier to understand and maintain.

Reference instances

After you configure the for_each argument, Terraform distinguishes between the block and the resources or module instances that are associated with the block. An instance is identified by a map key or set element. The key is from the value provided to for_each.

<TYPE>.<NAME> or module.<NAME>. For example, alicloud_resource_manager_resource_group.rg references a block.
<TYPE>.<NAME>[<KEY>] or module.<NAME>[<KEY>]. For example, alicloud_resource_manager_resource_group.rg["group1"] or alicloud_resource_manager_resource_group.rg["group2"] references a single instance.

This is different from resources and modules without count or for_each, which can be referenced without an index or key.

Similarly, resources from child modules with multiple instances are prefixed with module.<NAME>[<KEY>] when displayed in plan outputs and elsewhere in the UI. If a module does not contain the count or for_each argument, the address does not contain the module index because the module can be referenced by name.

Note: In a nested provisioner or connection block, the special self object references the current resource instance, not the entire resource block.

Use a set

Terraform does not have a literal syntax for set values. However, you can use the toset function to explicitly convert a list of strings to a set:

locals {
  vswitch_ids = toset([
    "vsw-abc123456",
    "vsw-def789012",
  ])
}

resource "alicloud_instance" "server" {
  for_each = local.vswitch_ids

  instance_name   = "server-${each.key}"
  instance_type   = "ecs.s6-c1m2.small"
  image_id        = "aliyun_2_1903_x64_20G_alibase_20230523.vhd"
  vswitch_id=each.key# Note: For sets, each.key and each.value are the same.
  security_groups = ["sg-xyz123456"]

  tags = {
    Name = "Server ${each.key}"
  }
}

The list-to-set conversion ignores the order of the elements in the list and removes any duplicate elements. toset(["b", "a", "b"]) will generate a set containing only "a" and "b", in no particular order and the second "b" is discarded.

If you are writing a module whose input variables will be used as a set of strings for for_each, you can set its type to set(string) to avoid the need for explicit type conversion:

variable "vswitch_ids" {
  type = set(string)
}

resource "alicloud_instance" "server" {
  for_each = var.vswitch_ids

  instance_name   = "server-${each.key}"
  instance_type   = "ecs.s6-c1m2.small"
  image_id        = "aliyun_2_1903_x64_20G_alibase_20230523.vhd"
  vswitch_id      = each.key
  security_groups = ["sg-xyz123456"]
  
  # ... (Other arguments are the same as above)
}

Examples

Create multiple RAM users and AccessKey pairs

variable "users" {
  description = "Map of RAM users to create"
  type        = map(object({
    email     = string
    role      = string
  }))
  default     = {
    "dev-user1" = {
      email    = "dev1@example.com"
      role     = "developer"
    },
    "dev-user2" = {
      email    = "dev2@example.com" 
      role     = "developer"
    },
    "admin-user" = {
      email    = "admin@example.com"
      role     = "administrator"
    }
  }
}

resource "alicloud_ram_user" "users" {
  for_each = var.users
  
  name         = each.key
  display_name = each.key
  email        = each.value.email
}

resource "alicloud_ram_access_key" "keys" {
  for_each = alicloud_ram_user.users
  
  user_name   = each.value.name
  status      = "Active"
  secret_file = "./credentials/ak-${each.value.name}.txt"
}

# Assign different policies to users
resource "alicloud_ram_user_policy_attachment" "policy" {
  for_each = var.users
  
  policy_name = each.value.role == "administrator" ?  "AdministratorAccess" : "ReadOnlyAccess"
  policy_type = "System"
  user_name   = alicloud_ram_user.users[each.key].name
}

Use a nested data structure to create a multi-region resource

locals {
  # A nested structure that contains regions and resource definitions for each region
  regions = {
    hangzhou = {
      display_name = "Hangzhou"
      zone_id      = "cn-hangzhou-i"
      instances    = ["small", "medium"]
    }
    beijing = {
      display_name = "Beijing"
      zone_id      = "cn-beijing-h"
      instances    = ["medium", "large"]
    }
  }
  
  # Use the for expression and flatten to create a flat list of instance configurations
  instance_configs = flatten([
    for region_key, region in local.regions : [
      for instance_type in region.instances : {
        region_key    = region_key
        region_name   = region.display_name
        zone_id       = region.zone_id
        instance_type = instance_type
        instance_name = "${region_key}-${instance_type}"
      }
    ]
  ])
  
  # Convert to a map that can be used by for_each
  instance_map = {
    for config in local.instance_configs :
    config.instance_name => config
  }
}

# Create multiple instances using the converted map
resource "alicloud_instance" "multi_region_servers" {
  for_each = local.instance_map
  
  instance_name   = each.value.instance_name
  instance_type   = each.value.instance_type == "small" ?  "ecs.s6-c1m2.small" : (
                     each.value.instance_type == "medium" ?  "ecs.s6-c1m4.xlarge" : "ecs.s6-c1m8.2xlarge")
  image_id        = "aliyun_2_1903_x64_20G_alibase_20230523.vhd"
  
  # Assume that you have created a VPC and a vSwitch for each region.
  vswitch_id = "${vsw_id}_each.value.region_key" # The actual vSwitch ID
  security_groups = ["sg-1231454254"] # The actual security group ID
  
  tags = {
    Name     = each.value.instance_name
    Region   = each.value.region_name
    Type     = each.value.instance_type
  }
}

Use setproduct to create a multi-dimensional resource combination

locals {
  environments   = ["dev", "staging", "prod"]
  instance_types = ["web", "app", "db"]

  # Use setproduct to create all possible combinations
  combinations = [
    for pair in setproduct(local.environments, local.instance_types) : {
      env  = pair[0]
      type = pair[1]
      name = "${pair[0]}-${pair[1]}"
    }
  ]

  # Convert to a map that can be used by for_each
  combination_map = {
    for combo in local.combinations :
    combo.name => combo
  }
}

# Create a security group for each combination
resource "alicloud_security_group" "by_env_and_type" {
  for_each = local.combination_map

  security_group_name = "sg-${each.key}"
  description         = "Security group for ${each.value.type} in ${each.value.env} environment"
  vpc_id = "vpc-abc123456" # Use the existing VPC ID.
}

# Create security group rules that are suitable for each environment and type.
resource "alicloud_security_group_rule" "rules" {
  for_each = local.combination_map

  security_group_id = alicloud_security_group.by_env_and_type[each.key].id
  type              = "ingress"
  ip_protocol       = "tcp"

  # Configure different port rules based on the instance type.
  port_range = each.value.type == "web" ?  "80/80" : (
  each.value.type == "app" ?  "8080/8080" : "3306/3306")

  # Only allow access from specific IP addresses in the production environment
  cidr_ip = each.value.env == "prod" ?  "10.0.0.0/8" : "0.0.0.0/0"

  description = "Default rule for ${each.value.type} in ${each.value.env}"
}

By using for_each, you can manage multiple similar resource instances more flexibly and ensure that resource identities remain stable even if the order of elements in the configuration changes. This enables Terraform to plan changes more accurately and reduce unnecessary resource deletion and recreation.

Note: This topic is based on the official Terraform topic for_each. For more information, see the official documentation.