Use Terraform to create and manage consumer groups - Terraform

This topic describes how to use the alicloud_alikafka_consumer_group resource provided by Terraform to create and manage consumer groups in ApsaraMQ for Kafka.

Note

You can run the sample code in this topic with a few clicks. Click here to run the sample code.

Before you start

By default, an Alibaba Cloud account has full permissions on all resources that belong to this account. Security risks may arise if the credentials of an Alibaba Cloud account are leaked. We recommend that you use a Resource Access Management (RAM) user and create an AccessKey pair for the RAM user. For more information, see Create a RAM user and Create an AccessKey pair.

You must use RAM to manage access permissions on cloud resources in an efficient manner. This helps meet the requirements for multi-user collaboration and allows you to grant permissions to users based on the principle of least privilege to prevent security vulnerabilities caused by excessive permissions. For more information, see Grant permissions to RAM users.

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "vpc:DescribeVpcAttribute",
                "vpc:DescribeRouteTableList",
                "vpc:DescribeVSwitchAttributes",
                "vpc:DeleteVpc",
                "vpc:DeleteVSwitch",
                "vpc:CreateVpc",
                "vpc:CreateVSwitch"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "bss:ModifyAgreementRecord",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "bss:DescribeOrderList",
                "bss:DescribeOrderDetail",
                "bss:PayOrder",
                "bss:CancelOrder"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ecs:CreateSecurityGroup",
                "ecs:ModifySecurityGroupPolicy",
                "ecs:DescribeSecurityGroups",
                "ecs:ListTagResources",
                "ecs:DeleteSecurityGroup",
                "ecs:DescribeSecurityGroupAttribute",
                "ecs:AuthorizeSecurityGroup",
                "ecs:RevokeSecurityGroup"
            ],
            "Resource": "*"
        },
        {
            "Action": "alikafka:*",
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": "ram:CreateServiceLinkedRole",
            "Resource": "*",
            "Effect": "Allow",
            "Condition": {
                "StringEquals": {
                    "ram:ServiceName": [
                        "connector.alikafka.aliyuncs.com",
                        "instanceencryption.alikafka.aliyuncs.com",
                        "alikafka.aliyuncs.com",
                        "etl.alikafka.aliyuncs.com"
                    ]
                }
            }
        }
    ]
}

Prepare the runtime environment for Terraform by using one of the following methods:
- Use Terraform in Terraform Explorer: Alibaba Cloud provides Terraform Explorer, an online runtime environment for Terraform. You can use Terraform after you log on to Terraform Explorer without the need to install Terraform. This method is suitable for scenarios in which you want to use and debug Terraform in a fast and convenient manner at no additional costs.
- Use Terraform in Cloud Shell: Terraform is preinstalled in Cloud Shell, and identity credentials are configured. You can run Terraform commands in Cloud Shell. This method is suitable for scenarios in which you want to use and debug Terraform in a fast and convenient manner at low costs.
- Install and configure Terraform on your on-premises machine: This method is suitable for scenarios in which network connections are unstable or a custom development environment is required.

Required resources

Note

You are charged for specific resources required in this topic. If you no longer require the resources, release or unsubscribe from the resources at the earliest opportunity.

alicloud_vpc: creates a virtual private cloud (VPC).
alicloud_vswitch: creates a vSwitch in a VPC.
alicloud_security_group: creates a security group.
alicloud_alikafka_instance: creates an ApsaraMQ for Kafka instance.
alicloud_alikafka_consumer_group: creates a consumer group.
alicloud_alikakfa_consumer_groups: queries consumer groups.

Create a consumer group

In the following example, a consumer group named tf-example is created in the China (Shenzhen) region. After you create the consumer group, you cannot change the name of the consumer group.

Create a working directory and a configuration file named main.tf in the directory. Copy the following code to the main.tf configuration file:

Create the required resources.

variable "region" {
  default = "cn-shenzhen"
}

variable "instance_name" {
  default = "alikafkaInstanceName"
}

variable "zone_id" {
  default = "cn-shenzhen-f"
}

provider "alicloud" {
  region = var.region
}

# Create a VPC.
resource "alicloud_vpc" "default" {
  vpc_name   = "alicloud"
  cidr_block = "172.16.0.0/16"
}

# Create a vSwitch.
resource "alicloud_vswitch" "default" {
  vpc_id     = alicloud_vpc.default.id
  cidr_block = "172.16.192.0/20"
  zone_id    = var.zone_id
}

# Create a security group. 
resource "alicloud_security_group" "default" {
  vpc_id = alicloud_vpc.default.id
}

# Create an instance. The disk type is ultra disk, the disk capacity is 500 GB, and the traffic specification is alikafka.hw.2xlarge. 
# Deploy the instance. 
resource "alicloud_alikafka_instance" "default" {
  name           = var.instance_name
  partition_num  = 50
  disk_type      = 0
  disk_size      = 500
  deploy_type    = 5
  io_max_spec    = "alikafka.hw.2xlarge"
  vswitch_id     = alicloud_vswitch.default.id
  security_group = alicloud_security_group.default.id
}

In the main.tf configuration file, add the resource "alicloud_alikafka_consumer_group" "default" configuration item.

variable "group_name" {
  default = "tf-example"
}

resource "alicloud_alikafka_consumer_group" "default" {
  consumer_id = var.group_name
  instance_id = alicloud_alikafka_instance.default.id
}

Run the following command to initialize the runtime environment for Terraform:

terraform init

If the following information is returned, Terraform is initialized.

Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "alicloud" (hashicorp/alicloud) 1.90.1...
...

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other

Create an execution plan and preview the changes.
```
terraform plan
```

Run the following command to create the resources:

terraform apply

During the execution, enter yes as prompted and press the Enter key. Wait until the command is executed. If the following information is returned, the command is run.

alicloud_vpc.default: Refreshing state... [id=vpc-****]
alicloud_security_group.default: Refreshing state... [id=sg-****]
alicloud_vswitch.default: Refreshing state... [id=vsw-****]
alicloud_alikafka_instance.default: Refreshing state... [id=alikafka_post-cn-****]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  + create

Terraform will perform the following actions:

  # alicloud_alikafka_consumer_group.default will be created
  + resource "alicloud_alikafka_consumer_group" "default" {
      + consumer_id = "tf-example"
      + id          = (known after apply)
      + instance_id = "alikafka_post-cn-****"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

alicloud_alikafka_consumer_group.default: Creating...
alicloud_alikafka_consumer_group.default: Creation complete after 2s [id=alikafka_post-cn-****:tf-example]

Apply complete!  Resources: 1 added, 0 changed, 0 destroyed.

Verify the result.

Run the terraform show command

Run the terraform show command to view information about the consumer group.

terraform show

# alicloud_alikafka_consumer_group.default:
resource "alicloud_alikafka_consumer_group" "default" {
    consumer_id = "tf-example"
    description = null
    id          = "alikafka_post-cn-****:tf-example"
    instance_id = "alikafka_post-cn-****"
}

Use the ApsaraMQ for Kafka console

Log on to the ApsaraMQ for Kafka console to view information about the consumer group.

Query a consumer group

In the main.tf configuration file, add the data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" configuration item and configure the configuration item based on the following code snippet:
```
data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" {
  instance_id = alicloud_alikafka_instance.default.id
}
```
Create an execution plan and preview the changes.
```
terraform plan
```

Run the following command to create the resources:

terraform apply

During the execution, enter yes as prompted and press the Enter key. Wait until the command is executed. If the following information is returned, the command is run.

alicloud_vpc.default: Refreshing state... [id=vpc-****]
alicloud_security_group.default: Refreshing state... [id=sg-****]
alicloud_vswitch.default: Refreshing state... [id=vsw-****]
alicloud_alikafka_instance.default: Refreshing state... [id=alikafka_post-cn-****]
alicloud_alikafka_consumer_group.default: Refreshing state... [id=alikafka_post-cn-****:tf-example]
data.alicloud_alikafka_consumer_groups.consumer_groups_ds: Reading...
data.alicloud_alikafka_consumer_groups.consumer_groups_ds: Read complete after 1s [id=****]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are
needed.

Apply complete!  Resources: 0 added, 0 changed, 0 destroyed.

Verify the result. Run the terraform show command to view information about the consumer group.

terraform show

# data.alicloud_alikafka_consumer_groups.consumer_groups_ds:
data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" {
    groups      = [
        {
            consumer_id = "tf-example"
            id          = "alikafka_post-cn-****:tf-example"
            instance_id = "alikafka_post-cn-****"
            remark      = null
            tags        = {}
        },
    ]
    id          = "****"
    ids         = [
        "alikafka_post-cn-****:tf-example",
    ]
    instance_id = "alikafka_post-cn-****"
    names       = [
        "tf-example",
    ]
}

Clear resources

If you no longer require the preceding resources that are created or managed by using Terraform, run the following command to release the resources. For more information about the terraform destroy command, see Common commands.

terraform destroy

Sample code

Note

You can run the sample code with a few clicks. Click here to run the sample code.

Sample code

variable "region" {
  default = "cn-shenzhen"
}

variable "instance_name" {
  default = "alikafkaInstanceName"
}

variable "zone_id" {
  default = "cn-shenzhen-f"
}

variable "group_name" {
  default = "tf-example"
}

provider "alicloud" {
  region = var.region
}

# Create a VPC.
resource "alicloud_vpc" "default" {
  vpc_name   = "alicloud"
  cidr_block = "172.16.0.0/16"
}

# Create a vSwitch.
resource "alicloud_vswitch" "default" {
  vpc_id     = alicloud_vpc.default.id
  cidr_block = "172.16.192.0/20"
  zone_id    = var.zone_id
}

# Create a security group. 
resource "alicloud_security_group" "default" {
  vpc_id = alicloud_vpc.default.id
}

# Create an instance. The disk type is ultra disk, the disk capacity is 500 GB, and the traffic specification is alikafka.hw.2xlarge. 
# Deploy the instance. 
resource "alicloud_alikafka_instance" "default" {
  name           = var.instance_name
  partition_num  = 50
  disk_type      = 0
  disk_size      = 500
  deploy_type    = 5
  io_max_spec    = "alikafka.hw.2xlarge"
  vswitch_id     = alicloud_vswitch.default.id
  security_group = alicloud_security_group.default.id
}

resource "alicloud_alikafka_consumer_group" "default" {
  consumer_id = var.group_name
  instance_id = alicloud_alikafka_instance.default.id
}

data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" {
  instance_id = alicloud_alikafka_instance.default.id
  depends_on  = [alicloud_alikafka_consumer_group.default]
}

References

You can also manage consumer groups in the ApsaraMQ for Kafka console or by calling API operations. For more information, see Step 3: Create resources and CreateConsumerGroup.
For information about other arguments provided by the alicloud_alikafka_consumer_group resource, see alicloud_alikafka_consumer_group.