Create and manage consumer groups with Terraform - Terraform

This topic describes how to use the Terraform resource alicloud_alikafka_consumer_group to create and manage consumer groups for ApsaraMQ for Kafka.

Note

You can run the sample code in this topic with one click. Run with one click

Prerequisites

An Alibaba Cloud account has full access to all its resources. Leaked credentials for this account pose a significant security risk, so we recommend using a RAM user and creating an AccessKey for that user. For more information, see Create a RAM user and Create an AccessKey.

RAM helps you manage access permissions for your cloud resources. Granting permissions based on the principle of least privilege supports multi-user collaboration and helps prevent security risks that arise from excessive permissions. For more information, see Manage RAM user permissions.

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "vpc:DescribeVpcAttribute",
                "vpc:DescribeRouteTableList",
                "vpc:DescribeVSwitchAttributes",
                "vpc:DeleteVpc",
                "vpc:DeleteVSwitch",
                "vpc:CreateVpc",
                "vpc:CreateVSwitch"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "bss:ModifyAgreementRecord",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "bss:DescribeOrderList",
                "bss:DescribeOrderDetail",
                "bss:PayOrder",
                "bss:CancelOrder"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ecs:CreateSecurityGroup",
                "ecs:ModifySecurityGroupPolicy",
                "ecs:DescribeSecurityGroups",
                "ecs:ListTagResources",
                "ecs:DeleteSecurityGroup",
                "ecs:DescribeSecurityGroupAttribute",
                "ecs:AuthorizeSecurityGroup",
                "ecs:RevokeSecurityGroup"
            ],
            "Resource": "*"
        },
        {
            "Action": "alikafka:*",
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": "ram:CreateServiceLinkedRole",
            "Resource": "*",
            "Effect": "Allow",
            "Condition": {
                "StringEquals": {
                    "ram:ServiceName": [
                        "connector.alikafka.aliyuncs.com",
                        "instanceencryption.alikafka.aliyuncs.com",
                        "alikafka.aliyuncs.com",
                        "etl.alikafka.aliyuncs.com"
                    ]
                }
            }
        }
    ]
}

Set up a Terraform runtime environment using one of the following methods.
- Use Terraform in Terraform Explorer: Alibaba Cloud offers Terraform Explorer, a free online environment that lets you use Terraform without local installation. It is ideal for quick trials and debugging.
- Use Terraform to create resources quickly: Cloud Shell has Terraform pre-installed and credentials pre-configured, allowing you to run Terraform commands directly. This method is convenient and low-cost.
- Install and configure Terraform locally: This method is suitable if you have poor network connectivity or need a customized development environment.

Resources used

Note

Some of the resources used in this tutorial incur charges. To avoid unexpected costs, release or unsubscribe from these resources when they are no longer needed.

alicloud_vpc: Creates a Virtual Private Cloud (VPC).
alicloud_vswitch: Creates a vSwitch.
alicloud_security_group: Creates a security group.
alicloud_alikafka_instance: Creates an ApsaraMQ for Kafka instance.
alicloud_alikafka_consumer_group: Creates a consumer group.
alicloud_alikafka_consumer_groups: Queries consumer groups.

Create a consumer group

This example creates a consumer group named tf-example in the China (Shenzhen) region. The consumer group name cannot be changed after creation.

Create a new working directory and a file named main.tf in it. Copy the following code into the main.tf file.

Create the prerequisite resources.

variable "region" {
  default = "cn-shenzhen"
}

variable "instance_name" {
  default = "alikafkaInstanceName"
}

variable "zone_id" {
  default = "cn-shenzhen-f"
}

provider "alicloud" {
  region = var.region
}

# Create a VPC.
resource "alicloud_vpc" "default" {
  vpc_name   = "alicloud"
  cidr_block = "172.16.0.0/16"
}

# Create a vSwitch.
resource "alicloud_vswitch" "default" {
  vpc_id     = alicloud_vpc.default.id
  cidr_block = "172.16.192.0/20"
  zone_id    = var.zone_id
}

# Create a security group.
resource "alicloud_security_group" "default" {
  vpc_id = alicloud_vpc.default.id
}

# Create an instance with an ultra disk of 500 GB and a traffic specification of alikafka.hw.2xlarge.
# Deploy the instance.
resource "alicloud_alikafka_instance" "default" {
  name           = var.instance_name
  partition_num  = 50
  disk_type      = 0
  disk_size      = 500
  deploy_type    = 5
  io_max_spec    = "alikafka.hw.2xlarge"
  vswitch_id     = alicloud_vswitch.default.id
  security_group = alicloud_security_group.default.id
}

In the main.tf file, add the resource "alicloud_alikafka_consumer_group" "default" configuration block.

variable "group_name" {
  default = "tf-example"
}

resource "alicloud_alikafka_consumer_group" "default" {
  consumer_id = var.group_name
  instance_id = alicloud_alikafka_instance.default.id
}

Run the following command to initialize the Terraform runtime environment.

terraform init

The following output indicates that the initialization is successful.

Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "alicloud" (hashicorp/alicloud) 1.90.1...
...

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other

Create an execution plan to preview the changes.
```
terraform plan
```

Run the following command to create the resources.

terraform apply

During the execution, enter yes and press the Enter key when prompted. Wait for the command to finish. If the following information is displayed, the operation was successful.

alicloud_vpc.default: Refreshing state... [id=vpc-****]
alicloud_security_group.default: Refreshing state... [id=sg-****]
alicloud_vswitch.default: Refreshing state... [id=vsw-****]
alicloud_alikafka_instance.default: Refreshing state... [id=alikafka_post-cn-****]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the
following symbols:
  + create

Terraform will perform the following actions:

  # alicloud_alikafka_consumer_group.default will be created
  + resource "alicloud_alikafka_consumer_group" "default" {
      + consumer_id = "tf-example"
      + id          = (known after apply)
      + instance_id = "alikafka_post-cn-****"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

alicloud_alikafka_consumer_group.default: Creating...
alicloud_alikafka_consumer_group.default: Creation complete after 2s [id=alikafka_post-cn-****:tf-example]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Verify the results.

Run terraform show

Run the terraform show command to view the consumer group.

terraform show

# alicloud_alikafka_consumer_group.default:
resource "alicloud_alikafka_consumer_group" "default" {
    consumer_id = "tf-example"
    description = null
    id          = "alikafka_post-cn-****:tf-example"
    instance_id = "alikafka_post-cn-****"
}

Log on to the console

Log on to the ApsaraMQ for Kafka console to view the consumer group.

Query a consumer group

In the preceding main.tf file, add the data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" configuration block. The configuration is as follows.
```
data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" {
  instance_id = alicloud_alikafka_instance.default.id
}
```
Create an execution plan to preview the changes.
```
terraform plan
```

Run the following command to query resources.

terraform apply

During execution, when prompted, enter yes and press the Enter key. The command has run successfully if the following message is displayed.

alicloud_vpc.default: Refreshing state... [id=vpc-****]
alicloud_security_group.default: Refreshing state... [id=sg-****]
alicloud_vswitch.default: Refreshing state... [id=vsw-****]
alicloud_alikafka_instance.default: Refreshing state... [id=alikafka_post-cn-****]
alicloud_alikafka_consumer_group.default: Refreshing state... [id=alikafka_post-cn-****:tf-example]
data.alicloud_alikafka_consumer_groups.consumer_groups_ds: Reading...
data.alicloud_alikafka_consumer_groups.consumer_groups_ds: Read complete after 1s [id=****]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are
needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Verify the results. Run terraform show to view the Kafka consumer group information.

terraform show

# data.alicloud_alikafka_consumer_groups.consumer_groups_ds:
data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" {
    groups      = [
        {
            consumer_id = "tf-example"
            id          = "alikafka_post-cn-****:tf-example"
            instance_id = "alikafka_post-cn-****"
            remark      = null
            tags        = {}
        },
    ]
    id          = "****"
    ids         = [
        "alikafka_post-cn-****:tf-example",
    ]
    instance_id = "alikafka_post-cn-****"
    names       = [
        "tf-example",
    ]
}

Clean up resources

When you no longer need the resources created in this tutorial, run the following command to destroy them. For more information about the terraform destroy command, see Common Terraform commands.

terraform destroy

Complete example

Note

You can run the sample code in this topic with one click. Run with one click

Sample code

variable "region" {
  default = "cn-shenzhen"
}

variable "instance_name" {
  default = "alikafkaInstanceName"
}

variable "zone_id" {
  default = "cn-shenzhen-f"
}

variable "group_name" {
  default = "tf-example"
}

provider "alicloud" {
  region = var.region
}

# Create a VPC.
resource "alicloud_vpc" "default" {
  vpc_name   = "alicloud"
  cidr_block = "172.16.0.0/16"
}

# Create a vSwitch.
resource "alicloud_vswitch" "default" {
  vpc_id     = alicloud_vpc.default.id
  cidr_block = "172.16.192.0/20"
  zone_id    = var.zone_id
}

# Create a security group.
resource "alicloud_security_group" "default" {
  vpc_id = alicloud_vpc.default.id
}

# Create an instance with an ultra disk of 500 GB and a traffic specification of alikafka.hw.2xlarge.
# Deploy the instance.
resource "alicloud_alikafka_instance" "default" {
  name           = var.instance_name
  partition_num  = 50
  disk_type      = 0
  disk_size      = 500
  deploy_type    = 5
  io_max_spec    = "alikafka.hw.2xlarge"
  vswitch_id     = alicloud_vswitch.default.id
  security_group = alicloud_security_group.default.id
}

resource "alicloud_alikafka_consumer_group" "default" {
  consumer_id = var.group_name
  instance_id = alicloud_alikafka_instance.default.id
}

data "alicloud_alikafka_consumer_groups" "consumer_groups_ds" {
  instance_id = alicloud_alikafka_instance.default.id
  depends_on  = [alicloud_alikafka_consumer_group.default]
}

Terraform:Manage Kafka consumer groups with Terraform

Prerequisites

Resources used

Create a consumer group

Run terraform show

Log on to the console

Query a consumer group

Clean up resources

Complete example

Sample code

Related documentation