Use Terraform to create, query, and delete ApsaraMQ for Kafka instances - ApsaraMQ for Kafka

The alicloud_alikafka_instance resource of Terraform provides arguments to manage ApsaraMQ for Kafka instances. You can use Terraform to automatically create and deploy ApsaraMQ for Kafka instances. This shortens the deployment time, ensures configuration consistency in multiple environments, and reduces errors caused by manual operations.

Note

You can run the sample code in this topic with a few clicks.

Before you start

An Alibaba Cloud account has full permissions on all resources that belong to this account. If the credentials of an Alibaba Cloud account are leaked, security risks may arise. We recommend that you use a Resource Access Management (RAM) user and create an AccessKey pair for the RAM user. For more information, see Create a RAM user and Create an AccessKey pair.

You must use RAM to manage access permissions on cloud resources in an efficient manner. This helps meet the requirements for multi-user collaboration and allows you to grant permissions to users based on the principle of least privilege to prevent security vulnerabilities caused by excessive permissions. For more information, see Grant permissions to RAM users.

{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "vpc:DescribeVpcAttribute",
                "vpc:DescribeRouteTableList",
                "vpc:DescribeVSwitchAttributes",
                "vpc:DeleteVpc",
                "vpc:DeleteVSwitch",
                "vpc:CreateVpc",
                "vpc:CreateVSwitch"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "bss:ModifyAgreementRecord",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "bss:DescribeOrderList",
                "bss:DescribeOrderDetail",
                "bss:PayOrder",
                "bss:CancelOrder"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ecs:CreateSecurityGroup",
                "ecs:ModifySecurityGroupPolicy",
                "ecs:DescribeSecurityGroups",
                "ecs:ListTagResources",
                "ecs:DeleteSecurityGroup",
                "ecs:DescribeSecurityGroupAttribute",
                "ecs:AuthorizeSecurityGroup",
                "ecs:RevokeSecurityGroup"
            ],
            "Resource": "*"
        },
        {
            "Action": "alikafka:*",
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": "ram:CreateServiceLinkedRole",
            "Resource": "*",
            "Effect": "Allow",
            "Condition": {
                "StringEquals": {
                    "ram:ServiceName": [
                        "connector.alikafka.aliyuncs.com",
                        "instanceencryption.alikafka.aliyuncs.com",
                        "alikafka.aliyuncs.com",
                        "etl.alikafka.aliyuncs.com"
                    ]
                }
            }
        }
    ]
}

Prepare the runtime environment for Terraform by using one of the following methods:
- Use Terraform in Terraform Explorer: Alibaba Cloud provides Terraform Explorer, an online runtime environment for Terraform. You can use Terraform after you log on to Terraform Explorer without the need to install Terraform. This method is suitable for scenarios in which you want to use and debug Terraform in a fast and convenient manner at no additional costs.
- Use Terraform in Cloud Shell: Terraform is preinstalled in Cloud Shell and identity credentials are configured. You can directly run Terraform commands in Cloud Shell. This method is suitable for scenarios in which you want to use and debug Terraform in a fast and convenient manner at low costs.
- Install and configure Terraform on your on-premises machine: This method is suitable for scenarios in which network connections are unstable or a custom development environment is required.

Required resources

Note

You are charged for specific resources required in this topic. If you no longer require the resources, release or unsubscribe from the resources at the earliest opportunity.

alicloud_vpc: creates a virtual private cloud (VPC).
alicloud_vswitch: creates a vSwitch in a VPC.
alicloud_security_group: creates a security group.
alicloud_alikafka_instance: creates an ApsaraMQ for Kafka instance.
alicloud_alikakfa_instances: queries ApsaraMQ for Kafka instances.

Create and deploy an instance

Note

You cannot use Terraform to delete subscription ApsaraMQ for Kafka instances.

In the following example, an ApsaraMQ for Kafka instance named alikafkaInstanceName is created in the China (Shenzhen) region. The instance is a VPC-connected instance that uses the alikafka.hw.2xlarge specification.

Create a working directory and a file named main.tf under the directory. Then, copy the following content to the main.tf file.

variable "region" {
  default = "cn-shenzhen"
}

variable "instance_name" {
  default = "alikafkaInstanceName"
}

variable "zone_id" {
  default = "cn-shenzhen-f"
}

provider "alicloud" {
  region = var.region
}

# Create a VPC.
resource "alicloud_vpc" "default" {
  vpc_name   = "alicloud"
  cidr_block = "172.16.0.0/16"
}

# Create a vSwitch.
resource "alicloud_vswitch" "default" {
  vpc_id     = alicloud_vpc.default.id
  cidr_block = "172.16.192.0/20"
  zone_id    = var.zone_id
}

# Create a security group. 
resource "alicloud_security_group" "default" {
  vpc_id = alicloud_vpc.default.id
}

# Create an instance. The disk type is ultra disk, the disk capacity is 500 GB, and the traffic specification is alikafka.hw.2xlarge. 
# Deploy the instance. 
resource "alicloud_alikafka_instance" "default" {
  name           = var.instance_name
  partition_num  = 50
  disk_type      = 0
  disk_size      = 500
  deploy_type    = 5
  io_max_spec    = "alikafka.hw.2xlarge"
  vswitch_id     = alicloud_vswitch.default.id
  security_group = alicloud_security_group.default.id
}

Run the following command to initialize the runtime environment for Terraform:

terraform init

If the following information is returned, Terraform is initialized.

Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "alicloud" (hashicorp/alicloud) 1.90.1...
...

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other

Create an execution plan and preview the changes.
```
terraform plan
```

Run the following command to create the resources:

terraform apply

During the execution, enter yes as prompted and press the Enter key. Wait until the command is executed. If the following information is returned, the command is run.

...

alicloud_alikafka_instance.default: Destroying... [id=alikafka_post-cn-****]
alicloud_alikafka_instance.default: Still destroying... [id=alikafka_post-cn-****, 10s elapsed]
alicloud_alikafka_instance.default: Destruction complete after 11s
alicloud_security_group.default: Destroying... [id=sg-****]
alicloud_vswitch.vsw: Destroying... [id=vsw-****]
alicloud_security_group.default: Destruction complete after 1s
alicloud_vpc.default: Creating...
alicloud_vswitch.vsw: Destruction complete after 4s
alicloud_vpc.vpc: Destroying... [id=vpc-****]
alicloud_vpc.vpc: Destruction complete after 1s
alicloud_vpc.default: Creation complete after 4s [id=vpc-****]
alicloud_security_group.default: Creating...
alicloud_vswitch.default: Creating...
alicloud_security_group.default: Creation complete after 1s [id=sg-****]
alicloud_vswitch.default: Creation complete after 4s [id=vsw-****]
alicloud_alikafka_instance.default: Creating...
alicloud_alikafka_instance.default: Still creating... [10s elapsed]
...
alicloud_alikafka_instance.default: Creation complete after 3m42s [id=alikafka_post-cn-****]

Apply complete!  Resources: 4 added, 0 changed, 0 destroyed.

Verify the result.

Use the terraform show command

Run the terraform show command to view information about the ApsaraMQ for Kafka instance.

terraform show

# alicloud_alikafka_instance.default:
resource "alicloud_alikafka_instance" "default" {
    config            = jsonencode(
        {
            "cloud.maxTieredStoreSpace"       = "0"
            "enable.acl"                      = "false"
            "enable.compact"                  = "true"
            "enable.tiered"                   = "false"
            "enable.vpc_sasl_ssl"             = "false"
            "kafka.log.retention.hours"       = "72"
            "kafka.message.max.bytes"         = "1048576"
            "kafka.offsets.retention.minutes" = "10080"
            "kafka.ssl.bit"                   = "1024"
        }
    )
    deploy_type       = 5
    disk_size         = 500
    disk_type         = 0
    eip_max           = 0
    end_point         = "172.16.201.166:9092,172.16.201.167:9092,172.16.201.165:9092"
    group_left        = 2100
    group_used        = 0
    id                = "alikafka_post-cn-jfj40boei007"
    io_max            = 20
    io_max_spec       = "alikafka.hw.2xlarge"
    is_partition_buy  = 1
    kms_key_id        = null
    name              = "alikafkaInstanceName"
    paid_type         = "PostPaid"
    partition_left    = 1050
    partition_num     = 50
    partition_used    = 0
    resource_group_id = "rg-acfmykd63gtpfpa"
    security_group    = "sg-wz95ccy5lzy46acczhsm"
    service_version   = "2.2.0"
    spec_type         = "normal"
    status            = 5
    topic_left        = 1050
    topic_num_of_buy  = 1050
    topic_quota       = 1050
    topic_used        = 0
    vpc_id            = "vpc-wz9gv3xgeoyxte8t4zab0"
    vswitch_id        = "vsw-wz9u6icvx3k1hy2d44emk"
    zone_id           = "zonef"
}

Use the ApsaraMQ for Kafka console

Log on to the ApsaraMQ for Kafka console to view information about the ApsaraMQ for Kafka instance.

Query an instance

In the following example, the first listed ApsaraMQ for Kafka instance in your Alibaba Cloud account is queried.

In the main.tf configuration file, add the data "alicloud_alikafka_instances" "instances_ds" configuration item and configure the configuration item based on the following code snippet:

data "alicloud_alikafka_instances" "instances_ds" {
  output_file = "instances.txt"
}

output "first_instance_name" {
  # The index [0] indicates the first item in the list. 
  value = data.alicloud_alikafka_instances.instances_ds.instances.0.name
}

Create an execution plan and preview the changes.
```
terraform plan
```

Run the following command to create the resources:

terraform apply

During the execution, enter yes as prompted and press the Enter key. Wait until the command is executed. If the following information is returned, the command is run.

alicloud_vpc.default: Refreshing state... [id=vpc-****]
alicloud_security_group.default: Refreshing state... [id=sg-****]
alicloud_vswitch.default: Refreshing state... [id=vsw-****]
alicloud_alikafka_instance.default: Refreshing state... [id=alikafka_post-cn-****]
data.alicloud_alikafka_instances.instances_ds: Reading...
data.alicloud_alikafka_instances.instances_ds: Read complete after 0s [id=****]

Changes to Outputs:
  + first_instance_name = "alikafkaInstanceName"

You can apply this plan to save these new output values to the Terraform state, without changing any real
infrastructure.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes


Apply complete!  Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

first_instance_name = "alikafkaInstanceName"

Verify the result.

In the instances.txt file, view information about the ApsaraMQ for Kafka instance.

[
	{
		"config": "{\"enable.vpc_sasl_ssl\":\"false\",\"kafka.log.retention.hours\":\"72\",\"kafka.offsets.retention.minutes\":\"10080\",\"enable.tiered\":\"false\",\"cloud.maxTieredStoreSpace\":\"0\",\"enable.acl\":\"true\",\"kafka.ssl.bit\":\"1024\",\"enable.compact\":\"true\",\"kafka.message.max.bytes\":\"1048576\"}",
		"create_time": 1698908400000,
		"deploy_type": 5,
		"disk_size": 900,
		"disk_type": 0,
		"domain_endpoint": "alikafka-post-cn-uax3go6z****-1-vpc.alikafka.aliyuncs.com:9092,alikafka-post-cn-****-2-vpc.alikafka.aliyuncs.com:9092,alikafka-post-cn-****-3-vpc.alikafka.aliyuncs.com:9092",
		"eip_max": 0,
		"end_point": "192.168.XX.XX:9092,192.168.XX.XX:9092,192.168.XX.XX:9092",
		"expired_time": 2014527600000,
		"id": "alikafka_post-cn-****",
		"io_max": 20,
		"msg_retain": 72,
		"name": "alikafka_post-cn-****",
		"paid_type": "PostPaid",
		"sasl_domain_endpoint": "alikafka-post-cn-****-1-vpc.alikafka.aliyuncs.com:9094,alikafka-post-cn-****-2-vpc.alikafka.aliyuncs.com:9094,alikafka-post-cn-****-3-vpc.alikafka.aliyuncs.com:9094",
		"security_group": "sg-****",
		"service_status": 5,
		"service_version": "2.2.0",
		"spec_type": "professional",
		"ssl_domain_endpoint": null,
		"ssl_end_point": "",
		"tags": {},
		"upgrade_service_detail_info": [
			{
				"current2_open_source_version": "2.2.0"
			}
		],
		"vpc_id": "vpc-****",
		"vswitch_id": "vsw-****",
		"zone_id": "zonek"
	},
]

Clear resources

If you no longer require the preceding resources that are created or managed by using Terraform, run the following command to release the resources. For more information about the terraform destroy command, see Common commands.

terraform destroy

Sample code

Note

You can run the sample code with a few clicks.

Sample code

variable "region" {
  default = "cn-shenzhen"
}

variable "instance_name" {
  default = "alikafkaInstanceName"
}

variable "zone_id" {
  default = "cn-shenzhen-f"
}

provider "alicloud" {
  region = var.region
}

# Create a VPC.
resource "alicloud_vpc" "default" {
  vpc_name   = "alicloud"
  cidr_block = "172.16.0.0/16"
}

# Create a vSwitch.
resource "alicloud_vswitch" "default" {
  vpc_id     = alicloud_vpc.default.id
  cidr_block = "172.16.192.0/20"
  zone_id    = var.zone_id
}

# Create a security group. 
resource "alicloud_security_group" "default" {
  vpc_id = alicloud_vpc.default.id
}

# Create an instance. The disk type is ultra disk, the disk capacity is 500 GB, and the traffic specification is alikafka.hw.2xlarge. 
# Deploy the instance. 
resource "alicloud_alikafka_instance" "default" {
  name           = var.instance_name
  partition_num  = 50
  disk_type      = 0
  disk_size      = 500
  deploy_type    = 5
  io_max_spec    = "alikafka.hw.2xlarge"
  vswitch_id     = alicloud_vswitch.default.id
  security_group = alicloud_security_group.default.id
}

# Query instances.
data "alicloud_alikafka_instances" "instances_ds" {
  depends_on = [alicloud_alikafka_instance.default]
  output_file = "instances.txt"
}

output "first_instance_name" {
  # The index [0] indicates the first item in the list. 
  value = data.alicloud_alikafka_instances.instances_ds.instances.0.name
}

References

You can also manage instances in the ApsaraMQ for Kafka console or by calling API operations. For more information, see Step 2: Purchase and deploy an instance and CreatePostPayOrder.
For information about other arguments provided by the alicloud_alikafka_instance resource, see alicloud_alikafka_instance.