This topic describes how to deploy Apache Kafka on a single Elastic Compute Service (ECS) instance by creating a stack in the Resource Orchestration Service (ROS) console.
Background information
Apache Kafka is an open source stream processing platform that is written in the Scala and Java programming languages. Apache Kafka is a high-throughput distributed publish-subscribe messaging system for processing streaming data of customer behaviors on websites.
You can use the Installs Kafka on an ECS instance (existing VPC) sample template to create an ECS instance based on existing resources and associate an elastic IP address (EIP) with the instance. The existing resources include a virtual private cloud (VPC), vSwitch, and security group. The following software versions are used in the sample template:
JDK 1.8.0: Java Development Kit (JDK)
Scala 2.12: the programming language
Apache Kafka 0.10.2.2: the compute engine
The Apache Kafka data in the sample template is stored in a data disk. The default storage directory of Apache Kafka data is /home/software/. The default storage directory of Apache Kafka binaries is /home/software/kafka/bin.
After you create a stack by using the sample template, you can obtain the value of KafkaManagerUrl and log on to the Apache Kafka management console. KafkaManagerUrl specifies the URL of the Apache Kafka management console. If you want to access the URL over the Internet, you must configure an inbound rule for the security group to allow traffic on port 9000. For more information, see Add a security group rule.
Step 1: Create a stack
Log on to the ROS console.
In the left-side navigation pane, choose .
Find the Installs Kafka on an ECS instance (existing VPC) template.
Click Create Stack.
In the Configure Parameters step, configure Stack Name and the following parameters.
Parameter
Description
Example
VSwitch Zone ID
The zone ID of the vSwitch in the VPC.
Qingdao Zone C
Existing VPC Instance ID
The VPC ID.
For more information about how to create and query a VPC, see Create and manage a VPC.
vpc-bp1m6fww66xbntjyc****
VSwitch ID
The ID of the vSwitch that resides in the VPC.
For more information about how to create and query a vSwitch, see Create and manage a vSwitch.
vsw-bp183p93qs667muql****
Business Security Group ID
The ID of the ECS security group.
For more information about how to query the ID of a security group, see Search for security groups.
sg-bp15ed6xe1yxeycg7o****
Instance Type
The ECS instance type.
Select a valid instance type. For more information, see Overview of instance families.
ecs.c5.large
Image ID
The image ID of the ECS instance.
For more information, see Find an image.
centos_7
Instance Password
The password of the ECS instance.
Test_12****
Public IP Bandwidth
The bandwidth of the public IP address.
Unit: Mbit/s.
Valid values: 1 to 100.
5
Disk Type
Valid values:
cloud_efficiency: ultra disk
cloud_ssd: standard SSD
cloud_essd: Enterprise SSD (ESSD)
cloud: basic disk
ephemeral_ssd: local SSD
For more information, see Disks.
cloud_efficiency
Data Disk Space
The data disk size of the ECS instance.
Valid values: 20 to 32786.
Unit: GB.
20
Kafka Listeners Port
The listener port. The port number must be greater than 1000.
Default value: 9092.
9092
Message Retention Time
The maximum retention period of a message.
Default value: 24.
Unit: hour.
24
Topic Specifications
The number of topics.
Default value: 50.
50
Click Create.
On the Stack Information tab, view the status of the stack. Wait until the stack is created. Then, click Outputs to obtain the value of KafkaManagerUrl.
Use the value of KafkaManagerUrl to log on to the Apache Kafka management console.
Step 2: View resources
In the left-side navigation pane, click Stacks.
On the Stacks page, click the name of the created stack.
Click the Resources tab to view the information about resources in the stack.
The following table describes the resources in this example.
Resource
Quantity
Description
Specifications
ALIYUN::ECS::Instance
1
Creates an ECS instance to deploy Apache Kafka.
An ECS instance that has the following specifications is created:
Instance type: ecs.c5.large.
Disk category: ultra disk.
System disk size: 40 GB.
Public IP address: A public IP address is allocated.
Note For more information about the resource charges, see the pricing schedule on the official website or the product pricing documentation.