This topic describes how to deploy Apache Kafka on a single Elastic Compute Service (ECS) instance by creating a stack in the Resource Orchestration Service (ROS) console.

Background information

Apache Kafka is an open source stream processing platform that is written in Scala and Java. It is a high-throughput distributed publish-subscribe messaging system for processing streaming data of customer behaviors on websites.

The Deploy Kafka on an ECS Instance in an Existing VPC sample template creates an ECS instance based on existing resources such as a virtual private cloud (VPC), a vSwitch, and a security group, and associates an elastic IP address (EIP) with the instance. The following software versions are used in the sample template:

  • JDK: 1.8.0
  • Scala: 2.12
  • Kafka: 0.10.2.2

The Kafka data for the sample template is stored in a data disk. The default storage directory of the Kafka data is /home/software/. The default storage directory of Kafka binaries is /home/software/kafka/bin.

After a stack is created by using the sample template, you can obtain the Kafka Manager URL and log on to the Kafka management console. If you want to access the Kafka Manager URL over the Internet, add inbound rules to the security group to allow traffic on port 9000. For more information, see Add security group rules.

Step 1: Create a stack

  1. Log on to the ROS console.
  2. In the left-side navigation pane, choose Templates > Sample Templates.
  3. Find the Deploy Kafka on an ECS Instance in an Existing VPC template.
  4. Click Create Stack.
  5. In the Configure Template Parameters step, set Stack Name and the following parameters.
    Parameter Description Example
    VSwitch Zone ID The zone ID of the vSwitch in the VPC. Qingdao Zone C
    Existing VPC Instance ID The ID of the VPC.

    For more information about how to create and query a VPC, see Create a VPC.

    vpc-bp1m6fww66xbntjyc****
    VSwitch ID The ID of the vSwitch in the VPC.

    For more information about how to create and query a vSwitch, see Create a vSwitch.

    vsw-bp183p93qs667muql****
    Business Security Group ID The ID of the ECS security group.

    For more information about how to query security groups, see Query security groups.

    sg-bp15ed6xe1yxeycg7o****
    Instance Type The instance type of the ECS instance.

    Select a valid instance type. For more information, see Instance families.

    ecs.c5.large
    Instance Password The password that is used to log on to the ECS instance. Test_12****
    Public IP Bandwidth The public IP bandwidth.

    Unit: Mbit/s.

    5
    Disk Type The disk category of the ECS instance. Valid values:
    • cloud_efficiency: ultra disk
    • cloud_ssd: standard SSD
    • cloud_essd: enhanced SSD (ESSD)
    • cloud: basic disk
    • ephemeral_ssd: local SSD

    For more information, see Cloud disks.

    cloud_efficiency
    Data Disk Space The data disk size of the ECS instance.

    Valid values: 20 to 32786.

    Unit: GB.

    20
    Kafka Listeners Port The Kafka listener port. The port number must be greater than 1000.

    Default value: 9092.

    9092
    Message Retention Time The maximum period of time that a message can be retained.

    Default value: 24.

    Unit: hours.

    24
    Topic Specifications The number of topics.

    Default value: 50.

    50
  6. Click Create.
  7. View the stack status on the Stack Information tab of the stack management page. After the stack is created, click Outputs to view the Kafka Manager URL.
  8. Use the URL to log on to the Kafka management console.

Step 2: View resources

  1. Log on to the ROS console.
  2. In the left-side navigation pane, click Stacks.
  3. On the Stacks page, click the stack that you created.
  4. On the stack management page, click the Resources tab to view the resource list.
    The following table describes the resources in this example.
    Resource type Quantity Description Specifications
    ALIYUN::ECS::Instance 1 Creates an ECS instance to deploy Apache Kafka. A single instance of the following specifications is created:
    • InstanceType: ecs.c5.large
    • SystemDiskCategory: cloud_efficiency
    • SystemDiskSize: 40
    • AllocatePublicIP: true
    Note For more information about the resource charges, see the pricing schedule on the official website or the product pricing documentation.