This topic describes how to create and deploy an ApsaraMQ for Kafka instance and connect to it from a virtual private cloud (VPC). By following these steps, you can obtain a running Kafka instance and its endpoint.
Before you begin
A VPC and a vSwitch must be available in the destination region. If not, you can create them during the deployment process.
When you deploy a Kafka instance for the first time, a dialog box appears and prompts you to grant permissions to the service-linked role AliyunServiceRoleForAlikafka. Click OK. This authorization is required for the service to function correctly.
Step 1: Purchase an instance
Log on to the ApsaraMQ for Kafka console.
In the Quick Start section on the right side of the page, click Purchase Instance.
In the Select A Billing Method For The Instance You Want To Create panel, select Instance Edition and Billing Method:
Billing Method
Scenario
Pay-as-you-go
Development, testing, or workloads with fluctuating traffic
Subscription
Production environments or scenarios with predictable costs
Serverless
Pay for what you use. Supports automatic scaling.
Then, click OK.
On the purchase page, configure the instance parameters.
Parameter
Description
Specification Type
Standard Edition (High-write) is suitable for most scenarios.
Region
Select the same region as your application servers to reduce network latency.
Instance Type
Select VPC-connected instance.
Traffic Specification
Select a specification based on your estimated throughput. For example,
alikafka.hw.2xlargesupports 20 MB/s for reads and writes.Disk Type
Ultra Disk is suitable for general use. SSD is suitable for high-throughput scenarios or scenarios with heavy message accumulation.
Disk Capacity
Estimate the required capacity based on the message retention period. Start with 300 GB to 900 GB.
Click Buy Now and complete the payment.
After the purchase is successful, an instance in the Not Deployed state appears on the Instances page.
Step 2: Get VPC information
Before you deploy the instance, obtain the VPC ID and the vSwitch ID.
Log on to the VPC console.
In the navigation pane on the left, click vSwitches.
In the top navigation bar, select the destination region.
On the vSwitches page, record the following information:
vSwitch ID: in the Instance ID/Name column
VPC ID: in the VPC column
The vSwitch determines the zone where the Kafka instance is deployed. To reduce network latency, select a zone that is the same as or close to your application servers.
Quickly create a VPC if you do not have one
In the VPC console, click Create VPC.
Enter a name and set the IPv4 CIDR block, such as
192.168.0.0/16.Create a vSwitch at the same time. Select a zone and set the subnet, such as
192.168.0.0/24.
Step 3: Deploy the instance
Go back to the Instances page in the ApsaraMQ for Kafka console and find the instance that is in the Not Deployed state.
In the Actions column, click Deploy.
In the Deploy Instance panel, configure the following parameters:
Parameter
Description
VPC ID
Select the VPC that you obtained in Step 2.
vSwitch ID
Select the vSwitch that you obtained in Step 2.
Version
Select
2.6.2(recommended) or2.2.0.Message Retention Period
The duration for which messages are stored on the server-side. The default value is 72 hours.
Maximum Message Size
The maximum size of a single message. The default value is 1 MB.
Consumer Offset Retention Period
The duration for which consumer offsets are stored. The default value is 10,080 minutes (7 days).
Disk Encryption
Enable this feature if you require at-rest encryption.
Additional configurations for Professional Edition:
Cross-zone Deployment: Deploy the instance across multiple zones to improve disaster recovery capabilities.
ACL: Enable access control for SASL authentication.
Click OK to start the deployment.
The deployment takes 10 to 30 minutes. The instance status changes from Deploying to In Service.
Step 4: View the instance endpoint
Once the instance is running:
On the Instances page, click the instance name to go to the Instance Details page.
In the Endpoint Information section, view the endpoint of the instance.
Type
Network
Protocol
Use
Default Endpoint
VPC
PLAINTEXT
Access from within the VPC
The following code provides an example of an endpoint:
alikafka-pre-public-intl-sg-xxxxx-1-vpc.alikafka.aliyuncs.com:9092, alikafka-pre-public-intl-sg-xxxxx-2-vpc.alikafka.aliyuncs.com:9092, alikafka-pre-public-intl-sg-xxxxx-3-vpc.alikafka.aliyuncs.com:9092Copy the endpoint and use it as the value for
bootstrap.serversin your Kafka client configuration.
Step 5: Configure a whitelist for the endpoint
By default, only clients in the same vSwitch can connect to the instance. To allow access from other CIDR blocks in the VPC, configure a whitelist.
In the Endpoint Information section, find the destination endpoint and click Manage Whitelist in the Actions column.
On the Whitelist Management page, click Add Whitelist Group.
Configure the whitelist:
Group Name: Enter an easy-to-identify name, such as
app-servers.Whitelist: Enter the IP addresses or CIDR blocks that are allowed to access the instance.
Click OK.
Verify network connectivity
Run the following command on a server in the VPC:
telnet alikafka-pre-public-intl-sg-xxxxx-1-vpc.alikafka.aliyuncs.com 9092A successful connection indicates that the network is correctly configured.
What to do next
After the instance is ready, you can perform the following operations:
Create a topic: Create a topic in the Topic Management section of the instance details page.
Create a group: Create a consumer group.
Connect an application: Use the endpoint to configure your Kafka client.