Connect to ApsaraMQ for Kafka to send and receive messages using the Spring Cloud framework - ApsaraMQ for Kafka

Spring Cloud simplifies building message-driven microservice applications with built-in service discovery, configuration management, and load balancing. Integrate it with ApsaraMQ for Kafka to send and receive messages in your distributed systems.

Prerequisites

JDK 1.8 or later is installed.
Maven 2.5 or later is installed.
You have downloaded the kafka-spring-stream-demo package and uploaded it to your Linux system.
Ensure your ApsaraMQ for Kafka instance is version 2.x or later. For more information, see Upgrade an instance version.
You have created topics and consumer groups.

Internet access (authentication and encryption required)

For Internet access, messages are authenticated and encrypted over the SASL_SSL protocol. Clients connect to ApsaraMQ for Kafka through an SSL endpoint. For more information about endpoints, see Endpoint Comparison.

In this example, the demo package is uploaded to /home/doc/project/aliware-kafka-demos/kafka-spring-stream-demo.

Log on to the Linux system and run the following command to go to the demo package directory /home/doc/project/aliware-kafka-demos/kafka-spring-stream-demo.
```
cd /home/doc/project/aliware-kafka-demos/kafka-spring-stream-demo
```
Run the following command to navigate to the configuration file path.
```
cd sasl-ssl/src/main/resources/
```

Run the following command to edit the application.properties file and configure the instance information based on the parameter list.

vi application.properties

## Replace the placeholder values with your instance information.
kafka.bootstrap-servers=alikafka-pre-cn-zv**********-1.alikafka.aliyuncs.com:9093,alikafka-pre-cn-zv**********-2.alikafka.aliyuncs.com:9093,alikafka-pre-cn-zv**********-3.alikafka.aliyuncs.com:9093
kafka.consumer.group=test-spring
kafka.output.topic.name=test-output
kafka.input.topic.name=test-input
kafka.ssl.truststore.location=/home/doc/project/aliware-kafka-demos/kafka-spring-stream-demo/sasl-ssl/src/main/resources/kafka.client.truststore.jks

### The following binding parameters associate ApsaraMQ for Kafka with the Spring Cloud Stream binder. You can use the default values.
spring.cloud.stream.bindings.MyOutput.destination=${kafka.output.topic.name}
spring.cloud.stream.bindings.MyOutput.contentType=text/plain
spring.cloud.stream.bindings.MyInput.group=${kafka.consumer.group}
spring.cloud.stream.bindings.MyInput.destination=${kafka.input.topic.name}
spring.cloud.stream.bindings.MyInput.contentType=text/plain

### The binder is the Spring Cloud abstraction for message middleware. You can use the default values for the following parameters.
spring.cloud.stream.kafka.binder.autoCreateTopics=false
spring.cloud.stream.kafka.binder.brokers=${kafka.bootstrap-servers}
spring.cloud.stream.kafka.binder.configuration.security.protocol=SASL_SSL
spring.cloud.stream.kafka.binder.configuration.sasl.mechanism=PLAIN
spring.cloud.stream.kafka.binder.configuration.ssl.truststore.location=${kafka.ssl.truststore.location}
spring.cloud.stream.kafka.binder.configuration.ssl.truststore.password=KafkaOnsClient
### If not present in the demo, add this parameter to disable server hostname verification.
### Server hostname verification checks whether the hostname in the SSL certificate matches the server's hostname. The default value is HTTPS.
spring.cloud.stream.kafka.binder.configuration.ssl.endpoint.identification.algorithm=

Table 1. Parameter list
Parameter	Description
kafka.bootstrap-servers	The endpoint of the ApsaraMQ for Kafka instance. You can obtain this endpoint from the Endpoint Information section on the Instance Details page of the ApsaraMQ for Kafka console.
kafka.consumer.group	A Group is used to subscribe to messages. You can create a group on the Groups page in the ApsaraMQ for Kafka console. For more information, see Step 3: Create resources.
kafka.output.topic.name	The topic for produced messages. The demo application periodically sends messages with fixed content to this topic. You can create a topic on the Topics page in the ApsaraMQ for Kafka console. For more information, see Step 3: Create resources.
kafka.input.topic.name	The topic from which messages are consumed. You can send messages to this topic from the console. The demo application consumes these messages and prints them to the log.
kafka.ssl.truststore.location	The path to the SSL root certificate kafka.client.truststore.jks.

Run the following command to open the kafka_client_jaas.conf file and configure the username and password for the instance.
```
vi kafka_client_jaas.conf
```
Note
- If the access control list (ACL) feature is disabled for the instance, you can obtain the username and password for the default user from the Instance Details page in the ApsaraMQ for Kafka console.
- If the access control list (ACL) feature is enabled for the instance, make sure that the SASL user uses the PLAIN mechanism and has permissions to send and receive messages. For more information, see Grant permissions to a SASL user.
```
KafkaClient {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="your-username"
  password="your-password";
};
```
Navigate to the /home/doc/project/aliware-kafka-demos/kafka-spring-stream-demo/sasl-ssl directory and run the following command to run the demo.
```
sh run_demo.sh
```
The program prints the following information, which indicates that it received the message sent by the console program to the topic specified in kafka.output.topic.name.
```
Send: hello world !!
Send: hello world !!
Send: hello world !!
Send: hello world !!
```
Log on to the ApsaraMQ for Kafka console to verify that messages are sent and received successfully.
- Check whether the topic specified for the kafka.output.topic.name parameter has received messages from the console application. For more information, see Query messages.
- Send a message to the topic that is configured in kafka.input.topic.name and check the logs of the demo program to verify that the message is printed. For more information, see Send messages.

VPC access (no authentication or encryption)

In a VPC, messages are transmitted over the PLAINTEXT protocol without authentication or encryption. Clients connect to ApsaraMQ for Kafka through the default endpoint. For more information about endpoints, see Endpoint comparison.

In this example, the demo package is uploaded to the /home/doc/project/aliware-kafka-demos/kafka-spring-stream-demo directory.

Log on to the Linux system and run the following command to navigate to the /home/doc/project/aliware-kafka-demos/kafka-spring-stream-demo directory that contains the demo package.
```
cd /home/doc/project/aliware-kafka-demos/kafka-spring-stream-demo
```
Run the following command to navigate to the configuration file path.
```
cd vpc/src/main/resources/
```

Run the following command to edit the application.properties file and configure the instance information based on the parameter list.

vi application.properties

### Modify the following parameters based on your instance information.
kafka.bootstrap-servers=alikafka-pre-cn-zv**********-1-vpc.alikafka.aliyuncs.com:9092,alikafka-pre-cn-zv**********-2-vpc.alikafka.aliyuncs.com:9092,alikafka-pre-cn-zv**********-3-vpc.alikafka.aliyuncs.com:9092
kafka.consumer.group=test-spring
kafka.output.topic.name=test-output
kafka.input.topic.name=test-input

Navigate to the /home/doc/project/aliware-kafka-demos/kafka-spring-stream-demo/vpc directory and run the following command to run the Demo.
```
sh run_demo.sh
```
The program prints the following output.
```
Send: hello world !!
Send: hello world !!
Send: hello world !!
Send: hello world !!
```
Log on to the ApsaraMQ for Kafka console to verify that messages are sent and received successfully.
- Check whether the topic specified for the kafka.output.topic.name parameter has received messages from the console application. For more information, see Query messages.
- Send a message to the topic that is configured in kafka.input.topic.name and check the logs of the demo program to verify that the message is printed. For more information, see Send messages.

ApsaraMQ for Kafka:Send and receive messages with Spring Cloud

Prerequisites

Internet access (authentication and encryption required)

VPC access (no authentication or encryption)

Related documents