Deploy SMC-R and Redis on enterprise-level CPU-based instances with eRDMA enabled - Elastic Compute Service

Elastic Remote Direct Memory Access (eRDMA) accelerates the processing of requests, achieving ultra-low latency. This topic describes how to deploy Shared Memory Communication (SMC) and Redis based on eRDMA and use SMC to test the request processing capability of Redis.

Background information

SMC is a high-performance protocol stack that operates in kernel space, uses shared memory technology, and is compatible with the socket layer. It is classified into two variations based on shared memory technology: Shared Memory Communications - Direct Memory Access (SMC-D), which uses internal shared memory (ISM) technology, and Shared Memory Communications over Remote Direct Memory Access (SMC-R), which uses RDMA technology. For more information, see SMC applicability.
Alibaba Cloud Linux 3 provides SMC-R acceleration capabilities based on eRDMA. To use SMC-R on Alibaba Cloud, you must create eRDMA-capable ECS instances.

Preparations

Create two eRDMA-capable instances, select the Auto-install eRDMA Driver option, and then enable the eRDMA Interface (ERI) feature for the primary elastic network interfaces (ENIs). For more information, see Create an instance on the Custom Launch tab.

The ECS instances serve as a server and a client with the following configurations:

Instance type: ecs.g8i.8xlarge
Image: Alibaba Cloud Linux 3.2104 LTS 64-bit
Network: The ECS instances belong to the same security group and can communicate with each other over the internal network.
Primary private IP addresses of the primary ENIs: 192.168.0.25 for the instance used as the server and 192.168.0.24 for the instance used as the client. Replace the IP addresses as needed.
Note
- In the test in this topic, ECS instances on which the ERI feature is enabled for the primary ENIs are used. The primary private IP address of the primary ENI on the ECS instance used as the server is 192.168.0.25.
- If you use ECS instances on which the ERI feature is enabled for secondary ENIs for the test, replace the preceding IP addresses with the primary private IP addresses of the secondary ENIs. For more information, see Step 3: Bind ERIs to an ECS instance.

Example on how to configure specific parameters during ECS instance creation

When you create the ECS instances, take note of the following parameters or options. For information about other parameters on the ECS instance buy page, see Create an instance on the Custom Launch tab.

Instances & Images: Select an instance type that supports eRDMA and install the eRDMA driver.
- Instance: For more information, see the Limits section of this topic.
- Image: Click the Public Images tab, select a public image, and then select Auto-install eRDMA Driver. The system automatically installs the eRDMA driver when the instance is started.
ENI: Select the eRDMA Interface option on the right side of Primary ENI to bind an ERI to the ECS instance.
Note
When you create an enterprise-level instance, you can enable the ERI feature only for the primary elastic network interface (ENI). You can enable the ERI feature for a secondary ENI in the ECS console or by calling an API operation. For more information, see ERIs.

Note

When you create an enterprise-level instance, you can enable the ERI feature only for the primary elastic network interface (ENI). You can enable the ERI feature for a secondary ENI in the ECS console or by calling an API operation. For more information, see ERIs.

Step 1: Deploy SMC

Connect to the ECS instances.
For more information, see Use Workbench to connect to a Linux instance over SSH.
Run the following command on the ECS instances to deploy the SMC-R module:
```
modprobe smc && modinfo smc
```
Run the following command on the ECS instances to disable IPv6, allowing eRDMA connections to be established on ERIs.
Important
- Alibaba Cloud eRDMA devices and SMC do not support IPv6 addresses. If applications use IPv6, SMC falls back to TCP. For more information, see SMC falls back to TCP when IPv6 addresses are used.
- Starting from kernel version ANCK 5.10.134-17.3, SMC supports IPv4-mapped IPv6 addresses.
```
sysctl net.ipv6.conf.all.disable_ipv6=1
```
Run the following command on the ECS instances to enable the SMC eRDMA feature for traffic to be preferentially transmitted over eRDMA.
After transparent socket conversion is enabled for the net namespace, subsequent new TCP sockets in the net namespace are converted to SMC sockets and existing TCP sockets are not affected.
```
sysctl net.smc.tcp2smc=1
```
Run the following command on the ECS instances to install smc-tools.
smc-tools is a toolkit from IBM that allows you to monitor and diagnose SMC-R from multiple aspects. It consists of the following modules:
- smcr: displays statistics on SMC-R resources.
- smcss: displays information about SMC sockets.
```
yum install -y smc-tools
```
After smc-tools is installed, run the smcss -a command to query the current traffic path.

Step 2: Deploy Redis

Deploy Redis on the ECS instances.

Connect to the ECS instances.
For more information, see Use Workbench to connect to a Linux instance over SSH.
Run the following command on the ECS instances to install Redis:
```
sudo yum install -y redis
```
Run the following command on the ECS instance used as the server to start the Redis server:
```
redis-server --bind 192.168.0.25 --port 6379 --protected-mode no --save
```
Note
192.168.0.25 is the primary private IP address of the primary ENI on the instance used as the server, and 6379 is the port number on which Redis listens. Replace the primary private IP address and port number with actual values.
Connect to the ECS instance used as the client and perform a stress test.
- Run the following command to connect to the Redis server:
```
redis-cli -h 192.168.0.25 -p 6379
```
- Perform a stress test by using redis-benchmark.
  Run the following command to simulate 100 clients sending 1,000,000 SET commands to the server:
```
redis-benchmark -h 192.168.0.25 -p 6379 -n 1000000 -t set -c 100
```

You can also start multiple stress test processes to perform a combined test after you enable SMC-R. Repeat the preceding steps to deploy another Redis client, start multiple stress test processes on the two Redis clients, and then view the number of operations per second (OPS) on the Redis server.

Sample command to start eight stress test processes for SET commands:

redis-benchmark -h 192.168.0.25 -p 6379 -n 100000000 -t set --threads 8 -c 100

Sample command to start eight stress test processes for GET commands:

redis-benchmark -h 192.168.0.25 -p 6379 -n 1000000 -t get --threads 8 -c 100

Sample command to view the number of OPS on the Redis server:
```
redis-cli -h 192.168.0.25 -p 6379 info | grep instantaneous_ops_per_sec
```
Note
Open a new connection window and then enter the preceding command.