All Products
Search
Document Center

Elastic Compute Service:Using Shared Memory Communication (SMC)

Last Updated:Apr 10, 2026

This topic explains how to enable Shared Memory Communication (SMC), define its acceleration scope, and configure its interfaces for optimal performance.

Use SMC on Alibaba Cloud ECS

Follow these steps to use SMC-R.

Prerequisites

  1. Create an Elastic Compute Service (ECS) Instance that supports the Elastic RoCE Infrastructure (ERI) feature.

    To use SMC-R on Alibaba Cloud, you need an Elastic Compute Service (ECS) Instance that has Elastic Remote Direct Memory Access (eRDMA) capability. For more information, see Enable eRDMA on an enterprise-level instance.

    Important
    • Alibaba Cloud eRDMA devices and SMC do not currently support IPv6 addresses. If your application uses an IPv6 address, SMC falls back to TCP. For more information, see SMC falls back to TCP with IPv6 addresses.

    • Starting from ANCK 5.10.134-17.3, SMC supports IPv4-mapped IPv6 addresses.

  2. Load the smc and smc_diag Kernel Modules.

    sudo modprobe smc
    sudo modprobe smc_diag

    Run the dmesg command to view kernel log messages. If you see output similar to the following, the modules are loaded successfully.

    smc: smc: load SMC module with reserve_mode
    NET: Registered protocol family 43
    smc: netns <netns ID> reserved ports [65500 ~ 65515] for eRDMA OOB
    smc: adding ib device erdma_0 with port count 1
    smc:    ib device erdma_0 port 1 has pnetid

    (Optional) To unload the SMC modules if you no longer need them, run the following commands:

    sudo rmmod smc_diag
    sudo rmmod smc
  3. Run the following commands to install the smc-tools and aliyun-smc-extensions utilities.

    sudo yum install -y smc-tools
    sudo yum install -y aliyun-smc-extensions

Run TCP applications on the SMC stack

Alibaba Cloud Linux 3 SMC-R supports transparent socket protocol conversion at two levels.

Net namespace-level socket conversion

Alibaba Cloud Linux 3 provides a feature to transparently convert sockets at the net namespace level. You can use the sysctl net.smc.tcp2smc switch to convert all TCP sockets in a net namespace to SMC sockets if they meet the following conditions:

  • The family is AF_INET.

  • The type is SOCK_STREAM.

  • The protocol is IPPROTO_IP(0) or IPPROTO_TCP(6).

The following figure shows the conversion process.

image

Follow these steps to enable transparent conversion at the net namespace level.

  1. Run the following command to enable the global net.smc.tcp2smc switch for the net namespace.

    Newly created TCP sockets are converted to SMC sockets. Existing TCP sockets are not affected.

    sudo sysctl net.smc.tcp2smc=1

    By default, sysctl net.smc.tcp2smc is set to 0, which means the feature is disabled.

  2. Run any TCP socket application in the current net namespace.

    Replace <foo> in the example with your application name.

    ./<foo>

    The TCP sockets created by the foo application are transparently replaced with SMC sockets, and the SMC-R Protocol Stack handles the application's network behavior. As described in the Overview, if the peer also supports the SMC-R protocol and the negotiation is successful, both ends transfer data over the RDMA network. Otherwise, communication safely falls back to the TCP network.

  3. (Optional) Run the following command to disable the global replacement switch for the net namespace. Newly created TCP sockets are no longer converted. Existing SMC sockets are not affected.

    sudo sysctl net.smc.tcp2smc=0

Process-level socket conversion

Alibaba Cloud Linux 3 also provides process-level transparent protocol conversion, which is enabled using the smc-tools utility.

When you run an application with the smc_run script, it uses the LD_PRELOAD environment variable to prioritize loading the libsmc-preload.so dynamic library. The libsmc-preload.so library converts TCP sockets that meet the following conditions in the application and its child processes into SMC sockets:

  • The family is AF_INET.

  • The type is SOCK_STREAM.

  • The protocol is IPPROTO_IP(0) or IPPROTO_TCP(6).

Note

smc_run uses LD_PRELOAD to intercept the socket() system call in glibc. Therefore, it does not work for applications that are statically linked or do not use glibc.

The following figure shows the replacement process.

image

Follow these steps to enable transparent conversion at the process level.

Run the following command, adding the smc_run prefix before the command for your foo application.

Replace <foo> in the example with your process name.

smc_run ./<foo>

The TCP sockets created by the foo application are transparently replaced with SMC sockets, and the SMC-R Protocol Stack handles the application's network behavior. As described in the Overview, if the peer also supports the SMC-R protocol and the negotiation is successful, both ends transfer data over the RDMA network. Otherwise, communication safely falls back to the TCP network.

SMC negotiation control with BPF

In practice, enabling SMC at the net namespace or process level can sometimes be too coarse-grained. For example, a server might have multiple listening ports in a net namespace. You may want to use SMC only for connections on ports that require performance acceleration, while connections to other ports, such as management ports, should safely fall back to TCP.

To address this, Alibaba Cloud Linux 3 supports using BPF to control SMC negotiation for connections after transparent conversion is enabled. The typical process is as follows:

  1. Enable and configure BPF policies to set up granular control for SMC negotiation.

  2. Enable SMC at the net namespace or process level. For more information, see Run TCP applications on the SMC stack.

As described in the Overview, communicating parties use a special TCP option during the TCP handshake to declare and confirm mutual SMC-R support. If the negotiation succeeds, subsequent network transmission between the two parties is based on the RDMA network. Otherwise, the connection safely falls back to the TCP network.

By default, an SMC socket always initiates and responds to this special TCP option. However, you can use BPF to gain more granular control over SMC enablement by applying policies based on ports or IPv4 addresses.

Alibaba Cloud Linux 3 provides the smc-ebpf tool (part of smc-tools) to configure these BPF policies.

Run the following command to check if smc-ebpf is installed successfully.

smc-ebpf policy help

If the following output is displayed, smc-ebpf is installed successfully.

smc-ebpf policy help
 Usage: smc-ebpf policy COMMAND [OPTIONS]
        smc-ebpf policy load [OPTIONS]    load policy
    --init                                load policy with pre-defination config
        smc-ebpf policy stop              stop policy
        smc-ebpf policy unload            unload policy
        smc-ebpf policy init              init policy with default config
        smc-ebpf policy clear             clear all policy config
        smc-ebpf policy dump              display all policy config
        smc-ebpf policy config [OPTIONS]  config policy
        smc-ebpf policy delete [OPTIONS]  delete policy
    --ip [IPv4]                           target IPv4 address
    --port                                target port
    --mode [auto|disable|enable]          target mode
 Examples:
    smc-ebpf policy load
    #disable port 80 to use smc
    smc-ebpf policy config --port 80 --mode disable
    #delete ip xxx.xxx.x.x/24 policy
    smc-ebpf policy delete --ip xxx.xxx.x.x --mask 24

Load smc-ebpf

Run the following command to load smc-ebpf.

sudo smc-ebpf policy load
Warning
  • smc-ebpf does not affect connections established before it is loaded.

  • The policies configured by smc-ebpf are system-wide and cannot be configured for specific net namespaces (containers).

  • The smc-ebpf features are still being standardized in the upstream community. The interfaces may change in the future and are currently considered experimental.

  • The following output indicates that the command was successful:

    # sudo smc-ebpf policy load
    Registered smc_sock_negotiator_ops anolis_smc id xxx
  • Otherwise, you cannot use this feature in the current environment. Possible reasons include:

    • Confirm that the operating system Kernel Version is ANCK 5.10.134-016 or later. You can run the uname -r command to check the Kernel Version.

    • Confirm that you have permission to load BPF programs. A common issue is that the pod container you are using does not have the capability to load BPF programs, or your user privilege level is insufficient. You can consult your environment provider for more information.

Default behavior of port policies

By default, after smc-ebpf is loaded, it denies SMC negotiation for any port that does not match a policy.

For example:

  • If smc-ebpf is loaded without any port policies configured, SMC negotiation is denied for all ports.

  • If smc-ebpf is loaded and only one policy is configured to allow SMC negotiation on port 80, SMC negotiation is denied for any connection using port 8080 because no policy matches port 8080.

You can change this default behavior by running the following command to set port 0 to enable. With this setting, if no port policy matches a target port, SMC negotiation is allowed for that port.

sudo smc-ebpf policy config --port 0 --mode enable

Similarly, you can revert to the original default behavior by setting port 0 to disable. This will again deny SMC negotiation for any port that does not have a matching policy.

sudo smc-ebpf policy config --port 0 --mode disable

Port-based policies

In addition to the default behavior, you can add policies for specific ports.

For example:

  • Example 1: Allow SMC negotiation only on port 80, and deny it for all other ports.

    Run the following command to add a policy that allows SMC negotiation on port 80.

    sudo smc-ebpf policy config --port 80 --mode enable

    Run the following command to view the port policy configuration.

    sudo smc-ebpf policy dump

    Example output:

    • "key": 80 indicates that the policy applies to port 80.

    • "mode" indicates whether SMC negotiation is allowed or denied. 2 means allowed, and 0 means denied.

    # sudo smc-ebpf policy dump
    [{
            "key": 80,
            "value": {
                "mode": 2,
                [Other fields can be ignored by general users]
            }
        }
    ]

    If this policy is no longer needed, run the following command to delete the policy for port 80.

    sudo smc-ebpf policy delete --port 80

    Run the dump command again. The following output indicates that the configuration has been successfully deleted.

    # sudo smc-ebpf policy dump
    []
  • Example 2: Deny SMC negotiation only on port 80, and allow it for all other ports.

    Run the following commands to change the default behavior to allow SMC negotiation for unmatched ports, and then add a policy to specifically deny SMC negotiation for port 80.

    sudo smc-ebpf policy config --port 0 --mode enable
    sudo smc-ebpf policy config --port 80 --mode disable

    After this, SMC negotiation is denied only for port 80. All other ports that do not match a specific policy are allowed to use SMC negotiation.

Default behavior of IPv4 address policies

Unlike port policies, IPv4 address policies apply only to client sockets. They control whether a client uses SMC negotiation based on the server's IPv4 address when establishing a connection.

Note

All configured port and IPv4 address policies are combined with AND logic. SMC negotiation is used only if all matching policies allow it. If any matching policy denies SMC negotiation, it will not be used.

By default, after smc-ebpf is loaded, it allows SMC negotiation for any server IP that does not match an IPv4 address policy.

For example:

  • If smc-ebpf is loaded without any IPv4 address policies configured, client sockets are allowed to use SMC negotiation when connecting to any server IP.

  • If smc-ebpf is loaded and only one policy is configured to deny SMC negotiation for the 192.168.1.0/24 network, a connection to 192.168.3.11 will be allowed to use SMC negotiation because no policy matches 192.168.3.11.

You can change this default behavior by running the following command to set 0.0.0.0/32 to disable. With this setting, if no IPv4 address policy matches a target server IP, SMC negotiation is denied for that server IP.

sudo smc-ebpf policy config --ip 0.0.0.0 --mask 32 --mode disable

Similarly, you can restore the default behavior by running the following command. This will again allow SMC negotiation for any server IP that does not have a matching policy.

sudo smc-ebpf policy config --ip 0.0.0.0 --mask 32 --mode enable

IPv4 address-based policies

In addition to the default behavior, you can configure policies for specific IPv4 addresses to control whether a client socket uses SMC negotiation when connecting to a specific server IP.

Note

The configured IPv4 address filtering rules only control SMC negotiation for client sockets connecting to a server; they do not affect server sockets.

For example:

  • Example 1: Allow a client to use SMC negotiation only when connecting to servers in the 192.168.2.0/24 network. Deny it for all other server addresses.

    Run the following commands to change the default behavior to deny SMC negotiation for unmatched server IPs, and then add a policy to allow it for the 192.168.2.0/24 network.

    sudo smc-ebpf policy config --ip 0.0.0.0 --mask 32 --mode disable
    sudo smc-ebpf policy config --ip 192.168.2.0 --mask 24 --mode enable

    Run the following command to view the IPv4 policy configuration.

    sudo smc-ebpf policy dump

    Example output:

    • key: The target IPv4 address for the policy.

    • value: Indicates whether SMC negotiation is allowed or denied. pass means allowed, and denied means denied.

    # sudo smc-ebpf policy dump
    key:     0.0.0.0/32           value:   "denied"
    key:     192.168.2.0/24       value:   "pass"

    If these policies are no longer needed, run the following commands to delete the policies for 0.0.0.0/32 and 192.168.2.0/24.

    sudo smc-ebpf policy delete --ip 192.168.2.0 --mask 24
    sudo smc-ebpf policy delete --ip 0.0.0.0 --mask 32
  • Example 2: Deny SMC negotiation for a client only when connecting to the 192.168.2.0/24 network. Allow it for all other server addresses.

    Run the following command to add a policy that disables SMC negotiation for the 192.168.2.0/24 server address range.

    sudo smc-ebpf policy config --ip 192.168.2.0 --mask 24 --mode disable

    No other action is required. By default, after smc-ebpf is loaded, SMC negotiation is allowed for any server IP that does not match a policy. Therefore, SMC negotiation is allowed for all server addresses except those in the 192.168.2.0/24 range.

Clear policies

Run the following command to clear all configured policies.

sudo smc-ebpf policy clear

Running this command deletes all configurations. The behavior reverts to the default state: SMC negotiation is denied for all ports, but allowed for all IPv4 addresses. Due to the AND logic between the two policy types, this prevents all connections from using SMC negotiation.

Use SMC on Alibaba Cloud ACK

In Container Service for Kubernetes (ACK), you can enable SMC through the ACK eRDMA Controller component. The eRDMA Controller manages eRDMA network cards, scheduling, and network capabilities for Pods. For more information, see Use SMC-R to transparently accelerate application networks.

Note

Similar to using SMC on Alibaba Cloud ECS, when using SMC on Alibaba Cloud ACK, you can also configure granular, BPF-based SMC enablement policies on the Node.

Parameters

The SMC Protocol Stack provides configuration interfaces through sysfs and user-space tools like smc-tools. The following sections describe the configurable features of SMC.

Sysfs parameters

Kernel parameter

Description

Kernel version requirements

Recommendations

net.smc.autocorking_size

SMC-R provides the autocork feature, similar to TCP autocork. It coalesces multiple small data packets into a larger packet for a single transmission. This improves throughput for small-packet workloads without affecting ping-pong latency.

The autocorking_size parameter sets the upper limit for the size of aggregated packets.

Default value: 65535.

Valid values: 0 to 4294967295. A value of 0 disables the feature.

Kernel 5.10.112-11 and later.

For small-packet workloads that prioritize bandwidth, adjust this parameter to an appropriate value to achieve optimal throughput.

For pipeline send/receive workloads that prioritize latency, set this value to 0 to prevent delays introduced by autocork.

net.smc.autosplit_size

SMC provides an autosplit feature that splits a large data packet into multiple smaller packets for batch transmission. This improves latency performance in large-packet scenarios. When a packet size exceeds 1.3 times the autosplit_size, it is split.

Default value: 131072.

Valid values: 32768 to 536870912.

Kernel versions:

  • 5.10.134-18 and later.

  • 5.10.134-17.3 and later in the 17 series.

  • 5.10.134-16.5 and later in the 16 series.

In large-packet, latency-sensitive scenarios, you can adjust this parameter to achieve optimal latency.

net.smc.experiment_vendor_options

Alibaba Cloud experimental feature options.

Default value: 4294967295 (0xFFFFFFFF).

Kernel 5.10.134-16 and later.

We recommend that you do not modify this parameter.

net.smc.global_mem

Controls the memory watermark for SMC across the entire system. When the total size of send and receive Buffers maintained by the SMC Protocol Stack reaches global_mem[2], all new SMC connections will fall back to TCP.

Default value: [25% of total system memory, 50% of total system memory, 75% of total system memory].

Kernel versions:

  • 5.10.134-18 and later.

  • 5.10.134-17.3 and later in the 17 series.

  • 5.10.134-16.5 and later in the 16 series.

You can configure global_mem[2] according to the desired memory watermark.

net.smc.limit_smc_hs

Controls whether to proactively fall back to TCP when connection establishment pressure is high.

Default value: 1.

Valid values:

  • 0: Disabled

  • 1: Enabled

Kernel versions:

  • 5.10.134-18 and later.

  • 5.10.134-17.3 and later in the 17 series.

  • 5.10.134-16.5 and later in the 16 series.

We recommend setting this to 1 (enabled).

In specific cases where you do not want the SMC protocol stack to automatically fall back based on connection pressure, you can set it to 0.

net.smc.mem

Controls the memory watermark for SMC in the current net namespace. When the total size of send and receive Buffers for all SMC connections in the net namespace reaches mem[2], all new SMC connections in this net namespace will fall back to TCP.

Default value: [25% of total system memory, 50% of total system memory, 75% of total system memory].

Kernel versions:

  • 5.10.134-18 and later.

  • 5.10.134-17.3 and later in the 17 series.

  • 5.10.134-16.5 and later in the 16 series.

You can configure mem[2] according to the desired memory watermark.

net.smc.rmem

The default receive Buffer size for an SMC socket. This value is used when the receive Buffer size (SO_RCVBUF) is not explicitly set using setsockopt().

Default value: 262144.

Valid values:

  • SMC-R: 16384 (16 KB) to 536870912 (512 MB)

  • SMC-D: 16384 (16 KB) to 1048576 (1 MB)

Kernel 5.10.134-14 and later.

  • If the local Rx/Buffer full metric is high, increase the local receive Buffer size.

  • If the local Tx/Buffer full(remote) and Tx/Buffer too small(remote) metrics are high, increase the peer receive Buffer size.

  • If memory resources are tight and reducing the Buffer size does not affect network performance, you can decrease the Buffer size.

For more information, see SMC monitoring.

net.smc.wmem

The default send Buffer size for an SMC socket. This value is used when the send Buffer size (SO_SNDBUF) is not explicitly set using setsockopt().

Default value: 262144.

Valid values:

  • SMC-R: 16384 (16 KB) to 536870912 (512 MB)

  • SMC-D: 16384 (16 KB) to 1048576 (1 MB)

Kernel 5.10.134-14 and later.

  • If the local Tx/Buffer full and Tx/Buffer too small metrics are high, increase the local send Buffer size.

  • If memory resources are tight and reducing the Buffer size does not affect network performance, you can decrease the Buffer size.

For more information, see SMC monitoring.

net.smc.smcr_buf_type

The memory type for SMC-R send and receive Buffers. Physically contiguous memory provides better performance, but it is often difficult to allocate, which can cause Buffer sizes to be smaller than expected. In contrast, virtually contiguous memory is easier to allocate but provides slightly lower performance.

Changes to this value take effect on SMC connections carried by newly created LGRs. Existing LGRs are not affected.

Default value: 2.

Valid values:

  • 0: Physically contiguous memory.

  • 1: Virtually contiguous memory.

  • 2: Prioritize physically contiguous memory, but use virtually contiguous memory if it cannot be obtained.

Kernel 5.10.134-12 and later.

We recommend that you do not modify this parameter.

net.smc.smcr_max_conns_per_lgr

The maximum number of SMC connections that an LGR can carry in SMC-R.

Default value: 32.

Valid values:

  • 1 to 255 (5.10.134-18 and later, 5.10.134-17.3 and later in the 17 series, 5.10.134-16.5 and later in the 16 series).

  • 16 to 255 (other versions).

Kernel 5.10.134-16.1 and later.

Modify with caution.

  • For high-throughput scenarios, you can appropriately reduce the number of SMC connections per LGR to ensure that each SMC connection gets more RDMA bandwidth.

  • We do not recommend increasing this value.

net.smc.smcr_max_links_per_lgr

The number of RDMA RC connections (SMC Links) that an LGR contains in SMC-R.

Default value: 1.

Valid values: 1 to 2.

Kernel 5.10.134-16.1 and later.

We recommend that you do not modify this parameter.

net.smc.smcr_testlink_time

The heartbeat packet interval (in seconds) for an RDMA RC connection (SMC Link) in SMC-R. When there is no data transmission on the SMC Link, 16 bytes of data are sent every smcr_testlink_time seconds to confirm link availability.

Default value: 30.

Valid values: 0 to 2147483647. A value of 0 disables the heartbeat check.

Kernel 5.10.134-13 and later.

We recommend that you do not modify this parameter.

net.smc.tcp2smc

Enables or disables transparent TCP-to-SMC conversion within the current net namespace. When enabled, new TCP sockets created in this net namespace are intercepted in the kernel. This converts them to SMC sockets by replacing their family and protocol parameters, which then run on the SMC Protocol Stack.

Default value: 0.

Valid values:

  • 0: Disables net namespace SMC replacement.

  • 1: Enables net namespace SMC replacement.

Kernel 5.10.134-16 and later.

  • Set to 1 to enable TCP-to-SMC transparent replacement at the net namespace level.

  • Set to 0 to disable TCP-to-SMC transparent replacement at the net namespace level.

EID parameters

Introduced in the SMCv2 protocol, the Enterprise ID (EID) ensures that only systems with a matching EID can communicate via SMCv2; otherwise, they fall back to TCP. A system can have up to eight EIDs.

image

On Alibaba Cloud Linux 3, eRDMA devices can only be used with the SMCv2 protocol. Alibaba Cloud Linux 3 systems are initially configured with the EID SMCV2-DEFAULT-UEID. Therefore, by default, all Alibaba Cloud Linux 3 nodes can communicate with each other using SMCv2 with eRDMA without any manual configuration.

If you need to control the communication scope by modifying the EID in special circumstances, follow these steps.

  1. View existing EIDs.

    smcr ueid show
  2. Add a new EID.

    An EID can contain up to 32 characters, including uppercase letters (A-Z), numbers (0-9), hyphens (-), and periods (.). The first character must be a letter or a number, and periods (.) cannot be used consecutively.

    sudo smcr ueid add <EID>
  3. Delete an existing EID.

    sudo smcr ueid del <EID>

Use case: Avoid cross-AZ SMC-R communication with EIDs

For optimal SMC-R acceleration, we recommend using it for communication within the same Availability Zone and using TCP for cross-Availability Zone communication. By adding the Availability Zone ID as an EID, you can achieve this behavior automatically. Follow these steps to configure it.

  • Method 1: Configure the EID step by step

    1. Run the following command to get the Availability Zone ID from the Alibaba Cloud Elastic Compute Service (ECS) Instance Metadata. For more information, see Instance Metadata.

      ZONE_ID=$(curl -s -m 1 100.100.100.200/latest/meta-data/zone-id | tr "[:lower:]" "[:upper:]")
    2. Run the following command to add the Availability Zone ID as an EID.

      sudo smcr ueid add $ZONE_ID
    3. Run the following command to delete the default SMCV2-DEFAULT-UEID.

      smcr ueid | grep SMCV2-DEFAULT-UEID > /dev/null && sudo smcr ueid del SMCV2-DEFAULT-UEID
  • Method 2: Use the aliyunsmc-ueid service from the aliyun-smc-extensions tool for one-click configuration

    1. Run the following command to start the aliyunsmc-ueid service. This service automatically adds the Availability Zone ID as an EID and removes the default EID.

      sudo systemctl start aliyunsmc-ueid
    2. (Optional) Run the following command to enable the aliyunsmc-ueid service to start on boot. This is necessary because the EID configuration is not persistent and resets on OS restart.

      sudo systemctl enable aliyunsmc-ueid

PNET ID parameters

TCP traffic flows through an Elastic Network Interface (ENI). When this traffic is transparently converted to SMC-R, the SMC-R Protocol Stack uses the RDMA network interface (ERI) associated with that ENI.

This association can be established in two ways.

  • The ERI is obtained by directly enabling the eRDMA capability on the target ENI. In this case, the SMC-R protocol stack automatically associates this ENI with the ERI.

    In this scenario, the SMC-R protocol stack can automatically identify and use the ERI associated with the ENI, requiring no additional operations.

  • The ERI is obtained by enabling the eRDMA capability on a different ENI. In this case, you must use a PNET ID to associate the ERI with the target ENI.

    Consider the following scenario:

    A Host has two Ethernet interfaces, eth0 and eth1. Only eth0 has an enabled RDMA interface, erdma_0.

    If TCP traffic is sent and received through eth0, switching to SMC-R allows the system to automatically find the associated erdma_0 and use the RDMA network for communication. This is the automatic association scenario described above.

    However, if TCP traffic is sent and received through eth1, switching to SMC-R fails to find the erdma_0 interface, causing a fallback to TCP. To resolve this, associate eth1 and erdma_0 with the same PNET ID, which instructs SMC-R to use erdma_0 for eth1's traffic.

    image

    Follow these steps to associate an ENI and an ERI by using a PNET ID:

    • Run the following commands to configure the same PNET ID for the target ENI and ERI:

      • Configure a PNET ID for the ENI.

        sudo smc_pnet -a <PNET ID> -I <eth_interface>
      • Configure a PNET ID for the eRDMA interface (ERI).

        sudo smc_pnet -a <PNET ID> -D <rdma_interface>

      A PNET ID can be up to 16 uppercase alphanumeric characters without spaces.

    • Run the following command to check the configured PNET ID parameters.

      # sudo smc_pnet

      Example output:

      # sudo smc_pnet
      00163E0CD751 n/a erdma_0 1
      00163E0CD751 eth1 n/a 255

      In this example, erdma_0 is bound to PNET ID 00163E0CD751, and eth1 is also bound to PNET ID 00163E0CD751. This allows TCP communication that originally used the eth1 interface to use the erdma_0 interface after enabling SMC-R.

Use case: Associate pod veth with host eRDMA

In scenarios where multiple pods on a single Host share an ERI, you can accelerate a pod's veth interface traffic using RDMA. To do this, assign the same PNET ID to the pod's veth Ethernet interface and the Host's eRDMA interface. This allows the kernel Protocol Stack inside the container to correctly find the Host's eRDMA interface when SMC-R is enabled.

image

Follow these steps to configure it.

  1. In the container's net namespace, configure a PNET ID for the container's Ethernet interface (for example, eth0).

    sudo ip netns exec <pod netns> smc_pnet -a <PNET ID> -I eth0
  2. In the Host's net namespace, configure the same PNET ID for the eRDMA interface (for example, erdma_0).

    sudo smc_pnet -a <PNET ID> -D erdma_0

Use case: Use SMC with Redis on an ECS instance

  1. Create two Elastic Compute Service (ECS) Instances. One will serve as the Redis client and the other as the Redis server. For more information about the parameters, see Custom launch ECS instances.

  2. Load the smc and smc_diag Kernel Modules.

    sudo modprobe smc
    sudo modprobe smc_diag
  3. Run the following command on both instances to install Redis.

    sudo yum install redis -y
  4. Run the following command on both instances to configure EIDs to avoid cross-Availability Zone SMC-R communication.

    sudo systemctl start aliyunsmc-ueid
  5. Run the following commands on both instances to configure port-based BPF policies. This allows SMC negotiation only for connections where the server IP address is within the vswitch CIDR block and the service port is 6379.

    1. Run the following command to load smc-ebpf.

      sudo smc-ebpf policy load
    2. Configure a port-based policy to allow SMC negotiation only for port 6379.

      sudo smc-ebpf policy config --port 6379 --mode enable
    3. Configure an IPv4 address-based policy to allow SMC negotiation only for server IPs within the vswitch CIDR block.

      sudo smc-ebpf policy config --ip 0.0.0.0 --mask 32 --mode disable
      cidr=$(curl -s -m 1 100.100.100.200/latest/meta-data/vswitch-cidr-block)
      sudo smc-ebpf policy config --ip \
        $(echo ${cidr} | awk -F'/' '{print $1}') --mask \
        $(echo ${cidr} | awk -F'/' '{print $2}') --mode enable
  6. Run the following command on both instances to configure transparent protocol conversion at the net namespace level.

    sudo sysctl -w net.smc.tcp2smc=1
  7. On the Redis server instance, run the following command to start the Redis service.

    Replace <IP> with the private IP address of the server instance's primary Elastic Network Interface (ENI).

    redis-server --bind <IP> --port 6379 --protected-mode no --save
  8. On the Redis client instance, connect to or test the Redis server.

    • Run the following command to connect to the Redis server.

      redis-cli -h <IP> -p 6379
    • Run the following command to perform a benchmark test by using redis-benchmark.

      redis-benchmark -h <IP> -p 6379 -n 1000000 -t set -c 100