Object Storage Service:Model broadcasting

Option 1: Purchase and configure Tair (Recommended)

Tair is Alibaba Cloud's fully managed cloud database service, compatible with the Redis protocol.

1. Create a Tair instance (version 6.0 or later, minimum specifications). Quick start overview.

2. Configure a whitelist so that inference nodes can access the Tair instance.

3. Note the Connection Address, Port Number, Username, and Password for the model broadcasting configuration.

Option 2: Deploy a standalone Redis service

You can also deploy Redis in a Kubernetes cluster.

The following YAML deploys Redis with ACL authentication.

1. Create an ACL configuration file and generate a Kubernetes secret.

   # Create ACL content
   cat > users.acl << EOF
   user default off -@all
   user Username on >Password ~* &* +@all
   EOF
   
   # Create a secret
   kubectl create secret generic redis-acl-secret \
     --from-file=users.acl \
     --dry-run=client -o yaml | kubectl apply -f -

   Note

   Replace Username and Password with your actual username and password.

2. Apply the following YAML to deploy Redis.

   # redis-service.yaml
   apiVersion: v1
   kind: Service
   metadata:
     name: redis
   spec:
     selector:
       app: model-redis
     ports:
       - protocol: TCP
         port: 6379
         targetPort: 6379
   
   ---
   # redis-deployment.yaml
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: model-redis-deployment
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: model-redis
     template:
       metadata:
         labels:
           app: model-redis
       spec:
         containers:
         - name: redis
           image: mirrors-ssl.aliyuncs.com/redis:8.4.0
           ports:
           - containerPort: 6379
           command: ["redis-server"]
           args:
           - "--aclfile"
           - "/etc/redis/users.acl"
           - "--maxmemory"
           - "900mb"
           - "--maxmemory-policy"
           - "volatile-lru"
           - "--save"
           - ""
           - "--appendonly"
           - "no"
           - "--loglevel"
           - "notice"
           resources:
             requests:
               memory: "1Gi"
               cpu: "100m"
             limits:
               memory: "1Gi"
               cpu: "200m"
           volumeMounts:
           - name: acl-config
             mountPath: /etc/redis/users.acl
             subPath: users.acl
         volumes:
         - name: acl-config
           secret:
             secretName: redis-acl-secret

3. Deploy the Redis service.

   kubectl apply -f redis-service.yaml

Test in Beijing region

Test environment

Unlimited cache size

• Only one back-to-source stream is used; all other data transfers via P2P, minimizing OSS bandwidth pressure.

• Average model-ready time stays near O(1) regardless of node count, demonstrating excellent horizontal scaling.

Limited cache size

The model-ready time was tested for 1, 10, 50, and 100 nodes starting simultaneously with the cache size unlimited and limited to 100, 60, 40, 20, and 0 GB.

• Model broadcasting works as expected across all cache size limits.

• Cache limit impact is consistent across concurrency levels. A cache of 40 GB or more has no significant effect on model-ready time. Performance declines noticeably at 20 GB or less.

Test in Ulanqab region

Test environment

The model-ready time was tested for 1, 10, 50, and 100 nodes starting simultaneously with the cache size unlimited and limited to 60 and 0 GB.

Even with limited OSS download bandwidth, model broadcasting maintains excellent horizontal scaling and minimizes source bandwidth pressure.