OSS Connector for AI/ML in Kubernetes - Object Storage Service: Central de Documentação do Alibaba Cloud

Instale e configure o OSS Connector for AI/ML no Kubernetes para carregar dados de modelos do OSS em serviços de inferência.

Instalar o componente Connector

Instale o componente Connector no contêiner da aplicação com um dos métodos a seguir.

Método 1: Init Container

Baixe e extraia o Connector para um volume compartilhado antes que o contêiner da aplicação seja iniciado.

initContainers:
  # Init Container: runs before the application container to prepare dependencies
  - name: install-connector
    image: busybox
    command:
      - sh
      - -c
      - |
        # Download the Connector DEB package
        wget -q https://gosspublic.alicdn.com/oss-connector/oss-connector-lib-1.2.0.x86_64.deb -O /tmp/connector.deb
        # Create a temp directory and extract the DEB package
        mkdir -p /tmp/extract && cd /tmp/extract
        ar x /tmp/connector.deb
        # Extract only the required .so file to the shared directory
        # The /shared directory is mounted as a volume visible to the main container
        mkdir -p /shared/usr/local/lib
        tar -xf data.tar.gz -O ./usr/local/lib/libossc_preload.so > /shared/usr/local/lib/libossc_preload.so
    volumeMounts:
      # Mount the connector-lib volume at /shared
      # The main container also mounts this volume for file sharing
      - name: connector-lib
        mountPath: /shared

containers:
  - name: vllm
    image: vllm/vllm-openai:latest
    volumeMounts:
      # Mount the connector-lib volume at /usr/local/lib in the container
      # This makes the .so file extracted by the Init Container available
      - name: connector-lib
        mountPath: /usr/local/lib
        subPath: usr/local/lib

volumes:
  # Shared volume for passing files between the Init Container and the main container
  - name: connector-lib
    emptyDir: {}

Método 2: Instalação na inicialização

Baixe e instale o Connector diretamente no comando de inicialização do contêiner.

containers:
  - name: vllm
    image: vllm/vllm-openai:latest
    command: ["/bin/bash", "-c"]
    args:
      - |
        # Download and install the Connector DEB package
        wget https://gosspublic.alicdn.com/oss-connector/oss-connector-lib-1.2.0.x86_64.deb
        dpkg -i oss-connector-lib-1.2.0.x86_64.deb
        # Start the model serving process
        ENABLE_CONNECTOR=1 python3 -m vllm.entrypoints.openai.api_server --model ${MODEL_DIR} ...

Método 3: Dockerfile personalizado

Pré-instale o Connector em uma imagem personalizada baseada em vllm/vllm-openai.

FROM vllm/vllm-openai:latest
RUN wget https://gosspublic.alicdn.com/oss-connector/oss-connector-lib-1.2.0.x86_64.deb && \
    dpkg -i oss-connector-lib-1.2.0.x86_64.deb

Crie e envie a imagem:

docker build -t myregistry/vllm-with-connector:latest .
docker push myregistry/vllm-with-connector:latest

Use a imagem personalizada:

containers:
  - name: vllm
    image: myregistry/vllm-with-connector:latest

Comparação dos métodos de instalação

Método	Caso de uso	Vantagens	Desvantagens
Init Container / Instalação na inicialização	Testes, validação ou desenvolvimento sem modificar a imagem base	Dispensa imagem personalizada, implantação flexível, configuração simples	Baixa e extrai arquivos a cada inicialização, cold start mais lento, requer acesso à rede externa
Dockerfile personalizado	Ambientes de produção, serviços de longa duração, clusters de grande escala	Inicialização mais rápida, imagem autossuficiente, alta estabilidade, reutilizável	Exige manutenção de uma imagem personalizada, versão fixa

Implantar um serviço de inferência de modelo

Este exemplo implanta um serviço de inferência que carrega dados de modelos do OSS pelo método Init Container com vllm/vllm-openai.

# ConfigMap: stores the Connector configuration file
# Purpose: mounts the Connector config as a file inside the container
# Mount path: /etc/oss-connector/config.json
apiVersion: v1
kind: ConfigMap
metadata:
  name: connector-config
data:
  config.json: |
    {
        "logLevel": 1,
        "logPath": "/var/log/oss-connector/connector.log",
        "auditPath": "/var/log/oss-connector/audit.log",
        "expireTimeSec": 120,
        "prefetch": {
            "vcpus": 16,
            "workers": 16
        }
    }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: model-connector-deployment
spec:
  selector:
    matchLabels:
      app: model-connector
  template:
    metadata:
      labels:
        app: model-connector
    spec:
      # Init Container: runs before the application container to prepare dependencies
      initContainers:
        - name: install-connector
          image: busybox
          command: ["/bin/sh", "-c"]
          args:
            - |
              # Download the Connector DEB package
              wget -q https://gosspublic.alicdn.com/oss-connector/oss-connector-lib-1.2.0.x86_64.deb -O /tmp/connector.deb

              # Create a temp directory and extract the DEB package
              mkdir -p /tmp/extract && cd /tmp/extract
              ar x /tmp/connector.deb

              # Extract only the required .so file to the shared directory
              # The /shared directory is mounted as a volume visible to the main container
              mkdir -p /shared/usr/local/lib
              tar -xf data.tar.gz -O ./usr/local/lib/libossc_preload.so > /shared/usr/local/lib/libossc_preload.so

          volumeMounts:
            # Mount the connector-lib volume at /shared
            # The main container also mounts this volume for file sharing
            - name: connector-lib
              mountPath: /shared
      containers:
      - name: vllm
        image: vllm/vllm-openai:latest
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: "16"
            memory: "70Gi"
          limits:
            cpu: "20"
            memory: "80Gi"
        command: ["/bin/bash", "-c"]
        args:
          - |
            # Add the Connector library to LD_PRELOAD to intercept file system calls
            export LD_PRELOAD="/usr/local/lib/libossc_preload.so${LD_PRELOAD:+:$LD_PRELOAD}"
            # Start the vLLM server
            # ENABLE_CONNECTOR=1 enables OSS Connector for accelerated model loading
            # The model path ${MODEL_DIR}/qwen/Qwen3-8B/ points to an OSS path,
            # intercepted and redirected by the Connector
            ENABLE_CONNECTOR=1 python3 -m vllm.entrypoints.openai.api_server \
              --model ${MODEL_DIR}/qwen/Qwen3-8B/ \
              --trust-remote-code \
              --tensor-parallel-size 1 \
              --disable-custom-all-reduce
        env:
        # OSS access configuration: specify the internal endpoint and region
        - name: OSS_ENDPOINT
          value: "oss-cn-beijing-internal.aliyuncs.com"
        - name: OSS_REGION
          value: "cn-beijing"

        # Root path on OSS. The Connector maps local paths to this OSS path
        - name: OSS_PATH
          value: "oss://examplebucket/"

        # Local mount directory (intercepted by the Connector, data is loaded from OSS)
        - name: MODEL_DIR
          value: "/var/model"

        # Read OSS access credentials from the Secret (oss-access-key-connector)
        # Create the Secret in advance:
        #   kubectl create secret generic oss-access-key-connector \
        #     --from-literal=key=<OSS_ACCESS_KEY_ID> \
        #     --from-literal=secret=<OSS_ACCESS_KEY_SECRET>
        - name: OSS_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef:
              name: oss-access-key-connector
              key: key
        - name: OSS_ACCESS_KEY_SECRET
          valueFrom:
            secretKeyRef:
              name: oss-access-key-connector
              key: secret
        volumeMounts:
          # Mount the Connector configuration file
          - name: connector-config
            mountPath:  /etc/oss-connector/
          # Mount the connector-lib volume at /usr/local/lib in the container
          # This makes the .so file extracted by the Init Container available
          - name: connector-lib
            mountPath: /usr/local/lib
            subPath: usr/local/lib
      terminationGracePeriodSeconds: 10
      volumes:
      # Connector configuration ConfigMap
      - name: connector-config
        configMap:
          name: connector-config
      # Shared volume for passing files between the Init Container and the main container
      - name: connector-lib
        emptyDir: {}

Implantar um serviço de broadcast de modelo com múltiplas instâncias

Este exemplo usa uma imagem pré-construída (myregistry/vllm-with-connector) para fazer o broadcast do modelo entre várias réplicas. Para obter mais informações sobre broadcast de modelos, consulte Broadcast de modelos.

# ConfigMap: stores the Connector configuration template
# Purpose: provides a template (config.json.tmpl) that the init container renders
#          into the final config file for the main container
# Template mount path (init container): /tmpl/config.json.tmpl
# Rendered output path (main container): /etc/oss-connector/config.json
apiVersion: v1
kind: ConfigMap
metadata:
  name: connector-config
data:
  config.json.tmpl: |
    {
        "logLevel": 1,
        "logPath": "/var/log/oss-connector/connector.log",
        "auditPath": "/var/log/oss-connector/audit.log",
        "expireTimeSec": 120,
        "prefetch": {
            "vcpus": 16,
            "workers": 16
        },
        "broadcast": {
            "enableBroadcast": true,
            "tenant": "${REDIS_TENANT}",
            "db": {
                "host": "${REDIS_HOST}",
                "port": 6379,
                "username": "${REDIS_USERNAME}",
                "password": "${REDIS_PASSWORD}"
            }
        },
        "bindPort": 19989
    }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: model-connector-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: model-connector
  template:
    metadata:
      labels:
        app: model-connector
    spec:
      # Init container: renders the config template into the final config.json
      #                  before the main container starts
      initContainers:
      - name: render-config
        image: busybox
        command: ["/bin/sh", "-c"]
        args:
          # Use sed to replace template placeholders with actual environment variable values
          - |
            sed -e "s|\${REDIS_HOST}|$REDIS_HOST|g" \
                -e "s|\${REDIS_USERNAME}|$REDIS_USERNAME|g" \
                -e "s|\${REDIS_PASSWORD}|$REDIS_PASSWORD|g" \
                -e "s|\${REDIS_TENANT}|$REDIS_TENANT|g" \
                /tmpl/config.json.tmpl > /etc/oss-connector/config.json
        env:
        # Read Redis connection details from the Secret (redis-secret)
        # Create the Secret in advance:
        #   kubectl create secret generic redis-secret \
        #     --from-literal=host=<host> \
        #     --from-literal=username=<username> \
        #     --from-literal=password=<password>
        - name: REDIS_HOST
          valueFrom:
            secretKeyRef:
              name: redis-secret
              key: host
        - name: REDIS_USERNAME
          valueFrom:
            secretKeyRef:
              name: redis-secret
              key: username
        - name: REDIS_PASSWORD
          valueFrom:
            secretKeyRef:
              name: redis-secret
              key: password
        - name: REDIS_TENANT
          value: "broadcast-demo"
        volumeMounts:
          # Mount the ConfigMap template as the input for sed
          - name: connector-config
            mountPath: /tmpl
          # Mount the shared emptyDir to write the rendered config for the main container
          - name: rendered-config
            mountPath: /etc/oss-connector
      containers:
      - name: vllm
        image: myregistry/vllm-with-connector:latest
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: "16"
            memory: "70Gi"
          limits:
            cpu: "20"
            memory: "80Gi"
        command: ["/bin/bash", "-c"]
        args:
          - |
            # Add the Connector library to LD_PRELOAD to intercept file system calls
            export LD_PRELOAD="/usr/local/lib/libossc_preload.so${LD_PRELOAD:+:$LD_PRELOAD}"
            # Start the vLLM server
            # ENABLE_CONNECTOR=1 enables OSS Connector for accelerated model loading
            ENABLE_CONNECTOR=1 python3 -m vllm.entrypoints.openai.api_server \
              --model ${MODEL_DIR}/qwen/Qwen3-8B/ \
              --trust-remote-code \
              --tensor-parallel-size 1 \
              --disable-custom-all-reduce
        env:
        # OSS access configuration: specify the internal endpoint and region
        - name: OSS_ENDPOINT
          value: "oss-cn-beijing-internal.aliyuncs.com"
        - name: OSS_REGION
          value: "cn-beijing"

        # Root path on OSS. The Connector maps local paths to this OSS path
        - name: OSS_PATH
          value: "oss://examplebucket/"

        # Local mount directory (intercepted by the Connector, data is loaded from OSS)
        - name: MODEL_DIR
          value: "/var/model"

        # Read OSS access credentials from the Secret (oss-access-key-connector)
        - name: OSS_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef:
              name: oss-access-key-connector
              key: key
        - name: OSS_ACCESS_KEY_SECRET
          valueFrom:
            secretKeyRef:
              name: oss-access-key-connector
              key: secret
        volumeMounts:
          # Connector configuration (rendered by init container)
          - name: rendered-config
            mountPath: /etc/oss-connector/
      terminationGracePeriodSeconds: 10
      volumes:
      # Connector config template ConfigMap (mounted in init container)
      - name: connector-config
        configMap:
          name: connector-config
      # Rendered config (shared from init container to main container)
      - name: rendered-config
        emptyDir: {}