在Kubernetes中啟用Connector功能 - Object Storage Service

本文介紹如何在Kubernetes部署中安裝和使用OSS Connector for AI/ML組件，實現模型資料從OSS載入的推理服務部署。

安裝Connector組件

要在Kubernetes部署中使用Connector功能，需要在業務容器中安裝Connector組件。根據使用情境，選擇以下任一方式安裝。

方式一：Init Container安裝

在業務容器啟動前，通過Init Container下載並解壓Connector組件到共用目錄。

initContainers:
  # Init Container：在業務容器啟動前運行，用於準備依賴檔案
  - name: install-connector
    image: busybox
    command:
      - sh
      - -c
      - |
        # 下載 Connector 的 deb 安裝包
        wget -q https://gosspublic.alicdn.com/oss-connector/oss-connector-lib-1.2.0.x86_64.deb -O /tmp/connector.deb
        # 建立臨時目錄並解壓 deb 包
        mkdir -p /tmp/extract && cd /tmp/extract
        ar x /tmp/connector.deb
        # 僅提取需要的 so 檔案到共用目錄
        # 共用目錄 /shared 通過 volumeMounts 掛載到主容器可見的卷
        mkdir -p /shared/usr/local/lib
        tar -xf data.tar.gz -O ./usr/local/lib/libossc_preload.so > /shared/usr/local/lib/libossc_preload.so
    volumeMounts:
      # 將名為 connector-lib 的卷掛載到 /shared
      # 主容器也會掛載同一個卷，實現檔案分享權限設定
      - name: connector-lib
        mountPath: /shared

containers:
  - name: vllm
    image: vllm/vllm-openai:latest
    volumeMounts:
      # 將 connector-lib 卷掛載到容器的 /usr/local/lib 目錄
      # 這樣 Init Container 提取的 so 檔案就在主容器的 /usr/local/lib 下可用
      - name: connector-lib
        mountPath: /usr/local/lib
        subPath: usr/local/lib

volumes:
  # 定義共用卷，用於 Init Container 和主容器之間傳遞檔案
  - name: connector-lib
    emptyDir: {}

方式二：啟動時動態安裝

在容器啟動命令中下載並安裝Connector組件。

containers:
  - name: vllm
    image: vllm/vllm-openai:latest
    command: ["/bin/bash", "-c"]
    args:
      - |
        # 下載並安裝 Connector 的 deb 安裝包
        wget https://gosspublic.alicdn.com/oss-connector/oss-connector-lib-1.2.0.x86_64.deb
        dpkg -i oss-connector-lib-1.2.0.x86_64.deb
        # 啟動模型服務
        ENABLE_CONNECTOR=1 python3 -m vllm.entrypoints.openai.api_server --model ${MODEL_DIR} ...

方式三：自訂Dockerfile

以vLLM官方鏡像vllm/vllm-openai為例，構建預裝Connector組件的自訂鏡像。

FROM vllm/vllm-openai:latest
RUN wget https://gosspublic.alicdn.com/oss-connector/oss-connector-lib-1.2.0.x86_64.deb && \
    dpkg -i oss-connector-lib-1.2.0.x86_64.deb

構建並推送鏡像：

docker build -t myregistry/vllm-with-connector:latest .
docker push myregistry/vllm-with-connector:latest

使用安裝了Connector組件的鏡像啟動：

containers:
  - name: vllm
    image: myregistry/vllm-with-connector:latest

安裝方式對比

安裝方式	適用情境	優點	缺點
Init Container安裝/啟動時動態安裝	臨時測試、快速驗證、開發調試、不想修改基礎鏡像	無需自訂鏡像，部署靈活，配置簡單，隨用隨裝	每次啟動都需下載解壓，啟動時間較長，依賴外部網路
自訂Dockerfile	生產環境、長期啟動並執行服務、大規模叢集	啟動最快，鏡像自包含，穩定性高，可複用	需要維護自訂鏡像，綁定版本

部署模型推理服務

以vLLM官方鏡像vllm/vllm-openai為例，通過Init Container安裝Connector組件，實現模型資料從OSS載入的推理服務部署。

# ConfigMap：儲存 Connector 組件的設定檔
# 作用：將 Connector 的配置以檔案形式掛載到容器內，供其讀取
# 掛載路徑：/etc/oss-connector/config.json
apiVersion: v1
kind: ConfigMap
metadata:
  name: connector-config
data:
  config.json: |
    {
        "logLevel": 1,
        "logPath": "/var/log/oss-connector/connector.log",
        "auditPath": "/var/log/oss-connector/audit.log",
        "expireTimeSec": 120,
        "prefetch": {
            "vcpus": 16,
            "workers": 16
        }
    }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: model-connector-deployment
spec:
  selector:
    matchLabels:
      app: model-connector
  template:
    metadata:
      labels:
        app: model-connector
    spec:
      # Init Container：在業務容器啟動前運行，用於準備依賴檔案
      initContainers:
        - name: install-connector
          image: busybox
          command: ["/bin/sh", "-c"]
          args:
            - |
              # 下載 Connector 的 deb 安裝包
              wget -q https://gosspublic.alicdn.com/oss-connector/oss-connector-lib-1.2.0.x86_64.deb -O /tmp/connector.deb

              # 建立臨時目錄並解壓 deb 包
              mkdir -p /tmp/extract && cd /tmp/extract
              ar x /tmp/connector.deb

              # 僅提取需要的 so 檔案到共用目錄
              # 共用目錄 /shared 通過 volumeMounts 掛載到主容器可見的卷
              mkdir -p /shared/usr/local/lib
              tar -xf data.tar.gz -O ./usr/local/lib/libossc_preload.so > /shared/usr/local/lib/libossc_preload.so

          volumeMounts:
            # 將名為 connector-lib 的卷掛載到 /shared
            # 主容器也會掛載同一個卷，實現檔案分享權限設定
            - name: connector-lib
              mountPath: /shared
      containers:
      - name: vllm
        image: vllm/vllm-openai:latest
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: "16"
            memory: "70Gi"
          limits:
            cpu: "20"
            memory: "80Gi"
        command: ["/bin/bash", "-c"]
        args:
          - |
            # 將 Connector 庫添加到 LD_PRELOAD, 攔截檔案系統調用
            export LD_PRELOAD="/usr/local/lib/libossc_preload.so${LD_PRELOAD:+:$LD_PRELOAD}"
            # 啟動 vLLM 服務
            # ENABLE_CONNECTOR=1 啟用 OSS Connector 加速模型載入
            # 模型路徑 ${MODEL_DIR}/qwen/Qwen3-8B/ 實際指向 OSS 路徑，由 Connector 攔截並重新導向
            ENABLE_CONNECTOR=1 python3 -m vllm.entrypoints.openai.api_server \
              --model ${MODEL_DIR}/qwen/Qwen3-8B/ \
              --trust-remote-code \
              --tensor-parallel-size 1 \
              --disable-custom-all-reduce
        env:
        # OSS 訪問配置：指定內網 Endpoint 和 Region
        - name: OSS_ENDPOINT
          value: "oss-cn-beijing-internal.aliyuncs.com"
        - name: OSS_REGION
          value: "cn-beijing"

        # 映射 OSS 上的根路徑，Connector 會將本地路徑映射到此 OSS 路徑
        - name: OSS_PATH
          value: "oss://examplebucket/"

        # 本機對應目錄（被 Connector 攔截，實際從 OSS 載入）
        - name: MODEL_DIR
          value: "/var/model"

        # 從 Secret(oss-access-key-connector) 中讀取 OSS 訪問憑證
        # 需提前建立：kubectl create secret generic oss-access-key-connector \
        #   --from-literal=key=<OSS_ACCESS_KEY_ID> \
        #   --from-literal=secret=<OSS_ACCESS_KEY_SECRET>
        - name: OSS_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef:
              name: oss-access-key-connector
              key: key
        - name: OSS_ACCESS_KEY_SECRET
          valueFrom:
            secretKeyRef:
              name: oss-access-key-connector
              key: secret
        volumeMounts:
          # Connector 設定檔掛載
          - name: connector-config
            mountPath:  /etc/oss-connector/
          # 將 connector-lib 卷掛載到容器的 /usr/local/lib 目錄
          # 這樣 Init Container 提取的 so 檔案就在主容器的 /usr/local/lib 下可用
          - name: connector-lib
            mountPath: /usr/local/lib
            subPath: usr/local/lib
      terminationGracePeriodSeconds: 10
      volumes:
      # Connector 配置 ConfigMap
      - name: connector-config
        configMap:
          name: connector-config
      # 定義共用卷，用於 Init Container 和主容器之間傳遞檔案
      - name: connector-lib
        emptyDir: {}

部署多執行個體模型廣播服務

以預裝Connector組件的鏡像myregistry/vllm-with-connector為例，開啟多副本執行個體間的模型廣播。關於模型廣播的詳細介紹，請參見模型廣播。

# ConfigMap：儲存 Connector 組件的配置模板檔案
# 作用：以模板形式（config.json.tmpl）提供配置，由 init container 渲染後供主容器讀取
# 模板掛載路徑（init container）：/tmpl/config.json.tmpl
# 渲染輸出路徑（主容器）：/etc/oss-connector/config.json
apiVersion: v1
kind: ConfigMap
metadata:
  name: connector-config
data:
  config.json.tmpl: |
    {
        "logLevel": 1,
        "logPath": "/var/log/oss-connector/connector.log",
        "auditPath": "/var/log/oss-connector/audit.log",
        "expireTimeSec": 120,
        "prefetch": {
            "vcpus": 16,
            "workers": 16
        },
        "broadcast": {
            "enableBroadcast": true,
            "tenant": "${REDIS_TENANT}",
            "db": {
                "host": "${REDIS_HOST}",
                "port": 6379,
                "username": "${REDIS_USERNAME}",
                "password": "${REDIS_PASSWORD}"
            }
        },
        "bindPort": 19989
    }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: model-connector-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: model-connector
  template:
    metadata:
      labels:
        app: model-connector
    spec:
      # init container：在主容器啟動前，將配置模板渲染為最終的 config.json
      initContainers:
      - name: render-config
        image: busybox
        command: ["/bin/sh", "-c"]
        args:
          # 用 sed 逐一替換模板中的預留位置為實際環境變數值
          - |
            sed -e "s|\${REDIS_HOST}|$REDIS_HOST|g" \
                -e "s|\${REDIS_USERNAME}|$REDIS_USERNAME|g" \
                -e "s|\${REDIS_PASSWORD}|$REDIS_PASSWORD|g" \
                -e "s|\${REDIS_TENANT}|$REDIS_TENANT|g" \
                /tmpl/config.json.tmpl > /etc/oss-connector/config.json
        env:
        # 從 Secret(redis-secret) 中讀取 Redis 串連資訊
        # 需提前建立：kubectl create secret generic redis-secret \
        #   --from-literal=host=<host> \
        #   --from-literal=username=<username> \
        #   --from-literal=password=<password>
        - name: REDIS_HOST
          valueFrom:
            secretKeyRef:
              name: redis-secret
              key: host
        - name: REDIS_USERNAME
          valueFrom:
            secretKeyRef:
              name: redis-secret
              key: username
        - name: REDIS_PASSWORD
          valueFrom:
            secretKeyRef:
              name: redis-secret
              key: password
        - name: REDIS_TENANT
          value: "broadcast-demo"
        volumeMounts:
          # 掛載 ConfigMap 中的模板檔案，作為 sed 的輸入
          - name: connector-config
            mountPath: /tmpl
          # 掛載共用 emptyDir，將渲染結果寫入此處供主容器使用
          - name: rendered-config
            mountPath: /etc/oss-connector
      containers:
      - name: vllm
        image: myregistry/vllm-with-connector:latest
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: "16"
            memory: "70Gi"
          limits:
            cpu: "20"
            memory: "80Gi"
        command: ["/bin/bash", "-c"]
        args:
          - |
            # 將 Connector 庫添加到 LD_PRELOAD, 攔截檔案系統調用
            export LD_PRELOAD="/usr/local/lib/libossc_preload.so${LD_PRELOAD:+:$LD_PRELOAD}"
            # 啟動 vLLM 服務
            # ENABLE_CONNECTOR=1 啟用 OSS Connector 加速模型載入
            ENABLE_CONNECTOR=1 python3 -m vllm.entrypoints.openai.api_server \
              --model ${MODEL_DIR}/qwen/Qwen3-8B/ \
              --trust-remote-code \
              --tensor-parallel-size 1 \
              --disable-custom-all-reduce
        env:
        # OSS 訪問配置：指定內網 Endpoint 和 Region
        - name: OSS_ENDPOINT
          value: "oss-cn-beijing-internal.aliyuncs.com"
        - name: OSS_REGION
          value: "cn-beijing"

        # 映射 OSS 上的根路徑，Connector 會將本地路徑映射到此 OSS 路徑
        - name: OSS_PATH
          value: "oss://examplebucket/"

        # 本機對應目錄（被 Connector 攔截，實際從 OSS 載入）
        - name: MODEL_DIR
          value: "/var/model"

        # 從 Secret(oss-access-key-connector) 中讀取 OSS 訪問憑證
        - name: OSS_ACCESS_KEY_ID
          valueFrom:
            secretKeyRef:
              name: oss-access-key-connector
              key: key
        - name: OSS_ACCESS_KEY_SECRET
          valueFrom:
            secretKeyRef:
              name: oss-access-key-connector
              key: secret
        volumeMounts:
          # Connector 設定檔掛載（rendered by init container）
          - name: rendered-config
            mountPath: /etc/oss-connector/
      terminationGracePeriodSeconds: 10
      volumes:
      # Connector 配置模板 ConfigMap（掛載到 init container）
      - name: connector-config
        configMap:
          name: connector-config
      # init container 渲染後的設定檔（共用給主容器）
      - name: rendered-config
        emptyDir: {}