All Products
Search

This Product

    Platform For AI:Optimize and deploy models with Blade EAS Plugin

    Document Center

    Platform For AI:Optimize and deploy models with Blade EAS Plugin

    Last Updated:Apr 02, 2026

    Blade EAS Plugin automatically optimizes your TensorFlow or PyTorch model for faster inference before EAS deploys the service — no code changes required. Enable it by adding a plugins field to your EAS service configuration file.

    How it works

    1. Submit a service configuration file with the plugins field to the eascmd client.

    2. Before deployment, Blade EAS Plugin runs the optimization. This takes 3 to 10 minutes depending on model complexity.

    3. EAS deploys the service using the optimized model.

    Optimization runs once on initial deployment. Scale-out and scale-in operations reuse the optimized model without re-running the plugin.

    Prerequisites

    Before you begin, ensure that you have:

    Blade EAS Plugin is only supported when creating services with the eascmd client.

    Verify that the plugin ran

    After running eascmd create, check the output for this line:

    [OK] Executing plugin eas-plugin-<id>: registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:<tag>

    If this line is absent, the plugin configuration was not recognized. Check that the plugins field is correctly formatted and that you used eascmd create (not the console).

    Configure Blade EAS Plugin

    Add a plugins field to your EAS service configuration file. The field is a list of one or more plugin objects. For the full list of service configuration fields, see Create a service.

    Step 1: Choose your device target

    The command and image values depend on whether you are optimizing for CPU or GPU.

    CPU

    Key Value
    command blade --mode eas_plugin --optimize_for cpu
    image registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:cpu_latest
    processor tensorflow_cpu_1.15 (TensorFlow) or pytorch_cpu_1.6 (PyTorch)

    GPU

    Key Value
    command blade --mode eas_plugin --optimize_for gpu
    image

    CUDA 10.0: registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:gpu_latest

    CUDA 11.0: registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:cu110_latest
    processor

    TensorFlow 1.15 (CUDA 10.0): tensorflow_gpu_1.15

    TensorFlow 2.4 (CUDA 11.0): tensorflow_gpu_2.4

    PyTorch 1.6 (CUDA 10.0): pytorch_gpu_1.6

    PyTorch 1.7 (CUDA 11.0): pytorch_gpu_1.7

    Step 2: Set the plugins field

    The minimum configuration for a CPU service looks like this:

    {
  "name": "blade_eas_plugin_test",
  "model_path": "oss://<yourBucket>/<pathToYourModel>/",
  "processor": "tensorflow_cpu_1.15",
  "metadata": {
    "instance": 1,
    "memory": 4000
  },
  "plugins": [
    {
      "command": "blade --mode eas_plugin --optimize_for cpu",
      "image": "registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:cpu_latest"
    }
  ],
  "resource": "<yourEASResource>"
}

    For GPU services, add resource and gpu inside the plugin object. These fields are required for GPU optimization — resource specifies the resource group that runs the plugin (separate from the top-level resource field, which specifies the EAS service resource group). The plugin resource group must use the same GPU card type as the EAS service resource group.

    {
  "name": "blade_eas_plugin_test",
  "metadata": {
    "cpu": 4,
    "gpu": 1,
    "instance": 1,
    "memory": 4096,
    "cuda": "10.0"
  },
  "model_path": "oss://<yourBucket>/<pathToYourModel>/",
  "plugins": [
    {
      "command": "blade --mode eas_plugin --optimize_for gpu",
      "image": "registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:gpu_latest",
      "resource": "T4_8CORE",
      "gpu": 1
    }
  ],
  "processor": "tensorflow_gpu_1.15",
  "resource": "<yourEASResource>"
}

    Supported GPU resource groups by region:

    Region Supported resource groups
    China (Hangzhou) T4_8CORE
    China (Shanghai) T4_8CORE, V100_8CORE, P4_8CORE

    Step 3: Deploy the service

    Save the configuration file and run:

    eascmd create service.json

    Expected output:

    +-------------------+-------------------------------------------------------------------------------------------------+
| Internet Endpoint | http://123456789012****.cn-shanghai.pai-eas.aliyuncs.com/api/predict/blade_eas_plugin_test       |
| Intranet Endpoint | http://123456789012****.vpc.cn-shanghai.pai-eas.aliyuncs.com/api/predict/blade_eas_plugin_test   |
|             Token | owic823JI3kRmMDZlOTQzMTA3ODhmOWIzMmVkZmZmZGQyNmQ1N2M5****                                        |
+-------------------+-------------------------------------------------------------------------------------------------+
[OK] Service is now creating
[OK] Fetching processor from [http://pai-blade.oss-cn-zhangjiakou.aliyuncs.com/release/3.18.0/py3.6.8_cpu_tf1.15.0_torch1.6.0_abiprecxx11/TENSORFLOW_SDK_CPU.d12d3dc-91024d0-1.15.0-Linux.tar.gz]
[OK] Successfully downloaded all artifacts
[OK] Executing plugin eas-plugin-73d70d54: registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:cpu_latest
[OK] Building image ...
[OK] Pushing image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117172259
[OK] Successfully pushed image registry-vpc.cn-shanghai.aliyuncs.com/eas/blade_eas_plugin_test_cn-shanghai:v0.0.1-20211117172259
[OK] Successfully created ingress
[OK] Successfully patch resources
[OK] Waiting [Total: 1, Pending: 1, Running: 0]
[OK] Running [Total: 1, Pending: 0, Running: 1]
[OK] Service is running

    Advanced optimization options

    Choose an optimization level

    The optimization_level key controls the precision vs. speed trade-off. Set it in config.model_info.<model_file>:

    Level Behavior When to use
    o1 Lossless precision. Attempts FP32 or FP16 optimization based on hardware. Default. Most models, especially when precision matters
    o2 INT8 quantization. Reduces model size and increases throughput. Hardware that supports INT8 (such as T4 GPU); when a small precision loss is acceptable and maximum inference speed is the priority

    Provide model information for better results

    Providing input/output node names, shapes, and test data lets Blade tailor the optimization. Add these under config.model_info.<model_file>:

    "plugins": [
  {
    "command": "blade --mode eas_plugin --optimize_for gpu",
    "image": "registry.cn-shanghai.aliyuncs.com/eas/pai-blade-deploy:gpu_latest",
    "resource": "T4_8CORE",
    "gpu": 1,
    "config": {
      "model_info": {
        "frozen.pb": {
          "optimization_level": "o1",
          "inputs": ["input_ids_a_1"],
          "outputs": ["l2_normalize"],
          "test_data": "test_len9240_bc1.npy"
        }
      }
    }
  }
]

    frozen.pb is the model file name. The config field currently supports only the model_info subkey for configuring one model per plugin object.

    All optimization parameters

    Parameter Description
    optimization_level o1 (default, lossless) or o2 (INT8 quantization). See Choose an optimization level.
    test_data Test data file for O1 optimization. Makes optimization more targeted. The file must be included in the path or compressed package specified by model_path. For TensorFlow models, especially useful for tuning operator selection. For PyTorch models, especially useful for accurate optimization.
    calibration_dataset Calibration data file for O2 optimization. If omitted, Blade performs online INT8 quantization. If provided, Blade performs offline INT8 quantization for better accuracy. Provide more than 100 pieces of calibration data. The file must be included in model_path.
    inputs List of input node name strings. If omitted, Blade uses nodes with no upstream connections. Not required for PyTorch models.
    outputs List of output node name strings. If omitted, Blade uses nodes with no downstream connections. Not required for PyTorch models.
    input_shapes Possible input tensor shapes. Helps Blade optimize for specific input dimensions. Each inner list element corresponds to one input tensor and uses the format '<dim1>*<dim2>'. Add multiple inner lists for multiple shape scenarios. Example for a model with two inputs: [['1*512', '3*256'], ['5*512', '9*256']]
    input_ranges Value ranges for input tensor elements. Format: square brackets with real numbers or characters, such as '[1,2]', '[0.3,0.9]', or '[a,f]'. Structure mirrors input_shapes. Example: [['[0.1,0.4]', '[a,f]'], ['[1.1,1.4]', '[h,l]']]
    quantization JSON object. Currently supports the weight_adjustment key ("true" or "false"). Adjusts model parameters to reduce quantization precision loss.

    Generate auxiliary data files

    Both test_data and calibration_dataset must follow the Blade auxiliary data format. The format differs by framework.

    TensorFlow

    Auxiliary data is a list of feed dicts. Each feed dict maps input node name strings to numpy ndarrays. Save as an .npy file.

    import numpy as np

calib_data = list()
for i in range(10):
    feed_dict = {
        'image_placeholder:0': np.ones((8, 3, 224, 224), dtype=np.float32),
        'threshold_placeholder:0': np.float32(0.5),
    }
    calib_data.append(feed_dict)
np.save("calib_data.npy", calib_data)

    PyTorch

    Auxiliary data is a list of tensor tuples. Save as a .pth file using torch.save.

    import numpy as np
import torch

calib_data = list()
for i in range(10):
    image = torch.ones(8, 3, 224, 224)
    threshold = torch.tensor(0.5)
    feed_tuple = (image, threshold)
    calib_data.append(feed_tuple)
torch.save(calib_data, 'calib_data.pth')

    After generating the file, include it in the path or compressed package specified by model_path, then reference the filename in test_data or calibration_dataset.

    What's next