Alibaba Cloud Model Studio：微調視頻產生模型

圖生視頻-基於首幀

curl --location 'https://dashscope-intl.aliyuncs.com/api/v1/fine-tunes' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model":"wan2.6-i2v",
    "training_file_ids":[
        "<替換為訓練資料集的檔案id>"
    ],
    "training_type":"efficient_sft",
    "hyper_parameters":{
        "n_epochs":400,
        "batch_size":2,
        "learning_rate":2e-5,
        "split":0.9,
        "eval_epochs": 50,
        "max_pixels": 36864
    }
}'

圖生視頻-基於首尾幀

curl --location 'https://dashscope-intl.aliyuncs.com/api/v1/fine-tunes' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model":"wan2.2-kf2v-flash",
    "training_file_ids":[
        "<替換為訓練資料集的檔案id>"
    ],
    "training_type":"efficient_sft",
    "hyper_parameters":{
        "n_epochs":400,
        "batch_size":4,
        "learning_rate":2e-5,
        "split":0.9,
        "eval_epochs": 50,
        "max_pixels": 262144
    }
}'

圖生視頻-基於首幀

curl --location 'https://dashscope-intl.aliyuncs.com/api/v1/deployments' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model_name": "<替換為模型名稱model_name>",
    "aigc_config": {
        "use_input_prompt": false,
        "prompt": "Provide a video description based on the image content. The description must include the phrase “Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she/he stretches her/his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.“\nOutput Template:\nThe video begins with a shot of [subject description]. [Environment description]. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she/he stretches her/his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.\nExample:\nThe video begins showing a young woman standing in front of a brick wall covered with ivy. She has long, smooth reddish-brown hair, wearing a white sleeveless dress, a shiny silver necklace, and a smile on her face. The brick wall in the background is covered with green vines, appearing rustic and natural. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she stretches her arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.",
        "lora_prompt_default": "Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she/he stretches her/his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain."
    },
    "capacity": 1,
    "plan": "lora"
}'

圖生視頻-基於首尾幀

curl --location 'https://dashscope-intl.aliyuncs.com/api/v1/deployments' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model_name": "<替換為模型名稱model_name>",
    "aigc_config": {
        "use_input_prompt": false,
        "prompt": "Provide a video description based on the image content. The description must include the phrase “Then she/he begins the s86b5p transformation.“\nOutput Template:\nThe video begins with a shot of [subject description]. [Environment description]. Then she/he begins the s86b5p transformation.\nExample:\nThe video begins with a young woman in an outdoor setting. She has short, curly dark brown hair and a friendly smile. She is wearing a black Polo shirt with colorful floral embroidery. The background features green vegetation and distant mountains. Then she begins the s86b5p transformation.",
        "lora_prompt_default": "Then she/he begins the s86b5p transformation."
    },
    "capacity": 1,
    "plan": "lora"
}'

圖生視頻-基於首幀

訓練集包括首幀映像、訓練視頻和標註檔案（data.jsonl）。

• 訓練集範例：wan-i2v-training-dataset.zip。

• zip包目錄結構：

  wan-i2v-training-dataset.zip
  ├── data.jsonl        # 必須固定命名為data.jsonl，最大支援 20MB
  ├── image_1.jpeg      # 映像最大解析度為4096*4096，支援BMP、JPEG、PNG、WEBP格式
  ├── video_1.mp4       # 視頻最大解析度為4096*4096，支援MP4、MOV格式
  ├── image_2.jpeg
  └── video_2.mp4

• 標註檔案（data.jsonl）：每一行代表一條訓練資料，必須為JSON 對象。一行訓練資料的結構如下：

  {
      "prompt": "The video begins showing a young woman standing in front of a brick wall covered with ivy. She has long, smooth reddish-brown hair, wearing a white sleeveless dress, a shiny silver necklace, and a smile on her face. The brick wall in the background is covered with green vines, appearing rustic and natural. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she stretches her arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.",
      "first_frame_path": "image_1.jpg",
      "video_path": "video_1.mp4"        
  }

圖生視頻-基於首尾幀

訓練集包括首幀映像、尾幀映像、訓練視頻和標註檔案（data.jsonl）。

• 訓練集範例：wan-kf2v-training-dataset.zip。

• zip包目錄結構：

  wan-kf2v-training-dataset.zip
  ├── data.jsonl                # 必須固定命名為data.jsonl，最大支援 20MB
  ├── image/                    # 存放首幀和尾幀映像
  │   ├── image_1_first.jpg     # 映像最大解析度為4096*4096，支援BMP、JPEG、PNG、WEBP格式
  │   └── image_1_last.png
  └── video/                    # 存放作為“訓練目標”的視頻檔案
      ├── video_1.mp4           # 視頻最大解析度為4096*4096，支援MP4、MOV格式
      └── video_2.mov

• 標註檔案（data.jsonl）：每一行代表一條訓練資料，必須為JSON 對象。一行訓練資料的結構如下：

  {
      "prompt": "The video begins by showing a young woman in an outdoor setting. She has short, curly dark brown hair, a smile on her face, and looks very friendly. She is wearing a black polo shirt with colorful floral embroidery, with a background of green vegetation and distant mountains. Then she begins the s86b5p transformation.",
      "first_frame_path": "image/image_1_first.jpg",
      "last_frame_path": "image/image_1_last.jpg", 
      "video_path": "video/video_1.mp4"  
  }

圖生視頻-基於首幀

驗證集包括首幀映像和標註檔案（data.jsonl），無需提供視頻。訓練任務會在每個評估節點，自動調用模型服務，使用驗證集的映像和 Prompt 產生預覽視頻。

• 驗證集範例：wan-i2v-valid-dataset.zip。

• zip包目錄結構：

  wan-i2v-valid-dataset.zip
  ├── data.jsonl       # 必須固定命名為data.jsonl，最大支援 20MB
  ├── image_1.jpeg     # 映像最大解析度為4096*4096，支援BMP、JPEG、PNG、WEBP格式
  └── image_2.jpeg

• 標註檔案（data.jsonl）：每一行代表一條驗證資料，必須為JSON 對象。一行驗證資料的結構如下：

  {
      "prompt": "The video begins showing a scene of a young man standing in front of a cityscape. He is wearing a black and white checkered jacket over a black hoodie, with a smile on his face and a confident expression. The background is a city skyline at sunset, with a famous domed building and layered roofs visible in the distance, the sky filled with clouds showing warm orange-yellow hues. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding him. The bills continue to fall while the camera slowly zooms in, he stretches his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.",
      "first_frame_path": "image_1.jpg"
  }

圖生視頻-基於首尾幀

驗證集包括首幀映像、尾幀映像和標註檔案（data.jsonl），無需提供視頻。訓練任務會在每個評估節點，自動調用模型服務，使用驗證集的映像和 Prompt 產生預覽視頻。

• 驗證集範例：wan-kf2v-valid-dataset.zip。

• zip包目錄結構：

  wan-kf2v-valid-dataset.zip
  ├── data.jsonl                 # 必須固定命名為data.jsonl，最大支援 20MB
  └── image/                     # 存放首幀和尾幀映像
      ├── image_1_first.jpg      # 映像最大解析度為4096*4096，支援BMP、JPEG、PNG、WEBP格式
      └── image_1_last.jpg

• 標註檔案（data.jsonl）：每一行代表一條驗證資料，必須為JSON 對象。一行驗證資料的結構如下：

  {
      "prompt": "The video begins showing a scene of a young man standing in front of a cityscape. He is wearing a black and white checkered jacket over a black hoodie, with a smile on his face and a confident expression. The background is a city skyline at sunset, with a famous domed building and layered roofs visible in the distance, the sky filled with clouds showing warm orange-yellow hues. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding him. The bills continue to fall while the camera slowly zooms in, he stretches his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.",
      "first_frame_path": "image/image_1_first.jpg",
      "last_frame_path": "image/image_1_last.jpg",
  }