Alibaba Cloud Model Studio:Penyempurnaan model pembuatan video

Image-to-video berbasis frame pertama

curl --location 'https://dashscope-intl.aliyuncs.com/api/v1/fine-tunes' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model":"wan2.6-i2v",
    "training_file_ids":[
        "<replace_with_training_dataset_file_id>"
    ],
    "training_type":"efficient_sft",
    "hyper_parameters":{
        "n_epochs":400,
        "batch_size":2,
        "learning_rate":2e-5,
        "split":0.9,
        "eval_epochs": 50,
        "max_pixels": 36864
    }
}'

Image-to-video berbasis frame pertama dan terakhir

curl --location 'https://dashscope-intl.aliyuncs.com/api/v1/fine-tunes' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model":"wan2.2-kf2v-flash",
    "training_file_ids":[
        "<replace_with_training_dataset_file_id>"
    ],
    "training_type":"efficient_sft",
    "hyper_parameters":{
        "n_epochs":400,
        "batch_size":4,
        "learning_rate":2e-5,
        "split":0.9,
        "eval_epochs": 50,
        "max_pixels": 262144
    }
}'

Image-to-video berbasis frame pertama

curl --location 'https://dashscope-intl.aliyuncs.com/api/v1/deployments' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model_name": "<replace_with_model_name>",
    "aigc_config": {
        "use_input_prompt": false,
        "prompt": "Provide a video description based on the image content. The description must include the phrase “Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she/he stretches her/his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.“\nOutput Template:\nThe video begins with a shot of [subject description]. [Environment description]. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she/he stretches her/his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.\nExample:\nThe video begins showing a young woman standing in front of a brick wall covered with ivy. She has long, smooth reddish-brown hair, wearing a white sleeveless dress, a shiny silver necklace, and a smile on her face. The brick wall in the background is covered with green vines, appearing rustic and natural. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she stretches her arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.",
        "lora_prompt_default": "Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she/he stretches her/his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain."
    },
    "capacity": 1,
    "plan": "lora"
}'

Image-to-video berbasis frame pertama dan terakhir

curl --location 'https://dashscope-intl.aliyuncs.com/api/v1/deployments' \
--header "Authorization: Bearer $DASHSCOPE_API_KEY" \
--header 'Content-Type: application/json' \
--data '{
    "model_name": "<replace_with_model_name>",
    "aigc_config": {
        "use_input_prompt": false,
        "prompt": "Provide a video description based on the image content. The description must include the phrase “Then she/he begins the s86b5p transformation.“\nOutput Template:\nThe video begins with a shot of [subject description]. [Environment description]. Then she/he begins the s86b5p transformation.\nExample:\nThe video begins with a young woman in an outdoor setting. She has short, curly dark brown hair and a friendly smile. She is wearing a black Polo shirt with colorful floral embroidery. The background features green vegetation and distant mountains. Then she begins the s86b5p transformation.",
        "lora_prompt_default": "Then she/he begins the s86b5p transformation."
    },
    "capacity": 1,
    "plan": "lora"
}'

Image-to-video berbasis frame pertama

Set pelatihan mencakup gambar frame pertama, video pelatihan, dan file anotasi (data.jsonl).

• Set pelatihan sampel: wan-i2v-training-dataset.zip.

• Struktur direktori paket ZIP:

  wan-i2v-training-dataset.zip
  ├── data.jsonl        # Harus dinamai data.jsonl, ukuran maks 20 MB
  ├── image_1.jpeg      # Resolusi gambar maks 4096x4096, mendukung format BMP, JPEG, PNG, WEBP
  ├── video_1.mp4       # Resolusi video maks 4096x4096, mendukung format MP4, MOV
  ├── image_2.jpeg
  └── video_2.mp4

• File anotasi (data.jsonl): Setiap baris merepresentasikan entri data pelatihan dan harus berupa objek JSON. Struktur entri data pelatihan adalah sebagai berikut:

  {
      "prompt": "The video begins showing a young woman standing in front of a brick wall covered with ivy. She has long, smooth reddish-brown hair, wearing a white sleeveless dress, a shiny silver necklace, and a smile on her face. The brick wall in the background is covered with green vines, appearing rustic and natural. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding her. The bills continue to fall, she stretches her arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.",
      "first_frame_path": "image_1.jpg",
      "video_path": "video_1.mp4"        
  }

Image-to-video berbasis frame pertama dan terakhir

Set pelatihan mencakup gambar frame pertama, gambar frame terakhir, video pelatihan, dan file anotasi (data.jsonl).

• Set pelatihan sampel: wan-kf2v-training-dataset.zip.

• Struktur direktori paket ZIP:

  wan-kf2v-training-dataset.zip
  ├── data.jsonl                # Harus dinamai data.jsonl, ukuran maks 20 MB
  ├── image/                    # Menyimpan gambar frame pertama dan terakhir
  │   ├── image_1_first.jpg     # Resolusi gambar maks 4096x4096, mendukung format BMP, JPEG, PNG, WEBP
  │   └── image_1_last.png
  └── video/                    # Menyimpan file video sebagai "target pelatihan"
      ├── video_1.mp4           # Resolusi video maks 4096x4096, mendukung format MP4, MOV
      └── video_2.mov

• File anotasi (data.jsonl): Setiap baris merepresentasikan entri data pelatihan dan harus berupa objek JSON. Struktur entri data pelatihan adalah sebagai berikut:

  {
      "prompt": "The video begins by showing a young woman in an outdoor setting. She has short, curly dark brown hair, a smile on her face, and looks very friendly. She is wearing a black polo shirt with colorful floral embroidery, with a background of green vegetation and distant mountains. Then she begins the s86b5p transformation.",
      "first_frame_path": "image/image_1_first.jpg",
      "last_frame_path": "image/image_1_last.jpg", 
      "video_path": "video/video_1.mp4"  
  }

Image-to-video berbasis frame pertama

Set validasi mencakup gambar frame pertama dan file anotasi (data.jsonl). Anda tidak perlu menyediakan video. Pada setiap node evaluasi, pekerjaan pelatihan secara otomatis memanggil layanan model untuk menghasilkan video pratinjau menggunakan gambar dan prompt dari set validasi.

• Set validasi sampel: wan-i2v-valid-dataset.zip.

• Struktur direktori paket ZIP:

  wan-i2v-valid-dataset.zip
  ├── data.jsonl       # Harus dinamai data.jsonl, ukuran maks 20 MB
  ├── image_1.jpeg     # Resolusi gambar maks 4096x4096, mendukung format BMP, JPEG, PNG, WEBP
  └── image_2.jpeg

• File anotasi (data.jsonl): Setiap baris merepresentasikan entri data validasi dan harus berupa objek JSON. Struktur entri data validasi adalah sebagai berikut:

  {
      "prompt": "The video begins showing a scene of a young man standing in front of a cityscape. He is wearing a black and white checkered jacket over a black hoodie, with a smile on his face and a confident expression. The background is a city skyline at sunset, with a famous domed building and layered roofs visible in the distance, the sky filled with clouds showing warm orange-yellow hues. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding him. The bills continue to fall while the camera slowly zooms in, he stretches his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.",
      "first_frame_path": "image_1.jpg"
  }

Image-to-video berbasis frame pertama dan terakhir

Set validasi mencakup gambar frame pertama, gambar frame terakhir, dan file anotasi (data.jsonl). Anda tidak perlu menyediakan video. Pada setiap node evaluasi, pekerjaan pelatihan secara otomatis memanggil layanan model untuk menghasilkan video pratinjau menggunakan gambar dan prompt dari set validasi.

• Set validasi sampel: wan-kf2v-valid-dataset.zip.

• Struktur direktori paket ZIP:

  wan-kf2v-valid-dataset.zip
  ├── data.jsonl                 # Harus dinamai data.jsonl, ukuran maks 20 MB
  └── image/                     # Menyimpan gambar frame pertama dan terakhir
      ├── image_1_first.jpg      # Resolusi gambar maks 4096x4096, mendukung format BMP, JPEG, PNG, WEBP
      └── image_1_last.jpg

• File anotasi (data.jsonl): Setiap baris merepresentasikan entri data validasi dan harus berupa objek JSON. Struktur entri data validasi adalah sebagai berikut:

  {
      "prompt": "The video begins showing a scene of a young man standing in front of a cityscape. He is wearing a black and white checkered jacket over a black hoodie, with a smile on his face and a confident expression. The background is a city skyline at sunset, with a famous domed building and layered roofs visible in the distance, the sky filled with clouds showing warm orange-yellow hues. Then the s86b5p money rain effect begins, countless huge-sized US dollar bills (beige background/dark green patterns) pour down like a torrential rain, densely hitting and surrounding him. The bills continue to fall while the camera slowly zooms in, he stretches his arms upward, neck slightly tilted back, expression surprised, completely immersed in this wild money rain.",
      "first_frame_path": "image/image_1_first.jpg",
      "last_frame_path": "image/image_1_last.jpg",
  }