Video sprite generation - Intelligent Media Management

This topic describes how to generate video sprites using the media transcoding feature of Intelligent Media Management (IMM).

Feature introduction

Video sprite generation is a technique that combines multiple keyframes from a video into a single image in the form of thumbnails. By generating sprites, users can quickly browse different scenes of a video in one image, allowing them to select content to watch more effectively.

Scenarios

Video preview and content navigation: Sprites provide users with a quick preview of video content, allowing them to clearly browse the keyframes in the video and easily select parts of interest.
Video playlists: In playlists, sprites can be used to display thumbnails for each video, making the entire list interface more concise and visually appealing, and enhancing the user experience.
Social media sharing: When sharing videos on social media, sprites can serve as cover images, helping users quickly understand the video content, thereby increasing click-through rates and views.
Live streaming and event playback: In live event broadcasts or recorded content playback, sprites help viewers quickly locate highlights of interest, enhancing viewing convenience.

Supported audio and video formats

Category	Format
Audio	Mainstream audio formats, such as AAC, MP3, WAV, FLAC, WMA, AC3, and Opus.
Video	Mainstream video formats, such as MP4, MPEG-TS, MKV, MOV, AVI, FLV, M3U8, WebM, WMV, RM, and VOB.

Procedure

Prerequisites

An AccessKey pair is created. For more information, see Create an AccessKey pair.
Object Storage Service (OSS) is activated, and a bucket is created. For more information, see Create a bucket.
IMM is activated. For more information, see Activate IMM.
A project is created in the IMM console. For more information, see Create a project.
Note
- You can call the CreateProject operation to create a project.
- You can call the ListProjects operation to list all project information in the specified region.

Step 1: Upload files

Use the OSS console to upload media files to a bucket in the same region as your IMM project.

Step 2: Use IMM to generate video sprites

Call the CreateMediaConvertTask operation to create a video sprite generation task.

Parameter examples

The process of generating video sprites typically includes the following steps:

Video frame extraction: Extract selected keyframes from the video.
Image processing: Resize and convert the extracted frames to maintain consistency and meet output requirements.
Sprite generation: Arrange the processed frames in a grid and merge them into a single image to form a sprite.

The following example uses the IMM project test-project and the file oss://test-bucket/video-demo/酿酒.mov to generate video sprites.

For more information about media processing features and usage, see Media transcoding.

Note

You can use the OpenAPI Explorer to process video files using the media transcoding operation. You can refer to the SDK code.
Do not set the Target.URI and Target.Container parameters for video sprite generation.
Multiple sprite images will be generated, with the exact number depending on the number of sub-images and the row and column configuration of the sprite.
It is recommended to configure the sprite output URI in the format "oss://bucket/objectPrefix-{index}.{autoext}". If the index variable is not set, sprite images may overwrite each other. For related variables, see TargetURI template.

Generate sprites by capturing frames every 2 seconds throughout video

Generate sprites by capturing frames every 2 seconds throughout the video, with each sprite having a 3×3 grid layout and sub-image resolution of 200×150.

Transcoding information

Format: JPG
Frame capture interval: 2 seconds
Sprite sub-image layout: 3×3
Single sub-image resolution: 200×150
Padding between sub-images (Pad): 0
Margin between sub-images and sprite edges (Margin): 0
Sub-image scaling method: Scale and crop.

Output file save path: oss://test-bucket/video-demo/sprites-%d.jpg
Frame capture completion notification: Send MNS message to the MNS topic "test-mns-topic"

For SDK examples, see OpenAPI Explorer. The example parameters have been automatically filled in for you. You can modify them as needed before debugging.

Request parameters

{
  "ProjectName": "test-project",
  "Notification": {
    "MNS": {
      "TopicName": "test-mns-topic"
    }
  },
  "Sources": [
    {
      "URI": "oss://test-bucket/video-demo/酿酒.mov"
    }
  ],
  "Targets": [
    {
      "Image": {
        "Sprites": [
          {
            "Format": "jpg",
            "Interval": 2,
            "Margin": 0,
            "Pad": 0,
            "ScaleHeight": 150,
            "ScaleType": "crop",
            "ScaleWidth": 200,
            "TileHeight": 3,
            "TileWidth": 3,
            "URI": "oss://test-bucket/video-demo/sprites-{index}.{autoext}"
          }
        ]
      }
    }
  ]
}

Generate sprites by capturing frames every 2 seconds from the 5s to 30s portion of the video

Generate sprites by capturing frames every 2 seconds from the 5s to 30s portion of the video, with each sprite having a 3×5 grid layout and sub-image dimensions at 1/10 of the source video.

Transcoding information

Format: JPG
Frame capture start time: 5 seconds
Frame capture duration: 25 seconds
Frame capture interval: 2 seconds
Sprite sub-image layout: 3×5
Single sub-image resolution: 1/10 of the source video
Padding between sub-images (Pad): 4
Margin between sub-images and sprite edges (Margin): 5
Sub-image scaling method: Scale without preserving black bars.

Output file save path: oss://test-bucket/video-demo/sprites-%d.jpg
Frame capture completion notification: Send MNS message to the MNS topic "test-mns-topic"

For SDK examples, see OpenAPI Explorer. The example parameters have been automatically filled in for you. You can modify them as needed before debugging.

Request parameters

{
  "ProjectName": "test-project",
  "Notification": {
    "MNS": {
      "TopicName": "test-mns-topic"
    }
  },
  "Sources": [
    {
      "Duration": 25,
      "StartTime": 5,
      "URI": "oss://test-bucket/video-demo/酿酒.mov"
    }
  ],
  "Targets": [
    {
      "Image": {
        "Sprites": [
          {
            "Format": "jpg",
            "Interval": 2,
            "Margin": 5,
            "Pad": 4,
            "ScaleHeight": 0.1,
            "ScaleType": "fit",
            "ScaleWidth": 0.1,
            "TileHeight": 5,
            "TileWidth": 3,
            "URI": "oss://test-bucket/video-demo/sprites-{index}.{autoext}"
          }
        ]
      }
    }
  ]
}

Generate sprites by evenly capturing frames based on quantity

Use the DetectMediaMeta operation to generate sprites by evenly capturing frames based on quantity.

Step 1: Obtain video duration

Call the DetectMediaMeta operation to obtain the video duration. For SDK examples, see OpenAPI Explorer. The example parameters have been automatically filled in for you. You can modify them as needed before debugging.

Request parameters

{
  "ProjectName": "test-project",
  "SourceURI": "oss://test-bucket/video-demo/酿酒.mov"
}

Response parameters (obtain the Duration which is the video duration)

{
  "Addresses": [],
  "AudioStreams": [],
  "Bitrate": 45521134,
  "Duration": 31.64,
  "FormatLongName": "QuickTime / MOV",
  "FormatName": "mov,mp4,m4a,3gp,3g2,mj2",
  "ProduceTime": "2014-12-19T14:20:38Z",
  "RequestId": "94A9AF46-B403-51D5-988F-D9B4939DEF39",
  "Size": 180036088,
  "StreamCount": 2,
  "Subtitles": [],
  "VideoHeight": 1080,
  "VideoStreams": [
    {
      "AverageFrameRate": "25/1",
      "BitDepth": 8,
      "Bitrate": 45517646,
      "CodecLongName": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
      "CodecName": "h264",
      "CodecTag": "0x31637661",
      "CodecTagString": "avc1",
      "ColorPrimaries": "bt709",
      "ColorRange": "tv",
      "ColorSpace": "bt709",
      "ColorTransfer": "bt709",
      "Duration": 31.64,
      "FrameCount": 791,
      "FrameRate": "25/1",
      "HasBFrames": 1,
      "Height": 1080,
      "Language": "eng",
      "Level": 50,
      "PixelFormat": "yuv420p",
      "Profile": "Main",
      "TimeBase": "1/2500",
      "Width": 1920
    }
  ],
  "VideoWidth": 1920
}

Step 2: Generate sprites by evenly capturing frames based on video duration

The video duration is 31.64s. To evenly capture 9 sub-images, the frame capture interval is 31.64/(9 - 1) = 3.955s. The sprite sub-image layout is 3×3, with sub-image dimensions at 1/10 of the source video.

Transcoding information

Format: JPG
Frame capture interval: 3.955 seconds
Sprite sub-image layout: 3×3
Single sub-image resolution: 1/10 of the source video
Padding between sub-images (Pad): 4
Margin between sub-images and sprite edges (Margin): 5
Sub-image scaling method: Scale and crop.

Output file save path: oss://test-bucket/video-demo/sprites-%d.jpg
Frame capture completion notification: Send MNS message to the MNS topic "test-mns-topic"

For SDK examples, see OpenAPI Explorer. The example parameters have been automatically filled in for you. You can modify them as needed before debugging.

Request parameters

{
  "ProjectName": "test-project",
  "Notification": {
    "MNS": {
      "TopicName": "test-mns-topic"
    }
  },
  "Sources": [
    {
      "URI": "oss://test-bucket/video-demo/酿酒.mov"
    }
  ],
  "Targets": [
    {
      "Image": {
        "Sprites": [
          {
            "Format": "jpg",
            "Interval": 3.955,
            "Margin": 5,
            "Pad": 4,
            "ScaleHeight": 0.1,
            "ScaleType": "fit",
            "ScaleWidth": 0.1,
            "TileHeight": 3,
            "TileWidth": 3,
            "URI": "oss://test-bucket/video-demo/sprites-{index}.{autoext}"
          }
        ]
      }
    }
  ]
}