Machine Learning Platform for AI provides the video classification and object marking templates. When you create a labeling job, select a template that meets your requirements.

Video classification

Video classification is used to find one or more labels from a set of labels to match the content of an input video and attach the labels to the video. The template supports single-label and multi-label video classification.

  • Scenarios

    Video surveillance, live streaming recommendation, and short video recommendation.

  • Data structure
    • Input data
      Each row in the manifest file contains a topic. The topic must contain the videoUrl field. 
      {"data":{"videoUrl":"oss://xxxxxxxxx.oss-cn-shanghai-internal.aliyuncs.com/video/English.mp4"}}
    • Output data
      Each row in the manifest file contains a topic and the labeling result. The following code provides an example of the JSON string in each row: 
      {
  "data": {
    "videoUrl": "oss://xxxxxxxxxxx.oss-cn-shanghai-internal.aliyuncs.com/video/show.webm"
  },
  "label-bo0pc51ok7olwe****": {
    "results": [{
      "data": [{
        "data": ["Car promotion", "Promotion video"],
        "id": 165****,
        "type": "survey/multivalue"
      }],
      "id": "165****",
      "type": "survey"
    }]
  }
}

Object marking

Object marking is used to locate and mark specific objects in a frame or specific frames of a video. The rectangle selection tool and polygon selection tool are commonly used.

  • Scenarios

    Autonomous driving, security surveillance, and video recommendation.

  • Data structure
    • Input data
      Each row in the manifest file contains a topic. The topic must contain the videoUrl field. 
      {"data":{"videoUrl":"oss://xxxxxxxxx.oss-cn-shanghai-internal.aliyuncs.com/video/English.mp4"}}
    • Output data
      Each row in the manifest file contains a topic and the labeling results of one or more frames. The following code provides an example of the JSON string in each row: 
      {
  "data": {
    "videoUrl": "oss://xxxxxxxxx.oss-cn-shanghai-internal.aliyuncs.com/video/English.mp4"
  },
  "label-rv0ih5l409u9x9****": {
    "results": [{
      "data": [{
        "speed_of_play": 1,
        "duration": 300.733375,
        "width": 960,
        "number_of_frames": 9022,
        "type": "video/meta",
        "frame_rate": 30,
        "height": 540
      }, {
        "frames": {
          "frame-443": {
            "L6G-5Oc5STCSOuzKb****": {
              "rotation": 0,
              "x": 336.0483870967742,
              "width": 488.4677419354839,
              "y": 108.87096774193549,
              "height": 349.83870967741933
            }
          }
        },
        "type": "video/frames"
      }, {
        "frames": [443],
        "custom": {
          "Video type": "English learning",
          "Video name": "English learning",
          "Category": ["Education"]
        },
        "id": "L6G-5Oc5STCSOuzKb****",
        "type": "video/rectangleLabel",
        "value": "",
        "labelColor": "#ff7700"
      }],
      "id": "129****",
      "type": "video"
    }]
  }
}