Introduction to offline prediction

Overview

EasyVision can read data from and write the results to MaxCompute tables. EasyVision can also read OSS files for prediction, and then write the results back to OSS files. The offline prediction process can be viewed as an assembly line process. Each atomic operation can be processed asynchronously and concurrently in multiple threads on each worker. During the I/O operation, each worker obtains input data by using shards and write data to corresponding output shards. For example, when images are read from a table for model prediction, the system splits data in the input table based on the number of workers. Each worker reads its own data, decodes Base64 data, performs model prediction, and writes the results to the output table. Base64 decoding and model prediction are performed asynchronously by multiple threads. This can make full use of the CPU and GPU computing power for concurrent processing. The following figure shows the Base64 decoding and model prediction processes.

EasyVision provides video-level prediction models to process video data. You can also call image-related models to predict video frames. The offline processing framework of EasyVision automatically decodes video data, predicts single-frame images, and summarizes all video frame results.

You may want to load trained models for offline prediction. The default prediction code provided by EasyVision cannot meet such requirements. Therefore, EasyVision allows you to customize prediction code and reuse the existing I/O feature of ev_predict. You can perform the offline prediction of your model by downloading and decoding data. EasyVision also allows you to insert a custom process before prediction. Input data is processed before being sent to the predictor. For more information, see Custom input data.

Input data formats

• Data read from MaxCompute tables
  The input table can contain one or more columns. One column is for the Base64-encoded URLs or binary data of images. The data in this column is of the string type. Schema example:

  +------------------------------------------------------------------------------------+
  | Field           | Type       | Label | Comment                                     |
  +------------------------------------------------------------------------------------+
  | id              | string     |       |                                             |
  | url             | string     |       |                                             |
  +------------------------------------------------------------------------------------+

• Data read from OSS files
  Each column of the OSS input file is a URL or an OSS endpoint. Example:

  oss://your/path/to/image.jpg
  http://your.path/to/image.jpg

• Custom input data

  When data is read from a MaxCompute table, only the URL or Base64-encoded data of the image is obtained. When data is read from an OSS file, only the URL of the image is obtained for you to download and decode the image. Either method can only obtain NumPy arrays of the image. The {"image": np.ndarray} format is used by various processes and predictors during prediction. As more and more users employ custom predictors and processes, this single input format cannot meet requirements. Therefore, the OSS reading mode is modified to support custom data formats.

  The custom format can be the original OSS file format or JSON string format. Each line in the OSS file is a JSON string. You can enter multiple key-value pairs. All key-value pairs are saved in a dictionary and transferred to custom predictors and processes. You can easily obtain the corresponding values based on custom keys.

  If the value is an OSS path or URL, the system automatically uses multiple threads to download the file content and converts the value to a Python file-like object. You can directly call the file method such as read() or readlines() to obtain the file content. If the value points to a file with an image extension, the system automatically decodes the image. You obtain the value from the input_data dictionary based on the key. The value is of the numpy.ndarray type.

  Input data example:

  {"image":"http://your/path/to/image.jpg", "prior":"oss://your/path/to/prior.txt", "config": {"key1":1, "key2":"value2"}}
  {"image":"http://your/path/to/image.jpg", "prior":"oss://your/path/to/prior.txt", "config": {"key2":1, "key2":"value2"}}

  The preceding input data will be converted into data in the input_data dictionary.

  • The value of the image field is the decoded data of an image.
  • The value of the prior field is a file.
  • The value of the config field is a dictionary of JSON strings.

  The input_data dictionary is in the following format. For all custom processes and predictors, you can find their values based on their keys.

  input_dict = {
    "image": np.ndarray,
    "prior" : file_like_object,
    "config": {"key1":1, "key2":"value2"}
  }

  Note All built-in predictors of EasyVision use the image key to obtain input images. If you want to use a custom input format to call the built-in predictors of EasyVision, the image key must be used for image data.

Parameters