Platform For AI:Convert images into TFRecord files

Option

Required

Description

Format

Default value

model_type

Yes

The type of models to which the converted data is applicable. Valid values:

• CLASSIFICATION: single-label or multi-label image classification

• DETECTION: object detection

• SEGMENTATION: semantic image segmentation

• INSTANCE_SEGMENTATION: instance segmentation

• TEXT_END2END: end-to-end optical character recognition (OCR)

• TEXT_RECOGNITION: single-line text recognition

• TEXT_DETECTION: text detection

• VIDEO_CLASSIFICATION: video classification

• SELF_DEFINED: custom conversion

STRING

N/A

class_list_file

No

The OSS path of the category file. The file contains a list of category names. The category names may be presented in the following format: Category name: Name of the mapping category.

oss://path/to/your/classlit

"

test_ratio

No

The ratio that is used to divide the set of test data into different subsets. If the value is set to 0, the entire set of test data is used for training. If the value is set to 0.1, 10% of the test data is used for verification.

FLOAT

0.1

max_image_size

No

The maximum pixel value for the longer side of the images. If you configure this parameter and the size of an image exceeds the upper limit, the image is resized and saved to the TFRecord file to reduce storage space and accelerate data reading.

INT

N/A

max_test_image_size

No

The maximum pixel value for the longer side of the image that is used for testing. The value of this parameter is the same as the value of the max_image_size parameter. This parameter is used to configure test data.

INT

${max_image_size}

default_class

No

The name of the default category. A category whose name is not included in class_list is considered the default category.

STRING

None

error_class

No

The name of the invalid category. Objects or bounding boxes that belong to this category are not used for training.

STRING

N/A

ignore_class

No

The name of the category that is used only for model detection. Bounding boxes that belong to this category are not used for training.

STRING

N/A

converter_class

No

The name of the conversion class. Valid values:

• pai itag labeling format: the format of labeled files generated by iTAG

• pai labeling format(old version): the previous format of labeled files generated by iTAG

• qince labeling format: the format of labeled files generated by the qince platform

• ssl labeling format: the file format defined by PAI for self-supervised image learning

STRING

pai labeling format(old version)

separator

No

The separator that is used in the split() method to divide the label file into substrings.

STRING

N/A

image_format

No

The encoding format of the images in the TFRecord file. The following image encoding formats are commonly used:

• jpg

• png

• bmp

STRING

jpg

read_parallel_num

No

The number of concurrent reads.

INT

10

write_parallel_num

No

The number of concurrent writes to the TFRecord file.

INT

1

num_samples_per_tfrecord

No

The number of images saved in each TFRecord file.

INT

256

user_defined_converter_path

No

The HTTP or OSS path of the custom converter code. Example: http://path/to/your/converter.py.

STRING

N/A

user_defined_generator_path

No

The HTTP or OSS path of the custom generator code. Example: http://path/to/your/generator.py.

STRING

N/A

generator_class

No

The name of the custom generator class.

STRING

N/A

char_replace_map_path

No

The OSS path of the CSV file for character map replacement. The file contains two columns named original and replaced.

• The original column contains a list of original strings.

• The replaced column contains a list of strings that are used to replace the original strings.

STRING

N/A

default_char_dict_path

No

The OSS path of the file that is used to map characters to IDs. In the file, each character occupies a line. The ID of a character is equal to the line number minus 1.

STRING

N/A

decode_type

No

The format used for decoding video. Valid values:

• 1: intra only

• 2: keyframe only

• 3: without bidir

• 4: decode all

INT

4

sample_fps

No

The number of frames that are extracted for sampling per second.

FLOAT

5

reshape_size

No

The size of the output frames. Unit: pixels.

INT

224

decode_batch_size

No

The number of frames that are decoded at the same time.

INT

10

decode_keep_size

No

The number of overlapped frames in different batches.

INT

0