Platform For AI:Create a dataset for a labeling job

Item

Method 1: From cloud service

Method 2: Local upload

Procedure

1. Create a local .manifest or .txt file using the format described below.

2. Upload the file to OSS. Upload files.

3. Create a dataset from a cloud service. Create a dataset: From an Alibaba Cloud cloud service.

1. Create a local .csv or .xlsx file using the format described below.

2. Go to iTAG.

   1. Log on to the PAI console.

   2. In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the target workspace.

   3. In the left-side navigation pane, choose Data Preparation>iTAG.

3. On the iTAG page, click Go to Task Center or Go to Management Page.

4. On the Data Management tab, click Create Original Dataset.

5. On the Create Original Dataset page, configure the following parameters:

   • For Import Data, select Local Upload.

   • For Import Format, select File.

   • Configure OSS Bucket and File Path in OSS.

   • Click Upload file and select the .csv or .xlsx file that you created.

6. Click Create.

File name extension

A .manifest or .txt file.

A .csv or .xlsx file.

File format

{"data":{"source":"text sample 1"}}
{"data":{"source":"text sample 2"}}
{"data":{"source":"text sample 3"}}

The source parameter specifies the sample content to label. Replace the source value with your text content.

A column in the .csv or .xlsx file contains the text to be labeled.

File example

textDemo1.manifest

textDemo2.csv

Item

Method 1: Scan folder

Method 2: Local upload

Procedure

1. Upload the image files to an OSS bucket to generate their URLs. Upload files.

2. Create a dataset by scanning a folder, which automatically generates a .manifest file. Create and manage datasets.

1. Create a local folder containing the images.

2. Go to iTAG.

   1. Log on to the PAI console.

   2. In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the target workspace.

   3. In the left-side navigation pane, choose Data Preparation>iTAG.

3. On the iTAG page, click Go to Task Center or Go to Management Page.

4. On the Data Management tab, click Create Original Dataset. In the Create Original Dataset panel, configure the following parameters:

   • For Import Data, select Local Upload.

   • For Import Format, select Folder.

   • Configure OSS bucket and File Path in OSS.

   • Click Upload Folder and upload the local folder.

5. Click Create.

File format

{"data":{"source":"oss://****.oss-cn-hangzhou.aliyuncs.com/iTAG/pic/1.jpg"}}
{"data":{"source":"oss://****.oss-cn-hangzhou.aliyuncs.com/iTAG/pic/10.jpg"}}
{"data":{"source":"oss://****.oss-cn-hangzhou.aliyuncs.com/iTAG/pic/11.jpg"}}

The source parameter specifies the sample to annotate. Set the source value to the OSS URL of the sample.

File example

• Image: pictureDemo.manifest

• Video: videoDemo.manifest

• Audio: audioDemo.manifest

Item

From cloud service

Procedure

1. Create a local .manifest or .txt file using the format described below.

2. Upload the file to OSS. Upload files.

3. Create a dataset from a cloud service. Create and manage datasets.

File name extension

A .manifest or .txt file.

File format

{"data":{"picture_url":"oss://****.oss-cn-hangzhou.aliyuncs.com/iTAG/pic/1.jpg","text":"Jack Ma and 17 other founders established Alibaba Group in a Hangzhou apartment. The group's first website was Alibaba.com, an English-language global wholesale marketplace."}}
{"data":{"picture_url":"oss://****.oss-cn-hangzhou.aliyuncs.com/iTAG/pic/10.jpg","text":"Alibaba Group held the first West Lake Summit, bringing together business and thought leaders from the internet industry to discuss key topics."}}
{"data":{"picture_url":"oss://****.oss-cn-hangzhou.aliyuncs.com/iTAG/pic/11.jpg","text":"Alibaba Group raised USD 82 million from several top-tier investment firms, which was the largest private equity financing in China's internet industry at the time."}}

Each line's "data" object represents one data item to label. The object can contain multiple key-value pairs for different data types, such as an image and text.

For example, the following line defines a data item with an image stored at oss://****.oss url 01 and text text sample1.

{"data":{"picture_url":"oss://****.oss url 01","text":"text sample1"}}

File example

multiModal.manifest