Platform For AI:Classify product tags with text analytics

Prepare the dataset

In the DataStudio module of DataWorks, create a table containing a single column named content and upload the dataset to the table. For more information, see Create a table and upload data.

Classify similar tags

1. Go to the Machine Learning Designer page.

   1. Log on to the PAI console.

   2. In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the workspace that you want to manage.

   3. In the left-side navigation pane, choose Model Training > Visualized Modeling (Designer).

2. Create a custom pipeline and open the pipeline page. For more information, see Create a custom pipeline.

3. Build and run the pipeline.

   1. From the component list on the left, drag the Read Table component from Source/Destination to the canvas and rename it shopping_data-1.

   2. From the component list on the left, drag the Split Word, Word Count, and Word2Vec components from Natural Language Processing > Basic NLP to the canvas.

   3. From the component list on the left, drag the Add ID Column and Type Transform components from the Data Preprocessing section to the canvas.

   4. From the component list on the left, drag the K-Means Clustering component from Machine Learning > Clustering to the canvas.

   5. From the component list on the left, drag the SQL Script component from the Custom Script category onto the canvas.

   6. Connect the components to build the pipeline. Configure the key parameters for each component as described in the following table, and then run the pipeline.

   

   No.	Description
   ①	Upload the shopping_data data and use the Split Word component to tokenize the data: On the canvas, click the shopping_data-1 component. On the Select Table tab, configure the prepared table name. On the canvas, click the Split Word-1 component. On the Field Settings tab, select the content column. Click the shopping_data-1 component and click Execute Node from the shortcut menu. After the component finishes running, run the Split Word-1 component in the same way.
   ②	Add an ID column. This is required because the uploaded data has only one field. The ID column serves as a primary key for each data entry. Click the Add ID Column-1 component and click Execute Node from the shortcut menu. After the component finishes running, run the Type Transform-1 component in the same way.
   ③	Count word frequencies to show the number of words that appear in each product description. On the canvas, click the Word Count-1 component. On the Field Settings tab, set Select Document ID Column to append_id and Select Document Content Column to content. Click the Word Count-1 component and click Execute Node.
   ④	Use the Word2Vec component to expand each word into vector dimensions based on its meaning to generate word embeddings. Word embeddings have the following properties: Words with a small vector distance have similar meanings. Distance differences between words have specific meanings. The Word2Vec component maps each word to a 100-dimensional space. On the canvas, click the Word2Vec-1 component. On the Field Settings tab, set Select Word Column to word. On the Parameters tab, select Use hierarchical softmax. Click the Word2Vec-1 component and click Execute Node.
   ⑤	Cluster word embeddings. Use the k-means clustering algorithm to calculate distances between word embeddings and automatically classify tag words by meaning. On the canvas, click the K-Means Clustering-1 component. On the Field Settings tab, set Feature Column(s) to f0 and Append Column(s) to word. Note The number of rows in the upstream input data table must be greater than or equal to the number of clusters set in the component parameters. Click the K-Means Clustering-1 component and click Execute Node.
   ⑥	Verify the result. Use the SQL Script-1 component to select a random cluster category to check whether tags of the same category are automatically classified together. This pipeline selects the 10th cluster group. On the canvas, click the SQL Script-1 component. On the Parameters tab, set SQL Script to select * from ${t1} where cluster_index=10. In the result, the system automatically classifies geography-related tags together. However, some tags such as nuts that clearly do not fit the category are mixed in. This may be because the training samples are insufficient. With sufficient training samples, the tag clustering result will be more accurate.