The Word2Vec component uses a neural network to map words to K-dimensional vectors. These vectors capture semantic relationships, allowing you to perform mathematical operations that correspond to linguistic meanings. The component accepts a word column or text corpus as input and generates a word vector table and a vocabulary.
Usage notes
Connect the Word2Vec component to the output of the word frequency statistics component.
The output of the word frequency statistics component is a set of triples that contain words and their counts. The Word2Vec component takes this output, converts the data into a list of single words, and then processes the entire list as a single document.
Configure the component
You can configure the Word2Vec component in one of the following ways.
Method 1: Configure in the GUI
Configure the component parameters on the pipeline page in Machine Learning Designer.
|
Tab |
Parameter |
Description |
|
Field Settings |
Select word column |
The word column to use for training. For best performance, we recommend that the number of words does not exceed 10 million. |
|
Parameter Settings |
Word feature dimension |
The number of dimensions for the word features. Valid values: 0 to 1,000. Default value: 100. |
|
Language model |
The language model to use for training. Valid values: skip-gram model and CBOW model. Default value: skip-gram model. |
|
|
Word window size |
The size of the word window. The value must be a positive integer. Default value: 5. |
|
|
Use random window |
Specifies whether to use a random window. Enabled by default. |
|
|
Minimum word frequency |
The value must be a positive integer. Default value: 5. |
|
|
Use hierarchical softmax |
Specifies whether to use hierarchical softmax. Enabled by default. |
|
|
Negative sampling |
The number of negative samples to use. A value of 0 disables this feature. Default value: 0. |
|
|
Downsampling threshold |
The threshold for downsampling frequent words. A value of 0 disables this feature. Default value: 0. |
|
|
Initial learning rate |
The value must be greater than 0. Default value: 0.025. |
|
|
Number of iterations |
The value must be 1 or greater. Default value: 1. |
|
|
Tuning |
Number of cores |
By default, the system allocates resources automatically. |
|
Memory size per core |
By default, the system allocates resources automatically. |
Method 2: Use a PAI command
Configure the component by using a PAI command. You can run a PAI command with the SQL Script component. For more information, see SQL Script.
pai -name Word2Vec
-project algo_public
-DinputTableName=w2v_input
-DwordColName=word
-DoutputTableName=w2v_output;|
Parameter |
Required |
Description |
Default |
|
inputTableName |
Yes |
The name of the input vocabulary table. |
None |
|
inputTablePartitions |
No |
The input table partitions for tokenization. Format: |
None |
|
wordColName |
Yes |
The name of the word column. Each row in this column must contain a single word. The </s> string represents a newline character. |
None |
|
inVocabularyTableName |
No |
The output table from a wordcount operation on the input vocabulary. |
The system performs a wordcount operation on the input table. |
|
inVocabularyPartitions |
No |
Partitions of the output table from the word count operation. |
All partitions of the table specified by inVocabularyTableName. |
|
layerSize |
No |
The number of dimensions for the word features. Valid values: 0 to 1,000. |
100 |
|
cbow |
No |
Specifies the training model. Set to 0 for the skip-gram model or 1 for the CBOW model. |
0 |
|
window |
No |
The size of the word window. Must be a positive integer. |
5 |
|
minCount |
No |
The minimum word frequency for truncation. Must be a positive integer. |
5 |
|
hs |
No |
Specifies whether to use hierarchical softmax. Set to 1 to enable or 0 to disable. |
1 |
|
negative |
No |
The number of negative samples. Must be a positive integer. A value of 0 disables this feature. |
0 |
|
sample |
No |
The threshold for downsampling. Valid values are from 1e-3 to 1e-5. A value of 0 disables this feature. |
0 |
|
alpha |
No |
The initial learning rate. Must be greater than 0. |
0.025 |
|
iterTrain |
No |
The number of training iterations. Must be 1 or greater. |
1 |
|
randomWindow |
No |
Specifies the word window type. A value of 0 indicates a fixed window size set by the window parameter. A value of 1 indicates a random window size between 1 and 5. |
1 |
|
outVocabularyTableName |
No |
The name of the output table for the vocabulary. |
None |
|
outputTableName |
Yes |
The name of the output table for the word vectors. |
None |
|
lifecycle |
No |
The lifecycle of the output table in days. Must be a positive integer. |
None |
|
coreNum |
No |
The number of cores. This parameter takes effect only when memSizePerCore is also set. Must be a positive integer. |
Automatically allocated |
|
memSizePerCore |
No |
The memory size per core in MB. This parameter takes effect only when coreNum is also set. Must be a positive integer. |
Automatically allocated |
FAQ
If the error message "Vocab size is zero! vocab_size: 0" appears, it indicates that the vocabulary is empty. To resolve this issue, decrease the value of the minCount parameter.