Data Conversion Module - Platform For AI - Alibaba Cloud Documentation Center

The Data Conversion Module transforms raw feature values into a format that downstream models can consume — normalized, discretized, indexed, or Weight of Evidence (WOE) encoded. Use it when your input features have mismatched scales, need bin-based encoding, or require WOE transformation for credit or risk modeling.

For example, if one feature ranges from 0 to 1 and another from 10,000 to 100,000, combining them directly can distort model training. Normalization resolves this by mapping all values to the [0, 1] range based on binning information.

Warning

By default, all columns in the input table are selected as feature columns. Review which columns to exclude — passing a label column or ID column through conversion will corrupt the output. Use the Columns to exclude from conversion parameter to specify any columns that should pass through unchanged.

Supported conversion types

Type	What it does	Output format
Normalization	Maps feature values to [0, 1] based on input binning information. Missing values are filled with 0.	Standard table
Discretization	Encodes features as dummy variables in key-value (KV) format, one entry per bin.	KV table
WOE conversion	Replaces each feature value with its Weight of Evidence (WOE) score from the binning result table.	Standard table
Index	Converts features to index values.	—

Configure the component

Method 1: Use the GUI

Open your workflow in Designer and click the Data Conversion Module component.

On the Fields Setting tab, configure the following parameters:

Parameter	Description
Feature columns in input table	The feature columns to convert. All columns are selected by default.
Columns to exclude from conversion	Columns passed through to the output unchanged. Specify your label column here to prevent it from being converted.
Data conversion type	Select Normalization, Discretization, WOE conversion, or Index.
Default WOE value	Applies only when Data conversion type is WOE conversion. If a sample value falls into a bin with no WOE score, this value is used as a fallback. If left blank, the algorithm returns an error for that sample.

On the Execution Tuning tab, configure resource allocation:

Parameter	Description
Number of cores	The number of CPU cores to use. The system allocates cores automatically by default.
Memory per core	The amount of memory per CPU core. The system allocates memory automatically by default.

Method 2: Use PAI commands

Run the following command in the SQL Script component. For more information about SQL Script, see SQL Script.

PAI -name data_transform
    -project algo_public
    -DinputFeatureTableName=feature_table
    -DinputBinTableName=bin_table
    -DoutputTableName=output_table
    -DmetaColNames=label
    -DfeatureColNames=feaname1,feaname2

Parameters

Parameter	Description	Required	Default
`inputFeatureTableName`	The input feature table.	Yes	—
`inputBinTableName`	The input binning result table.	Yes	—
`outputTableName`	The output table.	Yes	—
`inputFeatureTablePartitions`	The partitions to read from the input feature table.	No	Complete table
`featureColNames`	The feature columns to convert.	No	All columns
`metaColNames`	Columns excluded from conversion and passed through unchanged. Accepts multiple column names such as `label` and `sample_id`.	No	None
`transformType`	The conversion type. Valid values: `normalize`, `dummy` (discretization), `woe`.	No	`dummy`
`itemDelimiter`	The separator between features in the output. Applies to discretization only.	No	Comma (`,`)
`kvDelimiter`	The separator between keys and values in the output. Applies to discretization only.	No	Colon (`:`)
`lifecycle`	The lifecycle of the output table.	No	None
`coreNum`	The number of CPU cores to use.	No	System-calculated
`memSizePerCore`	The amount of memory per CPU core, in MB.	No	System-calculated

How normalization works

Normalization maps each feature value to the [0, 1] range based on the input binning information. Missing values are filled with 0.

if feature_raw_value == null or feature_raw_value == 0 then
    feature_norm_value = 0.0
else
    bin_index = FindBin(bin_table, feature_raw_value)
    bin_width = round(1.0 / bin_count * 1000) / 1000.0
    feature_norm_value = 1.0 - (bin_count - bin_index - 1) * bin_width

Output format

The output format depends on the conversion type:

Normalization and WOE conversion produce a standard table where each converted feature occupies one column.
Discretization produces a table in key-value (KV) format. Each feature value is mapped to a dummy variable named [{feaname}]_bin_{bin_id}. For example, for a feature named sns:
Condition Generated variable
sns falls into the second bin [sns]_bin_2
sns is null [sns]_bin_null
sns is not null but does not match any defined bin [sns]_bin_else

Condition	Generated variable
`sns` falls into the second bin	`[sns]_bin_2`
`sns` is null	`[sns]_bin_null`
`sns` is not null but does not match any defined bin	`[sns]_bin_else`