Dataphin:Configure the MaxCompute Output Component

Procedure

1. In the top menu bar on the Dataphin homepage, click Develop > Data Integration.

2. On the Integration page, select a Project in the top menu bar. In Dev-Prod mode, also select an environment.

3. In the navigation pane on the left, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline that you want to develop. The pipeline configuration page opens.

4. In the upper-right corner of the page, click Component Library to open the Component Library panel.

5. In the navigation pane on the left of the Component Library panel, click Output. In the output component list on the right, find the MaxCompute component and drag it onto the canvas.

6. Click and drag the icon of your input component to connect it to the MaxCompute output component.

7. Click the icon in the MaxCompute output component card to open the MaxCompute Output Configuration dialog box.

8. In the MaxCompute Output Configuration dialog box, configure the parameters.

   Parameter		Description
   Basic Settings	Step Name	The name of the MaxCompute output component. Dataphin generates a default name. You can change it based on your business needs. Use these naming rules: Use only Chinese characters, letters, underscores (_), and digits. Keep the name under 64 characters.
   	Datasource	This drop-down list shows all MaxCompute data sources. It includes data sources for which you have sync-write permissions and those for which you do not. Click the icon to copy the current data source name. If you lack sync-write permissions for a data source, click Request next to the data source to request sync-write permissions. For details, see Request, renew, or release data source permissions. If you do not have any MaxCompute data sources, click Create Data Source to create one. For details, see Create a MaxCompute data source.
   	Table	Select the target table for output data. If no target table exists in your MaxCompute data source, use the One-click Generate Target Table feature to create one quickly. Follow these steps: Click One-click Create Table. Select Standard Table or Delta Table. Standard Table is selected by default. Switching table formats updates the DDL in the code area below. Dataphin auto-generates the SQL script to create the target table. This includes the table name (defaulted to the source table name) and field types (converted from Dataphin fields). Adjust the script as needed, then click Create. After creation, Dataphin uses the new table as the output target.
   	Production Table Missing Policy	Specifies the policy for when the production table does not exist. You can select Take No Action or Automatic Creation. The default is Automatic Creation. If you select Take no action, the production table is not created when the task is published. If you select Automatic creation, a table with the same name is created in the target environment when the task is published. No Action: If the target table does not exist, the system warns you at submission time but still allows publishing. You must manually create the target table in the production environment before running the task. Automatic Creation: You need to Edit Table Creation Statement. The statement is pre-filled with the table creation statement of the selected table, and you can adjust it. The table name in the statement uses the placeholder ${table_name}, and only this placeholder is supported. It will be replaced with the actual table name during execution. If the target table does not exist, Dataphin first runs the table-creation statement. If creation fails, publishing fails. Fix the statement using the error message, then republish. If the target table already exists, no creation occurs. Note This setting applies only to projects in Dev-Prod mode.
   	Loading Policy	Choose how to write data into the target MaxCompute data source. Options are overwrite and append. Use cases: Overwrite: Replace historical data in the target table with data from the current source table. Append: Add new data to existing data in the target table without changing historical data. Available only for standard tables. Update: Update mapped fields in existing records when primary key conflicts occur. Available only for Delta tables.
   	Partition	If you select a partitioned table, enter the partition. Examples: ds=20230101 or ds=${bizdate}. You can add the /*dynamic*/ keyword before a partition to dynamically specify the write partition for each row of data based on source fields. For example, /*dynamic*/ds=$date, where date is the name of a source field; or /*dynamic*/ds=${bizdate},hh = $hour, where bizdate is a parameter and hour is the field name in the source table. Important Dynamic partitions are capped at 10,000. We recommend no more than 1,000 to avoid long run times or failures.
   	Preparation Statement (optional)	An SQL script that runs on the database before data import. For example, to keep service available, run these steps: First, create target table Target_A. Then, write data to Target_A. After writing completes, rename the live service table Service_B to Temp_C. Next, rename Target_A to Service_B. Finally, delete Temp_C.
   	Finalization Statement (optional)	An SQL script that runs on the database after data import.
   Field Mapping	Input Fields	Shows input fields from upstream components.
   	Output Fields	Lists all fields in the selected table. To skip sending some fields downstream, remove them: To delete a small number of fields, you can click the icon in the Actions column to delete the unnecessary fields. To remove many fields, click Field Management. In the Field Management page, select multiple fields, then click the icon to move Selected Input Fields to Unselected Input Fields.
   	Mapping	Map input fields from the source table to output fields in the target table. Two mapping types are supported: Name-based mapping: Maps fields with identical names. Row-based mapping: Maps fields by position when source and target field names differ.

9. Click Confirm to finish configuring the MaxCompute output component.