Dataphin:Configure the MySQL Input Component

Prerequisites

• You have created a MySQL data source. For more information, see Create a MySQL data source.

• The account used to configure the MySQL input component must have read-through permission for the data source. If you do not have this permission, request it for the data source. For more information, see Request Data Source Permissions.

Procedure

1. On the Dataphin homepage, in the top menu bar, choose Development > Data Integration.

2. On the integration page, in the top menu bar, choose Project. In Dev-Prod mode, select an environment.

3. In the navigation pane on the left, click Offline Integration. In the Offline Integration List, click the offline pipeline that you want to develop to open its configuration page.

4. In the upper-right corner of the page, click Component Library to open the Component Library panel.

5. In the left navigation pane of the Component Library panel, click Input. In the input component list on the right, locate the MySQL component and drag it to the canvas.

6. Click the icon on the MySQL Input component card to open the MySQL Input Configuration dialog box.

7. In the MySQL Input Configuration dialog box, configure the parameters.

   Parameter	Description
   Step Name	The name of the MySQL input component. Dataphin generates this name automatically, but you can modify it. Naming conventions: Can contain only Chinese characters, letters, underscores (_), and digits. Cannot exceed 64 characters.
   Source Table Quantity	Select the number of source tables. The options are Single Table and Multiple Tables: Single table: Applies to scenarios where business data from one table is synchronized to one target table. Multiple tables: Applies to scenarios where business data from multiple tables is synchronized to the same target table. When data from multiple tables is written to the same data table, the union algorithm is used.
   Datasource	The drop-down list displays all MySQL data sources in Dataphin, regardless of whether you have read-through permissions. If you select Single table for Source table quantity, select only one data source. Click the icon to copy the data source name. If you select Multiple tables for Source table quantity, select multiple data sources with the same time zone. For data sources without read-through permission, you can click Request next to the data source to request read-through permission for the data source. For more information, see Request data source permission. If you do not have a MySQL data source, click Create Data Source in the drop-down list to create one. For detailed instructions, see Creating a MySQL Data Source.
   Time Zone	Determines how time-formatted data is processed. Defaults to the time zone configured in the selected data source and cannot be modified. Note For tasks created before V5.1.2, you can select Data Source Default Configuration or Channel Configuration Time Zone. The default selection is Channel Configuration Time Zone. Data Source Default Configuration: The default time zone of the selected data source. Channel Configuration Time Zone: The time zone configured for the current integration task in Properties > Channel Configuration.
   Database (Optional)	Select the database where the table is located. If left blank, the database specified during data source registration is used. If you select Multiple tables for Source table quantity, select multiple databases. Click the icon to view all selected databases in the Database List dialog box.
   Table Matching Method	Choose General Rule or Database Regex. Note This option is supported only when Multiple tables is selected for Source table quantity.
   Table	Select the source table: If you select Single Table for Source Table Quantity, you can enter a table name keyword to search, or enter the exact table name and click Exact Match Search. After you select a table, the system automatically performs a table status check. Click the icon to copy the name of the currently selected table. If Multiple tables is selected for Source table quantity, add tables by entering different expressions based on the table matching method. If General Rule is selected for Table matching method: In the input box, enter a table expression to filter tables with the same structure. The system supports enumeration, regex-like, and mixed forms. For example, table_[001-100];table_102;table_abc*. If Database Regex is selected for Table matching method: In the input box, enter the regular expression supported by the current database. The system matches tables in the target database based on this regular expression. During task runtime, the system immediately matches new table ranges for synchronization based on the database regular expression. After entering the expression, click Precise Search to view the list of matched tables in the Confirm Match Details dialog box.
   Shard Key (Optional)	The system partitions data based on the shard key field. Combine this with concurrency settings to enable concurrent reads. You can use any column in the source table as the shard key. We recommend using a primary key or indexed column to ensure transfer performance. Important When you select a date and time type, the system identifies the maximum and minimum values and performs brute-force splitting based on the total time range and concurrency. This does not guarantee even distribution.
   Input Filter (Optional)	Filter conditions for extracted data. Configuration options: Configure static field: Extract corresponding data. For example, ds=20211111. Configure variable parameter: Extract a portion of data. For example, ds=${bizdate}.
   Output Fields	Displays all fields from the selected table that match the filter conditions. Supported operations: Field Management: If you do not need to output certain fields to downstream components, delete the corresponding fields: Single field deletion scenario: To delete a few fields, click the icon in the operation column to delete the unnecessary fields. Batch field deletion scenario: To delete many fields, click Field Management. In the Field Management dialog box, select multiple fields, then click the left shift icon to move the selected input fields to the unselected input fields, and click OK to complete the batch deletion of fields. Batch Add: Click Batch Add. Batch configuration is supported in JSON, TEXT, and DDL formats. Note After batch adding is complete, clicking OK will overwrite the configured field information. Batch configure in JSON format. For example: // Example: [{ "index": 1, "name": "id", "type": "int(10)", "mapType": "Long", "comment": "comment1" }, { "index": 2, "name": "user_name", "type": "varchar(255)", "mapType": "String", "comment": "comment2" }] Note Index indicates the column number of the specified object, name indicates the field name after import, and type indicates the field type after import. For example, "index":3,"name":"user_id","type":"String" indicates that the fourth column in the file is imported, the field name is user_id, and the field type is String. Batch configure in TEXT format. For example: // Example: 1,id,int(10),Long,comment1 2,user_name,varchar(255),Long,comment2 The row delimiter separates information for each field. The default is a line feed (\\n). Line feed (\\n), semicolon (;), and period (.) are supported. The column delimiter is used to separate field names from field types. The default value is a half-width comma (,), which can be ','. Field types can be omitted, and the default value for an omitted field type is ','. Batch configure in DDL format. For example: CREATE TABLE tablename ( user_id serial, username VARCHAR(50), password VARCHAR(50), email VARCHAR (255), created_on TIMESTAMP, ); Create Output Field: Click + Create Output Field, and fill in Column, Type, Description, and select Mapping Type as instructed on the page. After configuring the current row, click the icon to save.

8. Click Confirm to complete the property configuration of the MySQL input component.