MaxCompute:Migrate data from MaxCompute to OSS

Step 1: Create a table in MaxCompute

1. Log on to the DataWorks console and select a region in the upper-left corner.

2. In the left navigation pane, click Workspace.

3. On the Workspaces page, find your workspace and in the Actions column, choose Shortcuts > DataStudio.

4. On the DataStudio page, create a MaxCompute SQL node and run the following statement to create a table named transs:

   CREATE TABLE transs
   (name    STRING,
   id    STRING,
   gender    STRING);

Step 2: Import data into the transs table

1. Download demo_data.csv to your computer.

2. In the left navigation pane, choose Data Integration > Data Upload and Download.

3. Click Go to Data Upload and Download.

4. In the left navigation pane, click the upload icon and then click Upload Data.

5. Upload demo_data.csv to the MaxCompute table transs.

Step 3: Create a bucket and destination file in OSS

1. Log on to the OSS console.

2. In the left navigation pane, click Buckets, then click Create Bucket.

3. Create an empty file named qwee.csv on your computer and upload it to the bucket. The field names in qwee.csv must match those in the transs table (name, id, gender). For upload instructions, see Upload objects.

Step 4: Add a MaxCompute data source in DataWorks

1. Log on to the DataWorks console and select a region.

2. In the left navigation pane, click Workspace, then click the name of your workspace.

3. On the Workspace Details page, click Data Sources in the left navigation pane.

4. On the Data Source tab, click Add Data Source and select MaxCompute.

5. On the Add MaxCompute Data Source page, complete the Basic Information fields. For configuration details, see Configure a MaxCompute data source.

Step 5: Add an OSS data source in DataWorks

For configuration details, see Configure an OSS data source.

Step 6: Create and run a Batch Synchronization node

1. Log on to the DataWorks console and select a region.

2. In the left navigation pane, choose Data Development and O&M > Data Development.

3. In the Select Workspace section, click Go to Data Studio.

4. In the left pane, click the icon and select Create Node > Data Integration > Batch Synchronization.

5. On the node configuration page, set the following parameters:

   Parameter	Value
   Data Source > Source	The MaxCompute data source you added in step 4
   Data Source > Destination	The OSS data source you added in step 5
   Data Source > Table	transs
   Destination > Text Type	CSV
   Destination > Object Name (Path Included)	The OSS object path, for example qwee.csv or data/output/qwee.csv
   Runtime Resource	Your dedicated resource group

   Keep the default values for all other parameters.

   Alternatively, click the icon to switch to the code editor and paste the following script:

   {
       "order":{
           "hops":[
               {
                   "from":"Reader",
                   "to":"Writer"
               }
           ]
       },
       "setting":{
           "errorLimit":{
               "record":"0"
           },
           "speed":{
               "concurrent":1,
               "dmu":1,
               "throttle":false
           }
       },
       "steps":[
           {
               "category":"reader",
               "name":"Reader",
               "parameter":{
                   "column":[
                       "name",
                       "id",
                       "gender"
                   ],
                   "datasource":"odps_first",
                   "partition":[],
                   "table":"Transs"
               },
               "stepType":"odps"
           },
           {
               "category":"writer",
               "name":"Writer",
               "parameter":{
                   "datasource":"Trans",
                   "dateFormat":"yyyy-MM-dd HH:mm:ss",
                   "encoding":"UTF-8",
                   "fieldDelimiter":",",
                   "fileFormat":"csv",
                   "nullFormat":"null",
                   "object":"qwee.csv",
                   "writeMode":"truncate"
               },
               "stepType":"oss"
           }
       ],
       "type":"job",
       "version":"2.0"
   }

   The script uses writeMode: truncate, which overwrites qwee.csv if the file already exists in the bucket.

6. Click Run.