This topic describes how to use the Data Integration service of DataWorks to migrate JSON-formatted fields from MongoDB to MaxCompute.
Prerequisites
- MaxCompute and DataWorks are activated. For more information, see Activate MaxCompute and DataWorks .
- A workflow is created in DataWorks. In this example, a workflow is created in a workspace that is in basic mode. For more information, see Create a workflow.
Prepare test data in MongoDB
- Prepare an account. Create a user in your database to prepare information for creating a connection in DataWorks. In this example, run the following command:
db.createUser({user:"bookuser",pwd:"123456",roles:["user1"]})
In this example, a user named bookuser is created and the user password is 123456. The user is assigned a role that has the data access permission.
- Prepare data. Upload the data to the MongoDB database. In this example, an ApsaraDB for MongoDB instance in a virtual private cloud (VPC) is used. You must apply for a public endpoint for the ApsaraDB for MongoDB instance to communicate with the default resource group of DataWorks. The following test data is uploaded:
{ "store": { "book": [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95 }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99 }, { "category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99 } ], "bicycle": { "color": "red", "price": 19.95 } }, "expensive": 10 }
- Log on to the MongoDB database in the Data Management (DMS) console. In this example, the name of the database is admin, and the name of the collection is userlog. You can run the following command to view the uploaded data:
db.userlog.find().limit(10)
Migrate JSON-formatted data from MongoDB to MaxCompute by using DataWorks
- Login DataWorks console.
- Create a destination table in DataWorks. This table is used to store the data that is migrated from MongoDB.
- Create a MongoDB connection. For more information, see Add a MongoDB data source.
- Create a batch synchronization node.
Verify the migration result
- Right-click the workflow and choose .
- In create a node dialog box, enter node name, and click submit.
- On the configuration tab of the ODPS SQL node, enter the following statement:
SELECT * from mqdata;
- Click
icon to run the code.
- You can operation Log view the results.