The MongoDB Reader plug‑in uses MongoClient, the Java client of MongoDB, to read data from MongoDB. In the latest version of Mongo, the granularity of the DB lock has been reduced from the DB level to the document level. Combined with the powerful indexing function of MongoDB, it allows a high-performance reading of MongoDB.
If you are using ApsaraDB for MongoDB, a root account is provided by default. To make sure the security quality, Data Integration only supports using the relevant account of MongoDB for connection. Avoid using the root account as access account when adding and using the MongoDB data source.
MongoDB Reader reads data in parallel from MongoDB by means of Data Integration framework. Based on the specified rules, it partitions the data in MongoDB into multiple data fragments and reads them in parallel using the controlling Job program based on the specified rules, then converts the data types supported by MongoDB to the ones supported by Data Integration individually.
MongoDB Reader supports most data types in MongoDB. Check whether your data type is supported before using it.
MongoDB Reader converts MongoDB data types as follows:
|Category||MongoDB data type|
|Date and time||date|
|datasource||Data source name. It must be identical to the data source name added. Adding data source is supported in script mode.||Yes||None|
|collectionName||Collection name of MonogoDB.||Yes||None|
|column||An array of multiple column names of a file in MongoDB.
- name: The Column name.
- type: The Column type.
- splitter: MongoDB supports array, but CDP framework does not. Therefore, the data items read from MongoDB in an array format are joined into a string using this delimiter.
|query||Used to define the range of returned MongoDB data. For example, if you set it to
Development in wizard mode is unavailable currently.
Configure the data synchronization job to write data to MongoDB: