edit-icon download-icon

Configure MongoDB writer

Last Updated: Apr 03, 2018

The MongoDB Writer plug‑in uses MongoClient, the Java client of MongoDB, to write data into MongoDB. In the latest version of Mongo, the granularity of DB lock has been reduced from DB level to document level. Combined with the powerful indexing function of MongoDB, it allows writing data from the data source to MongoDB. In addition, you can configure the business primary key to update data. You must configure the data source before configuring the MongoDB Writer plug‑in. For more information, see MongoDB data source config.

Note:

  • If you are using ApsaraDB for MongoDB, a root account is provided by default.
  • To guarantee security, Data Integration only supports using the relevant account of MongoDB for connection. Avoid using the root account as the access account when you are adding and using the MongoDB data source.

MongoDB Writer acquires the protocol data generated by Reader by means of the Data Integration framework, and converts the data types supported by Data Integration to the ones supported by MongoDB individually. Data Integration does not support the array, but MongoDB does, and the array indexing is powerful.

To use the MongoDB array type, you must convert the string to the array in MongoDB by using special configurations of parameters before writing data into MongoDB.

MongoDB Writer supports most data types in MongoDB. Check whether your data type is supported before using it.

MongoDB Writer converts the MongoDB data types as follows.

Category MongoDB data type
Integer int, Long
Floating point double
String string, array
Date and time date
Boolean bool
Binary bytes

Parameter description

  • datasource

    • Description: Data source name. It must be identical to the data source name added. Adding data source is supported in script mode.

    • Required: Yes

    • Default value: None
  • collectionName

    • Description: The collection name of MonogoDB.

    • Required: Yes

    • Default value: None
  • column

    • Description: An array of multiple column names of a document in MongoDB.

      • name: The column name.
      • type: The column type.
      • splitter: Special delimiter. It is used only when a string to be processed is split into character arrays by delimiters. Strings are split using the delimiter specified by this parameter and stored into MongoDB arrays.
    • Required: Yes

    • Default value: None
  • writeMode

    • Description: It specifies whether to overwrite data during transmission.

      • isReplace: If this parameter is set to true, the data of the same replaceKey is overwritten. If it is set to false, the data is not overwritten.
      • replaceKey: It specifies the business primary key for each record entry. It is used to overwrite data.
    • Required: No

    • Default value: None

Development in wizard mode

Currently, development in wizard mode is not supported.

Development in script mode

Configure the data synchronization job to write data to MongoDB.

  1. {
  2. "type": "job",
  3. "version": "1.0",
  4. "configuration": {
  5. "reader": {},
  6. "writer": {
  7. "plugin": "mongodb",
  8. "parameter": {
  9. "datasource": "datasourceName",
  10. "collectionName": "tag_data",
  11. "column": [
  12. {
  13. "name": "unique_id",
  14. "type": "string"
  15. },
  16. {
  17. "name": "frontcat_id",
  18. "type": "Array",
  19. "splitter": " "
  20. },
  21. {
  22. "name": "property",
  23. "type": "string"
  24. },
  25. {
  26. "name": "scorea",
  27. "type": "int"
  28. },
  29. {
  30. "name": "online",
  31. "type": "bool"
  32. },
  33. {
  34. "name": "percentage",
  35. "type": "double"
  36. }
  37. ],
  38. "writeMode": {
  39. "isReplace": "true",
  40. "replaceKey": "unique_id"
  41. }
  42. }
  43. }
  44. }
  45. }
Thank you! We've received your feedback.