Maxgraph data source - DataWorks - Alibaba Cloud Documentation Center

DataWorks Data Integration provides Maxgraph Writer for you to write data from MaxCompute tables to Maxgraph. This topic describes the capabilities of synchronizing data to Maxgraph data sources.

Limits

You can use only the code editor to configure a batch synchronization node to write data to Maxgraph data sources.
The use scenario is synchronizing data from MaxCompute to Maxgraph.

Prepare a Maxgraph account that has the required permissions before data synchronization

If you want to use Maxgraph Writer to write data from MaxCompute tables to Maxgraph, grant the Maxgraph build account the read permissions on the MaxCompute tables in your MaxCompute project. Contact the Maxgraph administrator to obtain the Maxgraph build account.

Develop a data synchronization node

For more information about the configuration procedure, see Configure a batch synchronization node by using the code editor.
For information about all parameters that are configured and the code that is run when you use the code editor to configure a batch synchronization node, see Appendix: Code and parameters.

Appendix: Code and parameters

Appendix: Configure a batch synchronization node by using the code editor

If you use the code editor to configure a batch synchronization node, you must configure parameters for the reader and writer of the related data source based on the format requirements in the code editor. For more information about the format requirements, see Configure a batch synchronization node by using the code editor. The following information describes the configuration details of parameters for the reader and writer in the code editor.

Code for Maxgraph Writer

In the following code, synchronization nodes are configured to write data about vertices and edges to Maxgraph.

Configure a synchronization node to write data about vertices to Maxgraph

{
  "job": {
    "setting": {
      "speed": {
        "channel": 1 // Set channel to 1. 
      },
      "errorLimit": {
        "record": 1000
      }
    },
    "content": [
      {
        "reader": {
          "name": "odpsreader",
          "parameter": {
            "accessId": "*****",
            "accessKey": "*****",
            "project": "maxgraph_dev",
            "table": "maxgraph_demo_person",
            "column": [ // The names of columns in the MaxCompute table. The value of this parameter has a one-to-one mapping with the value of the column parameter of Maxgraph Writer. 
              "id",
              "name",
              "age"
            ],
            "packageAuthorizedProject": "biggraph_dev",
            "splitMode": "record",
            "odpsServer": "******"
          }
        },
        "writer": {
          "name": "maxgraphwriter",
          "parameter": {
            "endpoint": "http://graph.alibaba.net",
            "graphName": "xxx",
            "accessId": "xxx",
            "accessKey": "xxx",
            "label": "person",
            "labelType": "vertex",
            "onlineMode": "partition",
            "splitSize": "256",
            "column": [ // The names of vertex properties in Maxgraph. The value of this parameter has a one-to-one mapping with the value of the column parameter of MaxCompute Reader. 
              "id",
              "name",
              "age"
            ]
          }
        }
      }
    ]
  }
}

Configure a synchronization node to write data about edges to Maxgraph

{
  "job": {
    "setting": {
      "speed": {
        "channel": 1 // Set channel to 1. 
      },
      "errorLimit": {
        "record": 1000
      }
    },
    "content": [
      {
        "reader": {
          "name": "odpsreader",
          "parameter": {
            "accessId": "*****",
            "accessKey": "*****",
            "project": "maxgraph_dev",
            "table": "maxgraph_demo_knows",
            "column": [
              "person_id",
              "person_id2",
              "weight",
              "id"
            ],
            "packageAuthorizedProject": "biggraph_dev",
            "splitMode": "record",
            "odpsServer": "****"
          }
        },
        "writer": {
          "name": "maxgraphwriter",
          "parameter": {
            "endpoint": "http://graph.alibaba.net",
            "graphName": "xxx",
            "accessId": "xxx",
            "accessKey": "xxx",
            "label": "knows",
            "labelType": "edge",
            "srcLabel": "person",
            "dstLabel": "person",
            "onlineMode": "partition",
            "splitSize": "256",
            "column": [ 
              {
                "name": "id", // The name of the property in Maxgraph. 
                "propertyType": "srcPrimaryKey" // The type of the property. Valid values: srcPrimaryKey, dstPrimaryKey, and edgeProperty. 
              },
              {
                "name": "id",
                "propertyType": "dstPrimaryKey"
              },
              {
                "name": "weight",
                "propertyType": "edgeProperty"
              },
              {
                "name": "id",
                "propertyType": "edgeProperty"
              }
            ]
          }
        }
      }
    ]
  }
}

Parameters in code for Maxgraph Writer


Parameter	Description	Required	Default value
endpoint	The endpoint of Maxgraph.	Yes	No default value
graphName	The name of the Maxgraph instance.	Yes	No default value
accessId	The username.	Yes	No default value
accessKey	The password.	Yes	No default value
label	The label, which is the name of the vertex or edge.	Yes	No default value
labelType	The type of the label. Valid values: vertex and edge.	Yes	No default value
srcLabel	The label of the start vertex in an edge. This parameter takes effect only when you import data about edges.	Yes	No default value
dstLabel	The label of the end vertex in an edge. This parameter takes effect only when you import data about edges.	Yes	No default value
splitSize	The size of a shard in the MapReduce job. Unit: MB.	No	256MB
onlineMode	The mode in which data is uploaded to the storage of Maxgraph. Valid values: partition: When data is being uploaded, both existing data records and newly uploaded data records may be queried. However, the data consistency is ensured. The data upload speed in this mode is faster than that in type mode. type: When data is being uploaded, only existing data records can be queried. New data records can be queried only after data is uploaded. The data upload speed in this mode is slower than that in partition mode.	No	type
column	The name of the vertex property. This parameter takes effect only when you import data about vertices.	Yes	No default value
name	The name of the property.	Required only when you import data about edges	No default value
propertyType	The type of the property. Valid values: srcPrimaryKey, dstPrimaryKey, and edgeProperty.	Required only when you import data about edges	No default value
srcPrimaryKey	The primary key of the start vertex. This parameter takes effect only when you import data about edges.	Required only when you import data about edges	No default value
dstPrimaryKey	The primary key of the end vertex. This parameter takes effect only when you import data about edges.	Required only when you import data about edges	No default value
edgeProperty	The properties of the edge. This parameter can be left empty if the edge has no properties.	No	No default value