Sync MaxCompute Data to MaxGraph Using DataWorks - DataWorks

Limits

Maxgraph Writer supports only script mode.
Maxgraph Writer is primarily used to synchronize data from MaxCompute to Maxgraph.

Prerequisites

Before you begin, ensure that you have:

The name of the Maxgraph build account (contact your Maxgraph administrator to get it)
Read permission granted to the build account on the source MaxCompute table

Import data from MaxCompute to Maxgraph

Maxgraph Writer uses script mode only. For the procedure to configure a synchronization task, see Configure a task in script mode.

The import configuration differs for vertices and edges. Choose the section that matches what you are importing.

Import vertices

The following example imports person vertices from a MaxCompute table. The column array in the reader and the column array in the writer must be in the same order — each position maps one-to-one.

{
  "job": {
    "setting": {
      "speed": {
        "channel": 1
      },
      "errorLimit": {
        "record": 1000
      }
    },
    "content": [
      {
        "reader": {
          "name": "odpsreader",
          "parameter": {
            "accessId": "*****",
            "accessKey": "*****",
            "project": "maxgraph_dev",
            "table": "maxgraph_demo_person",
            "column": [
              "id",
              "name",
              "age"
            ],
            "packageAuthorizedProject": "biggraph_dev",
            "splitMode": "record",
            "odpsServer": "******"
          }
        },
        "writer": {
          "name": "maxgraphwriter",
          "parameter": {
            "endpoint": "http://graph.alibaba.net",
            "graphName": "xxx",
            "accessId": "xxx",
            "accessKey": "xxx",
            "label": "person",
            "labelType": "vertex",
            "onlineMode": "partition",
            "splitSize": "256",
            "column": [
              "id",
              "name",
              "age"
            ]
          }
        }
      }
    ]
  }
}

Import edges

The following example imports knows edges between person vertices. Each entry in the writer column array is an object with a name (the property name in the graph) and a propertyType that identifies whether it maps to the source vertex primary key, the destination vertex primary key, or an edge property.

{
  "job": {
    "setting": {
      "speed": {
        "channel": 1
      },
      "errorLimit": {
        "record": 1000
      }
    },
    "content": [
      {
        "reader": {
          "name": "odpsreader",
          "parameter": {
            "accessId": "*****",
            "accessKey": "*****",
            "project": "maxgraph_dev",
            "table": "maxgraph_demo_knows",
            "column": [
              "person_id",
              "person_id2",
              "weight",
              "id"
            ],
            "packageAuthorizedProject": "biggraph_dev",
            "splitMode": "record",
            "odpsServer": "****"
          }
        },
        "writer": {
          "name": "maxgraphwriter",
          "parameter": {
            "endpoint": "http://graph.alibaba.net",
            "graphName": "xxx",
            "accessId": "xxx",
            "accessKey": "xxx",
            "label": "knows",
            "labelType": "edge",
            "srcLabel": "person",
            "dstLabel": "person",
            "onlineMode": "partition",
            "splitSize": "256",
            "column": [
              {
                "name": "id",
                "propertyType": "srcPrimaryKey"
              },
              {
                "name": "id",
                "propertyType": "dstPrimaryKey"
              },
              {
                "name": "weight",
                "propertyType": "edgeProperty"
              },
              {
                "name": "id",
                "propertyType": "edgeProperty"
              }
            ]
          }
        }
      }
    ]
  }
}

Writer parameters

Common parameters

Parameter	Description	Required	Default
`endpoint`	The URL of Maxgraph.	Yes	None
`graphName`	The name of the graph instance.	Yes	None
`accessId`	The username.	Yes	None
`accessKey`	The password of the user.	Yes	None
`label`	The label name of the vertex or edge to import.	Yes	None
`labelType`	The type of graph entity to import. Use `vertex` when importing vertices; use `edge` when importing edges.	Yes	None
`splitSize`	The shard size (in MB) for the MapReduce job during data creation.	No	256
`onlineMode`	The publishing mode. See Choose a publishing mode.	No	`type`

Vertex parameters

Use these parameters when labelType is vertex.

Parameter	Description	Required	Default
`column`	An ordered list of vertex property names. The order must match the reader `column` list.	Yes	None

Edge parameters

Use these parameters when labelType is edge.

Parameter	Description	Required	Default
`srcLabel`	The label of the source vertex.	Yes	None
`dstLabel`	The label of the destination vertex.	Yes	None
`column`	An ordered list of property objects. Each object specifies a `name` (the property name in the graph) and a `propertyType`.	Yes	None
`name`	The property name in the graph, defined inside each `column` entry.	Yes	None
`propertyType`	The role of the property in the graph. Valid values: `srcPrimaryKey` (primary key of the source vertex), `dstPrimaryKey` (primary key of the destination vertex), `edgeProperty` (an edge property; optional when the edge has no properties).	Yes	None

Choose a publishing mode

The onlineMode parameter controls how Maxgraph publishes imported data. Choose based on whether your queries can tolerate a window of mixed old and new data during publishing.

Mode	Behavior during publishing	Speed	Use when
`partition`	Queries may return a mix of old and new data. Eventual consistency is guaranteed.	Faster	Data freshness is less critical and you want shorter import jobs.
`type`	Queries return only old data until publishing completes.	Slower	Queries must never see partially published data.

What's next

Configure a task in script mode