DataWorks:Graph Database

Tek bir tablonun verilerini senkronize etmek için toplu iş senkronizasyon görevi yapılandırma

• Yapılandırma yordamı hakkında daha fazla bilgi için Configure a batch synchronization task by using the codeless UI ve Configure a batch synchronization task by using the code editor konularına bakın.

• Toplu iş senkronizasyon görevini yapılandırmak için kod düzenleyiciyi kullandığınızda yapılandırılan tüm parametreler ve çalıştırılan kod hakkında bilgi edinmek için Ek: Kod ve parametreler bölümüne bakın.

Kod düzenleyici kullanarak toplu iş senkronizasyon görevi yapılandırma

Toplu iş senkronizasyon görevini kod düzenleyici kullanarak yapılandırmak istiyorsanız, komut dosyasındaki ilgili parametreleri birleşik komut dosyası biçimi gereksinimlerine göre yapılandırmanız gerekir. Daha fazla bilgi için Script mode configuration konusuna bakın. Aşağıdaki bilgiler, kod düzenleyiciyi kullanarak toplu iş senkronizasyon görevi yapılandırırken veri kaynakları için yapılandırmanız gereken parametreleri açıklar.

Reader komut dosyası demosu

Graph Database (GDB) veri yazmak için bir veri senkronizasyon işi yapılandırırken köşeleri ve kenarları ayrı ayrı yapılandırmanız gerekir:

• GDB örneğinden köşe verilerini okumak için senkronizasyon görevi yapılandırma

  {
      "order":{
          "hops":[
              {
                  "from":"Reader",
                  "to":"Writer"
              }
          ]
      },
      "setting":{
          "errorLimit":{
              "record":"100"  // Maximum number of dirty data records to tolerate.
          },
          "jvmOption":"",
          "speed":{
              "concurrent":3,
              "throttle":true, // If true, throttling is enabled. If false, throttling is disabled and the mbps parameter is ignored.
              "mbps":"12"// The throttling limit. 1 mbps is equal to 1 MB/s.
          }
      },
      "steps":[
          {
              "category":"reader",
              "name":"Reader",
              "parameter":{
                  "host": "gdb-xxxxxx.aliyuncs.com", // The endpoint of the GDB instance.
                  "port": 8182, // The port of the GDB instance.
                  "username": "gdb", // The username for accessing the GDB instance.
                  "password": "gdb", // The password for the username.
                  "labelType": "VERTEX", // The type of the label. VERTEX specifies a vertex.
                  "labels": ["label1", "label2"],  // A list of labels. If left empty, all vertices are exported.
                  "column": [
                      {
                          "name": "id",               // The field name.
                          "type": "string",           // The field type.
                          "columnType": "primaryKey"  // Field category. Specifies the vertex primary key, which must be of the STRING type in GDB.
                      },
                      {
                          "name": "label",              // The field name.
                          "type": "string",             // The field type.
                          "columnType": "primaryLabel"  // Field category. Specifies the vertex label, which must be of the STRING type in GDB.
                      },
                      {
                          "name": "age",                   // The property name.
                          "type": "int",                   // The property type.
                          "columnType": "vertexProperty"   // Field category. Specifies a basic property of the vertex in GDB.
                      }
                  ]
              },
              "stepType":"gdb"
          },
          {
              "category":"writer",
              "name":"Writer",
              "parameter":{
                  "print": true
              },
              "stepType":"stream"
          }
      ]
  }

• GDB örneğinden kenar verilerini okumak için senkronizasyon görevi yapılandırma

  {
      "order":{
          "hops":[
              {
                  "from":"Reader",
                  "to":"Writer"
              }
          ]
      },
      "setting":{
          "errorLimit":{
              "record":"100"  // Maximum number of dirty data records to tolerate.
          },
          "jvmOption":"",
          "speed":{
              "concurrent":3,
              "throttle":true,// If true, throttling is enabled. If false, throttling is disabled and the mbps parameter is ignored.
              "mbps":"12"// The throttling limit. 1 mbps is equal to 1 MB/s.
          }
      },
      "steps":[
          {
              "category":"reader",
              "name":"Reader",
              "parameter":{
                  "host": "gdb-xxxxxx.aliyuncs.com", // The endpoint of the GDB instance.
                  "port": 8182, // The port of the GDB instance.
                  "username": "gdb", // The username for accessing the GDB instance.
                  "password": "gdb", // The password for the username.
                  "labelType": "EDGE", // The type of the label. EDGE specifies an edge.
                  "labels": ["label1", "label2"],  // A list of labels. If left empty, all edges are exported.
                  "column": [
                      {
                          "name": "id",               // The field name.
                          "type": "string",           // The field type.
                          "columnType": "primaryKey"  // Field category. Specifies the edge primary key, which must be of the STRING type in GDB.
                      },
                      {
                          "name": "label",              // The field name.
                          "type": "string",             // The field type.
                          "columnType": "primaryLabel"  // Field category. Specifies the edge label, which must be of the STRING type in GDB.
                      },
                      {
                          "name": "srcId",               // The field name.
                          "type": "string",              // The field type.
                          "columnType": "srcPrimaryKey"  // Field category. Specifies the start vertex primary key, which must be of the STRING type in GDB.
                      },
                      {
                          "name": "srcLabel",               // The field name.
                          "type": "string",                 // The field type.
                          "columnType": "srcPrimaryLabel"   // Field category. Specifies the start vertex label, which must be of the STRING type in GDB.
                      },
                      {
                          "name": "dstId",                    // The field name.
                          "type": "string",                   // The field type.
                          "columnType": "dstPrimaryKey"       // Field category. Specifies the end vertex primary key, which must be of the STRING type in GDB.
                      },
                      {
                          "name": "dstLabel",                 // The field name.
                          "type": "string",                   // The field type.
                          "columnType": "dstPrimaryLabel"     // Field category. Specifies the end vertex label, which must be of the STRING type in GDB.
                      },
                      {
                          "name": "weight",               // The property name.
                          "type": "double",               // The property type.
                          "columnType": "edgeProperty"    // Field category. Specifies an edge property.
                      }
                  ]
              },
              "stepType":"gdb"
          },
          {
              "category":"writer",
              "name":"Writer",
              "parameter":{
                  "print": true
              },
              "stepType":"stream"
          }
      ]
  }

Reader komut dosyası parametreleri

Writer komut dosyası demosu

• GDB veritabanına köşe verilerini yazmak için senkronizasyon görevi yapılandırma

  {
      "order":{
          "hops":[
              {
                  "from":"Reader",
                  "to":"Writer"
              }
          ]
      },
      "setting":{
          "errorLimit":{
              "record":"100"  // Maximum number of dirty data records to tolerate.
          },
          "speed":{
               "throttle":true,// If true, throttling is enabled. If false, throttling is disabled and the mbps parameter is ignored.
              "concurrent":3, // The job concurrency.
              "mbps":"12"// The throttling limit. 1 mbps is equal to 1 MB/s.
          }
      },
      "steps":[
          {
              "category":"reader",
              "name":"Reader",
              "parameter":{
                  "column":[
                      "*"
                  ],
                  "datasource":"_ODPS",
                  "emptyAsNull":true,
                  "guid":"",
                  "isCompress":false,
                  "partition":[],
                  "table":""
              },
              "stepType":"odps"
          },
          {
              "category":"writer",
              "name":"Writer",
              "parameter": {
                  "datasource": "testGDB", // The name of the data source.
                  "label": "person", // The label, which is the name of the vertex.
                  "srcLabel": "", // Not applicable to vertices.
                  "dstLabel": "", // Not applicable to vertices.
                  "labelType": "VERTEX", // The type of the label. "VERTEX" specifies a vertex.
                  "writeMode": "INSERT", // The policy for handling duplicate primary keys.
                  "idTransRule": "labelPrefix", // The conversion rule for the vertex primary key.
                  "srcIdTransRule": "none", // Not applicable to vertices.
                  "dstIdTransRule": "none", // Not applicable to vertices.
                  "column": [
                      {
                          "name": "id", // The field name.
                          "value": "#{0}", // Takes the value from the source column at index 0. Concatenation is supported.
                          "type": "string", // The field type.
                          "columnType": "primaryKey" // The field category. `primaryKey` specifies the primary key.
                      }, // The primary key of the vertex. The field name must be `id`, the type must be STRING, and this record is required.
                      {
                          "name": "person_age",
                          "value": "#{1}", // Takes the value from the source column at index 1. Concatenation is supported.
                          "type": "int",
                          "columnType": "vertexProperty" // The field category. `vertexProperty` specifies a vertex property.
                      }, // A property of the vertex. Supported types: INT, LONG, FLOAT, DOUBLE, BOOLEAN, and STRING.
                      {
                          "name": "person_credit",
                          "value": "#{2}", // Takes the value from the source column at index 2. Concatenation is supported.
                          "type": "string",
                          "columnType": "vertexProperty"
                      } // A property of the vertex.
                  ]
              }
              "stepType":"gdb"
          }
      ],
      "type":"job",
      "version":"2.0"
  }

• GDB veritabanına kenar verilerini yazmak için senkronizasyon görevi yapılandırma

  {
      "order":{
          "hops":[
              {
                  "from":"Reader",
                  "to":"Writer"
              }
          ]
      },
      "setting":{
          "errorLimit":{
              "record":"100" // Maximum number of dirty data records to tolerate.
          },
          "jvmOption":"",
          "speed":{
              "throttle":true,// If true, throttling is enabled. If false, throttling is disabled and the mbps parameter is ignored.
              "concurrent":3, // The job concurrency.
              "mbps":"12"// The throttling limit. 1 mbps is equal to 1 MB/s.
          }
      },
      "steps":[
          {
              "category":"reader",
              "name":"Reader",
              "parameter":{
                  "column":[
                      "*"
                  ],
                  "datasource":"_ODPS",
                  "emptyAsNull":true,
                  "guid":"",
                  "isCompress":false,
                  "partition":[],
                  "table":""
              },
              "stepType":"odps"
          },
          {
              "category":"writer",
              "name":"Writer",
              "parameter": {
                  "datasource": "testGDB", // The name of the data source.
                  "label": "use", // The label, which is the name of the edge.
                  "labelType": "EDGE", // The type of the label. `EDGE` specifies an edge.
                  "srcLabel": "person", // The label of the start vertex.
                  "dstLabel": "software", // The label of the end vertex.
                  "writeMode": "INSERT", // The policy for handling duplicate primary keys.
                  "idTransRule": "labelPrefix", // The conversion rule for the edge primary key.
                  "srcIdTransRule": "labelPrefix", // The conversion rule for the primary key of the start vertex.
                  "dstIdTransRule": "labelPrefix", // The conversion rule for the primary key of the end vertex.
                  "column": [
                      {
                          "name": "id", // The field name.
                          "value": "#{0}", // Takes the value from the source column at index 0. Concatenation is supported.
                          "type": "string", // The field type.
                          "columnType": "primaryKey" // The field category. `primaryKey` specifies the primary key.
                      }, // The primary key of the edge. The field name must be `id` and the type must be STRING. This record is optional.
                      {
                          "name": "id",
                          "value": "#{1}", // Concatenation is supported. Ensure the mapping rule is consistent with the one used when importing vertices.
                          "type": "string",
                          "columnType": "srcPrimaryKey" // The field category. `srcPrimaryKey` specifies the primary key of the start vertex.
                      }, // The primary key of the start vertex. The field name must be `id`, the type must be STRING, and this record is required.
                      {
                          "name": "id",
                          "value": "#{2}", // Concatenation is supported. Ensure the mapping rule is consistent with the one used when importing vertices.
                          "type": "string",
                          "columnType": "dstPrimaryKey" // The field category. `dstPrimaryKey` specifies the primary key of the end vertex.
                      }, // The primary key of the end vertex. The field name must be `id`, the type must be STRING, and this record is required.
                      {
                          "name": "person_use_software_time",
                          "value": "#{3}", // Concatenation is supported.
                          "type": "long",
                          "columnType": "edgeProperty" // The field category. `edgeProperty` specifies an edge property.
                      }, // A property of the edge. Supported types: INT, LONG, FLOAT, DOUBLE, BOOLEAN, and STRING.
                      {
                          "name": "person_regist_software_name",
                          "value": "#{4}", // Concatenation is supported.
                          "type": "string",
                          "columnType": "edgeProperty"
                      }, // A property of the edge.
                      {
                          "name": "id",
                          "value": "#{5}", // Concatenation is supported.
                          "type": "long",
                          "columnType": "edgeProperty"
                      } // A property of the edge with the field name `id`. This is a regular property, not a primary key, and is optional.
                  ]
              }
              "stepType":"gdb"
          }
      ],
      "type":"job",
      "version":"2.0"
  }

Writer komut dosyası parametreleri

{"properties":[
    {"k":"name","t":"string","v":"tom"},
    {"k":"age","t":"int","v":"20"},
    {"k":"sex","t":"string","v":"male"}
]}