本文将为您介绍DataOS API的功能、输入、输出等详情,以及如何进行配置使用。

CheckMetaTable

  • 功能:判断table是否存在。
  • 输入:tableGuid(必选)。
  • 格式:odps.<project>.<table>
  • 输出:true或false。
  • 示例如下:
    • 输入:
      request.setTableGuid("odps.autotest.daily_test");
    • 输出:
      {"requestId":"0b85c9d915548770462378104e","errMsg":"success","errCode":0,"data":true}

GetMetaDB

  • 功能:获取MaxCompute项目的信息。
  • 输入:项目GUID(必选)。
  • 格式:odps.<project>
  • 输出:项目详情。
    参数 描述
    appGuid 项目唯一标识
    project 项目英文名称
    projectNameCn 项目名称
    comment 备注
    ownerId Owner的ID
    createTime 创建时间
    modifyTime 修改时间
  • 示例如下:
    • 输入:
      request.setDbGuid("odps.autotest");
    • 输出:
      {
          "requestId": "0bfaefec61500****",
          "errMsg": "success",
          "errCode": 0,
          "data": {
              "appGuid": "odps.meta",
              "projectName": "meta",
              "projectNameCn": "ODPS元仓",
              "comment": "",
              "ownerId": "13101879118",
              "createTime": "2014-02-18",
              "modifyTime": "2018-04-16"
        }
      }

GetMetaTable

  • 功能:获取MaxCompute表的信息。
  • 输入:tableGuid(必选)。
  • 格式:odps.<project>.<table>
  • 输出:表的详情。
    参数 描述
    appGuid 项目唯一标识
    tableGuid 表唯一标识
    tableName 表名称
    id 数据库ID
    ownerId owner的ID
    hasPart 是否为分区表
    dataSize 表数据的大小
    createTime 表的创建时间
    lastDdlTime 表DDL最后的更新时间
    lastModifyTime 表最后的修改时间
  • 示例如下:
    • 输入:
      request.setTableGuid(tableGuid);
    • 输出:
      {
          "requestId": "0b8906da817****",
          "errMsg": "success",
          "errCode": 0,
          "data": {
              "appGuid": "odps.meta",
              "tableGuid": "odps.meta.m_table",
              "tableName": "m_table",
              "id": 64809,
              "OwnerId": "dp-base-odps@aliyun-test.com",
              "hasPart": 1,
              "dataSize": 49397610904693,
              "createTime": "2014-12-10 21:20:23",
              "lastDdlTime": "2017-04-18 10:10:06",
              "lastModifyTime": "2019-04-09 20:24:08"
        }
      }

ListMetaTableColumn

  • 功能:获取MaxCompute的列信息。
  • 输入:tableGuid(必选)。
  • 格式:odps.<project>.<table>
  • 输出:列详情。
    参数 描述
    appGuid 项目唯一标识
    tableGuid 表唯一标识
    tableName 表名称
    columnGuid 列唯一标识,格式为odps.<project>.<table>.<col>
    columnName 列名
    columnType 列类型
    seqNumber 列编号(从1开始)
    isPartitionCol 是否为分区列
    comment 备注
    safeLevel 安全等级
  • 示例如下:
    • 输入:
      request.setTableGuid(tableGuid);
    • 输出:
      {
          "requestId": "0b8906d9796*****",
          "errCode": 0,
          "errMsg": "success",
          "columnList": [{
              "appGuid": "odps.meta",
              "tableGuid": "odps.meta.m_table",
              "tableName": "m_table",
              "columnGuid": "odps.meta.m_table.project_name",
              "columnName": "project_name",
              "columnType": "string",
              "seqNumber": 1,
              "isPartitionCol": 0,
              "comment": "Project名称",
              "safeLevel": "C2"
        }, 
      {
          "appGuid": "odps.meta",
          "tableGuid": "odps.meta.m_table",
          "tableName": "m_table",
          "columnGuid": "odps.meta.m_table.name",
          "columnName": "name",
          "columnType": "string",
          "seqNumber": 2,
          "isPartitionCol": 0,
          "isPrimaryKey": 0,
          "isNullable": 0,
          "comment": "表名",
          "safeLevel": "C2"
        } ... ]
      }

ListMetaTablePartition

  • 功能:获取MaxCompute的分区信息。
  • 输入:
    参数 说明
    tableGuid 格式为odps.<project>.<table>
    pageNum 页码
    pageSize 每页最多显示记录数
  • 输出:表分区的详情。
    参数 描述
    appGuid 项目唯一标识
    tableGuid 表唯一标识
    tableName 表名称
    partitionGuid 分区唯一标识,格式为odps.<project>.<table>.<partition>
    partitionName 分区名称
    createTime 分区的创建时间
    modifyTime 分区的修改时间
    dataSize 分区的数据大小
    records 分区的记录数
    pageNum 当前分页页码
    pageSize 当前分页大小
    totalNum 总记录数
  • 返回示例如下:
    {
        "requestId": "0baf3e0502****",
        "errCode": 0,
        "errMsg": "success",
        "pageNum": 1,
        "pageSize": 10,
        "totalNum": 1101,
        "partitionList": [{
            "appGuid": "odps.meta",
            "tableGuid": "odps.meta.m_table",
            "tableName": "m_table",
            "id": 168504514,
            "partitionGuid": "odps.meta.m_table.ds\u003d20190408",
            "partitionName": "ds\u003d20190408",
            "createTime": "2019-04-08 13:59:52",
            "modifyTime": "2019-04-08 19:54:51",
            "dataSize": 273248012568,
            "records": 720503170
      } ... ]
    }

SearchMetaTables

  • 功能:模糊查找表。
  • 输入:
    参数 说明
    keyword 表名称的关键字
    pageNum 页码
    pageSize 每页最多显示的记录数
  • 输出:
    参数 描述
    appGuid 项目唯一标识
    tableGuid 表唯一标识
    tableName 表名称
    ownerId Owner的ID
    createTime 表的创建时间
    lastDdlTime 表DDL最后的更新时间
    lastModifyTime 表最后的修改时间
  • 示例如下:
    • 输入:
      request.setKeyword("test");
    • 输出:
      {
          "message": null,
          "code": 200,
          "success": true,
          "data": {
              "requestId": "0be41b2227759****",
              "errCode": 0,
              "errMsg": "success",
              "pageNum": 1,
              "pageSize": 2,
              "totalNum": 5000,
          "data": [{
              "appGuid": null,
              "tableGuid": "odps.ant_p13n.finance_newsrec_tab_dataset_ds",
              "tableName": "finance_newsrec_tab_dataset_ds",
              "createTime": "2018-07-06 16:24:41",
              "lastModifyTime": "2019-04-26 10:49:23",
              "lastDdlTime": null,
              "lastAccessTime": null,
              "ownerId": "163585"
          }, 
          {
              "appGuid": null,
              "tableGuid": "odps.tbcdm.dws_tm_itm_cate_food_ftr_test_cm",
              "tableName": "dws_tm_itm_cate_food_ftr_test_cm",
              "createTime": "2017-11-23 17:06:18",
              "lastModifyTime": "2019-04-26 20:34:12",
              "lastDdlTime": null,
              "lastAccessTime": null,
              "ownerId": "108292"
          }]
        },
          "timestamp": 1556452227875,
          "sessionId": null
      }

GetDQCEntity

  • 功能:获取分区表达式信息。
  • 请求参数:
    参数 类型 是否必选 示例值 描述
    EnvType STRING MaxCompute 项目类型,包括MaxCompute、Hive和Streaming。
    MatchExpression STRING dt=${yyyymmdd-1} 分区表达式,可以不填写。如果不填写,则返回表下的所有分区表达式。
    ProjectName STRING autotest MaxCompute项目名称。
    TableName STRING test_dqc_decimal_1119_2 表名称。
  • 返回参数:
    参数 类型 示例值 描述
    ReturnCode STRING 0 返回码
    ReturnValue - - 返回的分区表达式列表
    Id LONG 4003918 分区表达式ID
    ProjectName STRING autotest MaxCompute项目名称
    TableName STRING test_dqc_decimal_1119_2 表名称
    EnvType STRING MaxCompute 项目类型,包括MaxCompute、Hive和Streaming
    MatchExpression STRING dt=${yyyymmdd-1} 分区表达式
    EntityLevel INTEGER 1 分区表达式级别
    • 0:SQL级别
    • 1:任务级别
    OnDuty STRING 50624 分区表达式责任人
    ModifyUser STRING 50624 最近修改人
    GmtCreate STRING 2018-11-26 23:18:34 创建时间
    GmtModify STRING 2018-11-26 23:18:34 修改时间
    Sql INTEGER 0 是否支持修改为SQL级别
    • 0:支持
    • -1:不支持
    Task INTEGER 0 是否支持修改为任务级别
    • 0:支持
    • -1:不支持
    Followers STRING 050624 订阅人,逗号分隔
    HasRelativeNode BOOLEAN false 是否关联调度
  • 请求示例:
    /?Action=GetDQCEntity
    &EnvType=odps
    &MatchExpression=dt=${yyyymmdd-1}
    &ProjectName=autotest
    &TableName=test_dqc_decimal_1119_2
    &<公共请求参数>
  • 正常返回示例:
    {
      "ReturnCode":"0",
      "ReturnValue":{
        "Entity":[
          {
            "EntityLevel":0,
            "EnvType":"odps",
            "Followers":"050624",
            "GmtCreate":"2018-11-26 15:06:32",
            "GmtModify":"2018-11-26 15:06:32",
            "HasRelativeNode":false,
            "Id":4003918,
            "MatchExpression":"dt=${yyyymmdd-3}",
            "OnDuty":"050624",
            "ProjectName":"autotest",
            "Sql":0,
            "TableName":"test_dqc_decimal_1119_2",
            "Task":0
          },
          {
            "EntityLevel":0,
            "EnvType":"odps",
            "Followers":"050624",
            "GmtCreate":"2018-11-26 22:31:13",
            "GmtModify":"2018-11-26 22:31:13",
            "HasRelativeNode":false,
            "Id":4003922,
            "MatchExpression":"dt=${yyyymmdd-1}",
            "OnDuty":"050624",
            "ProjectName":"autotest",
            "Sql":0,
            "TableName":"test_dqc_decimal_1119_2",
            "Task":0
          },
          {
            "EntityLevel":0,
            "EnvType":"odps",
            "Followers":"050624",
            "GmtCreate":"2018-11-26 23:18:34",
            "GmtModify":"2018-11-26 23:18:34",
            "HasRelativeNode":false,
            "Id":4003923,
            "MatchExpression":"dt=${yyyymmdd-2}",
            "OnDuty":"050624",
            "ProjectName":"autotest",
            "Sql":0,
            "TableName":"test_dqc_decimal_1119_2",
            "Task":0
          }
        ]
      }
    }
  • 异常返回示例:
    {
      "ReturnCode":"500"
    }

GetDQCFollower

  • 功能:获取告警订阅信息。
  • 请求参数:
    参数 类型 是否必选 示例值 描述
    EntityId LONG 4003922 分区表达式ID
    ProjectName STRING MaxCompute MaxCompute项目名称
  • 返回参数:
    参数 类型 示例值 描述
    ReturnCode STRING 0 返回码
    ReturnValue - - 返回的订阅列表
    Id LONG 4003918 分区表达式ID
    ProjectName STRING autotest MaxCompute项目名称
    TableName STRING test_dqc_decimal_1119_2 表名称
    EntityId STRING 4003922 分区表达式ID
    Followers STRING 050624 订阅人,逗号分隔
    AlarmMode INTEGER 1 包括以下4种告警模式:
    • 1:邮件
    • 2:邮件和短信
    • 3:钉钉群机器人/hook
    • 4:钉钉群机器人@ALL
  • 请求示例:
    /?Action=GetDQCFollower
    &EntityId=4003922
    &ProjectName=odps
    &<公共请求参数>
  • 正常返回示例:
    {
      "ReturnCode":"0",
      "ReturnValue":{
        "Follower":[{
          "AlarmMode":1,
          "EntityId":"4003922",
          "Follower":"050624",
          "Id":1726,
          "ProjectName":"autotest",
          "TableName":"test_dqc_decimal_1119_2"
        }]
      }
    }
  • 异常返回示例:
    {
      "ReturnCode":"500"
    }

GetDQCRule

  • 功能:获取规则详情。
  • 请求参数:
    参数 类型 是否必选 示例值 描述
    ProjectName STRING autotest MaxCompute项目名称
    EntityId LONG 4003922 分区表达式ID
  • 返回参数:
    一级参数 二级参数 三级参数 类型 示例值 描述
    ReturnCode - - STRING 0 返回码
    ReturnValue(返回的规则列表) TemplateRules(模板规则列表) Id INTEGER 4003918 分区表达式ID
    ProjectName STRING autotest MaxCompute项目名称
    TableName STRING test_dqc_decimal_1119_2 表名称
    EntityId INTEGER 4003922 分区表达式ID
    Property STRING table_count 参数
    MethodId INTEGER 8 采集方法ID
    MethodName STRING table_count 采集方法名称
    OnDuty STRING 050624 规则配置人
    RuleType INTEGER 0 规则类型
    BlockType INTEGER 1 强弱性
    • 0:弱规则
    • 1:强规则
    TemplateId INTEGER 7 模板ID
    TemplateName STRING SQL任务表行数,1、7、30天波动检测 模板名称
    RuleCheckerRelationId INTEGER 1008007 规则内部关联ID
    CheckerId INTEGER 7 校检器ID
    FixCheck BOOLEAN false 是否是固定值校验
    Trend STRING up 趋势
    WarningThreshold STRING 20 橙色阈值
    CriticalThreshold STRING 90 红色阈值
    HistoryWarningThreshold STRING history max:40%,history min:10% 历史橙色告警阈值
    HistoryCriticalThreshold STRING history max:40%,history min:10% 历史红色告警阈值
    PropertyKey STRING table_count 用于DQC前端规则联动,可以忽略
    MatchExpression STRING dt=${yyyymmdd-1} 分区表达式
    SelfserviceRules(自定义规则列表) ProjectName STRING autotest MaxCompute项目名称
    TableName STRING test_dqc_decimal_1119_2 表名
    Id INTEGER 279580664 规则ID
    EntityId INTEGER 4003922 分区表达式ID
    Property STRING table_count 参数
    MethodId INTEGER 21 采集方法ID
    MethodName STRING count或table_count 采集方法名称
    WhereCondition STRING id>10 过滤条件或自定义SQL
    OnDuty STRING 050624 规则配置人
    RuleType INTEGER 1 规则类型
    BlockType INTEGER 1 强弱性:
    • 0:弱规则
    • 1:强规则
    TemplateId INTEGER 7 模板ID
    TemplateName STRING SQL任务表行数,1、7、30天波动检测 模板名称
    RuleCheckerRelationId INTEGER 1008006 数据质量监控的内部规则映射表
    CheckerId INTEGER 6 校检器ID
    Checker INTEGER 9 校验器,此参数对应于前端的ID标识,需要由pkId转换
    CheckerName STRING compared with a fixed value 校检器名称
    FixCheck BOOLEAN false 是否是固定值校验
    Trend STRING abs 趋势
    CheckResult INTEGER 0 规则的校检结果,该接口不返回,可以忽略
    WarningThreshold STRING 20 橙色阈值
    CriticalThreshold STRING 60 红色阈值
    HistoryWarningThreshold STRING history max:40%,history min:10% 历史橙色告警阈值
    HistoryCriticalThreshold STRING history max:40%,history min:10% 历史红色告警阈值
    HistoryActualThreshold STRING history max:10%,history min:10% 历史真实波动率
    PropertyKey STRING table_count 用于DQC前端规则联动进行映射,可以忽略
    MatchExpression STRING dt=${yyyymmdd-1} 分区表达式

GetQualityByEntity

  • 功能:根据分区表达式获取质量汇总信息。
  • 请求参数:
    参数 类型 是否必选
    Action STRING
    EndDate STRING
    EntityId INTEGER
    PageSize INTEGER
    PageStart INTEGER
    ProjectName STRING
    StartDate STRING
  • 返回参数:
    参数 类型 描述
    Count INTEGER 搜索结果的总行数
    ReturnCode STRING 返回码
    ReturnValue - 分区下规则的校验详情
  • 请求示例:
    http(s)://[Endpoint]/?Action=GetQualityByEntity
    &EndDate=2019-08-25 00:00:00
    &EntityId=1526081
    &PageSize=15
    &PageStart=1
    &ProjectName=autotest
    &StartDate=2019-08-20 00:00:00
    &<公共请求参数>
  • 返回示例:
    {
      "returnCode": "0",
      "returnMessage": null,
      "returnErrorSolution": null,
      "returnErrorOper": null,
      "requestId": "40911dfc-a936-4092-bfe9-6fcd*****",
      "returnValue": [{
        "id": 787700746,
        "taskId": "15668171746013d72950384444****80",
        "entityId": 1508198,
        "ruleId": 28410562,
        "property": "table_count",
        "bizdate": 1566662400000,
        "dateType": "YMD",
        "actualExpression": "ds=20190826",
        "matchExpression": "ds=${yyyymmdd}",
        "blockType": 1,
        "checkResult": 0,
        "checkResultStatus": 0,
        "methodName": "count",
        "comment": null,
        "whereCondition": "",
        "beginTime": 1566817174000,
        "endTime": 1566817355000,
        "timeConsuming": "181s",
        "externalType": "CWF2",
        "externalId": "100033671",
        "discrete": false,
        "fixedCheck": true,
        "referenceValue": [{
          "bizDate": 32535100800000,
          "discreteProperty": "table_count",
          "value": 0.0,
          "threshold": null,
          "singleCheckResult": 0
        }],
        "sampleValue": [{
          "bizDate": 1566662400000,
          "discreteProperty": null,
          "value": 1306012.0
        }],
        "checkResultDetail": {
          "checkerId": null,
          "checkerType": null,
          "isDiscrete": false,
          "warningThreshold": null,
          "criticalThreshold": null,
          "op": ">",
          "expectValue": 0,
          "trend": null,
          "externalId": "100033671",
          "status": 0,
          "qualityStatus": 0,
          "actualResult": null,
          "detail": [{
            "currentSample": {
              "bizdate": 1566662400000,
              "property": "table_count",
              "distinctProperty": null,
              "value": 1306012,
              "detail": "1306012"
            },
            "historicalSamples": [{
              "bizdate": null,
              "property": null,
              "distinctProperty": null,
              "value": null,
              "detail": "--",
              "isExisted": null,
              "fluctuatedValue": null,
              "fluctuatedValueType": null,
              "checkResult": null
            }]
          }]
        },
        "trend": "abs",
        "warningThreshold": null,
        "criticalThreshold": null,
        "expectValue": 0.0,
        "op": ">",
        "projectName": null,
        "tableName": null,
        "templateId": null,
        "templateName": null,
        "resultStr": null,
        "checkerId": null,
        "checkerType": 0,
        "ruleName": null,
        "isPrediction": false,
        "upperValue": null,
        "lowerValue": null,
        "checkerName": "与固定值比较"
      }]
    }

GetQualityByRule

  • 功能:根据规则查询校验详情。
  • 请求参数:
    参数 类型 是否必选
    Action STRING
    EndDate STRING
    PageSize INTEGER
    PageStart INTEGER
    ProjectName STRING
    RuleId INTEGER
    StartDate STRING
  • 返回参数:
    参数 类型 描述
    Count INTEGER 数据总条数
    ReturnCode STRING 返回码
    ReturnValue - 返回的校验详情
  • 请求示例:
    http(s)://[Endpoint]/?Action=GetQualityByRule
    &EndDate=2019-08-26 00:00:00
    &PageSize=15
    &PageStart=1
    &ProjectName=cdo_meta
    &RuleId=28791246
    &StartDate=2019-08-22 00:00:00
    &<公共请求参数>
  • 返回示例:
    {
      "returnCode": "0",
      "returnMessage": null,
      "returnErrorSolution": null,
      "returnErrorOper": null,
      "requestId": "f34d529b-d559-4699-b850-9****b58",
      "returnValue": {
        "ruleCheckDto": [{
          "id": 787723200,
          "taskId": "156681910181215a8b8f9cd154****34e7d",
          "entityId": 1346320,
          "ruleId": 28791246,
          "property": "-",
          "bizdate": 1566662400000,
          "dateType": "YMD",
          "actualExpression": "NOTAPARTITIONTABLE",
          "matchExpression": "NOTAPARTITIONTABLE",
          "blockType": 1,
          "checkResult": 0,
          "checkResultStatus": 0,
          "methodName": "table_count",
          "comment": "监控表行数波动",
          "whereCondition": null,
          "beginTime": 1566819101000,
          "endTime": 1566819103000,
          "timeConsuming": "2s",
          "externalType": "CWF2",
          "externalId": "1607534",
          "discrete": false,
          "fixedCheck": false,
          "referenceValue": [{
            "bizDate": 32535100800000,
            "discreteProperty": null,
            "value": 3778.0,
            "threshold": 0.1852832186,
            "singleCheckResult": 0
          }, {
            "bizDate": 32535100800000,
            "discreteProperty": null,
            "value": 3772.0,
            "threshold": 0.3446447508,
            "singleCheckResult": 0
          }, {
            "bizDate": 32535100800000,
            "discreteProperty": null,
            "value": 3691.0,
            "threshold": 2.5,
            "singleCheckResult": 0
          }],
          "sampleValue": [{
            "bizDate": 1566662400000,
            "discreteProperty": null,
            "value": 3785.0
          }],
          "checkResultDetail": {
            "checkerId": null,
            "checkerType": null,
            "isDiscrete": false,
            "warningThreshold": 10,
            "criticalThreshold": 50,
            "op": null,
            "expectValue": null,
            "trend": null,
            "externalId": "1607534",
            "status": 0,
            "qualityStatus": 0,
            "actualResult": null,
            "detail": [{
              "currentSample": {
                "bizdate": 1566662400000,
                "property": "-",
                "distinctProperty": null,
                "value": 3785,
                "detail": "3785"
              },
              "historicalSamples": [{
                "bizdate": 32535100800000,
                "property": null,
                "distinctProperty": null,
                "value": 3778,
                "detail": "3778,  0.1852832186%",
                "isExisted": null,
                "fluctuatedValue": 0.185283,
                "fluctuatedValueType": null,
                "checkResult": 0
              }, {
                "bizdate": 32535100800000,
                "property": null,
                "distinctProperty": null,
                "value": 3772,
                "detail": "3772,  0.3446447508%",
                "isExisted": null,
                "fluctuatedValue": 0.344645,
                "fluctuatedValueType": null,
                "checkResult": 0
              }, {
                "bizdate": 32535100800000,
                "property": null,
                "distinctProperty": null,
                "value": 3691,
                "detail": "3691,  2.5%",
                "isExisted": null,
                "fluctuatedValue": 2.5,
                "fluctuatedValueType": null,
                "checkResult": 0
              }]
            }]
          },
          "trend": "abs",
          "warningThreshold": 10.0,
          "criticalThreshold": 50.0,
          "expectValue": null,
          "op": "abs",
          "projectName": null,
          "tableName": null,
          "templateId": 7,
          "templateName": null,
          "resultStr": null,
          "checkerId": null,
          "checkerType": 1,
          "ruleName": "表行数监控",
          "isPrediction": false,
          "upperValue": null,
          "lowerValue": null,
          "checkerName": null
        }],
        "count": 4325,
        "checkerType": 0
      }
    }

使用DataOS API

添加maven依赖如下所示。

<dependency>
    <groupId>com.aliyun</groupId>
    <artifactId>aliyun-java-sdk-dataworks-enterprise-ultimate</artifactId>
    <version>0.0.4</version>
</dependency>

<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.8.5</version>
</dependency>

<!-- 要求gson 2.8.5以上版本 -->
    <dependency>
        <groupId>com.aliyun</groupId>
        <artifactId>aliyun-java-sdk-core</artifactId>
        <version>4.4.0</version>
    </dependency>

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>
</dependency>

Java代码如下所示,其中创建IClientProfile时,需要指定云账号的AccessKeyID和AccessKeySecret,详情请参见下文的常见问题。

说明 将测试代码直接放在src/test/java目录下,会有UT快速启动入口。
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.CheckMetaTableRequest;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.CheckMetaTableResponse;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.GetMetaDBRequest;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.GetMetaDBResponse;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.GetMetaTableRequest;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.GetMetaTableResponse;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.ListMetaTableColumnRequest;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.ListMetaTableColumnResponse;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.ListMetaTablePartitionRequest;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.ListMetaTablePartitionResponse;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.SearchMetaTablesRequest;
import com.aliyuncs.dataworks_enterprise_ultimate.model.v20190424.SearchMetaTablesResponse;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.exceptions.ServerException;
import com.aliyuncs.profile.DefaultProfile;
import com.aliyuncs.profile.IClientProfile;
import com.google.gson.Gson;

import org.junit.BeforeClass;
import org.junit.Test;

public class Sample {
    private static final String REGION_ID = "cn-shanghai";
    private static IAcsClient client = null;

    @BeforeClass
    public static void setup() throws ClientException {
        String endpoint = "dataworks-ee-ue-share." + REGION_ID + ".aliyuncs.com";
        IClientProfile profile = DefaultProfile.getProfile(
            REGION_ID,
            "<!!!!your.accessId>",
            "<!!!!your.accessKey>");
        DefaultProfile.addEndpoint(REGION_ID, "dataworks-enterprise-ultimate", endpoint);
        client = new DefaultAcsClient(profile);
    }

    @Test
    public void testCheckMetaTable() throws ServerException, ClientException {
        String tableGuid = "odps.meta.m_table";

        CheckMetaTableRequest request = new CheckMetaTableRequest();
        request.setTableGuid(tableGuid);
        CheckMetaTableResponse response = client.getAcsResponse(request);
        System.out.println(new Gson().toJson(response));
    }

    @Test
    public void testGetProject() throws ServerException, ClientException {
        String appGuid = "odps.meta";

        GetMetaDBRequest request = new GetMetaDBRequest();
        request.setDbGuid(appGuid);
        GetMetaDBResponse getMetaDBResponse = client.getAcsResponse(request);
        System.out.println(new Gson().toJson(getMetaDBResponse));
    }

    @Test
    public void testGetPartitions() throws ServerException, ClientException {
        String tableGuid = "odps.meta.m_table";

        ListMetaTablePartitionRequest request = new ListMetaTablePartitionRequest();
        request.setTableGuid(tableGuid);
        request.setPageNum(1);
        request.setPageSize(10);
        ListMetaTablePartitionResponse response = client.getAcsResponse(request);
        System.out.println(new Gson().toJson(response));
    }

    @Test
    public void testSearchTables() throws ServerException, ClientException {
        SearchMetaTablesRequest request = new SearchMetaTablesRequest();
        request.setKeyword("test");
        request.setPageNum(1);
        request.setPageSize(10);
        SearchMetaTablesResponse response = client.getAcsResponse(request);
        System.out.println(new Gson().toJson(response));
    }

    @Test
    public void testGetColumns() throws ServerException, ClientException {
        String tableGuid = "odps.meta.m_table";

        ListMetaTableColumnRequest request = new ListMetaTableColumnRequest();
        request.setTableGuid(tableGuid);
        ListMetaTableColumnResponse response = client.getAcsResponse(request);
        System.out.println(new Gson().toJson(response));
    }

    @Test
    public void testGetTable() throws ServerException, ClientException {
        String tableGuid = "odps.meta.m_table";

        GetMetaTableRequest request = new GetMetaTableRequest();
        request.setTableGuid(tableGuid);
        GetMetaTableResponse response = client.getAcsResponse(request);
        System.out.println(new Gson().toJson(response));
    }

}

常见问题

  • 无法访问API,错误提示如下所示:
    Exception in thread "main" com.aliyuncs.exceptions.ClientException: InvalidApi.NotFound : Specified api is not found, please check your url and method.
    RequestId : B081CCF1-9F19-473E-9B99-68F20****

    错误原因:没有获取API权限。

  • 如何查询AccessKeyID和AccessKeySecret?
    单击页面右上角账号下的accesskeys,即可进行查询。查询
  • 调试时启动服务,配置项报错Circular Reference。
    java.lang.IllegalArgumentException:  Circular  placeholder  reference  'dataworks.dataos.auth.accessId'  in  property  definitions
    启动时需要添加如下图所示的配置参数。配置参数