You can use the Object Storage Service (OSS) object inspection feature to monitor OSS objects if descendant nodes can run only after OSS objects are generated in OSS. For example, you can run a node for synchronizing OSS data files to DataWorks only after the OSS data files are generated. In this case, you can use the OSS object inspection feature to monitor the OSS data files.

The OSS object inspection feature can monitor OSS objects of all tenants. To create an OSS Object Inspection node, follow these steps:
  1. Log on to the DataWorks console. In the left-side navigation pane, click Workspaces. On the Workspaces page, find the target workspace and click Data Analytics in the Actions column.
  2. Move the pointer over the Create icon and choose General > OSS Object Inspection.

    You can also find the target workflow, right-click General, and choose Create > OSS Object Inspection.

  3. In the Create Node dialog box that appears, enter the node name, select the target folder, and click Commit.
    Note A node name can be up to 128 characters in length.
  4. On the OSS Object Inspection page that appears, set the parameters.
    No. Parameter Description
    1 OSS Object The storage path of the OSS object. You can add a scheduling parameter to the storage path. For more information, see Scheduling parameters.
    2 Timeout The timeout period during which DataWorks checks whether the OSS object exists in OSS every five seconds. If the OSS object is not detected before the timeout period ends, the OSS Object Inspection node fails.
    3 Storage Address The storage space of the OSS object. Valid values:
    • Myself: detects the OSS object in the storage space of the current tenant.
    • Other: detects the OSS object in the storage space of another tenant.
    Note
    • When an OSS Object Inspection node is running, it monitors the OSS object through MaxCompute. Make sure that MaxCompute has the required permissions on the OSS bucket. For more information, see OSS STS mode authorization.
    • In the development or production environment, the node monitors the OSS object through the access identity of the development or production environment. Make sure that the access identity has the required permissions on the OSS bucket.
    • You cannot specify an OSS object by using the wildcard (*), nor can you use the system parameters cyctime and bizdate. However, you can use custom parameters.
  5. Grant MaxCompute the permission to access OSS in the Resource Access Management (RAM) console.
    MaxCompute uses RAM and Security Token Service (STS) of Alibaba Cloud to resolve security issues of accounts.
    • If the owners of MaxCompute and OSS are using the same Alibaba Cloud account, you can go to the RAM console and click Confirm Authorization Policy to authorize MaxCompute to access OSS.
    • If the owners of MaxCompute and OSS are using different Alibaba Cloud accounts, you can follow these steps to authorize MaxCompute to access OSS:
      1. Create a role in the RAM console.

        Create a role, such as AliyunODPSDefaultRole or AliyunODPSRoleForOtherUser, and set the following policy:

        --The owners of MaxCompute and OSS are using different Alibaba Cloud accounts.
        {
        "Statement": [
        {
        "Action": "sts:AssumeRole",
        "Effect": "Allow",
        "Principal": {
        "Service": [
        "The ID of the Alibaba Cloud account used by the owner of MaxCompute@odps.aliyuncs.com"
        ]
        }
        }
        ],
        "Version": "1"
        }
      2. Create the AliyunODPSRolePolicy permission policy that contains the permissions required for accessing OSS.
        {
        "Version": "1",
        "Statement": [
        {
        "Action": [
         "oss:ListBuckets",
         "oss:GetObject",
         "oss:ListObjects",
         "oss:PutObject",
         "oss:DeleteObject",
         "oss:AbortMultipartUpload",
         "oss:ListParts"
        ],
        "Resource": "*",
        "Effect": "Allow"
        }
        ]
        }
        --You can also add other permissions as required.
      3. Grant the AliyunODPSRolePolicy permission policy to the role.
  6. Go to Operation Center to view the runtime logs.
    If the following error information appears, the OSS object is not detected:
    <Error>
     <Code>NoSuchKey</Code>
     <Message>The specified key does not exist. </Message>
     <RequestId></RequestId>
     <HostId>OSS object</HostId>
     <Key>xc/111.txt</Key>
    </Error>