Realtime Compute provides a management page for various data storage systems, such as ApsaraDB for RDS and Table Store. It offers you a one-stop cloud-based management solution.

Data storage in Realtime Compute

In Realtime Compute, data storage has the following two meanings:
  • It refers to the data storage systems or database tables (hereinafter referred to as "storage resources") at the input and output nodes of Realtime Compute.
  • It also refers to how Realtime Compute manages the input and output storage resources (hereinafter referred to as "the data storage feature").
Note Before registering data storage resources with Realtime Compute, you must grant Realtime Compute the permission to access these resources in advance. For more information, see Grant a RAM role to Realtime Compute in exclusive mode.
Realtime Compute allows you to reference input and output storage resources by using plaintext AccessKeys or registering data storage resources. The following sections describe the two approaches in detail.

Use a plaintext AccessKey

To explicitly reference the input and output storage resources, you must set the accessId and accessKey parameters in the with clause of the corresponding Data Definition Language (DDL) statement. For more information, see DDL overview. In this mode, you can authorize an Alibaba Cloud account and the RAM users created under the account to access resources of the currrent or another Alibaba Cloud account. Assume that user A or a RAM user created under the Alibaba Cloud account of user A wants to use storage resources of user B in Realtime Compute. User A can set the AccessKey of user B in the following DDL statement in plaintext mode:

CREATE TABLE in_stream(
  a varchar,
  b varchar,
  c timestamp
)with(
type='datahub',
  endPoint='http://dh-cn-hangzhou.aliyuncs.com',
  project='<dataHubProjectName>',
  topic='<dataHubTopicName>',
  accessId='<accessIdOfUserB>',
  accessKey='<accessKeyOfUserB>'
);

Register a storage resource

Realtime Compute allows you to manage and reference input and output storage resources that have been registered with the Realtime Compute development platform. After a storage resource is registered, you can preview or sample the corresponding data, or obtain the DDL statements that are automatically generated for referencing the resource. This helps you manage your cloud-based storage resources in one-stop mode.

Note You can only register storage resources of the current Alibaba Cloud account. That means, user A or a RAM user created under the Alibaba Cloud account of user A can only register storage resources purchased by user A. You cannot register storage resources of other Alibaba Cloud accounts. To use storage resources of another Alibaba Cloud account, set the AccessKey of the specified Alibaba Cloud account in the corresponding DDL statement.
  • Register a storage resource
    To register input and output storage resources with the Realtime Compute development platform before referencing them, follow these steps:
    1. Log on to the Realtime Compute development platform.
    2. Click Development at the top.
    3. On the Development page, click Storage in the left-side navigation pane.
    4. On the Storage tab, click + Registration and Connection in the upper-right corner.
    5. In the Register Data Store and Test Connection dialog box, set parameters for storage resources as required.

      Realtime Compute allows you to register the following three types of storage resources. For more information about how to register storage resources of a specified type, click the corresponding link:

    • Preview data
      To preview data from registered storage resources, follow these steps:
      1. On the Development page, click Storage in the left-side navigation pane.
      2. On the Storage tab, navigate through cascaded folders and nodes to find the target table, and then double-click the name of the table.
      3. In the Table Details dialog box, view data from the storage resource in the Data Preview section.
    • Obtain the DDL statements that are automatically generated for referencing a storage resource
      To obtain the DDL statements that are automatically generated for referencing a storage resource, follow these steps:
      1. On the Development page, click Storage in the left-side navigation pane.
      2. On the Storage tab, navigate through cascaded folders and nodes to find the target table, and then double-click the name of the table.
      3. In the Table Details dialog box, click Reference as Source Table, Reference as Result Table, or Reference as Dimension Table as required. Then, you can obtain the DDL statements that are automatically generated for referencing the target table.
      Note The automatically generated DDL statements contain only basic parameters in the with clause to make sure the smooth connection between Realtime Compute and storage resources. You can add other parameters to the with clause as required.
    • Test the network connectivity
      The data storage feature supports network detection so that you can test the network connectivity between Realtime Compute and the target storage resource. To enable network detection, follow these steps:
      1. On the Development page, click Storage in the left-side navigation pane.
      2. On the Storage tab, click + Registration and Connection in the upper-right corner.
      3. In the Register Data Store and Test Connection dialog box, turn on Test Connection.