All Products
Search
Document Center

DataWorks:Getting started with DataService Studio

Last Updated:Jan 05, 2024

DataWorks provides DataService Studio to build a data service bus that individuals, teams, and enterprises can use to manage private and public APIs in a centralized manner. For example, you can use DataService Studio to quickly generate APIs based on data tables. You can also register existing APIs to DataService Studio to publish and manage the APIs in a centralized manner. This topic describes how to develop an API based on a MySQL data source.

Prerequisites

  • API Gateway is activated to manage and call APIs in a centralized manner. For more information, see API Gateway.

  • The data source that you want to use to generate APIs is added. For more information, see Configure a data source.

    In this example, a MySQL data source is added.

Background information

DataService Studio serves as a bridge between databases and data applications. DataService Studio provides comprehensive data processing and sharing capabilities for individuals, teams, and enterprises to help them manage APIs that belong to and do not belong to Alibaba Cloud in a centralized manner.

DataService Studio is connected to API Gateway. You can publish APIs to API Gateway with a few clicks. DataService Studio works together with API Gateway to provide a secure, stable, cost-effective, and easy-to-use data opening and sharing service. For more information about DataService Studio, see Overview.

Go to the DataService Studio page

Log on to the DataWorks console. In the left-side navigation pane, click DataService Studio. On the page that appears, select the desired workspace from the drop-down list and click Go to DataService Studio.

Procedure

  1. Step 1: Add a data source and establish a network connection between the data source and a DataWorks resource group for DataService Studio

    Before you use DataService Studio to create an API, you must add your database or data warehouse to DataWorks as a data source. Then, you can use the data source to create an API. During API development, DataService Studio accesses the data source to obtain the schema of a data table based on which you want to create an API to help you configure the request parameters and response parameters of the API. When you call the API, DataService Studio accesses the data source to execute a query request.

  2. Step 2: Create an API group

    DataService Studio provides external services based on API Gateway. In API Gateway, APIs are managed by group. To call an API, you must bind the domain name of the API calling address to the API group to which the API belongs. Before you use DataService Studio to create an API, you must create an API group in API Gateway.

  3. Step 3: Create a business process

    DataService Studio develops APIs by business process. DataService Studio also organizes and manages APIs and files that are used to create the APIs based on business processes. The files include functions. Before you use DataService Studio to create an API, you must create a business process.

  4. Step 4: Generate an API

    After you create the business process, you can create an API in the business process.

  5. Step 5: Test the API

    Before you publish the API, you must perform a test on the API and make sure that the API passes the test.

  6. Step 6: Publish the API

    After the API passes the test, you can publish the API to allow other services to call the API.

  7. Step 7: Call the API

    DataService Studio allows you to use the simple authentication or encrypted signature authentication method to call the API. You can select a method based on your business requirements.

  8. Step 8: View and manage the API

    On the Service Management tab of the DataService Studio page, you can view published APIs and manage the APIs based on your business requirements. For example, you can unpublish APIs, authorize other users to call APIs, and view the calling details of an API.

Step 1: Add a data source and establish a network connection between the data source and a DataWorks resource group for DataService Studio

Before you use DataService Studio to create an API, you must add your database or data warehouse to DataWorks as a data source. Make sure that a network connection is established between the data source and a DataWorks resource group for DataService Studio.

Note

After you associate a compute engine, such as a cluster or a database, with a DataWorks workspace as a compute engine instance, DataWorks creates a default data source of the compute engine type.

  1. Go to the SettingCenter page.

  2. In the left-side navigation pane of the page that appears, click Data Source. The Data Source page appears.

  3. Add a data source and test the network connectivity between the data source and the resource group for DataService Studio that you want to use.

    1. Click Add Data Source and select a data source type in the Add data source dialog box.

      In this example, MySQL is selected.

    2. Configure information about the data source.

      Configure information about the data source based on your business requirements. For more information, see Add a MySQL data source.

    3. Set the Resource Group connectivity parameter to Data Service and test the network connectivity between the data source and the resource group for DataService Studio that you want to use.

      In this example, the shared resource group for DataService Studio that is provided by DataWorks by default is used.

      Note

      If a large number of nodes must be run in parallel, exclusive resources are required to ensure highly parallel and frequent API calls. In this case, we recommend that you use an exclusive resource group for DataService Studio. For more information, see Create and use an exclusive resource group for DataService Studio. Exclusive resource groups for DataService Studio are available only in the China (Shanghai) region.

    4. Click Complete. The data source is added.

    In this example, a MySQL data source named rds_workshop_log is added, and the network connectivity between the data source and the shared resource group for DataService Studio is tested. The data source passes the network connectivity test. The following figure shows the parameter configurations.新增数据源

Step 2: Create an API group

DataService Studio provides external services based on API Gateway. In API Gateway, APIs are managed by group. To call an API, you must bind the domain name of the API calling address to the API group to which the API belongs. Before you use DataService Studio to create an API, you must create an API group in API Gateway.

  1. Log on to the API Gateway console.

  2. In the left-side navigation pane, choose Open API > API Groups.

  3. On the API Groups page, click Create Group to create an API group.

    In this example, an API group named Workshop is created.

Step 3: Create a business process

DataService Studio develops APIs by business process and organizes and manages APIs and files that are used to create the APIs based on business processes. The files include functions. Before you use DataService Studio to create an API, you must create a business process.

  1. Create a business process.

    You can use one of the following methods to create a business process in the Service Development pane of the DataService Studio page:

    • Method 1: Click the 新建 icon and select Create Business Process.

    • Method 2: Right-click Business Processes in the Service Development pane and select Create Business Process.

  2. Configure the parameters for the business process. The following table describes the parameters.

    Parameter

    Description

    Name

    The name of the business process. Specify a name based on your business requirements. In this example, a business process named DataService is created.

    API Group

    The API group that is used to manage APIs. In this example, the Workshop API group created in Step 2 is used.

    Note

    An API group is a collection of APIs that are used for a specific scenario or to accomplish a specific task. An API group is the smallest management unit for APIs in API Gateway. To create an API group, refer to Step 2: Create an API group.

  3. Click OK. The business process is created.

    After you create the business process, you can view the details of the business process in the Service Development pane.

Step 4: Generate an API

You can generate an API in the business process based on your business requirements.

  1. Generate an API.

    You can use one of the following methods to generate an API in the Service Development pane of the DataService Studio page:

    • Method 1: Click the 新建 icon and choose Create API > Generate API.

    • Method 2: In the Service Development pane, right-click the created business process and choose New > API > Generate API.

  2. Configure properties for the API.

    In the Generate API dialog box, configure properties for the API. The following figure shows sample configurations. 配置APIThe following table describes the parameters.

    Parameter

    Description

    API Mode

    The mode in which you want to create an API. Valid values: Wizard Mode and Script Mode.

    • Wizard Mode: allows you to quickly configure an API. You do not need to write code when you configure an API in this mode.

    • Script Mode: supports advanced features, such as multi-table joins, complex queries, and aggregate functions. In this mode, advanced users can write SQL statements to query an API based on their query requirements.

    In this example, Wizard Mode is selected. For more information about the two modes, see Overview.

    API Name

    The name of the API. In this example, the API named rpt_user_info_all_d is created.

    API Path

    The path of the API. The path is the API request path that is relative to the service host address. The API request path is used to concatenate an API calling address. In this example, the API request path is /user.

    Protocol

    The protocol over which you want to call the API. Valid values: HTTP and HTTPS.

    If you want to call the API over HTTPS, you must bind an independent domain name to the API group to which the API belongs in the API Gateway console after the API is published to API Gateway. You must also upload a Secure Sockets Layer (SSL) certificate in the API Gateway console. For more information, see Enable HTTPS for an API operation.

    Request Method

    The request method of the API. Valid values: GET and POST.

    Note
    • If you set the Request Method parameter to GET, you can select only QUERY for the Parameter Position parameter that is configured in the request parameters of the API.

    • If you set the Request Method parameter to POST, you can select QUERY or BODY for the Parameter Position parameter that is configured in the request parameters of the API.

    Response Type

    The format of the returned data. Only JSON can be selected.

    Visible Scope

    The visible scope of the API. Valid values: Workspace and Private.

    • Workspace: The API is visible to all members in the current workspace.

    • Private: The API is visible only to the API owner. Permissions on the API cannot be granted to other members.

    In this example, Workspace is selected.

    Destination Folder

    The folder that is used to store the API in the business process in the Service Development pane. If no business process exists, create one. You can refer to Step 3: Create a business process to create a business process.

  3. Configure the parameters for the API.

    On the configuration tab of the API, configure the parameters. The following figure shows sample configurations.配置API

    1. Configure the parameters in the Select Table section.

      In the Select Table section, select a table of a specific data source type based on which you want to generate the API. The following table describes the parameters.

      Parameter

      Description

      Datasource Type

      The type of data source based on which you want to create the API. In this example, MySQL is selected.

      Datasource Name

      The name of the data source of the selected type based on which you want to generate the API. In this example, the rds_workshop_log data source added in Step 1: Add a data source and establish a network connection between the data source and a DataWorks resource group for DataService Studio is selected.

      Data Source Environment

      The environment of the data source. If you use a workspace in standard mode, you can select the data source in the development or production environment. For information about the differences between the development and production environments, see Differences between workspaces in basic mode and workspaces in standard mode. In this example, Production is selected.

      Table Name

      The name of the data table based on which you want to generate the API. In this example, the rpt_user_info_all_d table in the rds_workshop_log data source is selected.

    2. Define request parameters and response parameters of the API.

      After you select a data table, all fields in the table are automatically displayed in the Select Parameters section. You can configure fields as the request parameters and response parameters of the API based on your business requirements. If you want to sort fields, you can add the desired fields to the Sort field section to adjust the order of the fields.

      In this example, the request parameter is uid and the response parameters are all fields in the table.

      Note

      In the right-side navigation pane of the configuration tab of the API, click Request Param or Response Param. On the tab that appears, modify the information about the fields that are configured as request or response parameters, such as the type, sample value, and default value. You can modify the information about the fields based on your business requirements.

Step 5: Test the API

After the API is generated, you can click Test in the upper-right corner of the configuration tab of the API to check whether the API request and return values meet your expectations. In this example, the value of the uid request parameter is set to 0016359810821. Check the returned result based on the value. For information about the test, see Test an API.测试API

Step 6: Publish the API

The API can be called by other services only after the API is submitted and published. To publish the API, select an API version and click Submit in the upper-right corner of the configuration tab of the API. You can view the published API in API Gateway. For more information about how to publish an API, see Publish an API.发布API

Step 7: Call the API

You can use one of the following authentication methods to call the API:

  • Simple authentication: provides low security and is suitable for scenarios such as reports and data dashboards.

  • Encrypted signature authentication: provides high security and is suitable for application development scenarios.

  1. In the top toolbar of the DataService Studio page, click Service Management. The Service Management tab appears.

  2. In the left-side navigation pane, click Call APIs.

    The API that is published in DataWorks is published to API Gateway. You can use API Gateway SDK to call the API. For more information, see Examples of calling APIs.

Step 8: View and manage the API

On the Service Management tab, you can view and manage published APIs.

Page

Description

Manage APIs

In the left-side navigation pane of the Service Management tab, click Manage APIs. On the Manage APIs page, you can view APIs that are published, APIs that you are authorized to call, and APIs that you authorize other users to call. You can also perform operations on the APIs on which you have permissions, such as unpublishing APIs, authorizing other users to call APIs, and testing APIs.

API Statistics

In the left-side navigation pane of the Service Management tab, click API Statistics. On the API Statistics page, you can perform different operations or view different information on the following tabs:

  • Statistics Dashboard: allows you to view the calling details of APIs from a global perspective, including the total number of APIs, distribution of HTTP status codes, distribution of error codes, overall statistics, and distribution of resources in resource groups.

  • Statistics Details: allows you to view statistics about each API in various charts and tables, including the API Gateway status codes, DataService Studio error codes, the number of application-initiated requests, traffic bandwidth, and average response time.

  • Log Analysis: allows you to analyze API requests. If an API request exception occurs, you can use the log analysis feature to quickly locate the cause of the exception and obtain diagnostic results and suggestions.