All Products
Search
Document Center

:Replace Lightning with Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) to connect to DataWorks DataService Studio

Last Updated:May 29, 2023

Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) is an online query acceleration service for Hologres scenarios. Based on a Hologres cloud-native architecture that separates storage from computing, this service uses shared cluster resources to accelerate queries on data in MaxCompute tables. This topic describes how to replace Lightning with Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) to connect to DataWorks DataService Studio.

Background information

Lightning will be discontinued and no longer maintained. You can consider Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) as an upgraded version of Lightning. The performance and services of Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) are better than those of Lightning. Lightning will be replaced by Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) in phases.

Process of using Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition)

The following figure compares the process of using Lightning to connect to DataWorks DataService Studio with the process of using Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) to connect to DataWorks DataService Studio.Process comparison

The process of using Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) to connect to DataWorks DataService Studio involves the following steps:

  • Purchase a Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) instance and verify the details of the purchased instance in the Hologres console.

  • Create a database and specify a permission model.

  • Add a Hologres data source in DataWorks DataService Studio. Then, create, configure, test, publish, and call an API in DataWorks DataService Studio.

Step 1: Purchase a Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) instance

Before you replace Lightning with Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition), you must purchase a Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) instance. Then, check whether the purchased instance can be used in the Hologres console.

  1. Go to the buy page of Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) and purchase an instance. Set the following parameters:
    • Product Type: Select Shared Cluster (BI-based MaxCompute Acceleration Edition).
    • Region: Select a region based on your business requirements.
    • Instance Name: The name must be 2 to 64 characters in length.
  2. Click Buy Now and confirm the order information.
  3. Select I have read and agree to Shared Hologres Cluster (BI-based MaxCompute Acceleration Edition) Agreement of Service and click Activate Now.
    It takes 5 to 10 minutes to complete the order of a Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) instance.
  4. After the instance is purchased, go to the Hologres console.
  5. In the left-side navigation pane, click Instances. On the Instances page, click the name of the desired instance.

    You can also click Manage in the Actions column of the instance. On the page that appears, you can view the details of the instance.

Step 2: Create a database

After the instance is created, the system automatically creates a default database named postgres. The default database is used to monitor and manage the instance that you have created. If you need to create other databases to process business data, perform the following steps:

  1. In the left-side navigation pane of the instance details page, click Database Management. On the Database Authorization page, click Create Database in the upper-right corner.
  2. In the Create Database dialog box, configure the Instance Name, Database Name, and SPM parameters based on your business requirements. If you want to simplify permission management, we recommend that you set the SPM parameter to Enable when you create a database.

    Hologres allows you to use the standard PostgreSQL authorization model and simple permission model (SPM) to manage user permissions.

    • The standard PostgreSQL authorization model is exactly the same as that in PostgreSQL. This model is also called the expert mode for short. For more information, see Standard PostgreSQL authorization model.
    • Backed by its understanding of customers' business and its practical experience, Alibaba Cloud introduced the SPM to Hologres to simplify the management of user permissions. For more information, see Overview.
  3. Click OK. You can view the created database on the Database Authorization page.

Step 3: Enable the feature of automatically creating foreign tables for MaxCompute tables

You can enable the feature of automatically creating foreign tables for MaxCompute tables. This way, you can accelerate data queries in MaxCompute tables. For more information about the feature of automatically creating foreign tables for MaxCompute tables, see Automatically create foreign tables for MaxCompute tables.
Note You can also manually create a MaxCompute foreign table by running the CREATE FOREIGN TABLE or IMPORT FOREIGN SCHEMA command. For more information, see Create a foreign table in Hologres to accelerate queries of MaxCompute data.

Step 4: Connect to DataWorks DataService Studio

Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) is compatible with PostgreSQL and can be directly connected to various business intelligence (BI) tools. If you use a BI tool other than DataWorks DataService Studio, you must replace the endpoint of Lightning with the endpoint of the Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) instance before you perform visualized data analysis.

In this example, DataWorks DataService Studio is used to connect to the Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) instance.

  1. Configure a Hologres data source.

    1. Log on to the DataWorks console. In the top navigation bar, select the region where your workspace resides. In the left-side navigation pane, click Workspaces.

    2. Find your workspace and click Data Integration in the Actions column.

    3. On the homepage of Data Integration, click Data Source in the left-side navigation pane. On the Data Sources page, click Create Data Source. In the Add data source dialog box, click Hologres in the Select the data source type section.

    4. In the Add Hologres data source dialog box, configure the parameters.

      The following table describes the parameters.

      Parameter

      Description

      Data Source Type

      Set this parameter to Alibaba Cloud Instance Mode.

      Data Source Name

      The name of the data source. The name can contain letters, digits, and underscores (_), and must start with a letter.

      Data Source Description

      The description of the data source. The description can be up to 80 characters in length.

      Environment

      • Development

      • Production

      Note

      If your workspace is created in standard mode, this parameter is required.

      Instance ID

      The ID of the Hologres instance to which you want to synchronize data.

      You can view the instance ID on the Hologres console.

      Database Name

      The name of the Hologres database.

      AccessKey ID

      You can obtain the AccessKey ID from the Security Management page.

      AccessKey Secret

      You can obtain the AccessKey secret from the Security Management page.

      Special Authentication Method

      Resource Group Connectivity

      You must ensure that shared resource groups can be connected to the data source.

  2. Create and configure an API.

    1. Log on to the DataWorks console. In the top navigation bar, select the region where your workspace resides. In the left-side navigation pane, click Workspaces.

    2. Find your workspace and click DataStudio in the Actions column.

    3. On the DataStudio page, click the Icon icon in the upper-left corner and choose All Products > Data Service > DataService Studio.

    4. In the Service Development pane, click the image icon and choose Create API > Generate API.Generate API

      In the Generate API dialog box, you can set the API Mode parameter to Wizard Mode or Script Mode.

  3. Test the API.

    1. On the configuration tab of the API, configure the parameters on the Properties, Request Param, and Response Param tabs in the right-side navigation pane. Then, save the settings.

    2. On the configuration tab of the API, click Test in the upper-right corner.

    3. In the Test APIs dialog box, check the input request parameters and click Test. TestIf the Tested successfully message appears in the lower part of the Test APIs dialog box, the API passes the test. You can also test the API in the Test APIs module of DataService Studio. For more information, see Test an API.

  4. Publish and view the API.

    On the configuration tab of the API, click Publish in the upper-right corner. Publish the API to API Gateway and make the API available on the API marketplace. For more information, see Publish an API. On the DataService Studio page, click Service Management on the right side of the top navigation bar. On the Published APIs tab, you can find the published API and click the API name to view the details of the API.

  5. Call the API.

    For more information about how to call the published API, see Call an API.

Migrate user permissions

Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition) and Lightning use different permission control methods. After you replace Lightning with Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition), you must grant necessary permissions to a RAM user to use Hologres Shared Cluster (BI-based MaxCompute Acceleration Edition).

For more information, see Grant the development permissions on a Hologres instance to RAM users.