All Products
Search
Document Center

DataWorks:DataWorks on EMR Serverless StarRocks best practices

Last Updated:Oct 09, 2026

DataWorks supports StarRocks data sources. Connecting to EMR Serverless StarRocks through a data source enables Data Integration, development, analytics, and DataService capabilities. This topic describes the procedure in DataWorks.

Background

About EMR Serverless StarRocks

StarRocks is a next-generation, high-performance MPP (massively parallel processing) database for fast, unified analytics.

EMR Serverless StarRocks is a fully managed Alibaba Cloud service based on open-source StarRocks. Use it to create and manage StarRocks instances and data. As a MySQL protocol-compatible OLAP engine, StarRocks offers high performance and models for multidimensional OLAP, data lake analytics, highly concurrent queries, and real-time analytics.

Important
  • Use Data Studio (New Version) workspaces can use Associate an EMR Serverless StarRocks computing resource to generate a data source of the same name, without creating one through the steps in this topic.

  • Workspaces that have not Use Data Studio (New Version) must create a StarRocks data source as described here to use StarRocks for development in DataWorks.

About DataWorks on EMR Serverless StarRocks

DataWorks is an end-to-end big data development and governance platform. Through a data source connection to EMR Serverless StarRocks, it supports integration and periodic scheduling of StarRocks jobs. Combined with the engine’s fast analytics and DataService capabilities, DataWorks helps you use StarRocks in many business scenarios.

Click to view core concepts and DataWorks modules

The following table summarizes DataWorks modules and concepts relevant to EMR Serverless StarRocks.

Concept or module

Description

Reference

Resource group

Use a DataWorks resource group to run tasks.

Data source

Create a data source to use it in DataWorks. For EMR Serverless StarRocks, create a StarRocks-type data source to develop and run tasks.

For data source details, see StarRocks data source.

Data Integration

DataWorks provides the Data Integration module to synchronize data across many data sources and scenarios.

For Data Integration details, see Data Integration.

Data Development and Operation Center

DataWorks provides the Data Development and Operation Center let you develop and debug tasks, then commit and publish them for periodic execution.

Data Analysis

Data Analysis in DataWorks lets you explore, edit, and share data online.

Data Analysis

DataService Studio

DataService Studio is a flexible, lightweight, secure, and reliable platform for building data APIs. It provides data services and sharing for individuals, teams, and businesses and manages APIs for internal and external use.

DataService Studio

Data Map

Data Map is an enterprise data catalog built on metadata. It provides global search, metadata details, data preview, lineage, and category management to help you find, understand, and use data.

Overview

Prerequisites

  • Activate DataWorks and create a workspace. For the procedure, see Purchase guide.

  • Purchase a resource group, bind it to the workspace, and configure its network. For details, see Resource Group Overview.

  • Create an EMR Serverless StarRocks instance. For the procedure, see Get started with a storage-compute integrated instance.

    Note

    After creating the instance, view its details in the EMR console and use EMR StarRocks Manager to connect and inspect databases and tables.

  • Add the DataWorks resource group IP addresses to the EMR Serverless StarRocks instance allowlist.

    To configure the instance allowlists:

    On the EMR Serverless StarRocks instance Instance Details page, click Security Group ID and then Private Network Allowlist to configure the private network allowlist. In the FE Details section, click Public Address and then Public Network Allowlist to configure the public network allowlist.

Create a data source

To use EMR Serverless StarRocks in DataWorks, first create a StarRocks data source connected to the existing database. Then use it in DataWorks modules.

Important
  • Use Data Studio (New Version) workspaces can use Associate an EMR Serverless StarRocks computing resource to generate a data source of the same name without using the following steps.

  • Workspaces that have not Use Data Studio (New Version) must create a StarRocks data source as described here to use StarRocks for development in DataWorks.

For data source details, see StarRocks data source. The data source entry point and key settings are:

    1. Log on to the DataWorks console. In the target region, click More > Management Center in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Management Center.

    2. On the Workspace Management page, click Data Sources in the left-side navigation pane to open the data source page.

  1. Click Add Data Source. Configure the following key settings; other parameters can retain their defaults.

    Choose how to create the data source based on connectivity between the StarRocks instance and DataWorks resource group. For connectivity options, see Overview.

    Private network connection

    Key parameter

    Description

    Configuration mode

    Select ApsaraDB for RDS.

    Alibaba Cloud Account

    • If the instance and DataWorks are under the same account, select Current Alibaba Cloud Account.

    • If the instance is under another Alibaba Cloud account, select Another Alibaba Cloud Account. After selecting Another Alibaba Cloud Account, also configure UID of Alibaba Cloud Account and RAM Role. For more about the other party’s RAM role, see Cross-account authorization (RDS, Hive, or Kafka).

    Region

    Select the region of the EMR Serverless StarRocks instance.

    Instance

    Select the specific StarRocks instance of the Serverless edition.

    Database Name

    Name of the database to connect. Use EMR StarRocks Manager and check Metadata Management after connecting to the instance.

    Username/Password

    Instance username and password.

    An admin user is created by default when the StarRocks instance is created. Its password is set at instance creation.

    Connection Configuration

    Test connectivity between the data source and purchased resource group. The status Connectable indicates that the network connection is available.

    Public network connection

    Key parameter

    Description

    Configuration mode

    Select User-created Data Store with Public IP Addresses.

    Host Address/IP Address

    The FE Public network address.

    public address format is fe-c-<instance-id>.starrocks.aliyuncs.com, with HTTP port 8030. Click Public Network Allowlist to configure an access allowlist, or Disable Public Access to stop public access.

    Port

    The FE Query port (9030 by default).

    Load URL

    StarRocks FE addresses are used for Stream Load. You can specify multiple FE addresses in the form FE public address:FE HTTP port, separated by commas.

    Database Name

    Name of the database to connect. Use EMR StarRocks Manager and check Metadata Management after connecting to the instance.

    Username/Password

    Instance username and password.

    An admin user is created by default when the StarRocks instance is created. Its password is set at instance creation.

    Connection Configuration

    Test connectivity between the data source and purchased resource group. The status Connectable indicates that the network connection is available.

Data Integration

DataWorks can synchronize MySQL, Hive, Kafka, OSS, HDFS, and other sources into EMR Serverless StarRocks tables. The following example shows how to batch synchronize MySQL data to such a table.

Note

For more StarRocks synchronization task settings, see StarRocks data source.

  1. Log on to the DataWorks console. In the target region, click Data Development and O&M > Data Development in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.

  2. Create a batch synchronization node. Set MySQL as the source and StarRocks as the destination.

    In the left navigation tree, expand Data Integration and click Create Node > Batch Synchronization. In the Network and Resource Configuration step, select a resource group and click Test Connectivity for both source and destination. Confirm that both show Connected.

  3. Test connectivity from the selected resource group to the source and destination data sources.

  4. Set a scheduling cycle, then commit and publish the node for periodic runs.

    After debugging, click Scheduling in the sidebar. Configure the scheduling cycle, rerun policy, other scheduling parameters, and resource group. Then click Submission and Deploy.

Development and scheduled operations

For EMR Serverless StarRocks jobs that require periodic scheduling, create a StarRocks node in DataStudio and select the connected data source. Write the StarRocks SQL and set a scheduling cycle. Key steps:

  1. Log on to the DataWorks console. In the target region, click Data Development and O&M > Data Development in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.

  2. Create a StarRocks node in DataStudio and select the connected StarRocks data source to write SQL.

    Right-click Database Nodes in a business process and select Create Node > StarRocks. In the SQL editor, select the data source and write SQL, such as SELECT * FROM testdb.testtb;.

  3. To debug StarRocks SQL in DataStudio, select the SQL statement, click Run, choose a resource group, and debug it.

  4. Set a scheduling cycle, then commit and publish the node for periodic runs.

    After debugging, click Scheduling in the sidebar. Configure the scheduling cycle, rerun policy, other scheduling parameters, and resource group. Then click Submission and Deploy.

Data Analysis

Use Data Analysis to analyze EMR Serverless StarRocks table data quickly:

  1. Go to the SQL Query page.

    Log on to the DataWorks console. In the target region, click Data Analysis and Service > DataAnalysis in the left-side navigation pane. Click Go to DataAnalysis to open the Data Analysis page. In the left-side navigation pane, click SQL Query.

  2. Click the image icon in the left sidebar, then click More > System Management to open the System Management page. Set the StarRocks query resource group to the group used by the task.

    In SQL Query, under Resource group settings, the StarRocks engine uses the public scheduling resource group by default. Change it to the target resource group and click Save.

  3. Return to SQL Query, switch the engine to StarRocks in the upper-right corner, select a data source, and run a query to analyze data.

    In the settings pane, also select a workspace and click OK.

DataService Studio

DataService Studio can generate APIs from StarRocks data sources.

  1. Log on to the DataWorks console. In the target region, click Data Analysis and Service > DataService Studio in the left-side navigation pane. Select a workspace from the drop-down list and click Go to DataService Studio.

  2. Generate an API and configure its parameters.

    DataService Studio offers wizard and script modes. Script mode can generate request and response parameters from SQL. The following wizard example shows key steps.

    In the left navigation tree, expand API under the business process, right-click, and select Create API > Generate API to open wizard configuration.

    Select StarRocks as the data source type, the created StarRocks data source and table, then configure request and response parameters.

  3. Click Service Resource Group in the right sidebar and select an exclusive resource group for DataService Studio.

    In the Exclusive Service Resource Group list, select a group such as test, and set API Gateway Instance Type to Shared Instance and , to Timeout to Maximum Rows per Request to 10000 rows.

  4. After testing the API, commit and publish it.

Data Map

Data Map supports metadata collection, search, and table details for StarRocks.

Metadata collection

  1. Log on to the DataWorks console. In the target region, click Data Governance > Data Map in the left-side navigation pane. On the page that appears, click Go to Data Map.

  2. Click the image icon in the left navigation pane, then click Management.

  3. Switch to the Data Sources for Which No Crawler Is Created tab. In the Operation column, click Metadata Collection.

  4. ConfigureResource Group Name and Test Connectivity and set Collection Plan. Click Confirm to finish.

    Note
    • For metadata collection details, see Metadata collection.

    • This task can run only on a Serverless resource group.

Search

  1. Log on to the DataWorks console. In the target region, click Data Governance > Data Map in the left-side navigation pane. On the page that appears, click Go to Data Map.

  2. Click the image icon. Select the StarRocks data source, then search for tables across data source types at the top.

    Note

    For search details, see Metadata retrieval.

Table details

  1. Log on to the DataWorks console. In the target region, click Data Governance > Data Map in the left-side navigation pane. On the page that appears, click Go to Data Map.

  2. Find a table in Data Map or search and click its name to open details.

  3. On the details page, view Table Basic Information and Technical Information and Business Information and Details and Output information and Lineage Information other details.

    Note
    • For table details, see Metadata retrieval.

    • Metadata and lineage analysis can be enabled for StarRocks Serverless clusters version 3.1.13, 3.2.9, and later. For configuration, see View lineage.

    The table details page also has a Usage tab. Details contains the Field Information and Change Records subtabs. The Field Information table shows field names, data types, and descriptions.