Quick start with a Shared-data instance - E-MapReduce - Alibaba Cloud Documentation Center

Prerequisites

Register an Alibaba Cloud account and complete identity verification.
As a RAM user, you must have the AliyunEMRStarRocksFullAccess system policy. For more information, see Grant permissions to a RAM user.

Note
The AliyunEMRStarRocksFullAccess system policy is required to create and manage StarRocks instances.

Precautions

You are responsible for managing and configuring the runtime environment for your code.

Procedure

Step 1: Create a Shared-data StarRocks instance

Go to the E-MapReduce Serverless StarRocks instance list page.
1. Log on to the E-MapReduce console.
2. In the navigation pane on the left, choose EMR Serverless > StarRocks.
3. In the top menu bar, select the required region.
On the Instances page, click Create Instance.

On the E-MapReduce Serverless StarRocks page, configure the instance.

Parameter	Example	Description
Product Type	pay-as-you-go	Select Pay-as-you-go. For billing details, see pay-as-you-go.
Region	China (Beijing)	The physical location of the instance. Important You cannot change the region after the instance is created. Select the region with care.
Network and zone	vpc_Hangzhou/vpc-bp1f4epmkvncimpgs** Zone I vsw_i/vsw-bp1e2f5fhaplp0g6p**	Select a virtual private cloud (VPC), a zone, and the corresponding vSwitch. VPC: An isolated network environment that you define in Alibaba Cloud. Select an existing VPC, or click Create a VPC to go to the VPC console and create one. For more information, see Create and manage a VPC. Note When you create a VPC, you must select an IPv4 CIDR block from one of the following three private network ranges defined in RFC 1918: `10.0.0.0/8` (10.0.0.0 to 10.255.255.255) `172.16.0.0/12` (172.16.0.0 to 172.31.255.255) `192.168.0.0/16` (192.168.0.0 to 192.168.255.255) If your Serverless StarRocks instance needs to access the Internet (for example, to import data or query foreign tables), make sure its VPC has Internet access. You can deploy an Internet NAT gateway in the VPC and enable the SNAT feature. For more information, see Use the SNAT feature of an Internet NAT gateway to access the Internet. Zone: The zone where the instance is located. vSwitch: A vSwitch is a basic network module of a VPC that connects different cloud resources. Select an existing vSwitch, or click . Create vSwitch to go to the VPC console and create one. For more information, see Create and manage a vSwitch.
Instance Type	Shared-data	Suited for workloads with lower query performance requirements, such as OLAP multidimensional analysis, data lake analytics, federated queries on external tables, real-time analytics, and data warehousing.
Instance Edition	Standard Edition	Supports Basic Edition and Standard Edition. For more information, see Instance edition overview. Note Basic Edition is available only in the China (Beijing), China (Shanghai), China (Shenzhen), and China (Hangzhou) regions.
Kernel Version	3.3	The StarRocks community version number.
FE Specifications	Specification type: Standard Specifications Compute units (CUs): 8 CU Data disk: ESSD PL1 High availability: Enabled by default Number of nodes: 3 Load balancing: Built-in PrivateZone	Specification type: The specification type of FE nodes varies with the Instance Edition of StarRocks. Basic Edition: Supports Standard Specifications. Standard Edition: Supports Standard Specifications and Memory-optimized Specifications specifications. Compute CUs: Select the number of compute units (CUs). Select the appropriate CU specification as needed. For more information about CU fees, see Billable items. Data Disk: Only PL1 ESSD is supported. The data disk size ranges from 100 GB to 65000 GB, with a step size of 100. For more information about cloud disks, see Enterprise SSDs. HA: Enabled by default. The Standard edition supports high availability. After you enable high availability, the number of StarRocks FE nodes increases from 1 to 3 to reduce the risk of failures. Important We strongly recommend that you enable high availability for production environments. Number of Nodes: The number of FE nodes. The value can be an odd number from 1 to 11. Load balancing: Supports the following methods. Built-in Private Zone: Balances traffic through PrivateZone domain name resolution at no extra cost. Suitable for lightweight or cost-sensitive scenarios. Recommended for non-production environments or services with low load balancing requirements. Load balancing SLB: Provides high-performance load balancing through the SLB service. Recommended for production environments with high performance and reliability requirements. The feature to remove the FE leader from query traffic is available only after SLB is activated. You must activate the SLB service, which incurs extra fees. For more information, see CLB billing overview.
CN Specifications	Specification type: Standard Specifications Compute units (CUs): 8 CU Data disk: ESSD PL1, 200 GB, 1 disk Number of nodes: 3	Specification type: The specification type of CN nodes varies with the Instance Edition of StarRocks. Basic Edition: Supports Standard Specifications. Standard Edition: Supports the following specifications. Standard Specifications: The default and recommended option. 1 CU = 1 CPU core + 4 GiB of memory. This specification uses ESSD for StarRocks storage. Memory-optimized Specifications: 1 RCU = 1 CPU core + 8 GiB of memory. This specification is suitable for memory-intensive scenarios, such as running a large number of complex queries or handling high-concurrency requests. It uses ESSD for StarRocks storage. Network-enhanced Specifications: 1 NCU = 1 CPU core + 4 GiB of memory, offering more than double the network bandwidth of the standard specifications. This specification is suitable for scenarios that involve scanning large amounts of data from external tables. It uses ESSD for StarRocks storage. High-performance Storage: You need to select detailed specifications for this type. It uses a local SSD for StarRocks storage and is ideal for scenarios with stringent I/O performance requirements. High-specification Storage: You need to select detailed specifications for this type. It uses a local HDD for StarRocks storage. This type is ideal for storing very large data volumes at a lower cost, but it has lower I/O performance. Compute CUs: Select the number of compute units (CUs). Select the appropriate CU specification as needed. For more information about CU fees, see Billable items. Data Disk: Supports ESSD PL0, ESSD PL1 Cloud Disk (Recommended), ESSD PL2, ESSD PL3, Elastic temporary disk (standard edition), and Elastic temporary disk (Advanced Edition). For more information about cloud disk and elastic ephemeral disk fees, see Billable items. For more information, see Enterprise SSDs and Elastic ephemeral disks. Note You can enter the required storage capacity, and the system automatically recommends a configuration. If the capacity you select exceeds the recommended threshold, a prompt is displayed to help you adjust for optimal performance. Only Standard Specifications, Memory-optimized Specifications, and Compute enhanced specifications support elastic ephemeral disks. Elastic ephemeral disks have limitations on regions and zones. Number of Nodes: The number of CN nodes. The value can range from 1 to 100.
Instance Name	Enter a custom instance name.	The instance name must be 1 to 64 characters in length and can contain Chinese characters, letters, digits, hyphens (-), and underscores (_).
Administrator	admin	The built-in administrator account for StarRocks management. The default username cannot be changed.
Password and Confirm Password	Enter a custom password.	The password for the built-in admin account of the StarRocks instance. Save this password for future use. If you forget the password, you can reset it. For more information, see How do I reset the password of an instance?

For more information about instance parameters, see Create an instance.

Select the service agreement, click Create Instance, and complete the payment as prompted.

After payment, return to the instance management page. The instance is created successfully when its Status changes to Running.

Step 2: Connect to a StarRocks instance

On the Instances page, find the target instance and click Connect in the Actions column.

You can also connect to a StarRocks instance in other ways.

Connect to the StarRocks instance.

On the New Connection tab, configure the following parameters.

After completing the configuration, click Test Network Connectivity to validate the connection, and then click OK to create it.

Parameter	Example	Description
Region	China (Hangzhou)	Select the region where your StarRocks instance resides.
Instance	StarRocks_Serverless	Select the StarRocks instance you want to connect to.
Name	Connection_Serverless	Enter a custom connection name. The name must be 1 to 64 characters in length and can contain Chinese characters, letters, digits, hyphens (-), and underscores (_).
Username	Enter a value based on your actual needs.	The default username is admin. You can also create other users as needed. For more information, see Manage Users and Data Authorization.
Password	Enter a value based on your requirements.	The password for the specified username.

Click Test Connectivity.
After the connection test is successful, click OK.

You are redirected to the SQL Editor page, where you can run SQL queries. For more information, see Connect to a StarRocks instance by using EMR StarRocks Manager.

Step 3: Run SQL queries

On the Queries page of the SQL Editor, click the icon.
In the New dialog box, click Confirm.

In the new file, enter the following commands. Select all commands and click Run.

/**Create a database**/
CREATE DATABASE IF NOT EXISTS load_test;

/**Use the database**/
USE load_test;

/**Create a table**/
CREATE TABLE insert_wiki_edit
(
    event_time DATETIME,
    channel VARCHAR(32) DEFAULT '',
    user VARCHAR(128) DEFAULT '',
    is_anonymous TINYINT DEFAULT '0',
    is_minor TINYINT DEFAULT '0',
    is_new TINYINT DEFAULT '0',
    is_robot TINYINT DEFAULT '0',
    is_unpatrolled TINYINT DEFAULT '0',
    delta INT SUM DEFAULT '0',
    added INT SUM DEFAULT '0',
    deleted INT SUM DEFAULT '0'
)
AGGREGATE KEY(event_time, channel, user, is_anonymous, is_minor, is_new, is_robot, is_unpatrolled)
PARTITION BY RANGE(event_time)
(
    PARTITION p06 VALUES LESS THAN ('2015-09-12 06:00:00'),
    PARTITION p12 VALUES LESS THAN ('2015-09-12 12:00:00'),
    PARTITION p18 VALUES LESS THAN ('2015-09-12 18:00:00'),
    PARTITION p24 VALUES LESS THAN ('2015-09-13 00:00:00')
)
DISTRIBUTED BY HASH(user) BUCKETS 10
PROPERTIES("replication_num" = "1");

/**Insert data**/
INSERT INTO insert_wiki_edit VALUES("2015-09-12 00:00:00","#en.wikipedia","GELongstreet",0,0,0,0,0,36,36,0),("2015-09-12 00:00:00","#ca.wikipedia","PereBot",0,1,0,1,0,17,17,0);

/**Query data**/
select * from insert_wiki_edit;

The query results are displayed.

Note

Run the SHOW CREATE TABLE load_test.insert_wiki_edit; command and check for the datacache.enable property to verify that the Shared-data instance is active. After creating the database and table, you can find their directories in your OSS bucket.

Run the SHOW CREATE TABLE load_test.insert_wiki_edit; command. The result includes the datacache.enable property, which confirms that storage-compute separation is active. The following example shows the key PROPERTIES section.

DISTRIBUTED BY HASH(user) BUCKETS 10 PROPERTIES ("replication_num" = "1", "datacache.enable" = "true", "storage_volume" = "builtin_storage_volume", "enable_async_write_back" = "false", "enable_persistent_index" = "false", "compression" = "LZ4");

Step 4: View table information

In the new file, run the following command to view the databases:
```
SHOW PROC '/dbs';
```
The result lists five databases: information_schema (DbId=1, TableNum=46), sys (DbId=100, TableNum=6), _starrocks_audit_db_ (DbId=10004, TableNum=1), _statistics_ (DbId=10023, TableNum=8), and load_test (DbId=10120, TableNum=1). The Quota for each is 8,388,608.000 TB, and the LastConsistencyCheckTime column is empty.
Run the following command to view detailed table information:
```
SHOW PROC '/dbs/10120';
```
The result includes columns such as TableId, TableName, IndexNum, PartitionColumnName, PartitionNum, State, Type, ReplicaCount, PartitionType, and StoragePath.

In storage-compute separation mode, a Type of CLOUD_NATIVE identifies a Shared-data table. The StoragePath shows where the table's data is stored.

Step 5: Demonstrate the cache feature

In the navigation pane on the left, choose Diagnostics and Analytics > SQL Tasks.
Find the query and click the query ID.
Click the Execution Details tab.

In the profile execution tree, locate the relevant nodes to find the following metrics: CompressedBytesReadLocalDisk (read from local cache) and CompressedBytesReadRemote (read from remote OSS).

In this example, the insert_wiki_edit_cache table has local caching enabled, and the metrics confirm that the query fully hit the local cache.

In the metrics panel of the CONNECTOR_SCAN node, the value of CompressedBytesReadLocalDisk is 729.52 MB, and the value of CompressedBytesReadRemote is 0.00.

The metrics for the insert_wiki_edit_nocache table show that the query did not hit the local cache and all data was read from remote OSS.

In the CONNECTOR_SCAN node for the insert_wiki_edit_nocache table, CompressedBytesReadLocalDisk is 0.00, and CompressedBytesReadRemote is 804.63 MB, confirming that all data was read from remote OSS.

Step 6: Run a performance test

This example compares query performance between storage-compute separation (with local cache) and storage-compute integration. For a more detailed comparison using the SSB test set, see SSB performance test instructions.

Prepare the data environment.
- Cluster resource configuration: 1 FE (8 CUs) + 3 CNs (Compute: 16 CUs | Storage: 1,000 GB).
- Cluster parameters: Use the default settings. Enable local cache for the storage-compute separation cluster.
- Data volume: 500 GB (sf=500)
Test results.
- Total time for storage-compute integration: 21.586s.
- Total time for storage-compute separation (with local cache enabled on the second execution): 27.364s.
- Total time for storage-compute separation (without local cache enabled): 117.529s.

After running the SSB test with the sh ssb_query.sh ssb script, the results show that with local cache enabled, storage-compute separation performs nearly identically to storage-compute integration and significantly outperforms the configuration without local cache.

SQL	Storage-compute integration	Storage-compute separation (cache enabled)	Storage-compute separation (cache disabled)
Q1.1	0m0.373s	0m0.380s	0m2.080s
Q1.2	0m0.303s	0m0.292s	0m2.141s
Q1.3	0m0.101s	0m0.097s	0m0.144s
Q2.1	0m2.461s	0m2.821s	0m14.401s
Q2.2	0m2.272s	0m2.735s	0m13.048s
Q2.3	0m2.168s	0m2.588s	0m13.957s
Q3.1	0m4.536s	0m4.864s	0m14.810s
Q3.2	0m2.371s	0m2.682s	0m11.292s
Q3.3	0m2.082s	0m2.648s	0m13.651s
Q3.4	0m0.195s	0m0.212s	0m0.572s
Q4.1	0m5.122s	0m5.847s	0m29.576s
Q4.2	0m1.141s	0m1.369s	0m1.465s
Q4.3	0m0.661s	0m0.829s	0m0.792s
Total	21.586s	27.364s	117.529s

(Optional) Step 7: Release the instance

Important

This operation deletes the instance and all its resources. This action cannot be undone. Proceed with caution.

To avoid incurring further charges, release an instance when you no longer need it.

On the Instances page, click Release in the Actions column of the instance.
In the dialog box that appears, click OK.

Contact us

For questions, join our support DingTalk group by searching for the ID 24010016636.

E-MapReduce:Quick start with a Shared-data instance