E-MapReduce:Using the new version of DataWorks

Case introduction

To develop effective business strategies, you can build user profiles from website activity data, such as geographical and social attributes, and analyze the profiles by time and location for fine-grained traffic operations. This case uses DataWorks with EMR Serverless Spark for data synchronization, processing, management, and consumption.

Data Studio

This tutorial requires the new Data Studio in DataWorks. To enable it:

• When you create a workspace, select Use Data Studio (New Version).

• To upgrade from an older version, click Upgrading at the top of the page.

• After February 18, 2025, the new Data Studio is enabled by default for accounts that create their first workspace in these regions:

  China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Thailand (Bangkok), Germany (Frankfurt), UK (London), US (Silicon Valley), US (Virginia)

Procedure

1. Step 1: Prepare environments

   Create the required EMR Serverless Spark and DataWorks workspaces, and configure resource group and network settings.

2. Step 2: Synchronize data

   Configure a DataWorks synchronization task to sync the basic user information and website access logs of the users provided in the tutorial to a Spark computing resource, then query the data.

3. Step 3: Process data

   Process the synchronized basic user information and access log data with EMR Spark SQL nodes in DataWorks to generate user profile data.

4. Step 4: Monitor data quality

   Set up table monitors to detect and intercept dirty data before it affects downstream processes.

5. Step 5: Manage data

   After the analysis completes, view the generated data tables in EMR and their lineages in Data Map.

6. Step 6: Consume data

   Use the data analytics module to visualize the processed data and extract key insights into business trends.