DataWorks integrates with E-MapReduce (EMR) for data warehouse development and analysis. This tutorial walks you through a user profile analysis case study that covers Data Integration, Data Studio, and Operation Center.
Experiment introduction
Effective business strategies require basic profile data of website users, including geographical and social attributes derived from user activities. By analyzing profile data by time and location, you can refine website traffic operations. DataWorks and EMR together support end-to-end data synchronization, processing, management, and consumption.
Read Experiment overview to understand the full process of the user profile analysis experiment before you start this tutorial.
Procedure
-
Step 1: Prepare the environment
Create an EMR cluster and a DataWorks workspace, and configure the environment.
-
Step 2: Synchronize data
Configure a data synchronization task in DataWorks to synchronize user information and website access logs to an Object Storage Service (OSS) data source, and create tables by using EMR Hive nodes to query the synchronized data.
-
Step 3: Process data
Use EMR Hive nodes in DataWorks to process the user information and access log data synchronized to OSS, and generate user profile results.
-
Step 4: Configure a monitor
Configure a monitor in DataWorks Data Quality for the dwd_log_info_di_emr table generated from the processed data.