Alibaba's data framework recommends structuring the data warehouse into three layers, from bottom to top: the Operational Data Store (ODS) layer, the Common Data Model (CDM) layer, and the Application Data Service (ADS) layer.
The following diagram illustrates the data warehouse layers and the purpose of each layer.
Operational Data Store (ODS) layer: This layer stores raw data from source systems. Its structure mirrors that of the source system and serves as the data staging area for the data warehouse. The primary purpose of the ODS layer is to ingest foundational data into MaxCompute and record its historical changes.
Common Data Model (CDM) layer: This layer, which processes data from the ODS layer, consists of the DIM, DWD, and DWS sub-layers. Its main purpose is to process and integrate data, establish conformed dimensions, build reusable detailed fact tables for analysis and statistics, and aggregate metrics at common granularities.
DIM layer: Based on the principles of dimensional modeling, this layer establishes conformed dimensions for the entire enterprise. This reduces the risk of inconsistent metric definitions and algorithms.
Tables in the DIM layer are often called logical dimension tables, where each dimension typically corresponds to a single logical dimension table.
DWS layer: Driven by analytical subjects and the metric requirements of upstream applications and products, this layer builds summary fact tables at a common granularity, materializing these models as wide tables. It also establishes statistical metrics with consistent naming conventions and definitions, provides shared metrics for upstream applications, and creates summary wide tables.
Tables in the DWS layer, known as summary logical tables, store derived metrics.
DWD layer: Driven by specific business processes, this layer builds detailed fact tables at the finest granularity. Depending on data usage patterns, important dimension attributes can be denormalized into the fact tables to create wide tables.
Tables in the DWD layer are often called logical fact tables.
Application Data Service (ADS) layer: This layer stores customized statistical metrics for data products and applications, generated by processing data from the CDM and ODS layers.
In this data architecture, the ODS layer is divided into three parts: a data staging area, an offline data area, and a near-real-time data area. The overall data architecture is illustrated in the diagram below.
In this tutorial, the DataWorks data integration service synchronizes data from a transaction data system to the ODS layer of the data warehouse. After the Data Development process generates wide fact tables, it aggregates the data by dimensions such as product and region.
The overall data flow is illustrated in the diagram below. MaxCompute performs the ETL (extract, transform, load) processing from the ODS layer to the DIM layer. After this process completes, the data is synchronized across all relevant storage systems. Data from the ODS and DWD layers is published to data middleware for subscription by downstream services. Data from the DWS and ADS layers is typically stored in online storage systems, which downstream applications access via API calls.