Create a logical model: fact table
A fact table stores large volumes of factual data or detailed values that reflect business activities. It is a table generated by aggregating data based on specific dimensions. For example, to analyze product sales, you can create a sales fact table that stores product dimensions (as foreign keys), time dimensions (as foreign keys), and total sales. This topic describes how to create a fact table.
The data modeling feature in DataWorks follows the Kimball dimensional modeling methodology. Design and create dimension tables, fact tables, aggregate tables, and application tables, publish models to development engines, and reverse-model existing physical tables into logical models.
Modeling perspective
Dimensional modeling organizes model tables into three levels: Common layer, Application layer, and Uncategorized. The Common layer is used to build reusable unified metrics, dimensions, and detailed fact data, and supports management from either data domain or business category perspectives. The Application layer addresses business-specific statistical needs and supports only the business category perspective. After selecting a level, you can create and manage model tables in the corresponding directory tree.
Prerequisites
-
You have created data layers. Data layers allow you to organize tables with different functions into unified layers to simplify discovery and use. A fact table is typically stored in the DWD layer. You can also store it in other data layers based on your business needs. For more information, see Define data warehouse layers.
-
You have created a business process. Creating a business process helps determine the specific business activity data that the fact table needs to store. For more information, see Business Process.
Feature overview
Sort and analyze data that is generated in each business process, and store the data in fact tables as fields. For example, you can create a fact table for the business process of placing an order, and record the following information as fields in the fact table: order ID, order creation time, commodity ID, number of commodities, and sales amount. You can deploy the fact tables in a data warehouse and perform ETL operations to summarize and store data in the format defined in the fact table. This allows business personnel to access the data for subsequent data analysis.
As shown in the preceding figure:
-
When creating a fact table:
-
You can specify the data layer in the data warehouse where the fact table's data is stored for dimensional modeling and analysis. Typically, a fact table is stored in the DWD layer.
-
You can associate the fact table with the business category and business process it describes. This makes it easier to find all fact tables related to a specific category or process.
-
-
After creating the fact table, you can add fields, configure associations and partitions, and apply unified field standards to ensure data consistency across the entire data domain.
-
After configuring the fact table, you can publish and materialize it to a compute engine. You can then use the materialized table for data analysis in the compute engine.
Create a fact table
Log on to the DataWorks console. In the target region, click in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Modeling.
-
In the top navigation bar of the Data Modeling page, click Dimensional Modeling to go to the Dimensional Modeling page.
-
Create a fact table.
-
On the Dimensional Modeling page, hover over the
icon and choose . -
Configure the basic information for the fact table.
Parameter
Description
Data Layer
Select a data layer. Only the DWD of the common layer is supported. The fact table's data is stored in this data layer. For more information, see Define data warehouse layers.
Business Category
Select an existing business category. For more information, see Business Category.
Business Process
Select an existing business process. For more information, see Business Process.
Storage Policy
Defines how data is stored in the fact table. Options include Synchronize Incremental Data Daily and Synchronize All Data Daily.
Naming Rule
Select a configured checker to verify that the table name complies with the specified naming rules. For more information, see Configure data warehouse layer checkers and Use checkers.
Table Name
The internal name of the table. If you select a checker for the Table naming rule parameter, the name must comply with the defined rules.
Table Display Name
The display name of the table.
Lifecycle
The retention period of the fact table in days. The maximum value is 36000.
Owner
The owner of the fact table. By default, this is the user who creates the table.
Description
The description of the fact table.
-
-
After you configure the parameters, click Save. The new table appears in the left-side directory tree.
Add table fields
After you create the model, you must add fields to it.
You can add fields in Shortcut Mode or Script Mode. In Shortcut Mode, you can select Import from Table/View to import fields from an existing physical table or view in a compute engine. Search for and select an existing physical table or view from the Search for existing table/view drop-down list to import its fields.
Shortcut mode
Currently, you can import fields only from tables or views in MaxCompute, Hologres, and EMR Hive.
-
In Shortcut Mode, click Expand next to Import from Table/View.
-
In the Search for existing table/view text box, enter a name to find the corresponding table or view. Then, choose to import all or some of its fields.
Note-
Fuzzy search is supported. You can enter a keyword to find all tables or views that contain the keyword in their names.
-
You can search for tables only in the production environment. Tables in the development environment cannot be searched.
-
Click the
icon to import all fields. -
Click the
icon to import specific fields.
-
-
If you choose to import specific fields, a dialog box appears. Select the desired fields and click Import.
-
If an imported field has an empty Field Display Name, you can use the field description as its display name as prompted on the page.
Script mode
You can also use FML statements to create fields, associations, and partitions. For more information, see Modeling in script mode.
Script Mode provides a code-based editing experience. When you click Script Mode, a dialog box appears, showing auto-generated modeling language based on your model's configuration. You can modify the code as needed, and then click OK.
-- After a model is published to create a physical table (before, during, or after approval), the table name cannot be modified.
CREATE DIM TABLE dim_ec_pub_department_df ALIAS 'Department dimension table'
(
id ALIAS 'id' STRING COMMENT 'id',
gmt_create ALIAS 'Creation time' TIMESTAMP COMMENT 'Creation time',
gmt_modified ALIAS 'Modification time' TIMESTAMP COMMENT 'Modification time',
name ALIAS 'Department name' STRING COMMENT 'Department name',
parent_id ALIAS 'Parent department ID' STRING COMMENT 'Parent department ID',
`level` ALIAS 'Level; 0: Group; 1: Subsidiary; 2: Business unit; 3: Department.' BIGINT COMMENT 'Level; 0: Group; 1: Subsidiary; 2: Business unit; 3: Department.',
ds ALIAS 'Business date, yyyymmdd' STRING COMMENT 'Business date, yyyymmdd'
)
COMMENT 'Department dimension table'
WITH('life_cycle'='1000');
Configure field information
After you add fields to the model, you can configure their attributes based on your business needs, such as Associated Field, Redundant Field, and Associated Granularity/Metric.
-
Set field attributes.
By default, the page displays basic attributes such as Field Name, Type, Field Display Name, Description, Primary Key, Not Null, and Actions. In the upper-right corner of the field list, click Field Display Settings to select the attributes you want to display and modify them as needed.
-
Set the Field Standard to Associate and Lookup Table to Associate for the fields. This helps standardize the content and range of values for the added fields.
-
Field Standard to Associate: Centrally manages data that has the same meaning but different field names, and defines value ranges, units of measure, and more.
-
Lookup Table to Associate: Defines the available values and value range for a field standard.
-
-
Set the Redundant Field.
In the Actions column for the desired field, click Redundant Field to configure its associated fields.
In a traditional star schema, dimensions are stored in dimension tables and are accessed through foreign keys in the fact table to reduce storage consumption. However, to improve downstream query performance and simplify data access, it is common to denormalize frequently used dimension attributes directly into the fact table. This reduces the number of required table joins. For example, the 'order creation detail table' denormalizes attributes from the 'shipping address dimension table', such as 'shipping address' and 'consignee phone number'.
In the Redundant field dialog box, select an Associated table/view name from MaxCompute, select the desired Associated fields from the list, and then click Save to complete the configuration.
-
After configuring the fields, click Save in the upper-left corner.
Next steps
After you create the fact table, you still need to configure its fields, associations, and partitions. Then, you can publish the table to the target environment. For more information, see Materialize a logical model.