The data warehouse development feature of DMS allows you to integrate, process, and visualize data, and extract value from data on a one-stop development platform. This topic describes the functionalities of data warehouse development and how to use data warehouse development.

Background information

Enterprises store a large number of valuable data, such as transaction data and customer information, in databases. The enterprises need a solution to easily analyze and process the data and convert the data into business intelligence. The data warehouse development feature of DMS provides such solutions. The data warehouse development feature of DMS uses a database as the compute engine and integrates various tools and services in the database ecosystem, such as Data Transmission Service (DTS) and Data Lake Analytics (DLA). You can use the data warehouse development feature to build, develop, and manage data warehouses.

Functionalities

Functionality Description
A variety of data warehouse engines You can select appropriate database engines based on your enterprise scale, data volume, and timeliness requirements. These database engines include AnalyticDB, DLA, PolarDB, and ApsaraDB RDS. DMS provides the same features for all these database engines.
Two development modes DMS provides the task orchestration and data warehouse development modes to meet different data warehouse development requirements. In the task orchestration mode, you do not need to specialize in data warehousing. You need to only focus on the business logic and know how to create task flows and edit SQL statements. The data warehouse development mode is designed for professional data warehouse developers. This mode provides various features such as theme management, hierarchical management, production publish, multi-person collaboration, data map, and data quality control, some of which are being developed. It offers a professional data warehouse development solution for enterprises.
Offline and real-time data warehouses You can use offline data synchronization and scheduled tasks to develop offline data warehouses in DMS. In addition, DMS is integrated with Alibaba Cloud DTS and AnalyticDB. You can use DTS to synchronize data in real time and use AnalyticDB as the computing engine to build real-time data warehouses. In this way, you can produce and consume data in DMS in real time.
Centralized management of online and offline data Based on the centralized database management and permission control capabilities, DMS centrally manages your online databases such as online transaction processing (OLTP) databases and offline databases such as online analytical processing (OLAP) databases. This ensures the data security of your systems. DMS can also track data lineage and analyze the impacts throughout the data lifecycle.

How to build and use a data warehouse

Procedure

  1. Data warehouse engine selection
    Purchase or select an appropriate database engine, such as PolarDB, ApsaraDB RDS, or AnalyticDB, based on your business needs and use it as the data warehouse engine.
  2. Data integration
    Synchronize the data that you want to analyze from an external data store such as a transaction database to the data warehouse.
  3. Data development
    Synchronize the data that you want to analyze from an external data store such as a transaction database to the data warehouse.
  4. Data application
    Provide consumption channels for processed data. For example, you can create data-related API operations or visualize the data in charts.
  5. Data governance
    Manage data throughout the development process of the data warehouse. For example, you can monitor the data quality, track data lineage, and analyze data impacts.