This topic describes the reasons and procedure for migrating the metadata of an E-MapReduce (EMR) cluster from the metadatabase of the cluster to Data Lake Formation (DLF).
Reasons for metadata migration
- An on-premises MySQL database is deployed in standalone mode, which cannot ensure high service availability and is prone to service interruption.
- The Unified Metabases storage type will be gradually discontinued in the future. You need to use the DLF Unified Metadata storage type that is provided in the new EMR console to store metadata.
DLF provides an O&M-free, highly available, and high-performance unified metadata service. The metadata service is compatible with multiple versions of a Hive metastore, can be seamlessly integrated with open source compute engines in EMR, and supports data profiling. In addition, DLF also provides features such as data exploration, data lake management, and data permission management, and can be seamlessly integrated with other Alibaba Cloud computing services, such as MaxCompute, Databricks DataInsight (DDI), and Hologres. This allows you to use DLF in a wide range of computing scenarios. For more information about DLF, see Overview.
Migration process
The Alibaba Cloud EMR and DLF teams provide a comprehensive migration process and technical support in the entire migration process to ensure fast and smooth data migration.
Phase | Operation | Participant | Estimated duration |
---|---|---|---|
1. Preparations |
| Alibaba Cloud EMR team and customers | 2 hours |
2. Migration |
| Alibaba Cloud EMR team and customers | 30 minutes |
3. Check | Observe the running of the tasks for one week or a longer period of time and view the results.
| Alibaba Cloud EMR team and customers | 1 week |
(Optional) 4. Rollback |
| Alibaba Cloud EMR team and customers | 30 minutes |
Contact technical support
The Alibaba Cloud EMR and DLF teams provide a comprehensive migration process and technical support in the entire migration process. If you want to migrate EMR metadata to DLF, join the DingTalk group for migration by searching for the DingTalk group number 33719678. Then, engineers will contact you to formulate a migration plan.