All Products
Search
Document Center

Migrate the advertising business data of a developing game company from HBase to Lindorm

Last Updated: Jul 14, 2021

Solution highlights: After the advertising business data is migrated to ApsaraDB for Lindorm (Lindorm), the advertising system performance is improved by at least 30 times. The average response time is reduced from hundreds of milliseconds to 20 milliseconds.

Customer feedback

The performance of our self-managed cluster is not stable and we cannot obtain fixed values to evaluate the performance. After the data is migrated to Lindorm, the system stability is significantly improved and the performance is improved by at least 30 times. The system still has room for stability improvement and performance improvement.

Challenges

  • The self-managed cluster is a hybrid cluster that provides multiple big data services. The services are affected by each other. This results in poor request stability and high response latency.

  • The costs of the self-managed cluster are high. The cluster uses more than thirty 32-core 128 GB Elastic Compute Service (ECS) instances.

  • The data size reaches more than 24 TB and the storage overhead of the self-managed cluster continues to rise.

Solutions

  • The company migrated the advertising business data from the self-managed hybrid cluster to an independent Lindorm cluster. The average request response time (RT) when the self-managed hybrid cluster was used was in a range from 500 to 800 milliseconds. After the data was migrated to Lindorm, the RT dropped to 20 milliseconds. This significantly improves the throughput of the advertising system.

  • The Lindorm expert team provided suggestions on how to optimize the business application. Lindorm experts suggested that the game company enable the data compression feature. If the feature is enabled, the data compression ratio can be up to 13:1. The storage overhead can be reduced to less than 5% of the storage overhead when HBase is used.

  • The garbage collection (GC) optimization feature and the triplicate architecture of Lindorm Log Consensus (LLC) provided by Lindorm help the advertising system significantly reduce the maximum response latency for 99.9% of the requests.

  • Lindorm provides a high-availability solution. The primary and secondary clusters are deployed in the same region. When the primary cluster fails, the workloads on the failed primary cluster are immediately switched to a secondary node.

Garbage collection (GC) optimization V2 provided by Lindorm

Benefits

  • After the advertising data is migrated to Lindorm, the system performance is significantly improved and the storage overhead is reduced. This significantly improves the business throughput.

  • After the data is migrated to Lindorm, the primary and secondary clusters can be deployed in the same region. This way, the business system has the capability of cross-zone disaster recovery. This helps ensure business continuity even in extreme cases.

  • After the data is migrated from the self-managed hybrid cluster to Lindorm, Alibaba Cloud provides service level agreement (SLA) guarantees and a complete portfolio of data management services to free the game company from O&M. Alibaba Cloud also offers a professional Lindorm expert team to provide free technical support. This helps the company focus on their business development.