This topic describes the solution for Neusoft Group to explain how ApsaraDB for Lindorm (Lindorm) makes O&M smart in the digital age. The business of Neusoft Group (Neusoft) relates to multiple industries, such as telecommunications, public services, automobiles, and healthcare.
- Lindorm can be used to store large amounts of O&M data from the entire mobile-to-cloud business link.
- Lindorm helps Neusoft analyze heterogeneous monitoring data such as metrics and logs from multiple sources in a combined manner.
- Lindorm provides a storage solution that delivers 99.99% data availability at a low cost.
Lindorm is a latest cloud native multi-model database service. Neusoft uses Lindorm to drive an intelligent O&M solution for the digital information processing systems of public service sectors and enterprises. Lindorm provides Neusoft with an all-in-one storage solution for volumetric real-time heterogeneous monitoring data. Therefore, Neusoft can analyze heterogeneous data such as metrics, logs, code traces, and network packets in a combined manner. Lindorm also ensures high data reliability and allows Neusoft to store all real-time monitoring data in the cloud. Lindorm provides powerful big data analytics capabilities for the digital O&M team of public service sectors and enterprises. Lindorm improves the performance of the digital systems for public service sectors and enterprises, and improves the digital experience of users who use online services. Lindorm also reduces the storage and management costs for low-value monitoring data.
Neusoft was founded in 1991. It is the first listed software company in China. The company has been committed to creating better lifestyles and promoting social development by leveraging innovative software. Neusoft provide software products and services for enterprises in the fields of smart cities, healthcare, smart cars, and software engineering. Neusoft has nearly 20,000 employees worldwide. It has established R&D centers, sales networks, and service networks in more than 60 cities in China. The company also has subsidiaries in regions outside China, such as the United States, Japan, and Europe. Neusoft has been titled as one of top 100 global software enterprises by PricewaterhouseCoopers for four consecutive times. Neusoft has also gained multiple reputations, such as one of the top 20 most globally competitive Chinese companies, one of the top 50 global challengers in China, the most appreciated knowledge-based enterprise in Asia, and the best employer in the Asia-Pacific region.
Business requirements and challenges
Internet-based information services rapidly develop. This increases the demand on the O&M systems of digital applications. The O&M systems must maintain the stability of the digital systems for public service sectors and enterprises and ensure superior digital experience for users. In Internet-based service scenarios, diverse types of data, such as time series metrics, logs, and code traces, are collected and monitored. As a result, the amount of O&M and monitoring data increases sharply. The O&M system must concurrently collect the monitoring data in real time. The existing O&M systems that use single-model engines, such as Round Robin Database (RRD), OpenTSDB, or Elasticsearch, cannot meet the requirements for storing and retrieving the low-value monitoring data.
In a Forrest report, 57% of enterprises have at least one critical error every week. A critical error is an error that affects the application performance and availability. The Forrest report also shows that 28% of enterprises have errors that occur every day. Enterprises to which applications are essential cannot tolerant these errors. More than half of the enterprises in the report believe that application performance problems reduce the work efficiency and waste the time of their users and IT departments. The Forrest report also shows that 42% of the enterprises believe that application performance problems directly reduce their income. In general, inefficient O&M is caused because of the following reasons:
- New demands drive the monitoring data of IT systems to surge. Mobile smart terminals are popularized and mobile applications are used in all aspects of work and life. This causes the number of applications to soar. Products increasingly become more intelligent and interconnected. The complexity and the number of IT systems and applications built by enterprises for their customers, partners and internal employees are increasing. To ensure the system availability, the types of data that the monitoring systems need to collect and the amount of data that needs to be stored increase too. The existing monitoring systems that use single-model engines, such as an RRD engine or a relational database engine, cannot meet the requirements for storing and retrieving monitoring data.
The digitization of products leads to more complex application structures and more diverse types of monitoring data. New technologies such as hybrid cloud, data analytics, IoT, Internet of Vehicles (IoV), and body area networks are continuously evolved. This leads to more complex application structures and makes the maintenance of the application performance more difficult. The Forrest statistics show that 52% of O&M teams scatter their investment in monitoring and management tools because of problems occurred. Single-model storage engines have limited features due to limited storage capabilities. These storage engines can provide only scattered and fragmented storage for data. In the future, O&M will be focused around a core application. Application upgrades and technological evolution will put forward higher requirements for O&M systems. The single-model storage engines cannot meet the new requirements. The storage and analysis capabilities of the existing IT O&Ms system must be upgraded.
The complexity of applications and the amount of monitoring data continuously increase. However, the existing O&M systems have limited capabilities to store and retrieve data. Neusoft faced problems and challenges on its IT O&M system for IoT scenarios and Internet services. The existing O&M system was difficult to implement combined analysis on multi-model data. The O&M system provided weak write concurrency and was not able to efficiently handle concurrent write requests from a large number of data collection terminals. The storage of monitoring data that is of a large size but of low value was not cost-effective. The self-managed data storage clusters that were built based on open source software had multiple problems, such as unstable performance and high O&M costs. Neusoft urgently needed a new big data storage engine to upgrade the O&M system.
Neusoft analyzed the pain points of the storage and analysis of volumetric monitoring data in IT O&M for the latest digital information systems in industries such as telecommunications, public sectors, and transportation. Then, Neusoft decided to use Lindorm to collect and store all of its O&M data. Lindorm is a cloud native multi-model database service provided by Alibaba Cloud. Lindorm can store multiple types of data for Neusoft, such as the digital footprints and digital user experience data generated when terminals such as mobile phones and tablet computers from multiple regions in China are connected to the application service systems. It can also store packet capture analysis data on the networks between the cloud and data centers, code trace data generated when applications run, and log data.
Neusoft analyzed the requirements of different companies such as telecommunication companies and automobile companies. Then, Neusoft decided to use Lindorm as the core service to improve the performance of the smart application O&M product RealSight APM and the cloud management product SaCa Aclome. Lindorm can be used to store the monitoring data for Neusoft in real time and at a low cost. Lindorm also provides the capabilities of high-throughput and high-concurrency. The built-in multi-model data processing engines of Lindorm meet the data storage and analysis requirements for IT O&M and monitoring . Neusoft uses Lindorm to build a new generation of smart IT O&M system for the Internet-based digital information systems of public service sectors and enterprises. In this architecture, Lindorm is the core engine for O&M data storage and retrieval. It can store all the heterogeneous monitoring data collected in real time from data sources such as mobile terminals, network sniffers, business monitoring agents, and log collectors. Lindorm allows Neusoft to concurrently write the monitoring data to databases at a high throughput. Users can retrieve large amounts of data in different O&M scenarios by using different methods such as real-time monitoring dashboards, statistical analysis reports, risk positioning, and detection tools. The Lindorm engines can be used to retrieve and analyze volumetric data.
- Use a dashboard to display real-time status.
- Retrieve volumetric application log data.
- Track the behavior of specific users.
- Trace and analyze all failure data.
- Use AI-powered tools to detect exceptions.
- Lindorm provides storage, retrieval, and analysis capabilities for structured, semi-structured, and unstructured data such as integrated metrics, logs, code traces, and network packets. This simplifies development and deployment.
- Lindorm provides low-cost cloud storage for volumetric data. This significantly reduces the total cost of ownership (TCO).
- Lindorm can concurrently write monitoring data to databases at a high throughput. This helps the O&M system monitor applications that have a large number of daily active users.
- Lindorm helps Neusoft monitor, trace, and analyze multi-model heterogeneous data such as real-time logs and metrics. Lindorm also simplifies the process of extracting information from low-value data.
- Lindorm guarantees a 99.99% data availability and reduces the risk of data loss.
- Lindorm can be connected anywhere. This simplifies the network configuration and management.
- Lindorm is an out-of-box service. It frees Neusoft from O&M. This reduces the system maintenance costs.
- Lindorm provides real-time and fine-grained storage and analysis capabilities to store and analyze all monitoring data. This helps Neusoft maintain the performance of the monitored systems and indirectly improves the digital experience of users.
Neusoft has deployed a system in Alibaba Cloud for a leading vehicle company. This system is used to store and retrieve user digital footprints and user experience data that are collected from global customer terminals. The system also collects the monitoring data that is generated by access requests from tens of thousands of users every day. The system collects 0.12 billion tuples of real-time monitoring metric data every day. The costs for data storage and system maintenance are reduced to 60% of the cost for using other engines.