In recent years, data plays an important role in various sectors of the national economy and undergoes many changes compared with data in the past. This topic describes the trends in data technology and the business challenges that enterprises encounter.
Trends in technology development
Explosive growth in data scale
The development of technologies, such as 8K, 5G, IoT, big data, and artificial intelligence (AI) spurs the explosive growth of data volumes. In the Data Age 2025 report, the International Data Corporation (IDC) predicts that the global datasphere can grow from 33 zettabytes in 2018 to 175 zettabytes by 2025, an increase of ten times compared with data generated in 2016. This indicates the arrival of an era that focuses on data value, which gradually replaces the previous period that emphasized the shift from data mocking to digitization. Therefore, it is essential to generate, utilize, and manage data to benefit consumers, government agencies, and enterprises in their daily lives and operations. Consumers and enterprises continue to generate, share, and access data across different devices and clouds. The volume of data grows at a pace faster than expected.
Real-time production and processing
Big data can be characterized by volume, variety, and velocity. Big data involves petabytes of static data. Fast data emphasizes variety and velocity based on this large data volume. The fast data solution allows you to process data in real time at a higher speed. In a recent white paper, IDC predicts that as global connectivity grows, more data can be generated, and an increasing portion of that data consists of real-time information. By 2025, nearly 30% of generated data can be consumed in real time. A recent study by Forrester indicates that over 75% of surveyed companies utilize fast data solutions. Among those surveyed, 88% expresses the need to perform analytics on data in near real time.
Intelligent production and processing
In addition to structured data, enterprises recognize the value of an increasing proportion of semi-structured data, such as logs, and unstructured data, such as audio and video files. In Data Age 2025, IDC predicts that 80% of worldwide data becomes unstructured by 2025 and the amount of unstructured data grows at an annual rate of 55%. Enterprises cannot gain insight into the value of their data without tools to analyze these large amounts of data. In this case, the demand for multimodal data analysis increases. Traditional big data technology can address the needs, but the diverse technology stack and inconsistent usage practices create challenges for most enterprises. Therefore, unified and standardized technical solutions are required.
Accelerated data migration to the cloud
Gartner predicts that by 2023, 75% of all databases are on the cloud. Enterprises and organizations deploy new applications in the cloud and accelerate the migration of existing data assets. This trend is expected to persist. Enterprises are increasingly inclined to deploy and innovate their data management system (DBMS) by using cloud-first strategies or pure cloud environments. Personnel who choose DBMS solutions agree that cloud-based DBMS represents the future direction of data analysis.
Business challenges
Scattered and inconsistent data
Traditional enterprises generate various types of data, such as structured, semi-structured, and unstructured data. Data sources include databases, logs, objects, and stock data from existing data warehouses. Data in different formats from various sources requires different access and analysis methods. SQL statements are used in most traditional enterprises that build their business systems on relational databases. This approach increases storage and usage costs.
Non-real-time analysis
Enterprise operations become diverse and include real-time recommendations, precision marketing, advertising effectiveness, real-time logistics, and risk control. Data timeliness plays an increasingly important role in enterprise operations. Real-time data processing capabilities become an important factor for enterprises to improve their competitiveness. Big data analysis is no longer limited to traditional T+1 scenarios. An increasing number of enterprises have higher requirements for real-time data analysis and processing. The traditional batch processing mode has a latency of hours or even days, which cannot meet the business requirements of T+0 business. Users want to analyze large amounts of data within seconds or even milliseconds.
Extremely complex system
Big data platforms involve complex operations and are difficult to use. Users want to concentrate on core business rather than on the underlying technology. A ready-to-use solution is preferred over high learning costs and complicated technologies. Users want a simple and easy-to-use platform. Solutions that combine big data platforms are also insufficient to meet the requirements for fine-grained access control, high reliability, disaster recovery, and high availability, especially for customers in industries such as finance.
High usage costs
The use of data in enterprises is cyclical and uncertain. Business rapidly changes, and the data scale also significantly changes. Some enterprises experience obvious peaks and troughs. Resources are often idle during off-peak hours. Such business characteristics require high resource scalability of the underlying capabilities. The scalability refers to the scalability of storage and computing capabilities. Users can select resources and change resource configurations at any time to maximize the return on invested resources based on business requirements.