Streaming data and big screen DataV

Our platform sales management large-screen real-time data display system adopts Alibaba Cloud's latest big data and stream computing technology to display the customer's login information, equipment information, sales volume, amount, customer information and vehicle information of the loading system, and shipping information. The geographic location, direction, etc., are presented to enterprises and related leaders in multi-dimensional forms such as flying line charts, heat maps, point maps, traditional bar charts, and pie charts. Powerful visual impact and highly visual graphics and digital presentations bring clear, intuitive, real and dynamic sales data.

One benefit of big data visualization is that it allows users to track the connection between operations and overall business performance. In a competitive environment, finding correlations between business functions and market performance is critical. Through the display of data visualization, it can help the business personnel of the e-commerce platform to understand and process their information faster, making it easier for stakeholders to evaluate the entire enterprise. Adjust and identify market changes more timely and accurately. Improve the competitive advantage of the e-commerce platform industry.

Visualization solves the problem of front-end display. In fact, it relies on powerful data processing, integration, analysis and prediction capabilities. The advantage of our platform is that it transforms a very difficult prediction problem into a relatively simple description problem.

Through the application of the big data platform, the e-commerce platform can retrieve the on-site delivery data in real time, predict the supply and demand relationship based on these data, and adjust the price in time, which ensures the interests of the enterprise and maintains the economic benefits of the market supply and demand balance.

For example, oil and gas prices are affected by a variety of domestic and international factors, and are prone to price fluctuations. The e-commerce platform collects the customer's login information, browsing information, purchase information, and retrieves the platform's sales volume, amount, customer information, vehicle information of the loading system, the geographical location and direction of the shipment, etc., through the DataV logistics screen, transaction The relevant technical components of the screen are displayed, which can monitor the access frequency, access method, real-time sales volume, sales amount, and real-time loading and shipping information of members to the platform in real time. Keep abreast of business dynamics, real-time early warning, and quickly adjust operating strategies and other scenarios. And timely feedback this information to enterprises to assist relevant enterprises to make secondary price adjustments or multiple price adjustments for hot-selling products with high attention. At the platform level, the e-commerce platform formulates various price lock strategies such as payment lock price, card production lock price, and entry lock price. It also has flexible price strategies, which not only ensures the interests of enterprises, but also maintains the relationship between supply and demand in the market.

The realization of äºï¼ digital large screen

The entire processing process includes three processes: data collection, data processing, and data display. Alibaba's DTS, DataHub, StreamCompute, RDS, and DataV products are used for distribution.

2.1 Data collection

We implement data transmission from RDS to DataHub through Data Synchronization (DTS) in Alibaba Data Transmission. Before configuring DTS, first create a project in DataHub (as shown in the figure below).

Then, you need to configure the source instance and target instance in Data Transfer - Data Synchronization of the Alibaba Cloud console and purchase it.

After the synchronization configuration is completed, the incremental update data of each table will be stored in the topic corresponding to the DataHub during the synchronization process. After the DTS configuration is completed, check whether the DTS tasks can be automatically executed and whether the delay is too large. Generally speaking, it is at the ms level.

Go to the console of Alibaba Cloud DataHub, click on the created project, and check whether the data structure created by DTS is missing.

Click Data Sampling to see if the latest data is coming into DataHub.

2.2 Data processing

First create a new RDS instance, and connect through the client to create the corresponding table structure in RDS according to the source table structure.

Then, find Big Data - Stream Computing - Development in the Ali console, click the button in the development job on the left menu to create a new development job, and then select Datahub data storage in the data storage on the left menu, and find the corresponding table , and then click Reference as Input Table on the right. In this way, the operation of referencing the data source in the development job is realized.

Then select the RDS datastore in the datastore, find the RDS for the response, and click Reference as Result.

Finally, insert the source table data into the target table through the Replace Into statement. In addition, if the data format does not match, the corresponding data format conversion work can also be performed here, such as using the from_unixtime function.

After the online operation is complete, click O&M at the top of the stream computing console to view the job status, stop or restart the job as needed, and view the service delay.

Then click the job name to view detailed business delay, calculation time, data input and other indicators. If the data transmission is abnormal, data skew may occur, and the system will throw an exception, and the specific response measures will be determined according to the thrown exception.

2.3 Data presentation

After the data is transferred to RDS, we can display the data through DataV attack.

Open the Alibaba Cloud console Big Data-DataV data visualization, select the appropriate template, add the corresponding components, and then adjust the style

The relevant data can then be organized according to actual business needs. Before data organization, first set the data source of DataV, click the arrow in the upper left corner to return to the main page of DataV. Select My Data, and then click the Add Data button to set the relevant data source as the target RDS of the previous stream data.

Then click My Visualization in the upper left menu, hover the mouse on the big screen you just created, and click the Edit button.

Then select the component to be edited, click the data in the upper right corner, then select the database, and write SQL according to the business requirements. If it takes effect, when the data changes in real time, it will be displayed on the big screen in real time.

If the style and data of DataV have been edited, you can publish the big screen, click the publish button in the upper right corner, click publish in the pop-up box, and a large screen connection that can be accessed by the public network will be generated. Set it up. Embed the link on a website, or access it directly from a large-screen device to achieve large-screen display.