Community Blog Real-World Implementation of Data Analytics with Alibaba Cloud: Adding Value with DataWorks (Part 3)

Real-World Implementation of Data Analytics with Alibaba Cloud: Adding Value with DataWorks (Part 3)

Part 3 of this article series discusses how Alibaba Cloud DataWorks helps process and work with big data in different scenarios.

By Shantanu Kaushik

Big data development has evolved from being a part of the process to a significant player in the strategic reforms for an organization. Big data has developed rapidly and has been the prime mover for multiple giants in every industry or trade to formulate better business decisions based on real-world data. Businesses are able to make better decisions because of the value extracted from basic raw data using a Big Data Development solution to perform data analytics.

In the previous articles of this series, we discussed how data analytics played an important role in extracting usable and valuable data from raw data and how it is evolving. In this article, we will discuss how Alibaba Cloud DataWorks helps process and work with big data in different scenarios.

DataWorks | Big Data | Alibaba Cloud

Let's start with the architecture for information flow that Alibaba Cloud DataWorks uses for big data:


Features | Benefits | DataWorks | Alibaba Cloud

Alibaba Cloud offers the DataWorks solution as a service. It is based on the Platform as a Service (PaaS) solution and offers many services:

DataWorks has been in the limelight for the unique capabilities that allow enterprises to use it as a one-stop solution for big data development and management. Alibaba Cloud DataWorks supports various compute engines and storage engines:

Alibaba Cloud DataWorks. DataWorks enables data processing features, such as data integration, conversion, and transmission. You can choose to import data from different sources and transmit it to another data system after the required processing.

With Alibaba Cloud DataWorks, you can:

  • Develop visualization using drag and drop functionality to showcase a workflow
  • Debug and edit your code online
  • Collaborate with other developers online
  • Work with multiple types of tasks:

    1 . Machine Learning

    1. Shell Tasks
    2. MaxCompute MR
    3. Data Integration
    4. MaxCompute SQL
  • Leverage strong scheduling with capabilities scaling to run millions of tasks at a time. You can opt for hourly, daily, weekly, and monthly task scheduling.
  • Real-time task monitoring and alarm system to get errors rectified for uninterrupted service

Alibaba Cloud DataWorks allows implementation wherever big requirements pop up. It can be the media or entertainment industry, meteorological data systems, large e-commerce platforms, or any other industry that has to process large datasets.

Along with that, DataWorks can be used for security implantation with big data. Alibaba Cloud's big data system deeply integrates all of the products under this umbrella while leveraging other products within the Alibaba Cloud tech umbrella.

Scenarios | DataWorks | Alibaba Cloud

Business Operations Refinement | DataWorks

Alibaba Cloud DataWorks helps you work alongside business operations to refine them and add to the overall value of operations. With the deep integration of Alibaba Cloud MaxCompute, DataWorks ensures high quality data extraction and development. With proper business data analysis, DataWorks helps process any business demands that pop-up.

Let's take a look at the flow of data on the chart below:


Alibaba Cloud DataWorks helps you monitor and analyze the business data to increase business efficiency. A large amount of data is processed and used to enrich the overall user experience. DataWorks responds to the need for data analysis to work in-sync with business intelligence products, such as QuickBI, to increase efficiency and cut down on the response time taken to react to customer demands.

Data Security | DataWorks

DataWorks helps identify sensitive data and tags it to classify this data based on custom rules set by the user. The user can easily set the masking rules to use for data masking when data is being presented. Along with that, Alibaba Cloud DataWorks offers risk monitoring functionality. As a user, you can visually monitor the data distribution and its usage to create a risk identification profile.

Let's take a look at this works using the chart below:


In this workflow, a custom Software as a Service (SaaS) based application is used to monitor and extract valuable data. Various Alibaba Cloud services, such as Object Storage Service (OSS), Elastic Compute Service (ECS), Server Load Balancer (SLB), MaxCompute, E-HPC, and others are used to extract information from this application.

In this scenario, we are collecting meteorological data and processing it using the Alibaba Cloud platform. The application processes this data and reports the necessary results to the administrator using an Elastic High-Performance Computing (E-HPC) node.

Big Data Analysis | Alibaba Cloud

Let's take a look at how this works on the chart below with a weather system application:


Big Data Development can only have productive results when the methodology applied is standard. DataWorks and Alibaba Cloud MaxCompute enable the integration and use of open-source MaxCompute plugins that help with data migration to the cloud.

When it comes to logging, Alibaba Cloud DataWorks helps sync log data to MaxCompute and run SQL statements for data analysis and processing, improving work cycle efficiency.

Wrapping Up

Alibaba Cloud DataWorks is the solution to building big data warehouses. With capabilities like data aggregation, processing, governance, integration, development, QA, and protection, Alibaba Cloud DataWorks checks all the boxes for a reliable and highly scalable big data solution.

It features separate environments for development and production to help debug code in the pre-production environment. Alibaba Cloud DataWorks is an end-to-end solution with great efficiency that doesn't require multiple tools for different workloads.

Based on industry-leading infrastructure support by Alibaba Cloud, this PaaS leverages some of the best tools, including ECS, OSS, Databases, and security systems from Alibaba Cloud. Multiple sandbox protection and alert systems protect your big data with layered security. Try DataWorks today for your Big Data Development needs.

Upcoming Articles

  1. Real-World Implementation of Data Analytics with Alibaba Cloud (Part 4): MaxCompute and Warehousing
  2. Alibaba Cloud and Apache Flink
0 0 0
Share on

Alibaba Clouder

2,600 posts | 750 followers

You may also like


Alibaba Clouder

2,600 posts | 750 followers

Related Products