Connect by using DataWorks

Updated at:
Copy as MD

DataWorks is a unified, end-to-end platform for big data development and governance, powered by compute engines like MaxCompute.

Background

With DataWorks, you can associate compute engines with a workspace. This allows you to create and periodically schedule computing tasks. You can connect to MaxCompute through DataWorks in the following ways:

  • The SQL Query feature in DataAnalysis:

    This feature lets you write and edit MaxCompute SQL, query and analyze data in a web-based spreadsheet, and share or download data. For more information about the SQL Query feature, see SQL Query (Legacy).

  • The ODPS node in Data Development:

    DataWorks provides various node types for different engine tasks. You can use these nodes to define data development jobs and combine them with resources and functions to create complex workflows. The ODPS node types include ODPS SQL node, ODPS Spark node, PyODPS 2 node, PyODPS 3 node, ODPS Script node, and ODPS MR node.

Use cases

Use cases for DataAnalysis

The following are common use cases for the SQL Query feature in DataAnalysis:

You can use the SQL Query feature to query data and analyze the results in a web-based spreadsheet. Downloading the query results to your computer reduces data transfer and improves data security.

Use cases for ODPS nodes

You can use DataStudio in DataWorks to develop periodic MaxCompute jobs. You can configure properties, such as scheduling cycles and dependencies, and then commit the jobs to Operation Center for scheduling.

Procedure

  1. Create a DataWorks workspace.

  2. Associate a MaxCompute computing resource or create a MaxCompute data source.

    The steps depend on whether you opted into the public preview of the new DataStudio when you created your workspace.

    You can determine which version your workspace uses from the Workspaces page. Find your target workspace and click Shortcuts in the Actions column.

    If you did not opt into the public preview of the new DataStudio

    If you opted into the public preview of the new DataStudio

    In the Actions column, click Shortcuts and select Data Development.

    The Data Development page has a navigation pane on the left. The title of the main content area is Data Development. The toolbar includes the Refresh and + New buttons. Below the toolbar are three tabs, All, Owned by me, and My Favorites, and a search box. The directory tree contains the expandable Solutions and Business Flows nodes.

    For more information about DataStudio (Data Development), see Data Development (DataStudio) (Legacy).

    In the Actions column, click Shortcuts and select Data Studio Public Preview.

    On the Data Development page, the left-side navigation pane shows the Data Development module, which includes the Project Directory and Personal Directory menus. The upper-right corner shows the current workspace name and a Simple Mode identifier. By default, the main content area shows the Welcome tab.

    For more information about the new version of DataStudio, see Data Development (Data Studio) (New).

  3. Use MaxCompute through DataWorks.

    • DataAnalysis

      You can access the SQL Query feature in DataAnalysis in one of the following ways:

      • In the left-side navigation pane of the MaxCompute console, choose Data Analytics > SQL Query to open the SQL Query page.

      • On the homepage of DataAnalysis, click SQL Query in the Shortcuts section to open the SQL Query page.

      • In the DataAnalysis module, click SQL Query in the left-side menu to open the SQL Query page.

      For more information about how to create and run an SQL query, see SQL Query (Legacy).

    • ODPS node

      For more information about how to create an ODPS node, see Overview of periodically scheduled jobs.