PyODPS enhances the result exploration and progress display features of Jupyter Notebook.

Result exploration

PyODPS provides a data exploration feature in Jupyter Notebook for SQLCell and DataFrame. You can use interactive data exploration tools to browse local data and create graphs.

  1. If the execution result is a DataFrame object, PyODPS reads the result and displays it in a paged table. You can click a page number, the Previous button, or the Next button to browse the data.
  2. You can select other display modes on the top of the table to display the result in a column chart, pie chart, line chart, or scatter chart. The following figure shows a scatter chart created based on the default fields, which are the first three fields.
  3. You can click the Settings icon in the upper-right corner of a graph to modify the settings. For example, set Groups to name, X Axis to petallength, and Y Axis to petalwidth. The result is shown in the following graph. The petallength-petalwidth settings display the data in a manner that is easy to understand.

    For column charts and pie charts, you can select an aggregate function for the value fields. The default aggregate function for column charts is sum, and that for pie charts is count. You can click the function name next to the name of the value field to select another function.

    For line charts, the values on the x-axis cannot be null. If any values are null, the graph may not be correctly displayed.
  4. After you make the graph, click the Download icon to save it.
Note To use this feature, you must install pandas and ipywidgets.

Progress display

The execution of large jobs takes extended periods of time. PyODPS provides progress bars to show the execution progress. When DataFrame jobs, machine learning jobs, or SQL statements that start with %sql are executed in Jupyter Notebook, a list of these jobs and their overall progress are displayed.
If you click a job name, a dialog box that shows the progress of each task in the job appears.
After the execution is completed, a message that indicates whether the job succeeded appears.