E-MapReduce:Develop a notebook

Prerequisites

A workspace and a notebook session are created. For more information, see Create a workspace and Manage notebook sessions.

Create a notebook

1. Go to the Data Development page

   1. Log on to the E-MapReduce (EMR) console.

   2. In the navigation pane on the left, choose EMR Serverless > Spark.

   3. On the Spark page, find the desired workspace and click the name of the workspace.

   4. In the navigation pane on the left of the EMR Serverless Spark page, click Development.

2. Create a notebook.

   1. On the Development tab, click the icon.

   2. Enter a name, set Type to Interactive Development > Notebook, and then click OK.

Edit and run the notebook

1. In the upper-right corner of the configuration tab of the notebook, select a notebook session that you created and started from the Notebook Sessions drop-down list.

   You can also click Create Notebook Session from the Notebook Sessions drop-down list to create a notebook session.

   Note

   A notebook session can be used by multiple notebooks at a time. This way, you do not need to seperately create a notebook session for each notebook.

2. Enter Python statements in the cell.

3. Click Execute All Cells or the icon in front of the cell to run the notebook.

Publish the notebook

1. Confirm that the job runs as expected. Then, in the upper-right corner of the configuration tab of the job, click Publish.

2. In the Publish dialog box, configure the Remarks parameter and click OK.

Export the notebook

On the Development tab of the Development page, move the pointer over the notebook name and choose > Export.

Import a notebook

1. On the Development tab of the Development page, move the pointer over a specific folder and click the icon.

   

2. In the dialog box that appears, click the area in the dotted line rectangle to select a local file, or drag the file to the area. Then, click OK.

Call other notebooks in a notebook

In a notebook, you can use the %run magic command to execute code from another notebook. This is useful when you need to share functions or variables between multiple notebooks.

For example, you have two notebook files: notebook_a.ipynb and notebook_b.ipynb. The notebook_a.ipynb file defines functions and variables that you want to use in notebook_b.ipynb.

1. In notebook_a, define a simple function greet and a variable message.

   # notebook_a.ipynb
   def greet(name):
       return f"Hello, {name}!"
   
   message = "Welcome to our Python session."
   

2. In notebook_b, use the %run command to call notebook_a.

   %run /dev/path/to/notebook_a
   

   Note

   The file path must start with /dev. Replace path/to/ with the actual folder path. In this example, notebook_a is in the test folder.

3. Use the content defined in notebook_a in the new cell.

   print(greet("EMR Serverless Spark"))
   print(message)

   The following information is returned.