MaxCompute:Use MaxFrame in an on-premises environment

Prerequisites

Before you use MaxFrame, make sure that the following preparations are complete:

• Python 3.7 or 3.11 is installed in your on-premises environment. Errors may occur if you use other versions.

• The pip tool is installed in your on-premises environment. If the pip tool is not pre-installed in your Python version, visit the Python official website to obtain the pip installation guide.

• A MaxCompute project is created. For more information, see Create a MaxCompute project.

Install MaxFrame

1. In the CLI of your system, such as the Command Prompt of the Windows operating system, run the pip command to install the MaxFrame client. Sample command:

   pip install --upgrade maxframe

2. Run the following command to check whether the MaxFrame client is installed. If no value or error message is returned, the installation is successful.

   python -c "import maxframe.dataframe as md"

   If an error is reported, query the default Python version and change it to Python 3.7 or 3.11. Sample commands:

   # Query the default Python version of the system.
   python --version 
   
   # Change the default Python version of the system to Python 3.7. In the following command, $path/python3.7 is the installation path of Python.
   $path/python3.7 -m pip install setuptools>=3.0

Use MaxFrame

1. Create a .py file in the on-premises environment, such as test.py, copy the following sample code to the test.py file, and then save the file.

   import os
   import maxframe.dataframe as md
   from odps import ODPS
   from maxframe import new_session
   
   # Create a MaxCompute entry.
   o = ODPS(
       # Set the ALIBABA_CLOUD_ACCESS_KEY_ID environment variable to the AccessKey ID of your Alibaba Cloud account. 
       # Set the ALIBABA_CLOUD_ACCESS_KEY_SECRET environment variable to the AccessKey secret of your Alibaba Cloud account. 
       # We recommend that you do not directly use the actual AccessKey ID and AccessKey secret. 
       os.getenv('ALIBABA_CLOUD_ACCESS_KEY_ID'),
       os.getenv('ALIBABA_CLOUD_ACCESS_KEY_SECRET'),
       project='your-default-project',
       endpoint='your-end-point',
   )
   table = o.create_table("test_source_table", "a string, b bigint", if_not_exists=True)
   with table.open_writer() as writer:
       writer.write([
           ["value1", 0],
           ["value2", 1],
       ])
   
   # Create a MaxFrame session.
   session = new_session(o)
   
   df = md.read_odps_table("test_source_table",index_col="b")
   df["a"] = "prefix_" + df["a"]
   
   # Print DataFrame data.
   print(df.execute().fetch())
   
   # Write data from MaxFrame DataFrame to a MaxCompute table.
   md.to_odps_table(df, "test_prefix_source_table").execute()
   
   # Destroy the MaxFrame session.
   session.destroy()

   Parameters:

   • ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET: Set these environment variables to the AccessKey ID and AccessKey secret of the Alibaba Cloud account that has the MaxCompute permissions on the objects that you want to manage in your MaxCompute project. You can obtain the AccessKey ID on the AccessKey Pair page in the Alibaba Cloud Management Console.

   • your-default-project: the name of your MaxCompute project. To view the name of the MaxCompute project, log on to the MaxCompute console and choose Workspace > Projects in the left-side navigation pane.

   • your_end_point: the endpoint of the region in which your MaxCompute project resides. For more information, see Endpoints.

2. Go to the path of the test.py file and run the file. Sample command:

   python test.py