All Products
Search

This Product

    MaxCompute:Best practices

    Document Center

    MaxCompute:Best practices

    Last Updated:Jan 10, 2026

    This topic describes the key capabilities and best practices of the MaxFrame framework for real-world development. It provides runnable code examples and scenario-based instructions to help you quickly understand and master the core features of MaxFrame. This helps lower the barrier to entry and improve development efficiency.

    Common features and core capabilities

    Feature module

    Core capability

    Typical application scenario

    Advantages and features

    apply / apply_chunk operators

    Supports the parallel execution of user-defined functions (UDFs) on DataFrames or data chunks.

    Complex user-defined computing logic that requires building a UDF. Large-scale data processing that requires the parallel processing of multiple input rows.

    Supports native calls to Python functions, automatic distributed scheduling, and concurrent batch processing to significantly improve processing efficiency.

    GU resource usage

    Supports hybrid scheduling of CPUs and GPUs for heterogeneous computing.

    Deep learning inference and multimodal data processing.

    Unified management of heterogeneous resources (CUs and GUs) lets you build a complete processing flow in a single job pipeline.

    AI Function on GU

    Supports built-in large language models (LLMs) and custom models. You can call the AI Function API that uses GU resources for LLM inference.

    Batch model inference scenarios, such as structured data extraction, text translation, data labeling, image classification, speech recognition, and vectorization.

    Includes built-in mainstream LLMs, such as Qwen3 and DeepSeek. Provides API calls, such as generate and task. Automatically loads models to GUs for low-latency, high-concurrency execution.

    OSS mounting and access

    Supports direct mounting of Object Storage Service (OSS) to read, write, and operate on massive amounts of data.

    Loading multimodal datasets.

    Supports mounting OSS without downloading, supports stream reading, and is compatible with standard file APIs.

    Specific scenarios

    Best practices for using the apply and apply_chunk operators

    Set the batch_rows parameter based on your data volume and available resources to prevent out-of-memory (OOM) errors.

    Feature description

    • apply applies a custom function to the rows or columns of a MaxFrame DataFrame. It supports vectorized operations on entire rows or columns.

    • apply_chunk is a low-level API provided by MaxFrame. It applies a custom function in parallel to each data chunk of a MaxFrame DataFrame. It is suitable for advanced scenarios that require direct operations on the physical shards of a distributed DataFrame and is often used for performance optimization or custom computing logic.

    Example scenarios

    • Use apply to standardize fields, such as masking phone numbers.

    • Use apply_chunk to process millions of image paths in parallel and extract metadata.

    Tutorials

    Best practices for using the MaxFrame apply_chunk operator

    Using MaxFrame GU resources

    Feature description

    Complex data processing and job pipelines often require CPU or GPU resources on different compute nodes. MaxFrame supports hybrid scheduling and computation of CU and GU resources. In a MaxFrame UDF, you can request GU Quota resources using resource tags for high-performance computing (HPC) tasks.

    Example scenarios

    • Image or video frame extraction and encoding

    • Complex data processing

    Tutorials

    Best practices for developing with AI Function on GU

    Select and purchase a GPU card with sufficient GPU memory for the number of parameters in the LLM that you use.

    Feature description

    MaxFrame AI Function is an end-to-end solution on the Alibaba Cloud MaxCompute platform for offline LLM inference scenarios. Its core features include the following:

    • Seamless integration of data processing and AI capabilities

      • Supports direct interaction with LLMs, such as Qwen3-4B, through MaxFrame DataFrames.

      • Provides generate and task APIs to balance flexibility and ease of use.

    • GPU resource scheduling (GU)

      • You can request GPU resources using gu_quota_name to accommodate models of different sizes. For example, a 4B model requires 2 GUs.

    • Managed LLM calls

      • Built-in model library, such as Qwen3-4B-Instruct-2507-FP8, with support for parameter tuning, such as temperature and max_tokens.

      • Supports large-scale concurrent scheduling to optimize batch inference performance.

    Example scenarios

    • Knowledge Q&A

      • Scenario description: Answers questions in fields such as natural science, history, and technology. It supports multiple languages and complex reasoning.

      • Typical applications:

        • Scientific computing: "What is the average distance between the Earth and the Sun?"

        • Historical events: "In what year did the American Revolutionary War begin?"

        • Technical principles: "What is the core mechanism of the Transformer model?"

    • Text translation

      • Scenario description: Translates text between languages. It supports Chinese-English translation and handles terminology in professional fields.

      • Typical applications:

        • Chinese to English: "How to relieve a headache?" → "How to relieve a headache?"

        • Legal/medical text translation: "The patient needs to take one aspirin tablet daily."

    • Structured data extraction

      • Scenario description: Extracts key entities, properties, or relationships from unstructured text.

      • Typical applications:

        • Entity extraction:

          • Input: "The iPhone 15 Pro is the latest flagship phone released by Apple."

          • Output: {"product": "iPhone 15 Pro", "brand": "Apple", "type": "flagship phone"}

        • Resume parsing:

          • Input: "Zhang San, 5 years of Java development experience, skilled in the Spring Boot framework."

          • Output: {"name": "Zhang San", "skills": ["Java", "Spring Boot"], "experience": 5}

    Tutorials

    • Best practices for developing with AI Function on GU

    Best practices for mounting and using OSS

    For improved performance, you can combine this feature with apply_chunk to implement parallel reading.

    Feature description

    In data analytics scenarios, you often need to use MaxFrame jobs with persistent object storage, such as OSS. For example:

    • Load raw data from OSS for cleaning or processing.

    • Write intermediate results to OSS for consumption by downstream tasks.

    • Share static resources such as trained model files and configuration files.

    Traditional read and write methods, such as pd.read_csv("oss://..."), are inefficient in distributed environments because of SDK performance limits and network overhead. File system-level mounting (FS Mount) lets you access OSS files in MaxCompute as if they were local disk files. This greatly improves development efficiency.

    Example scenario

    • Mount the OSS bucket directory oss://maxframe-datasets/images/ to the local MaxCompute path /data/imgs for subsequent processing.

    Tutorials