Release notes

Behavior change for DataFrameGroupBy.mf.apply_chunk

Starting from MaxFrame SDK 2.6, the behavior of DataFrameGroupBy.mf.apply_chunk is changing. This update simplifies the index parameter and unifies behavior across the DPE and MaxCompute SQL engines. The new behavior is controlled by the prepend_index_group_keys parameter, which defaults to False for backward compatibility but will switch to True in a future release.

• Before MaxFrame SDK 2.6

  MaxFrame DataFrameGroupBy.mf.apply_chunk required you to specify the full Index shape of the output DataFrame in the index parameter. For example:

  def process(df):
      df = df.copy()
      df["total"] += 1return df
  
  # In SDK 2.5 and earlier, you must specify all output fields. Here, year is the group key specified in groupby.
  expect_index = pd.MultiIndex.from_tuples([(0, 0)], names=["year", None])
  
  md_result_df = md_df.groupby(["year"]).mf.apply_chunk(
      process,
      batch_rows=10,
      dtypes=result_dtypes,
      index=expect_index,
      output_type="dataframe",
      skip_infer=True
  )
  

  Additionally, due to an implementation issue, the columns received by user functions in the MaxFrame DPE engine included group keys, while those in the MaxCompute SQL engine did not.

• MaxFrame SDK 2.6 and later

  The example above is rewritten as follows:

  def process(df):
      df = df.copy()
      df["total"] += 1return df
  
  md_result_df = md_df.groupby(["year"]).mf.apply_chunk(
      process,
      batch_rows=10,
      dtypes=result_dtypes,
      skip_infer=True,
      prepend_index_group_keys=True,
  )
  

  Setting prepend_index_group_keys to True enables the new behavior. With this behavior, the input DataFrame passed to user functions in both the DPE and MaxCompute SQL engines no longer includes group keys by default. Additionally, the index parameter passed to apply_chunk now represents only the index type of the user function output, not the Index of the apply_chunk output object. The relationship to the final Index depends on the groupby function parameters, as shown below:

  Parameter	group_keys=True (default)	group_keys=False
  as_index=True (default)	Prepends the input group columns as Index before the user Index	Does not prepend any columns as Index
  as_index=False	Prepends Group ID as Index before the user Index (DPE only)	(same as above)

Migration guide:

Add prepend_index_group_keys=True to your groupby.mf.apply_chunk calls and make the following changes:

1. If you specify the index parameter, remove group keys from it. For example, change pd.MultiIndex.from_tuples([(0, 0)], names=["year", None]) to pd.Index([0]). If your user function does not change the Index type, you can remove the index parameter entirely.

2. If you use groupby.mf.apply_chunk in the MaxFrame DPE engine, check your user function. If the function references group columns, or does not filter out group columns from the output, explicitly include group columns after groupby to prevent them from being excluded from the user function input. For example, change df.groupby("col").mf.apply_chunk(func) to df.groupby("col")[list(df.dtypes.index)].mf.apply_chunk(func).

MaxFrame Coding Skill

MaxFrame Coding Skill is officially released, empowering AI agents to develop MaxFrame jobs intelligently. This skill suite supports MaxFrame session management, operator recommendation, code generation, and interactive debugging, helping developers efficiently build large-scale data processing pipelines on MaxFrame.

AI agents can automatically orchestrate end-to-end jobs from data reading, data processing, to result writing. Combined with MaxFrame deferred execution and resource cleanup mechanisms, the Coding Skill ensures job stability and performance optimization. By describing requirements in natural language, you can generate runnable code, significantly lowering the barrier to distributed data processing for a seamless Data + AI development experience.

For more information, see MaxFrame Coding Skill.

MaxFrame AI Function enhancements

• Added support for Alibaba Cloud Model Studio commercial models (qwen3-max and text-embedding-v4).

• Introduced token-based billing (pay-as-you-go), further reducing the cost and barrier to large-scale offline model inference.

No need to manage model deployment or maintenance. Simply call Model Studio model resources through AI Function interfaces to perform batch inference on large-scale data. Combined with MaxFrame distributed data processing capabilities, this enables end-to-end Data+AI development.

For more information, see MaxFrame AI function.

Multimodal data processing operators

To meet growing multimodal data processing needs, MaxFrame introduces a new multimodal data processing operator module. This release adds built-in image processing operators that support distributed processing of large-scale image data.

• image.decode for decoding image data;

• image.width, image.height, image.size, image.mode, and image.format for extracting image properties;

• End-to-end image decoding, property extraction, size filtering, scaling, cropping, and format conversion.

No need to set up a separate image processing service. You can efficiently clean and preprocess image data directly within MaxFrame DataFrames.

Local Debug Mode

MaxFrame introduces Local Debug Mode. Previously, debugging functions such as apply() and apply_chunk() required submitting code to a remote cluster, and every code change required resubmission.

With Local Debug Mode:

• MaxFrame UDF functions can be executed directly in a local Python environment.

• IDE breakpoint debugging is supported, and the mode works fully offline.

• The same code runs seamlessly in both local debugging and cluster production environments without modification.

For more information, see MaxFrame local debug mode.

DPE engine custom image support

MaxFrame now supports custom images in the DPE (Distributed Python Engine), allowing you to use specific dependencies and runtime environments.

Specify an image name through configuration options, for example: {"odps.session.image": "maxframe_image_v1"}, to load a runtime environment with pre-installed dependencies in the MaxFrame distributed computing environment. This eliminates the need to install dependencies on every task execution, significantly improving development efficiency and flexibility.

For more information, see Custom images.

OSS mount enhancements

MaxFrame enhances its OSS data mount feature with support for mounting multiple OSS buckets and configuring path prefixes. The @with_fs_mount decorator now allows you to mount multiple OSS buckets or different path prefixes from the same bucket simultaneously, providing flexible adaptation to complex data organization structures. In addition, the new cache_memory_limit parameter lets you configure the memory cache limit for mount points, effectively controlling resource usage and improving performance and stability when reading large-scale files.

For more information, see OSS mount feature (Invitational Preview).

Network and security enhancements

The MaxFrame DPE engine adds network access whitelist configuration to control network access permissions during MaxFrame UDF execution.

Configuration options:

• substep.public_network_whitelist: Public network access whitelist. Allows access to external public network services such as APIs and model services.

• substep.internal_network_whitelist: Internal network access whitelist. Allows access to Alibaba Cloud internal network services such as OSS internal endpoints.

For more information, see MaxFrame flag configuration guide.

GU compute resources are now available for MaxFrame UDFs

Supported regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Ulanqab), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), US (Virginia), and Hangzhou Finance Cloud

Release date: December 30, 2025 (UTC+8)

MaxCompute introduces GPU Units (GU) as a new AI compute resource specification. Purchase independent GU compute resource quota groups based on your requirements. For more information, see Commercial availability of MaxCompute AI compute resources.

MaxFrame adds the maxframe.udf.with_running_options interface for GPU resource scheduling in user-defined functions (UDFs). Use the GU and gu_quota parameters to control the proportion and priority of GPU resources that compute tasks consume, enabling fine-grained management of heterogeneous computing resources.

GU resources are well-suited for compute-intensive jobs such as multimodal data pre-processing, where GPU acceleration reduces processing time and improves job iteration efficiency.

For more information, see Purchase and use MaxCompute AI compute resources.

MaxFrame AI Function now supports Qwen3, DeepSeek, and custom model hosting

Supported regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Ulanqab), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), US (Virginia), and Hangzhou Finance Cloud

Release date: December 30, 2025 (UTC+8)

AI Function deeply integrates large language models (LLMs) with MaxCompute. This release adds the following capabilities:

Model support:

• Out-of-the-box support for the Qwen3 series and DeepSeek models

• Custom model hosting for models not in the built-in catalog

• AI Function on GU, which accommodates a wider range of model sizes and improves offline inference efficiency

New interfaces:

• generate — supports custom prompt templates for flexible LLM invocation

• translate — built-in translation

• extract — structured data extraction

• embedding — vectorization

AI Function uses MaxCompute's elastic resources to run offline, high-concurrency LLM inference. Combined with DataFrame APIs and the Python ecosystem, it supports end-to-end pipelines that cover data cleaning, data processing, and AI inference within MaxCompute — without separate LLM deployment infrastructure. Typical use cases include multimodal data processing and large-scale offline LLM inference.

For more information, see MaxFrame AI Function.

MaxFrame supports mounting OSS buckets (invitational preview)

Supported regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Ulanqab), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), US (Virginia), and Hangzhou Finance Cloud

Release date: December 30, 2025 (UTC+8)

Mount OSS buckets to map them directly to the MaxCompute local file system. Use the @with_fs_mount declarative configuration to define a mount path and operate on OSS data through POSIX-like interfaces, without the boilerplate required by traditional SDK (software development kit) development patterns.

For more information, see Mount OSS buckets (invitational preview).

New DataFrame, tensor, and Scikit-learn APIs added

Supported regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Ulanqab), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), US (Virginia), and Hangzhou Finance Cloud

Release date: December 30, 2025 (UTC+8)

This release adds the following APIs:

DataFrame and Series:

• to_numeric — numeric data processing

• New GroupBy APIs

• Index.get_level_values and Index.to_series

MaxFrame extension APIs:

• DataFrame.mf.collect_kv — key-value collection

• DataFrame.mf.extract_kv — key-value fetching

• DataFrame.mf.map_reduce — MapReduce

• DataFrame.mf.rebalance — data rebalancing

Other additions:

• New Scikit-learn evaluation metric APIs

• New tensor APIs with support for 10 categories of mathematical operations: FFT (Fast Fourier Transform), indexing, linear algebra, data manipulation, mathematical functions, general routines, sorting, special functions, and statistics

For more information, see MaxFrame-specific APIs.

Partial job submission lets you retrieve data from failed jobs

Supported regions: All regions

Release date: July 17, 2025 (UTC+8)

Partial job submission tolerates minor failures within a job. Even when a job fails, the result data from successfully completed Fuxi instances remains retrievable.

For more information, see Partial job submission in MaxFrame.

AI Function adds built-in Qwen3 models (Qwen3-0.6B to Qwen3-14B)

Supported regions: All regions

Release date: July 17, 2025 (UTC+8)

MaxFrame AI Function now includes built-in models from the Qwen3 series, ranging from Qwen3-0.6B to Qwen3-14B.

For more information, see MaxFrame AI Function.

Worker-level CPU and memory monitoring is now available

Supported regions: All regions

Release date: July 17, 2025 (UTC+8)

Use Fuxi Sensor to view the actual CPU and memory consumption of individual workers.

For more information, see Fuxi Sensor.