By default, MaxCompute discards all result data when a job fails. MaxFrame partial submission retains the result data of successfully executed Fuxi instances when a job fails or times out, so you can retrieve partial results and avoid wasting computing resources.
Only MaxCompute MaxFrame jobs support partial submission. Enable it by setting odps.sql.always.commit.result to "true" in options.sql.settings.
Partial submission does not take effect for jobs that are actively canceled by users.
Configuration parameters
Set the following parameters at the job level through options.sql.settings:
| Parameter | Value | Description |
|---|---|---|
odps.sql.always.commit.result | "true" | Enable partial submission. Retains the result data of successfully executed Fuxi instances when a job fails. |
odps.sql.job.max.time.hours | String (hours), e.g. "1" | Set the job timeout period in hours. After this duration, the job is forcibly terminated. Optional -- only required for timeout-based partial result retrieval. |
Retrieve partial results from failed jobs
Enable partial submission and choose one of the following methods to access the results.
Write results to a destination table (recommended)
Direct the final output to a named destination table. If the job fails, query the destination table to retrieve the result data of partially successful executions.
options.sql.settings = {
"odps.sql.always.commit.result": "true" # Enable partial submission
}
df.mf.flatmap(
... # Customer business logic
).to_odps_table("data_output").execute()After execution, view the successfully completed Fuxi instances in the job Logview:
Query through temporary tables
If the final result data is not written to a destination table, MaxFrame automatically generates a temporary table. Get the temporary table name from Logview and query it to access the result data.
options.sql.settings = {
"odps.sql.always.commit.result": "true" # Enable partial submission
}
df.mf.flatmap(
...
).execute()View the MaxFrame temporary table name in Logview:
Set a job timeout to retrieve partial results
For jobs that may have long-tail tasks due to data skew, infinite loops, or other issues, set a timeout period in addition to enabling partial submission. After the timeout, the job is forcibly terminated, but the result data of successfully executed Fuxi instances is retained.
options.sql.settings = {
"odps.sql.always.commit.result": "true", # Enable partial submission
"odps.sql.job.max.time.hours": "1" # Set timeout to 1 hour
}
# Option A: Write to a destination table (recommended)
df.mf.flatmap(
... # Customer business logic
).to_odps_table("data_output").execute()
# Option B: Use a temporary table
# df.mf.flatmap(...).execute()
# Then retrieve the temporary table name from Logview.If the job fails before the timeout period elapses, the result data of successfully executed Fuxi instances is still retained.
Billing
Pay-as-you-go
MaxFrame jobs running in pay-as-you-go mode are billed based on actual CU-hours used.
Download job usage details from the Usage Details page. For more information, see MaxCompute bill usage details analysis.
Subscription
MaxFrame jobs running in subscription mode occupy subscription resource group resources according to the requested amount during execution. No additional fees apply.