All Products
Search
Document Center

AgentBay:Mobile Use

Last Updated:Aug 22, 2025

Open capabilities

Tool

Name

Description

Support status

OSS

oss_env_init

Creates and initializes OSS environment variables. Specify the Endpoint, Access Key ID, Access Key Secret, Security Token, and Region. Temporary security credentials are obtained through Security Token Service (STS). For more information, see Use temporary access credentials from STS to access OSS.

Supported

oss_upload

Uploads a local file or directory to a specified OSS Bucket. If you upload a directory, it is first compressed into a ZIP file. You can specify the object name in OSS. If you do not specify a name, the file or ZIP name is used by default.

Note

Before using this tool, call the oss_env_init tool to initialize the OSS environment variables.

Supported

oss_download

Downloads an object from a specified OSS Bucket to a local path. If the parent directory does not exist, it is automatically created. If the target file already exists, it is overwritten.

Note

Before using this tool, call the oss_env_init tool to initialize the OSS environment variables.

Supported

oss_upload_annon

Uploads a local file or directory to a specified URL using HTTP PUT. If you upload a directory, it is first compressed into a ZIP file. If the target already exists, it is overwritten.

Supported

oss_download_annon

Downloads a file from a specified URL to a local path. If the parent directory does not exist, it is automatically created. If the target file already exists, it is overwritten.

Supported

UI

click

Clicks a specific coordinate on the screen.

Supported

input_text

Inputs text.

Supported

send_key

Sends a key. Supported keys on the Android platform:

  • 3:HOME

  • 4:BACK

  • 24:VOLUME_UP

  • 25:VOLUME_DOWN

  • 26:POWER

  • 82:MENU

Supported

swipe

Performs a swipe gesture on the screen.

Supported

get_all_ui_elements

Gets all UI elements on the device within the timeout period, including non-interactive elements.

Supported

get_clickable_ui_elements

Gets all clickable UI elements within the timeout period.

Supported

App

get_installed_apps

Retrieves a list of applications installed on the system. Supports filtering by Start menu entries and desktop shortcuts, and you can choose to exclude system applications. Returns application details, including the name, start command, optional stop command, and working directory.

Supported

start_app

Starts a specified application using the provided command and optional working directory. Returns a list of processes associated with the launched application, including the process name, PID, and start command.

Supported

stop_app_by_cmd

Stops an application using the provided stop command. Use this with caution because it forcefully terminates the specified process.

Supported

Shell

shell

Executes a shell command on the Android platform and returns the output or an error.

Supported

OSS

Name

Description

Parameters

oss_env_init

Creates and initializes OSS environment variables. Specify the Endpoint, Access Key ID, Access Key Secret, Security Token, and Region. Temporary security credentials are obtained through Security Token Service (STS). For more information, see Use temporary access credentials from STS to access OSS.

{
  "inputSchema": {
    "properties": {
      "access_key_id": {
        "description": "The AccessKey ID for OSS authentication.",
        "type": "string"
      },
      "access_key_secret": {
        "description": "The AccessKey Secret for OSS authentication.",
        "type": "string"
      },
      "endpoint": {
        "description": "The OSS service endpoint. If not specified, the default is https://oss-cn-hangzhou.aliyuncs.com.",
        "type": "string"
      },
      "region": {
        "description": "The OSS region. If not specified, the default is cn-hangzhou.",
        "type": "string"
      },
      "security_token": {
        "description": "The security token for OSS authentication.",
        "type": "string"
      }
    },
    "required": ["access_key_id", "access_key_secret", "security_token"],
    "type": "object"
  }
}

oss_upload

Uploads a local file or directory to a specified OSS Bucket. If you upload a directory, it is first compressed into a ZIP file. You can specify the object name in OSS. If you do not specify a name, the file or ZIP name is used by default.

Note

Before using this tool, call the oss_env_init tool to initialize the OSS environment variables.

{
  "inputSchema": {
    "properties": {
      "bucket": {
        "description": "The OSS Bucket name.",
        "type": "string"
      },
      "object": {
        "description": "The object path in the OSS Bucket, for example, test/test.txt.",
        "type": "string"
      },
      "path": {
        "description": "The full path of the local file or non-empty directory to upload. For example, /tmp/test.txt or /tmp on Linux, or C:/tmp/test.txt or C:/tmp on Windows.",
        "type": "string"
      }
    },
    "required": ["bucket", "object", "path"],
    "type": "object"
  }
}

oss_download

Downloads an object from a specified OSS Bucket to a local path. If the parent directory does not exist, it is automatically created. If the target file already exists, it is overwritten.

Note

Before using this tool, call the oss_env_init tool to initialize the OSS environment variables.

{
  "inputSchema": {
    "properties": {
      "bucket": {
        "description": "The OSS Bucket name.",
        "type": "string"
      },
      "object": {
        "description": "The object path in the OSS Bucket, for example, test/test.txt.",
        "type": "string"
      },
      "path": {
        "description": "The full local path to save the downloaded file. For example, /tmp/test.txt on Linux or C:/tmp/test.txt on Windows.",
        "type": "string"
      }
    },
    "required": ["bucket", "object", "path"],
    "type": "object"
  }
}

oss_upload_annon

Uploads a local file or directory to a specified URL using HTTP PUT. If you upload a directory, it is first compressed into a ZIP file. If the target already exists, it is overwritten.

{
  "inputSchema": {
    "properties": {
      "path": {
        "description": "The full path of the local file or non-empty directory to upload. For example, /tmp/test.txt or /tmp on Linux, or C:/tmp/test.txt or C:/tmp on Windows.",
        "type": "string"
      },
      "url": {
        "description": "The HTTP/HTTPS URL to upload the file to.",
        "type": "string"
      }
    },
    "required": ["url", "path"],
    "type": "object"
  }
}

oss_download_annon

Downloads a file from a specified URL to a local path. If the parent directory does not exist, it is automatically created. If the target file already exists, it is overwritten.

{
  "inputSchema": {
    "properties": {
      "path": {
        "description": "The full local path to save the downloaded file. For example, /tmp/test.txt on Linux or C:/tmp/test.txt on Windows.",
        "type": "string"
      },
      "url": {
        "description": "The HTTP/HTTPS URL to download the file from.",
        "type": "string"
      }
    },
    "required": ["url", "path"],
    "type": "object"
  }
}

UI

Name

Description

Parameters

click

Clicks a specific coordinate on the screen.

{
  "inputSchema": {
    "properties": {
      "button": {
        "description": "The button type. Valid values: left, middle, and right. Default value: left.",
        "type": "string"
      },
      "x": {
        "description": "The X coordinate.",
        "type": "integer"
      },
      "y": {
        "description": "The Y coordinate.",
        "type": "integer"
      }
    },
    "required": ["x", "y", "button"],
    "type": "object"
  }
}

input_text

Inputs text.

{
  "inputSchema": {
    "properties": {
      "text": {
        "description": "The text entered by the client.",
        "type": "string"
      }
    },
    "required": ["text"],
    "type": "object"
  }
}

send_key

Sends a key. Supported keys on the Android platform:

  • 3:HOME

  • 4:BACK

  • 24:VOLUME_UP

  • 25:VOLUME_DOWN

  • 26:POWER

  • 82:MENU

{
  "inputSchema": {
    "properties": {
      "key": {
        "description": "The key sent by the client.",
        "type": "integer"
      }
    },
    "required": ["key"],
    "type": "object"
  }
}

swipe

Performs a swipe gesture on the screen.

{
  "inputSchema": {
    "properties": {
      "duration_ms": {
        "default": 300,
        "description": "The swipe duration in milliseconds. Default value: 300.",
        "type": "integer"
      },
      "end_x": {
        "description": "The ending X coordinate.",
        "type": "integer"
      },
      "end_y": {
        "description": "The ending Y coordinate.",
        "type": "integer"
      },
      "start_x": {
        "description": "The starting X coordinate.",
        "type": "integer"
      },
      "start_y": {
        "description": "The starting Y coordinate.",
        "type": "integer"
      }
    },
    "required": ["start_x", "start_y", "end_x", "end_y", "duration_ms"],
    "type": "object"
  }
}

get_all_ui_elements

Gets all UI elements on the device within the timeout period, including non-interactive elements.

{
  "inputSchema": {
    "properties": {
      "timeout_ms": {
        "default": 1000,
        "description": "The command execution timeout in milliseconds. If not specified, a default value, such as 1000, is used.",
        "type": "integer"
      }
    },
    "required": ["timeout_ms"],
    "type": "object"
  }
}

get_clickable_ui_elements

Gets all clickable UI elements within the timeout period.

{
  "inputSchema": {
    "properties": {
      "timeout_ms": {
        "default": 1000,
        "description": "The command execution timeout in milliseconds. If not specified, a default value, such as 1000, is used.",
        "type": "integer"
      }
    },
    "required": ["timeout_ms"],
    "type": "object"
  }
}

App

Name

Description

Parameters

get_installed_apps

Retrieves a list of applications installed on the system. Supports filtering by Start menu entries and desktop shortcuts, and you can choose to exclude system applications. Returns application details, including the name, start command, optional stop command, and working directory.

{
  "inputSchema": {
    "properties": {
      "desktop": {
        "default": false,
        "description": "Includes desktop shortcuts. Default value: false.",
        "type": "boolean"
      },
      "ignore_system_app": {
        "default": true,
        "description": "Excludes system applications. Default value: true.",
        "type": "boolean"
      },
      "start_menu": {
        "default": true,
        "description": "Includes Start menu applications. Default value: true.",
        "type": "boolean"
      }
    },
    "required": [],
    "type": "object"
  }
}

start_app

Starts a specified application using the provided command and optional working directory. Returns a list of processes associated with the launched application, including the process name, PID, and start command.

{
  "inputSchema": {
    "properties": {
      "activity": {
        "description": "The activity name on Android, for example, .xxActivity or com.xx/.xxActivity.",
        "type": "string"
      },
      "start_cmd": {
        "description": "The command to start the application. On Android, use the 'monkey -p ' syntax, for example, 'monkey -p <package_name> -c android.intent.category.LAUNCHER 1'.",
        "type": "string"
      },
      "work_directory": {
        "default": "",
        "description": "The working directory from which to start the application. If omitted, the default directory is used.",
        "type": "string"
      }
    },
    "required": ["start_cmd"],
    "type": "object"
  }
}

stop_app_by_cmd

Stops an application using the provided stop command. Use this with caution because it forcefully terminates the specified process.

{
  "inputSchema": {
    "properties": {
      "stop_cmd": {
        "description": "The command used to stop the application.",
        "type": "string"
      }
    },
    "required": ["stop_cmd"],
    "type": "object"
  }
}

Shell

Name

Description

Parameters

shell

Executes a shell command on the Android platform and returns the output or an error.

{
  "inputSchema": {
    "properties": {
      "command": {
        "description": "The command entered by the client.",
        "type": "string"
      },
      "timeout_ms": {
        "default": 1000,
        "description": "The command execution timeout in milliseconds. If not specified, a default value, such as 1000, is used.",
        "type": "integer"
      }
    },
    "required": ["command", "timeout_ms"],
    "type": "object"
  }
}

MCP Tool List

{
  "wuying_oss": {
    "tools": [
      {
        "name": "oss_env_init",
        "description": "Create and initialize OSS environment variables with the specified endpoint, access key ID, access key secret, security token, and region. The temporary security credentials obtained from the STS (Security Token Service). For more information, see: https://www.alibabacloud.com/help/zh/oss/developer-reference/use-temporary-access-credentials-provided-by-sts-to-access-oss?spm=a2c4g.11186623.help-menu-search-31815.d_1#9ab17afd7cs4t .",
        "inputSchema": {
          "properties": {
            "access_key_id": {
              "description": "The Access Key ID for OSS authentication",
              "type": "string"
            },
            "access_key_secret": {
              "description": "The Access Key Secret for OSS authentication",
              "type": "string"
            },
            "endpoint": {
              "description": "The OSS service endpoint, e.g.,  If not specified, the default is https://oss-cn-hangzhou.aliyuncs.com",
              "type": "string"
            },
            "region": {
              "description": "The OSS region, e.g., cn-hangzhou. If not specified, the default is cn-hangzhou",
              "type": "string"
            },
            "security_token": {
              "description": "The Security Token for OSS authentication",
              "type": "string"
            }
          },
          "required": [
            "access_key_id",
            "access_key_secret",
            "security_token"
          ],
          "type": "object"
        }
      },
      {
        "name": "oss_upload",
        "description": "Upload a local file or directory to the specified OSS bucket. If a directory is specified, it will be compressed into a ZIP file before uploading. The object name in OSS can be specified; if not, the file or ZIP name will be used by default. Note: You must call the oss_env_init tool to initialize OSS environment variables before using this tool.",
        "inputSchema": {
          "properties": {
            "bucket": {
              "description": "OSS bucket name",
              "type": "string"
            },
            "object": {
              "description": "Object path in OSS bucket, e.g., test/test.txt",
              "type": "string"
            },
            "path": {
              "description": "Local file or not empty directory full path to upload, e.g., /tmp/test.txt /tmp on Linux or C:/tmp/test.txt C:/tmp on Windows",
                            "type": "string"
                        }
                    },
                    "required": [
                        "bucket",
                        "object",
                        "path"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "oss_download",
                "description": "Download an object from the specified OSS bucket to the given local path. If the parent directory does not exist, it will be created automatically. If the target file already exists, it will be overwritten. Note: You must call the oss_env_init tool to initialize OSS environment variables before using this tool.",
                "inputSchema": {
                    "properties": {
                        "bucket": {
                            "description": "OSS bucket name",
                            "type": "string"
                        },
                        "object": {
                            "description": "Object path in OSS bucket, e.g., test/test.txt",
                            "type": "string"
                        },
                        "path": {
                            "description": "Local full path to save the downloaded file, e.g., /tmp/test.txt on Linux or C:/tmp/test.txt on Windows",
                            "type": "string"
                        }
                    },
                    "required": [
                        "bucket",
                        "object",
                        "path"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "oss_upload_annon",
                "description": "Upload a local file or directory to the specified URL using HTTP PUT. If a directory is specified, it will be compressed into a ZIP file before uploading. If the upload target already exists, it will be overwritten.",
                "inputSchema": {
                    "properties": {
                        "path": {
                            "description": "Local file or not empty directory full path to upload, e.g., /tmp/test.txt /tmp on Linux or C:/tmp/test.txt  C:/tmp on Windows",
                            "type": "string"
                        },
                        "url": {
                            "description": "The HTTP/HTTPS URL to upload the file to",
                            "type": "string"
                        }
                    },
                    "required": [
                        "url",
                        "path"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "oss_download_annon",
                "description": "Download a file from the specified URL to the given local path. If the parent directory does not exist, it will be created automatically. If the target file already exists, it will be overwritten.",
                "inputSchema": {
                    "properties": {
                        "path": {
                            "description": "The full local file path to save the downloaded file, e.g., /tmp/test.txt on Linux or C:/tmp/test.txt on Windows",
                            "type": "string"
                        },
                        "url": {
                            "description": "The HTTP/HTTPS URL to download the file from",
                            "type": "string"
                        }
                    },
                    "required": [
                        "url",
                        "path"
                    ],
                    "type": "object"
                }
            }
        ]
    },
    "wuying_ui": {
        "tools": [
            {
                "name": "click",
                "description": "Click on the screen at specific coordinates.",
                "inputSchema": {
                    "properties": {
                        "button": {
                            "description": "button type,Available values:left,middle,right,default:left",
                            "type": "string"
                        },
                        "x": {
                            "description": "X coordinate",
                            "type": "integer"
                        },
                        "y": {
                            "description": "Y coordinate",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "x",
                        "y",
                        "button"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "input_text",
                "description": "Input text",
                "inputSchema": {
                    "properties": {
                        "text": {
                            "description": "client input text",
                            "type": "string"
                        }
                    },
                    "required": [
                        "text"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "send_key",
                "description": "Send a key.Supported Keys on Android Platform: 3:HOME,4:BACK,24:VOLUME_UP,25:VOLUME_DOWN,26:POWER,82:MENU",
                "inputSchema": {
                    "properties": {
                        "key": {
                            "description": "client send key",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "key"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "swipe",
                "description": "Perform a swipe gesture on the screen.",
                "inputSchema": {
                    "properties": {
                        "duration_ms": {
                            "default": 300,
                            "description": "Duration of swipe in milliseconds,default:300",
                            "type": "integer"
                        },
                        "end_x": {
                            "description": "Ending X coordinate",
                            "type": "integer"
                        },
                        "end_y": {
                            "description": "Ending Y coordinate",
                            "type": "integer"
                        },
                        "start_x": {
                            "description": "Starting X coordinate",
                            "type": "integer"
                        },
                        "start_y": {
                            "description": "Starting Y coordinate",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "start_x",
                        "start_y",
                        "end_x",
                        "end_y",
                        "duration_ms"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "get_all_ui_elements",
                "description": "Get all UI elements from the device with timeout, including non-interactive elements.",
                "inputSchema": {
                    "properties": {
                        "timeout_ms": {
                            "default": 1000,
                            "description": "Command execution timeout (unit: milliseconds). If not specified, the default value (such as 1000 milliseconds) is used",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "timeout_ms"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "get_clickable_ui_elements",
                "description": "Get all clickable UI elements within timeout.",
                "inputSchema": {
                    "properties": {
                        "timeout_ms": {
                            "default": 1000,
                            "description": "Command execution timeout (unit: milliseconds). If not specified, the default value (such as 1000 milliseconds) is used",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "timeout_ms"
                    ],
                    "type": "object"
                }
            }
        ]
    },
    "wuying_app": {
        "tools": [
            {
                "name": "get_installed_apps",
                "description": "Retrieve a list of installed applications on the system. Supports filtering by Start Menu entries and Desktop shortcuts, with an option to exclude system applications. Returns application details including name, start command, optional stop command, and working directory.",
                "inputSchema": {
                    "properties": {
                        "desktop": {
                            "default": false,
                            "description": "Include Desktop shortcuts (default: false)",
                            "type": "boolean"
                        },
                        "ignore_system_app": {
                            "default": true,
                            "description": "Exclude system applications (default: true)",
                            "type": "boolean"
                        },
                        "start_menu": {
                            "default": true,
                            "description": "Include Start Menu applications (default: true)",
                            "type": "boolean"
                        }
                    },
                    "required": [],
                    "type": "object"
                }
            },
            {
                "name": "start_app",
                "description": "Start a specified application using the provided command and optional working directory. Returns a list of processes associated with the launched application, including their process names, PIDs, and startup commands.\n\n",
                "inputSchema": {
                    "properties": {
                        "activity": {
                            "description": "The activity name on android,like .xxActivity or com.xx/.xxActivity.",
                            "type": "string"
                        },
                        "start_cmd": {
                            "description": "The command to start the application. On Android, use the 'monkey -p ' syntax (e.g., 'monkey -p <package_name> -c android.intent.category.LAUNCHER 1').",
                            "type": "string"
                        },
                        "work_directory": {
                            "default": "",
                            "description": "The directory from which the application should be launched. If omitted, the default directory is used.",
                            "type": "string"
                        }
                    },
                    "required": [
                        "start_cmd"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "stop_app_by_cmd",
                "description": "Terminate an application using the provided stop command. Use with caution as this will forcefully terminate the specified process.",
                "inputSchema": {
                    "properties": {
                        "stop_cmd": {
                            "description": "The command used to terminate the application.",
                            "type": "string"
                        }
                    },
                    "required": [
                        "stop_cmd"
                    ],
                    "type": "object"
                }
            }
        ]
    },
    "wuying_shell": {
        "tools": [
            {
                "name": "shell",
                "description": "Executes an shell command with timeout and returns the output or an error on android platform.",
                "inputSchema": {
                    "properties": {
                        "command": {
                            "description": "client input command",
                            "type": "string"
                        },
                        "timeout_ms": {
                            "default": 1000,
                            "description": "Command execution timeout (unit: milliseconds). If not specified, the default value (such as 1000 milliseconds) is used",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "command",
                        "timeout_ms"
                    ],
                    "type": "object"
                }
            }
        ]
    }
}