API standards and multilingual preset SDKs
The OpenAPI of this product (emr-serverless-spark/2023-08-08) uses the ROA signature style. We have encapsulated SDKs for common programming languages for developers. Developers can download the SDK to directly call this product's OpenAPI without worrying about technical details. If the existing SDK does not meet your needs, you can use the signature mechanism for self-signing integration. Since the details of self-signing are very complex, it may take around 5 business days. Therefore, we recommend joining our DingTalk service group (147535001692) and conducting signature integration under expert guidance.
Before using the API, you need to prepare your identity account and access key (AccessKey) to effectively access the API through client tools (such as SDK and CLI). For details, see Obtain an AccessKey.
Custom signature scenarios
If your business scenario has special requirements and you need to integrate the API through self-signing, we recommend consulting our technical support team first (DingTalk service group: 147535001692) to obtain professional guidance and ensure efficient integration.
Account and security preparation
Alibaba Cloud accounts have full administrative permissions over all resources. Once an AccessKey is compromised, all associated resources will be at risk of unauthorized access. To ensure security, it is recommended to create a RAM user with only API access permissions and configure its AccessKey, while configuring RAM policies based on the principle of least privilege (PoLP). Use the Alibaba Cloud account only in specific scenarios where Alibaba Cloud account permissions are explicitly required.
Permission management
|
API |
Title |
Description |
| AddMembers | AddMembers | Adds a Resource Access Management (RAM) user or a RAM role to a workspace as a member. |
| ListMembers | ListMembers | Retrieves the list of members in the workspace. |
| GrantRoleToUsers | GrantRoleToUsers | Assigns a specified role to users. |
Session management
|
API |
Title |
Description |
| GetSessionCluster | GetSessionCluster | Retrieves session details. |
| CreateSessionCluster | CreateSessionCluster | Creates a session cluster. |
| StartSessionCluster | StartSessionCluster | Starts a session cluster. |
| ListSessionClusters | ListSessionClusters | Retrieves a list of session clusters. |
Gateway
|
API |
Title |
Description |
| Kyuubi Gateway | Kyuubi Gateway | |
| ListKyuubiSparkApplications | ListKyuubiSparkApplications | Lists Kyuubi Spark applications. |
| CancelKyuubiSparkApplication | CancelKyuubiSparkApplication | Cancels a Kyuubi Spark application. |
| DeleteKyuubiToken | DeleteKyuubiToken | Deletes a Kyuubi Gateway authentication token. |
| UpdateKyuubiToken | UpdateKyuubiToken | Updates a Kyuubi Gateway authentication token. |
| GetKyuubiToken | GetKyuubiToken | Obtains an authentication token for Kyuubi Gateway. |
| CreateKyuubiToken | CreateKyuubiToken | Creates a Kyuubi Gateway authentication token. |
| ListKyuubiToken | ListKyuubiToken | Lists KyuubiServer authentication tokens. |
| ListKyuubiServices | ListKyuubiServices | Lists Kyuubi Gateways. |
| CreateKyuubiService | CreateKyuubiService | Creates a Kyuubi gateway. |
| GetKyuubiService | GetKyuubiService | Retrieves the details of a Kyuubi Gateway. |
| UpdateKyuubiService | UpdateKyuubiService | Updates a Kyuubi Gateway. |
| DeleteKyuubiService | DeleteKyuubiService | Deletes a Kyuubi gateway. |
| StartKyuubiService | StartKyuubiService | Starts the Kyuubi Gateway. |
| StopKyuubiService | StopKyuubiService | Stops a Kyuubi Gateway. |
| Livy Gateway | Livy Gateway | |
| ListLivyComputeSessions | ListLivyComputeSessions | Lists the historical sessions for a Livy Gateway. |
| DeleteLivyCompute | DeleteLivyCompute | Deletes a Livy gateway. |
| CreateLivyCompute | CreateLivyCompute | Creates a Livy compute. |
| UpdateLivyCompute | UpdateLivyCompute | Updates a Livy Gateway. |
| ListLivyCompute | ListLivyCompute | Lists Livy Gateways. |
| StartLivyCompute | StartLivyCompute | Starts a Livy gateway. |
| RefreshLivyComputeToken | RefreshLivyComputeToken | Refreshes the token for a Livy Gateway. |
| GetLivyComputeToken | GetLivyComputeToken | Retrieves a token for a Livy Gateway. |
| CreateLivyComputeToken | CreateLivyComputeToken | Creates a token for a Livy Gateway. |
| DeleteLivyComputeToken | DeleteLivyComputeToken | Deletes a token for a Livy Gateway. |
| StopLivyCompute | StopLivyCompute | Stops a Livy Gateway. |
| ListLivyComputeToken | ListLivyComputeToken | Lists Livy Gateway tokens. |
| GetLivyCompute | GetLivyCompute | Retrieves information about a Livy Gateway. |
Data development
|
API |
Title |
Description |
| TerminateSqlStatement | TerminateSqlStatement | Terminates an SQL query task. |
| CreateSqlStatement | CreateSqlStatement | Creates an SQL statement. |
| GetSqlStatement | GetSqlStatement | Retrieves the details of an SQL query. |
| ListSqlStatementContents | ListSqlStatementContents | Retrieves the execution results of an SQL statement. |
Workspace management
|
API |
Title |
Description |
| ListWorkspaceQueues | ListWorkspaceQueues | Lists the queues in a workspace. |
| EditWorkspaceQueue | EditWorkspaceQueue | Updates a Workspace Queue. |
Spark version management
|
API |
Title |
Description |
| ListReleaseVersions | ListReleaseVersions | Retrieves a list of release versions. |
Spark job management
|
API |
Title |
Description |
| ListJobExecutors | ListJobExecutors | Lists the executors of a job. |
| ListLogContents | Query Log Content | Get Log Content |
| GetDoctorApplication | GetDoctorApplication | Obtains job analysis information on E-MapReduce (EMR) Doctor. |
| ListJobRuns | ListJobRuns | Call the ListJobRuns operation to retrieve a list of Spark jobs. |
| GetJobRun | GetJobRun | Get the details of a job. |
| StartJobRun | StartJobRun | Starts a Spark Job. |
| GetRunConfiguration | GetRunConfiguration | Retrieves the configuration of a Spark job. |
Ray job management
|
API |
Title |
Description |
| StartProcessInstance | StartProcessInstance | Starts a workflow manually. |
| UpdateProcessDefinitionWithSchedule | UpdateProcessDefinitionWithSchedule | Updates a workflow definition and its timed scheduling. |
| CreateProcessDefinitionWithSchedule | CreateProcessDefinitionWithSchedule | Creates a workflow definition. |
| GenerateTaskCodes | GenerateTaskCodes | Publishes a workflow and its scheduling configuration. |
Job orchestration
|
API |
Title |
Description |
| Ray cluster | Ray cluster | |
| UpdateRayCluster | UpdateRayCluster | Updates a Ray cluster. |
| GetRayCluster | GetRayCluster | Retrieves the details of a Ray cluster, including its configuration, runtime state, node information, and connection endpoints. |
| ListRayCluster | ListRayCluster | Lists Ray clusters. |
| StopRayCluster | StopRayCluster | Stops a Ray cluster. |
| StartRayCluster | StartRayCluster | Starts a Ray cluster. |
| DeleteRayCluster | DeleteRayCluster | Deletes a Ray cluster. |
| CreateRayCluster | CreateRayCluster | Creates a Ray cluster. |
Cluster management
|
API |
Title |
Description |
| ListNetworkServices | ListNetworkServices | Lists network connections. |
| CreateNetworkService | CreateNetworkService | Creates a network service. |
Network connectivity
|
API |
Title |
Description |
| ListCatalogs | ListCatalogs | View the list of data catalogs |
| ListExecutorLogs | ListExecutorLogs | Lists the log files for a job executor. |
| GetCacheCluster | Get cache cluster details | Retrieves the details of a Cache cluster. |
| StartCacheCluster | Start Cache Cluster | Start CacheCluster |
| StopCacheCluster | Stop a cache cluster | Stops a CacheCluster. |
| ActivateAICenter | ActivateAICenter | Activates the AI center. |
| ChangeResourceGroup | ChangeResourceGroup | Moves an instance to a different resource group. |
| CreateWorkspace | CreateWorkspace | Creates a workspace. |
| GetAICenterState | GetAICenterState | Gets the state of an AI center. |
| GetTemplate | GetTemplate | Retrieves a task template. |
| ListTemplate | ListTemplate | Lists job templates. |
| ListWorkspaces | ListWorkspaces | Call `ListWorkspaces` to get a list of workspaces. |
| UpdateWorkspace | UpdateWorkspace | Update workspace properties |
| CancelJobRun | CancelJobRun | Terminates a Spark job. |
| GetCuHours | GetCuHours | Queries the number of CU-hours consumed by a queue during a specified cycle. |
| GetRayLog | Get ray job log | Retrieves the log of a Ray job. |
| ListRayLogs | List ray logs | Lists Ray logs. |
| StopSessionCluster | StopSessionCluster | Stops a session. |