API standards and multilingual preset SDKs
The OpenAPI of this product (emr-serverless-spark/2023-08-08) uses the ROA signature style. We have encapsulated SDKs for common programming languages for developers. Developers can download the SDK to directly call this product's OpenAPI without worrying about technical details. If the existing SDK does not meet your needs, you can use the signature mechanism for self-signing integration. Since the details of self-signing are very complex, it may take around 5 business days. Therefore, we recommend joining our DingTalk service group (147535001692) and conducting signature integration under expert guidance.
Before using the API, you need to prepare your identity account and access key (AccessKey) to effectively access the API through client tools (such as SDK and CLI). For details, see Obtain an AccessKey.
Custom signature scenarios
If your business scenario has special requirements and you need to integrate the API through self-signing, we recommend consulting our technical support team first (DingTalk service group: 147535001692) to obtain professional guidance and ensure efficient integration.
Account and security preparation
Alibaba Cloud accounts have full administrative permissions over all resources. Once an AccessKey is compromised, all associated resources will be at risk of unauthorized access. To ensure security, it is recommended to create a RAM user with only API access permissions and configure its AccessKey, while configuring RAM policies based on the principle of least privilege (PoLP). Use the Alibaba Cloud account only in specific scenarios where Alibaba Cloud account permissions are explicitly required.
Permission management
|
API |
Title |
Description |
| AddMembers | AddMembers | Adds a Resource Access Management (RAM) user or a RAM role to a workspace as a member. |
| ListMembers | ListMembers | Retrieves the list of members in the workspace. |
| GrantRoleToUsers | GrantRoleToUsers | Assigns a specified role to users. |
Session management
|
API |
Title |
Description |
| GetSessionCluster | GetSessionCluster | Retrieves session details. |
| CreateSessionCluster | Create a session | Creates a session. |
| StopSessionCluster | StopSessionCluster | Stops a session. |
| StartSessionCluster | StartSessionCluster | Starts a session cluster. |
| ListSessionClusters | ListSessionClusters | Retrieves a list of session clusters. |
Gateway
|
API |
Title |
Description |
| Kyuubi Gateway | Kyuubi Gateway | |
| ListKyuubiSparkApplications | ListKyuubiSparkApplications | Lists Kyuubi Spark applications. |
| CancelKyuubiSparkApplication | CancelKyuubiSparkApplication | Cancels a Kyuubi Spark application. |
| DeleteKyuubiToken | DeleteKyuubiToken | Deletes a Kyuubi Gateway authentication token. |
| UpdateKyuubiToken | UpdateKyuubiToken | Updates a Kyuubi Gateway authentication token. |
| GetKyuubiToken | Retrieve kyuubi gateway authentication token | Retrieves the Kyuubi Gateway authentication token. |
| CreateKyuubiToken | CreateKyuubiToken | Creates a Kyuubi Gateway authentication token. |
| ListKyuubiToken | ListKyuubiToken | Lists KyuubiServer authentication tokens. |
| ListKyuubiServices | List kyuubi gateway information | Lists Kyuubi Gateways. |
| CreateKyuubiService | CreateKyuubiService | Creates a Kyuubi gateway. |
| GetKyuubiService | Retrieve a kyuubi gateway | Retrieves a Kyuubi Gateway. |
| UpdateKyuubiService | UpdateKyuubiService | Updates a Kyuubi Gateway. |
| DeleteKyuubiService | DeleteKyuubiService | Deletes a Kyuubi gateway. |
| StartKyuubiService | StartKyuubiService | Starts the Kyuubi Gateway. |
| StopKyuubiService | StopKyuubiService | Stops a Kyuubi Gateway. |
| Livy Gateway | Livy Gateway | |
| ListLivyComputeSessions | ListLivyComputeSessions | Lists the historical sessions for a Livy Gateway. |
| DeleteLivyCompute | DeleteLivyCompute | Deletes a Livy gateway. |
| CreateLivyCompute | CreateLivyCompute | Creates a Livy compute. |
| UpdateLivyCompute | UpdateLivyCompute | Updates a Livy Gateway. |
| ListLivyCompute | ListLivyCompute | Lists Livy Gateways. |
| StartLivyCompute | StartLivyCompute | Starts a Livy gateway. |
| RefreshLivyComputeToken | RefreshLivyComputeToken | Refreshes the token for a Livy Gateway. |
| GetLivyComputeToken | Retrieve the token for a livy gateway | Retrieves the token for a Livy Gateway. |
| CreateLivyComputeToken | CreateLivyComputeToken | Creates a token for a Livy Gateway. |
| DeleteLivyComputeToken | DeleteLivyComputeToken | Deletes a token for a Livy Gateway. |
| StopLivyCompute | StopLivyCompute | Stops a Livy Gateway. |
| ListLivyComputeToken | List livy gateway tokens | Lists the tokens of a Livy Gateway. |
| GetLivyCompute | Retrieve livy gateway information | Retrieves the Livy Gateway information. |
Data development
|
API |
Title |
Description |
| TerminateSqlStatement | TerminateSqlStatement | Terminates an SQL query task. |
| CreateSqlStatement | Create an SQL query | Creates an SQL query. |
| GetSqlStatement | GetSqlStatement | Retrieves the details of an SQL query. |
| ListSqlStatementContents | ListSqlStatementContents | Retrieves the execution results of an SQL statement. |
Workspace management
|
API |
Title |
Description |
| ListWorkspaceQueues | List workspace queues | Queries the list of queues in a workspace. |
| EditWorkspaceQueue | Edit a workspace queue | Edits a workspace queue. |
Spark version management
|
API |
Title |
Description |
| ListReleaseVersions | Retrieve spark version list | Retrieves a list of release versions. |
Spark task management
|
API |
Title |
Description |
| ListJobExecutors | ListJobExecutors | Lists the executors of a job. |
| ListLogContents | Query Log Content | Get Log Content |
| GetCuHours | GetCuHours | Queries the number of CU-hours consumed by a queue during a specified cycle. |
| GetDoctorApplication | GetDoctorApplication | Obtains job analysis information on E-MapReduce (EMR) Doctor. |
| CancelJobRun | CancelJobRun | Terminates a Spark job. |
| ListJobRuns | List spark jobs | Queries a list of Spark jobs. |
| GetJobRun | Retrieve spark job run details | Retrieves the details of a Spark job run by calling GetJobRun. |
| StartJobRun | StartJobRun | Starts a Spark Job. |
| GetRunConfiguration | GetRunConfiguration | Retrieves the configuration of a Spark job. |
Ray task management
|
API |
Title |
Description |
| ListRayLogs | List ray logs | Lists Ray logs. |
| GetRayLog | Get ray job log | Retrieves the log of a Ray job. |
| CancelRayJob | Stop a ray job | Stops a RayJob. |
| SubmitRayJob | Submit a ray job | Submits a Ray job. |
| ListRayJob | List ray job information | Lists Ray Job information. |
| GetRayJob | Retrieve ray job information | Retrieves Ray Job information. |
Task orchestration
|
API |
Title |
Description |
| StartProcessInstance | Manually run a workflow | Manually runs a workflow. |
| UpdateProcessDefinitionWithSchedule | UpdateProcessDefinitionWithSchedule | Updates a workflow definition and its timed scheduling. |
| CreateProcessDefinitionWithSchedule | Create a workflow definition and schedule | Creates a workflow definition. |
| GenerateTaskCodes | GenerateTaskCodes | Publishes a workflow and its scheduling configuration. |
Cluster management
|
API |
Title |
Description |
| Ray cluster | Ray cluster | |
| UpdateRayCluster | Update a ray cluster | Updates a Ray cluster. |
| GetRayCluster | Retrieve ray cluster information | Retrieves a Ray cluster. |
| ListRayCluster | ListRayCluster | Lists Ray clusters. |
| StopRayCluster | StopRayCluster | Stops a Ray cluster. |
| StartRayCluster | StartRayCluster | Starts a Ray cluster. |
| DeleteRayCluster | DeleteRayCluster | Deletes a Ray cluster. |
| CreateRayCluster | Create a ray cluster | Creates a Ray cluster. |
Network connectivity
|
API |
Title |
Description |
| ListNetworkServices | ListNetworkServices | Lists network connections. |
| CreateNetworkService | CreateNetworkService | Creates a network service. |
Others
|
API |
Title |
Description |
| ListCatalogs | ListCatalogs | View the list of data catalogs |
| CreateWorkspaceQueue | Create a workspace queue | Creates a workspace queue. |
| DeleteWorkspaceQueue | Delete a workspace queue | Deletes a workspace queue. |
| ListExecutorLogs | ListExecutorLogs | Lists the log files for a job executor. |
| GetCacheCluster | Get cache cluster details | Retrieves the details of a Cache cluster. |
| StartCacheCluster | Start Cache Cluster | Start CacheCluster |
| StopCacheCluster | Stop a cache cluster | Stops a CacheCluster. |
| ActivateAICenter | ActivateAICenter | Activates the AI center. |
| ChangeResourceGroup | ChangeResourceGroup | Moves an instance to a different resource group. |
| CreateWorkspace | CreateWorkspace | Creates a workspace. |
| GetAICenterState | Retrieve AI hub status | Retrieves the status of AI Hub. |
| GetTemplate | GetTemplate | Retrieves a task template. |
| ListTemplate | ListTemplate | Lists job templates. |
| ListWorkspaces | List workspaces | Queries a list of workspaces. |
| UpdateWorkspace | Update workspace | Updates the properties of a workspace. |
| QueryApmGrafanaData | Query job metrics | Queries APM Grafana panel data for Serverless Spark. |