MobileAgentSDK for Android - ECP SDK integration guide - Cloud Phone: Central de Documentação do Alibaba Cloud

Baixe o SDK

Uso

Importe o AAR

Copie o arquivo AAR para seu projeto e adicione a dependência no build.gradle:

dependencies {
    implementation(files('libs/sdk-release.aar'))
}

Uso básico

Opção 1: Usar Flow (recomendado)

O Flow oferece tratamento reativo de eventos para corrotinas do Kotlin:

import com.wuying.mobileagentsdk.MobileAgentSdk
import com.wuying.mobileagentsdk.MobileAgentEvent
import com.wuying.mobileagentsdk.MobileAgentState
import kotlinx.coroutines.flow.filterIsInstance
import kotlinx.coroutines.launch
import kotlinx.coroutines.flow.collectLatest

// Use try-with-resources to automatically manage the lifecycle.
MobileAgentSdk().use { sdk ->
    // Collect events in a coroutine.
    lifecycleScope.launch {
        sdk.events.collectLatest { event ->
            when (event) {
                is MobileAgentEvent.StateChanged -> {
                    when (event.state) {
                        MobileAgentState.CONNECTED -> {
                            Log.d("SDK", "Connected")
                            // Execute a task.
                            sdk.executeTask("Please help me open settings", 10)
                        }
                        MobileAgentState.DISCONNECTED -> {
                            Log.d("SDK", "Disconnected")
                        }
                        else -> {}
                    }
                }
                is MobileAgentEvent.TaskResult -> {
                    Log.d("SDK", "Task complete: ${event.resultText}")
                }
                is MobileAgentEvent.Stream -> {
                    Log.d("SDK", "Stream data: ${event.content}")
                }
                else -> {}
            }
        }
    }

    // You can also collect only specific types of events.
    lifecycleScope.launch {
        sdk.events
            .filterIsInstance<MobileAgentEvent.StateChanged>()
            .collect { event ->
                Log.d("SDK", "State: ${event.state}")
            }
    }

    lifecycleScope.launch {
        sdk.events
            .filterIsInstance<MobileAgentEvent.TaskResult>()
            .collect { event ->
                Log.d("SDK", "Task result: ${event.resultText}")
            }
    }

    // Connect to the server.
    sdk.connect("wss://localhost:30005/ws")

    // The SDK is automatically destroyed at the end of the use block.
}

Opção 2: Usar callback

Para projetos sem corrotinas:

import com.wuying.mobileagentsdk.MobileAgentSdk
import com.wuying.mobileagentsdk.MobileAgentCallback
import com.wuying.mobileagentsdk.MobileAgentState

MobileAgentSdk().use { sdk ->
    // Set the callback.
    sdk.setCallback(object : MobileAgentCallback {
        override fun onStateChanged(state: MobileAgentState) {
            when (state) {
                MobileAgentState.CONNECTED -> {
                    Log.d("SDK", "Connected")
                    sdk.executeTask("Please help me open settings", 10)
                }
                MobileAgentState.DISCONNECTED -> {
                    Log.d("SDK", "Disconnected")
                }
                else -> {}
            }
        }

        override fun onTaskResult(
            taskId: String,
            result: String,
            resultText: String,
            success: Boolean,
            actualSteps: Int
        ) {
            Log.d("SDK", "Task complete: $resultText")
        }

        override fun onStream(
            taskId: String,
            step: Int,
            agent: String,
            contentType: String,
            content: String
        ) {
            Log.d("SDK", "Stream data: $content")
        }

        // Implement other callbacks as needed.
    })

    // Connect to the server.
    sdk.connect("wss://localhost:30005/ws")
}

Gerenciamento de ciclo de vida

O SDK implementa AutoCloseable para limpar recursos automaticamente:

// Recommended: Automatically manage the lifecycle.
MobileAgentSdk().use { sdk ->
    sdk.connect("wss://localhost:30005/ws")
    // Use the SDK...
} // Automatically calls close() to release resources.

// Or, manage the lifecycle manually.
val sdk = MobileAgentSdk()
try {
    sdk.connect("wss://localhost:30005/ws")
    // Use the SDK...
} finally {
    sdk.close()  // or sdk.destroy()
}

API

Gerenciamento de conexão

connect(url: String)

Estabelece conexão com o servidor:

sdk.connect("wss://localhost:30005/ws")

connectWithTicket(ticket: String)

Conecta-se mediante ticket (requer suporte no servidor):

sdk.connectWithTicket("your-ticket-here")

disconnect()

Encerra a conexão com o servidor:

sdk.disconnect()

Gerenciamento de tarefas

executeTask(task: String, maxSteps: Int): String

Executa uma tarefa e retorna o ID correspondente:

val taskId = sdk.executeTask("Please help me open the settings app", 10)

pauseTask(taskId: String)

Pausa uma tarefa:

sdk.pauseTask(taskId)

resumeTask(taskId: String)

Retoma uma tarefa:

sdk.resumeTask(taskId)

cancelTask(taskId: String)

Cancela uma tarefa:

sdk.cancelTask(taskId)

Consulta de estado

getState(): MobileAgentState

Retorna o estado atual da conexão:

val state = sdk.getState()
when (state) {
    MobileAgentState.CONNECTED -> // Connected
    MobileAgentState.CONNECTING -> // Connecting
    MobileAgentState.DISCONNECTED -> // Disconnected
}

API Flow

O Flow events emite todos os tipos de eventos. Utilize filterIsInstance para filtrar eventos específicos:

// Collect all events.
sdk.events.collect { event ->
    when (event) {
        is MobileAgentEvent.StateChanged -> // Handle state changes.
        is MobileAgentEvent.TaskResult -> // Handle task results.
        is MobileAgentEvent.Stream -> // Handle stream data.
        // ...
    }
}

// Collect only specific types of events.
sdk.events
    .filterIsInstance<MobileAgentEvent.StateChanged>()
    .collect { event ->
        // Handle only state changes.
    }

Tipos de eventos:

Tipo de evento	Descrição
`MobileAgentEvent.StateChanged`	Estado da conexão alterado
`MobileAgentEvent.TodoWrite`	Lista de tarefas atualizada
`MobileAgentEvent.TaskStart`	Tarefa iniciada
`MobileAgentEvent.StepStart`	Etapa iniciada
`MobileAgentEvent.Stream`	Dados de stream recebidos
`MobileAgentEvent.StepEnd`	Etapa finalizada
`MobileAgentEvent.TaskResult`	Resultado da tarefa recebido

API Callback

Métodos de MobileAgentCallback. Todos possuem implementações padrão vazias; sobrescreva apenas o necessário:

Método	Descrição
`onStateChanged(state: MobileAgentState)`	Estado da conexão alterado
`onTodoWrite(todos: Array<TodoItem>, count: Int)`	Lista de tarefas atualizada
`onTaskStart(taskId: String, task: String, maxSteps: Int)`	Tarefa iniciada
`onStepStart(taskId: String, step: Int)`	Etapa iniciada
`onStream(taskId, step, agent, contentType, content)`	Dados de stream recebidos
`onStepEnd(taskId, step, status, summary)`	Etapa finalizada
`onTaskResult(taskId, result, resultText, success, actualSteps)`	Resultado da tarefa recebido
`onRawMessage(rawJson: String)`	Mensagem JSON bruta e não analisada

Classes de dados

MobileAgentState

Enumeração dos estados de conexão:

enum class MobileAgentState {
    DISCONNECTED,  // Disconnected
    CONNECTING,    // Connecting
    CONNECTED      // Connected
}

TodoItem

Classe de dados para itens de tarefa:

data class TodoItem(
    val id: String,
    val title: String,
    val status: String,
    val details: String
)

MobileAgentEvent

Classe selada para todos os tipos de eventos:

sealed class MobileAgentEvent {
    data class StateChanged(val state: MobileAgentState)
    data class TodoWrite(val todos: Array<TodoItem>, val count: Int)
    data class TaskStart(val taskId: String, val task: String, val maxSteps: Int)
    data class StepStart(val taskId: String, val step: Int)
    data class Stream(val taskId: String, val step: Int, val agent: String, val contentType: String, val content: String)
    data class StepEnd(val taskId: String, val step: Int, val status: String, val summary: String)
    data class TaskResult(val taskId: String, val result: String, val resultText: String, val success: Boolean, val actualSteps: Int)
}

Uso avançado

Analisar pensamentos do agente

O MobileAgent encapsula os pensamentos do agente em tags <conclusion> na saída do stream. Extraia esse conteúdo dos eventos Stream.

Formato de `<conclusion>`

Formato XML:

<conclusion>This is the agent's thought process...</conclusion>

Implementação em Kotlin

Analise os pensamentos durante a transmissão do stream:

import com.wuying.mobileagentsdk.MobileAgentSdk
import com.wuying.mobileagentsdk.MobileAgentEvent

class ConclusionParser {
    private val conclusionBuffer = StringBuilder()
    private var inConclusion = false

    /**
     * Processes stream data to extract content from <conclusion> tags.
     */
    fun processStream(content: String): String? {
        val result = StringBuilder()

        for (char in content) {
            if (!inConclusion) {
                // Not inside a conclusion; check for the start of <conclusion>.
                conclusionBuffer.append(char)

                // Check for <conclusion>.
                if (conclusionBuffer.length >= 12) {
                    if (conclusionBuffer.endsWith("<conclusion>")) {
                        inConclusion = true
                        conclusionBuffer.clear()
                        result.append("[Thought] ")
                    } else if (conclusionBuffer.length > 12) {
                        // Keep the last 11 characters for the next check.
                        conclusionBuffer.delete(0, conclusionBuffer.length - 11)
                    }
                }
            } else {
                // Inside a conclusion; check for the end tag.
                conclusionBuffer.append(char)

                // Check for </conclusion>.
                if (conclusionBuffer.length >= 13) {
                    if (conclusionBuffer.endsWith("</conclusion>")) {
                        // Found the end tag.
                        inConclusion = false
                        val conclusionContent = conclusionBuffer.toString()
                            .dropLast(13)  // Remove </conclusion>.
                            .trim()

                        if (conclusionContent.isNotEmpty()) {
                            result.append(conclusionContent).append("\n")
                        }

                        conclusionBuffer.clear()
                        continue
                    } else if (conclusionBuffer.length > 13) {
                        // Keep the last 12 characters for the next check.
                        conclusionBuffer.delete(0, conclusionBuffer.length - 12)
                    }
                }

                // If not part of a potential end tag, output immediately.
                if (!couldBeEndTag(conclusionBuffer.toString())) {
                    result.append(char)
                }
            }
        }

        return if (result.isNotEmpty()) result.toString() else null
    }

    /**
     * Checks if the end of the buffer could be a prefix of the end tag.
     */
    private fun couldBeEndTag(buffer: String): Boolean {
        val prefixes = listOf(
            "<", "</", "</c", "</co", "</con", "</conc",
            "</concl", "</conclu", "</conclus", "</conclusi",
            "</conclusio", "</conclusion"
        )

        return prefixes.any { buffer.endsWith(it) }
    }

    /**
     * Resets the parser state.
     */
    fun reset() {
        conclusionBuffer.clear()
        inConclusion = false
    }
}

// Sample usage
MobileAgentSdk().use { sdk ->
    val parser = ConclusionParser()

    lifecycleScope.launch {
        sdk.events
            .filterIsInstance<MobileAgentEvent.Stream>()
            .collect { event ->
                // Parse and print the output.
                val output = parser.processStream(event.content)
                if (output != null) {
                    print(output)
                }
            }
    }

    sdk.connect("wss://localhost:30005/ws")
}

Versão simplificada

Para analisar após a conclusão da tarefa, em vez de durante o stream:

/**
 * Extracts all content from <conclusion> tags in a complete string.
 */
fun extractConclusions(content: String): List<String> {
    val conclusions = mutableListOf<String>()
    val regex = "<conclusion>(.*?)</conclusion>".toRegex(RegexOption.DOT_MATCHES_ALL)

    for (match in regex.findAll(content)) {
        val conclusion = match.groupValues[1].trim()
        if (conclusion.isNotEmpty()) {
            conclusions.add(conclusion)
        }
    }

    return conclusions
}

// Sample usage
lifecycleScope.launch {
    sdk.events
        .filterIsInstance<MobileAgentEvent.TaskResult>()
        .collect { event ->
            // Extract the thought process from the task result.
            val conclusions = extractConclusions(event.resultText)

            println("Agent's thought process:")
            conclusions.forEachIndexed { index, conclusion ->
                println("${index + 1}. $conclusion")
            }
        }
}

Saída

[Thought] The user wants to open the Settings app. I need to find its package name and Activity.
[Thought] By querying the system app list, I found the Settings app: com.android.settings/.Settings
[Thought] Now I will start the Settings app.

Task complete: Successfully opened the settings app.

Usar Flow e callbacks simultaneamente

Ambas as abordagens funcionam em conjunto:

MobileAgentSdk().use { sdk ->
    // Use Flow to handle the main logic.
    lifecycleScope.launch {
        sdk.events
            .filterIsInstance<MobileAgentEvent.TaskResult>()
            .collect { event ->
                // Handle the task result.
            }
    }

    // Use a callback for logging.
    sdk.setCallback(object : MobileAgentCallback {
        override fun onRawMessage(rawJson: String) {
            // Log raw messages for debugging.
            Log.d("SDK_RAW", rawJson)
        }
    })

    sdk.connect("wss://localhost:30005/ws")
}

Tratamento de erros

try {
    MobileAgentSdk().use { sdk ->
        sdk.connect("wss://localhost:30005/ws")
    }
} catch (e: IllegalStateException) {
    Log.e("SDK", "Failed to create SDK", e)
} catch (e: Exception) {
    Log.e("SDK", "Connection error", e)
}

Verificar status do SDK

val sdk = MobileAgentSdk()

// Check if the SDK is destroyed.
if (sdk.isDestroyed()) {
    Log.w("SDK", "The SDK has been destroyed.")
}

// Check if the SDK is valid.
sdk.checkNotDestroyed()  // Throws an exception if destroyed.

Configuração do ProGuard

O SDK inclui proguard-rules.pro para preservar métodos nativos. Geralmente, configurações adicionais são desnecessárias.

Configuração manual:

# MobileAgentSDK
-keep class com.wuying.mobileagentsdk.** { *; }
-keepclassmembers class com.wuying.mobileagentsdk.MobileAgentSdk {
    private native <methods>;
}

Dependências

Dependência de runtime:

dependencies {
    // The only runtime dependency
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3")
}

O SDK exclui frameworks de UI (AppCompat, Material) para permanecer leve.