MobileAgentSDK for Android - ECP SDK integration guide - Cloud Phone

Download the SDK

Usage

Import the AAR

Copy the AAR file into your project and add the dependency to build.gradle:

dependencies {
    implementation(files('libs/sdk-release.aar'))
}

Basic usage

Option 1: Use Flow (recommended)

Flow provides reactive event handling for Kotlin coroutines:

import com.wuying.mobileagentsdk.MobileAgentSdk
import com.wuying.mobileagentsdk.MobileAgentEvent
import com.wuying.mobileagentsdk.MobileAgentState
import kotlinx.coroutines.flow.filterIsInstance
import kotlinx.coroutines.launch
import kotlinx.coroutines.flow.collectLatest

// Use try-with-resources to automatically manage the lifecycle.
MobileAgentSdk().use { sdk ->
    // Collect events in a coroutine.
    lifecycleScope.launch {
        sdk.events.collectLatest { event ->
            when (event) {
                is MobileAgentEvent.StateChanged -> {
                    when (event.state) {
                        MobileAgentState.CONNECTED -> {
                            Log.d("SDK", "Connected")
                            // Execute a task.
                            sdk.executeTask("Please help me open settings", 10)
                        }
                        MobileAgentState.DISCONNECTED -> {
                            Log.d("SDK", "Disconnected")
                        }
                        else -> {}
                    }
                }
                is MobileAgentEvent.TaskResult -> {
                    Log.d("SDK", "Task complete: ${event.resultText}")
                }
                is MobileAgentEvent.Stream -> {
                    Log.d("SDK", "Stream data: ${event.content}")
                }
                else -> {}
            }
        }
    }

    // You can also collect only specific types of events.
    lifecycleScope.launch {
        sdk.events
            .filterIsInstance<MobileAgentEvent.StateChanged>()
            .collect { event ->
                Log.d("SDK", "State: ${event.state}")
            }
    }

    lifecycleScope.launch {
        sdk.events
            .filterIsInstance<MobileAgentEvent.TaskResult>()
            .collect { event ->
                Log.d("SDK", "Task result: ${event.resultText}")
            }
    }

    // Connect to the server.
    sdk.connect("wss://localhost:30005/ws")

    // The SDK is automatically destroyed at the end of the use block.
}

Option 2: Use a callback

For projects without coroutines:

import com.wuying.mobileagentsdk.MobileAgentSdk
import com.wuying.mobileagentsdk.MobileAgentCallback
import com.wuying.mobileagentsdk.MobileAgentState

MobileAgentSdk().use { sdk ->
    // Set the callback.
    sdk.setCallback(object : MobileAgentCallback {
        override fun onStateChanged(state: MobileAgentState) {
            when (state) {
                MobileAgentState.CONNECTED -> {
                    Log.d("SDK", "Connected")
                    sdk.executeTask("Please help me open settings", 10)
                }
                MobileAgentState.DISCONNECTED -> {
                    Log.d("SDK", "Disconnected")
                }
                else -> {}
            }
        }

        override fun onTaskResult(
            taskId: String,
            result: String,
            resultText: String,
            success: Boolean,
            actualSteps: Int
        ) {
            Log.d("SDK", "Task complete: $resultText")
        }

        override fun onStream(
            taskId: String,
            step: Int,
            agent: String,
            contentType: String,
            content: String
        ) {
            Log.d("SDK", "Stream data: $content")
        }

        // Implement other callbacks as needed.
    })

    // Connect to the server.
    sdk.connect("wss://localhost:30005/ws")
}

Lifecycle management

The SDK implements AutoCloseable for automatic resource cleanup:

// Recommended: Automatically manage the lifecycle.
MobileAgentSdk().use { sdk ->
    sdk.connect("wss://localhost:30005/ws")
    // Use the SDK...
} // Automatically calls close() to release resources.

// Or, manage the lifecycle manually.
val sdk = MobileAgentSdk()
try {
    sdk.connect("wss://localhost:30005/ws")
    // Use the SDK...
} finally {
    sdk.close()  // or sdk.destroy()
}

API

Connection management

connect(url: String)

Connects to the server:

sdk.connect("wss://localhost:30005/ws")

connectWithTicket(ticket: String)

Connects with a ticket (requires server-side support):

sdk.connectWithTicket("your-ticket-here")

disconnect()

Disconnects from the server:

sdk.disconnect()

Task management

executeTask(task: String, maxSteps: Int): String

Executes a task and returns the task ID:

val taskId = sdk.executeTask("Please help me open the settings app", 10)

pauseTask(taskId: String)

Pauses a task:

sdk.pauseTask(taskId)

resumeTask(taskId: String)

Resumes a task:

sdk.resumeTask(taskId)

cancelTask(taskId: String)

Cancels a task:

sdk.cancelTask(taskId)

State query

getState(): MobileAgentState

Returns the current connection state:

val state = sdk.getState()
when (state) {
    MobileAgentState.CONNECTED -> // Connected
    MobileAgentState.CONNECTING -> // Connecting
    MobileAgentState.DISCONNECTED -> // Disconnected
}

Flow API

The events Flow emits all event types. Use filterIsInstance to filter specific events:

// Collect all events.
sdk.events.collect { event ->
    when (event) {
        is MobileAgentEvent.StateChanged -> // Handle state changes.
        is MobileAgentEvent.TaskResult -> // Handle task results.
        is MobileAgentEvent.Stream -> // Handle stream data.
        // ...
    }
}

// Collect only specific types of events.
sdk.events
    .filterIsInstance<MobileAgentEvent.StateChanged>()
    .collect { event ->
        // Handle only state changes.
    }

Event types:

Event type	Description
`MobileAgentEvent.StateChanged`	Connection state changed
`MobileAgentEvent.TodoWrite`	To-do list updated
`MobileAgentEvent.TaskStart`	Task started
`MobileAgentEvent.StepStart`	Step started
`MobileAgentEvent.Stream`	Stream data received
`MobileAgentEvent.StepEnd`	Step ended
`MobileAgentEvent.TaskResult`	Task result received

Callback API

MobileAgentCallback methods. All have empty default implementations — override only what you need:

Method	Description
`onStateChanged(state: MobileAgentState)`	Connection state changed
`onTodoWrite(todos: Array<TodoItem>, count: Int)`	To-do list updated
`onTaskStart(taskId: String, task: String, maxSteps: Int)`	Task started
`onStepStart(taskId: String, step: Int)`	Step started
`onStream(taskId, step, agent, contentType, content)`	Stream data received
`onStepEnd(taskId, step, status, summary)`	Step ended
`onTaskResult(taskId, result, resultText, success, actualSteps)`	Task result received
`onRawMessage(rawJson: String)`	Raw, unparsed JSON message

Data classes

MobileAgentState

Connection state enum:

enum class MobileAgentState {
    DISCONNECTED,  // Disconnected
    CONNECTING,    // Connecting
    CONNECTED      // Connected
}

TodoItem

To-do item data class:

data class TodoItem(
    val id: String,
    val title: String,
    val status: String,
    val details: String
)

MobileAgentEvent

Sealed class for all event types:

sealed class MobileAgentEvent {
    data class StateChanged(val state: MobileAgentState)
    data class TodoWrite(val todos: Array<TodoItem>, val count: Int)
    data class TaskStart(val taskId: String, val task: String, val maxSteps: Int)
    data class StepStart(val taskId: String, val step: Int)
    data class Stream(val taskId: String, val step: Int, val agent: String, val contentType: String, val content: String)
    data class StepEnd(val taskId: String, val step: Int, val status: String, val summary: String)
    data class TaskResult(val taskId: String, val result: String, val resultText: String, val success: Boolean, val actualSteps: Int)
}

Advanced usage

Parse agent thoughts

MobileAgent wraps agent thoughts in <conclusion> tags within stream output. Extract this content from Stream events.

`<conclusion>` format

XML format:

<conclusion>This is the agent's thought process...</conclusion>

Kotlin implementation

Parse thoughts as they stream in:

import com.wuying.mobileagentsdk.MobileAgentSdk
import com.wuying.mobileagentsdk.MobileAgentEvent

class ConclusionParser {
    private val conclusionBuffer = StringBuilder()
    private var inConclusion = false

    /**
     * Processes stream data to extract content from <conclusion> tags.
     */
    fun processStream(content: String): String? {
        val result = StringBuilder()

        for (char in content) {
            if (!inConclusion) {
                // Not inside a conclusion; check for the start of <conclusion>.
                conclusionBuffer.append(char)

                // Check for <conclusion>.
                if (conclusionBuffer.length >= 12) {
                    if (conclusionBuffer.endsWith("<conclusion>")) {
                        inConclusion = true
                        conclusionBuffer.clear()
                        result.append("[Thought] ")
                    } else if (conclusionBuffer.length > 12) {
                        // Keep the last 11 characters for the next check.
                        conclusionBuffer.delete(0, conclusionBuffer.length - 11)
                    }
                }
            } else {
                // Inside a conclusion; check for the end tag.
                conclusionBuffer.append(char)

                // Check for </conclusion>.
                if (conclusionBuffer.length >= 13) {
                    if (conclusionBuffer.endsWith("</conclusion>")) {
                        // Found the end tag.
                        inConclusion = false
                        val conclusionContent = conclusionBuffer.toString()
                            .dropLast(13)  // Remove </conclusion>.
                            .trim()

                        if (conclusionContent.isNotEmpty()) {
                            result.append(conclusionContent).append("\n")
                        }

                        conclusionBuffer.clear()
                        continue
                    } else if (conclusionBuffer.length > 13) {
                        // Keep the last 12 characters for the next check.
                        conclusionBuffer.delete(0, conclusionBuffer.length - 12)
                    }
                }

                // If not part of a potential end tag, output immediately.
                if (!couldBeEndTag(conclusionBuffer.toString())) {
                    result.append(char)
                }
            }
        }

        return if (result.isNotEmpty()) result.toString() else null
    }

    /**
     * Checks if the end of the buffer could be a prefix of the end tag.
     */
    private fun couldBeEndTag(buffer: String): Boolean {
        val prefixes = listOf(
            "<", "</", "</c", "</co", "</con", "</conc",
            "</concl", "</conclu", "</conclus", "</conclusi",
            "</conclusio", "</conclusion"
        )

        return prefixes.any { buffer.endsWith(it) }
    }

    /**
     * Resets the parser state.
     */
    fun reset() {
        conclusionBuffer.clear()
        inConclusion = false
    }
}

// Sample usage
MobileAgentSdk().use { sdk ->
    val parser = ConclusionParser()

    lifecycleScope.launch {
        sdk.events
            .filterIsInstance<MobileAgentEvent.Stream>()
            .collect { event ->
                // Parse and print the output.
                val output = parser.processStream(event.content)
                if (output != null) {
                    print(output)
                }
            }
    }

    sdk.connect("wss://localhost:30005/ws")
}

Simplified version

To parse after task completion instead of streaming:

/**
 * Extracts all content from <conclusion> tags in a complete string.
 */
fun extractConclusions(content: String): List<String> {
    val conclusions = mutableListOf<String>()
    val regex = "<conclusion>(.*?)</conclusion>".toRegex(RegexOption.DOT_MATCHES_ALL)

    for (match in regex.findAll(content)) {
        val conclusion = match.groupValues[1].trim()
        if (conclusion.isNotEmpty()) {
            conclusions.add(conclusion)
        }
    }

    return conclusions
}

// Sample usage
lifecycleScope.launch {
    sdk.events
        .filterIsInstance<MobileAgentEvent.TaskResult>()
        .collect { event ->
            // Extract the thought process from the task result.
            val conclusions = extractConclusions(event.resultText)

            println("Agent's thought process:")
            conclusions.forEachIndexed { index, conclusion ->
                println("${index + 1}. $conclusion")
            }
        }
}

Output

[Thought] The user wants to open the Settings app. I need to find its package name and Activity.
[Thought] By querying the system app list, I found the Settings app: com.android.settings/.Settings
[Thought] Now I will start the Settings app.

Task complete: Successfully opened the settings app.

Using Flow and callbacks together

Both approaches work simultaneously:

MobileAgentSdk().use { sdk ->
    // Use Flow to handle the main logic.
    lifecycleScope.launch {
        sdk.events
            .filterIsInstance<MobileAgentEvent.TaskResult>()
            .collect { event ->
                // Handle the task result.
            }
    }

    // Use a callback for logging.
    sdk.setCallback(object : MobileAgentCallback {
        override fun onRawMessage(rawJson: String) {
            // Log raw messages for debugging.
            Log.d("SDK_RAW", rawJson)
        }
    })

    sdk.connect("wss://localhost:30005/ws")
}

Error handling

try {
    MobileAgentSdk().use { sdk ->
        sdk.connect("wss://localhost:30005/ws")
    }
} catch (e: IllegalStateException) {
    Log.e("SDK", "Failed to create SDK", e)
} catch (e: Exception) {
    Log.e("SDK", "Connection error", e)
}

Check SDK status

val sdk = MobileAgentSdk()

// Check if the SDK is destroyed.
if (sdk.isDestroyed()) {
    Log.w("SDK", "The SDK has been destroyed.")
}

// Check if the SDK is valid.
sdk.checkNotDestroyed()  // Throws an exception if destroyed.

ProGuard configuration

The SDK bundles proguard-rules.pro to preserve native methods. Extra configuration is usually unnecessary.

Manual configuration:

# MobileAgentSDK
-keep class com.wuying.mobileagentsdk.** { *; }
-keepclassmembers class com.wuying.mobileagentsdk.MobileAgentSdk {
    private native <methods>;
}

Dependencies

Runtime dependency:

dependencies {
    // The only runtime dependency
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3")
}

The SDK excludes UI frameworks (AppCompat, Material) to stay lightweight.