:Data structures

Property

Type

Description

audioProfile

ARTCAICallAudioProfile

The audio encoding configuration. Default value: HighQualityMode.

audioScenario

ARTCAICallAudioScenario

The audio scenario configuration. Default value: ARTCAICallAudioSceneMusicMode.

Property

Type

Description

view

UIView

The rendering view.

viewMode

ARTCAICallAgentViewMode

The image rendering mode.

viewMirrorMode

ARTCAICallAgentViewMirrorMode

The image mirror mode.

viewRotationMode

ARTCAICallAgentViewRotationMode

The image rotation mode.

Property

Type

Description

preview

UIView?

The preview view. If this is empty, the stream is ingested without a preview.

viewMode

ARTCAICallAgentViewMode

The preview image rendering mode.

viewMirrorMode

ARTCAICallAgentViewMirrorMode

The preview image mirror mode.

viewRotationMode

ARTCAICallAgentViewRotationMode

The preview image rotation mode.

dimensions

CGSize

The stream ingest resolution.

frameRate

Int

The stream ingest frame rate.

bitrate

Int

The stream ingest bitrate.

keyFrameInterval

Int

Stream ingest: Keyframe interval (milliseconds)

useHighQualityPreview

Bool

Specifies whether to use high-definition preview. If not, the SDK automatically adjusts the resolution.

cameraCaptureFrameRate

Int

The preview resolution. Default value: 15 fps.

Property

Type

Description

text

String

The text parameter for requesting the multi-modal large language model.

enableASR

Bool

Specifies whether to pass the ASR result to the large language model as input.

isSingle

Bool

Specifies whether to perform a single frame capture.

eachDuration

UInt

The frame capture interval. Unit: seconds.

num

UInt

The number of images captured each time.

duration

UInt

The duration of continuous frame capture. Unit: seconds. This parameter is valid only for continuous frame capture.

userData

String?

A JSON string for custom business information.

Property

Type

Description

text

String

The text message to send to the agent, such as "What is this?"

Property

Type

Description

agentId

String

The agent ID.

agentType

ARTCAICallAgentType

The agent type. This must match the type of the agentId. Otherwise, the agent will fail to start.

agentUserId

String?

The agent's UID. If this is empty, the service assigns one.

region

String

The region where the agent service is located. This must match the region of the agentId. Otherwise, the agent will fail to start.

userId

String

The current user ID.

userJoinToken

String

The token for the current user to join the call.

userData

[String: Any]?

Custom user information that is passed to the agent.

agentConfig

ARTCAICallAgentConfig?

The agentConfig parameter for starting a call.

audioConfig

ARTCAICallAudioConfig?

The on-premises audio configuration.

videoConfig

ARTCAICallVideoConfig?

The on-premises video configuration. This takes effect only for VisionAgent or VideoAgent.

chatSyncConfig

ARTCAICallChatSyncConfig?

The associated chat agent configuration.

templateConfig

ARTCAICallTemplateConfig (deprecated)?

Deprecated. Use agentConfig instead.

Property

Type

Description

agentGreeting

String?

The agent's welcome message. If this is empty, the agent's configured value is used. The maximum length is 100 characters.

userOnlineTimeout

Int32

The timeout period for the agent to close the task if the user does not join the call. If the value is less than 0, the server's default value of 60s is used.

userOfflineTimeout

Int32

The timeout period for the agent to close the task after the user leaves the call. If the value is less than 0, the server's default value of 5s is used.

workflowOverrideParams

[String: Any]?

The workflow override parameters.

bailianAppParams

[String: Any]?

Application Center parameters in Model Studio

asrMaxSilence

Int32

The threshold for voice sentence segmentation detection. Valid values: 200 ms to 1200 ms. If the value is less than 0, the server's default value of 400 ms is used.

volume

Int32

The agent's speaking volume. Valid values: 0 to 400. Output volume = Voice output volume in the workflow × volume / 100. If the value is less than 0, the server's default value of 100 is used.

vadLevel

Int32

The sensitivity parameter for AIVad. Valid values: 0 to 10. Default value: 3.

enableVoiceInterrupt

Bool

Specifies whether to enable intelligent interruption.

agentVoiceId

String?

The agent's voice ID. If this is empty, the agent's configured value is used.

enableIntelligentSegment

Bool

Specifies whether to enable intelligent sentence segmentation and merging.

useVoiceprint

Bool

Specifies whether to use voiceprint-based noise suppression for the current sentence segmentation.

voiceprintId

String?

The voiceprint ID. If this is not empty, voiceprint-based noise suppression is enabled for the current call.

agentMaxIdleTime

Int32

The maximum idle time for the agent. Unit: seconds. If the value is less than 0, the server's default value of 600s is used.

llmHistoryLimit

Int32

The maximum number of historical conversation rounds to retain for the LLM or multi-modal LLM context. If the value is less than 0, the server's default value of 10 is used.

enablePushToTalk

Bool

Specifies whether to enable push-to-talk mode.

agentGracefulShutdown

Bool

Specifies whether to enable graceful shutdown. The agent finishes broadcasting the current sentence before stopping.

agentAvatarId

String?

The digital human model ID. If this is empty, the agent's configured value is used.

asrLanguageId

String?

The ASR language ID. If this is empty, the agent's configured value is used. Valid values:

• zh_mandarin: Chinese

• en: English

• zh_en: Chinese and English

• es: Spanish

• jp: Japanese

wakeUpQuery

String?

A user command given before the call starts. This allows the agent to respond immediately after the call begins.

llmSystemPrompt

String?

The LLM system prompt, such as "You are a friendly and helpful assistant..." Note: LLM nodes of the Alibaba Cloud Model Studio pipeline type are not supported.

asrHotWords

[String]?

The ASR hotword list. It can contain up to 500 words. Each word can have up to 10 characters.

interruptWords

[String]?

Specific words or phrases that trigger conversation interruption, such as "Excuse me" or "I know."

Property

Type

Description

sessionId

String

The ID of the associated chat agent session.

agentId

String

The ID of the associated chat agent. It must be in the same account and region.

receiverId

String

The user ID of the associated chat agent session.

Property

Type

Description

shareId

String?

The agent share ID.

agentType

ARTCAICallAgentType

Agent Workload Type

expireTime

Date?

The expiration time.

region

String?

The region where the agent resides.

templateConfig

String?

The template configuration as a JSON string.

userData

[String: Any]?

Custom user information that is passed to the agent.

Property

Type

Description

dimensions

CGSize

The stream ingest resolution.

frameRate

Int

The stream ingest frame rate.

bitrate

Int

The stream ingest bitrate.

keyFrameInterval

Int

The stream ingest GOP. Unit: milliseconds.

useHighQualityPreview

Bool

Specifies whether to use high-definition preview. If not, the SDK automatically adjusts the preview resolution based on the stream ingest resolution.

cameraCaptureFrameRate

Int

The preview resolution.

useFrontCameraDefault

Bool

Specifies whether to start the front camera by default.

Property

Type

Description

agentGreeting

String?

The agent's welcome message. If this is empty, the agent's configured value is used.

wakeUpQuery

String?

A user command given before the call starts. This allows the agent to respond immediately after the call begins.

agentMaxIdleTime

Int32

The maximum idle time for the agent before it automatically goes offline. Unit: seconds. Default value: 600s.

userOnlineTimeout

Int32

The timeout period for the agent to close the task if the user does not join the call. Default value: 60s.

userOfflineTimeout

Int32

The timeout period for the agent to close the task after the user leaves the call. Default value: 5s.

enablePushToTalk

Bool

Specifies whether to enable push-to-talk mode.

agentGracefulShutdown

Bool

Specifies whether to enable graceful shutdown.

volume

Int32

The agent's speaking volume. Valid values: 0 to 400. Default value: 100.

workflowOverrideParams

[String: Any]?

The workflow override parameters.

enableIntelligentSegment

Bool

The switch for intelligent sentence segmentation.

asrConfig

ARTCAICallAgentAsrConfig

The speech recognition configuration.

ttsConfig

ARTCAICallAgentTtsConfig

The speech synthesis configuration.

llmConfig

ARTCAICallAgentLlmConfig

The large language model configuration.

avatarConfig

ARTCAICallAgentAvatarConfig

The digital human configuration.

interruptConfig

ARTCAICallAgentInterruptConfig

The interruption configuration.

voiceprintConfig

ARTCAICallAgentVoiceprintConfig

The voiceprint-based noise suppression configuration.

turnDetectionConfig

ARTCAICallAgentTurnDetectionConfig

The turn detection configuration.

experimentalConfig

ARTCAICallExperimentalConfig

The non-production customized configuration.

vcrConfig

ARTCAICallAgentVcrConfig

The VCR configuration.

Property

Type

Description

asrLanguageId

String?

The ASR language ID. If this is empty, the agent's configured value is used.

asrMaxSilence

Int32

The threshold for voice sentence segmentation detection. A silence duration exceeding this threshold is considered a sentence break. Default value: 400 ms. Valid values: 200 ms to 1200 ms.

asrHotWords

[String]?

The ASR hotword list. It can contain up to 500 words. Each word can have up to 10 characters.

vadLevel

Int32

The sensitivity parameter for AIVad. Valid values: 0 to 10. Default value: 3.

customParams

String?

When you use a custom ASR, pass runtime parameters in a URL parameter format. Example: "mode=fast&sample=16000&format=wav".

vadDuration

Int32

The minimum duration for VAD to adjust interruption sensitivity. A value of 0 disables this feature. Valid values: 200 to 2000 ms. A common range is 200 to 500 ms, corresponding to one to four characters. If the value is less than 0, it is not sent to the server, where the feature is disabled by default.

Property

Type

Description

agentVoiceId

String?

The voice ID of the agent. If this parameter is empty, the value configured for the agent is used.

pronunciationRules

[[String: Any]]?

An array of pronunciation rules. A maximum of 20 rules are supported. If this parameter is nil or an empty array, no rules are used. Example:

 [
  {
      "Word": "Overlap",                       // Target word
      "Pronunciation": "chongdie",              // Replacement pronunciation
      "Type": "replacement"                // Polyphone rule
  },
  {
      "Word": "Action",
      "Pronunciation": "hangdong",
      "Type": "replacement"
  }
]

speechRate

Double

The Text-to-Speech (TTS) speech rate. This parameter is supported for all TTS types. Valid values range from 0.5 to 2.0, and the default value is 1.0. If you specify a value of less than 0, the value is not sent to the server. Instead, the value configured in the console is used.

languageId

String?

The TTS language code. This parameter is valid only when the TTS type is MiniMax.

emotion

String?

The TTS emotion type. This parameter is valid only when the TTS type is MiniMax.

modelId

String?

The TTS model ID. This parameter currently supports only MiniMax. Valid values are `speech-01-turbo` and `speech-02-turbo`.

Property

Type

Description

llmHistoryLimit

Int32

The maximum number of historical conversation rounds to retain. Default value: 10.

llmSystemPrompt

String?

The LLM system prompt.

bailianAppParams

[String: Any]?

Parameters for the Model Studio Application Center

llmCompleteReply

boolean

Specifies whether to send the complete LLM result.

openAIExtraQuery

String?

Additional query parameters for LLMs that use the OpenAI protocol.

Property

Type

Description

agentAvatarId

String?

The digital human model ID. If this is empty, the agent's configured value is used.

Property

Type

Description

enableVoiceInterrupt

Bool

Specifies whether to enable intelligent interruption.

interruptWords

[String]?

Specific words or phrases that trigger conversation interruption.

Property

Type

Description

useVoiceprint

Bool

Specifies whether to use voiceprint-based noise suppression for the current sentence segmentation.

voiceprintId

String?

The voiceprint ID. If this is not empty, voiceprint-based noise suppression is enabled for the current call.

Property

Type

Description

turnEndWords

[String]?

Specific words for sentence segmentation, such as "Over" or "I'm done."

mode

ARTCAICallTurnDetectionMode

The mode for determining whether the user has finished speaking. Default value: Semantic. This mode uses AI for semantic analysis to determine if the user has finished speaking.

semanticWaitDuration

Int32

The custom wait time for semantic sentence segmentation. Unit: milliseconds. Valid values: 0 to 10,000. If the value is less than 0, it is not sent to the server. The server then uses its default value of -1, which allows the AI to automatically determine the wait time.

Property

Type

Description

resultData

[String]?

All VCR detection results returned by the agent.

stillFrameMotionResult

FrameMotionResult?

The VCR still frame detection result.

invalidFrameMotionResult

FrameMotionResult?

The VCR invalid frame detection result.

peopleCountResult

PeopleCountResult?

The VCR real-time people count detection result.

equipmentResult

EquipmentResult?

The VCR electronic device detection result.

headMotionResult

HeadMotionResult?

The VCR head motion detection result.

Property

Type

Description

data

[String]?

Caches the JSON object that you pass. This object is then used to generate a JSON string, which allows for custom extensions.

stillFrameMotion

ARTCAICallAgentVcrFrameMotionConfig?

The VCR still frame detection configuration.

invalidFrameMotion

ARTCAICallAgentVcrFrameMotionConfig?

The VCR invalid frame detection configuration.

peopleCount

ARTCAICallAgentVcrBaseConfig?

The VCR real-time people count detection configuration.

equipment

ARTCAICallAgentVcrBaseConfig?

The VCR electronic device detection configuration.

headMotion

ARTCAICallAgentVcrBaseConfig?

The VCR head motion detection configuration.

Property

Type

Description

enable

Boolean

Specifies whether to enable this feature. Default value: true.

Property

Type

Description

callbackDelay

Int32

The delay in milliseconds before a callback is triggered. Default value: 3000.

Property

Type

Description

rtcSdkParams

[String: Any]?

RTC SDK parameters.

commonParams

[String: Any]?

Common parameters.