ApsaraVideo VOD:Duet recording

Supported editions

Key concepts

To better understand the feature, review the definitions of duet recording, track, and track layout.

Related classes

Procedure

The procedure for duet recording is similar to basic recording. The main difference is in the configuration of input and output parameters and the preview view.

Initialize and configure parameters

Initialize the AliyunIMixRecorder class, create a recording instance, and configure recording parameters. For more information about the parameters used in the code, see Related classes.

// Create a recording instance.
AliyunIMixRecorder recorder = AliyunMixRecorderCreator.createAlivcMixRecorderInstance(context);

// Release the recording instance.
// Ensure you release the instance when it is no longer needed or before your application exits.
// Do not release it while it is in use.
AliyunIMixRecorder.release();

// Set the quality of the recorded video.
AliyunIMixRecorder.setVideoQuality(quality);

// Set the bitrate of the recorded video.
AliyunIMixRecorder.setVideoBitrate(int bitrate); // Unit: kbps

// Set the output parameters for the recorded video.
// `inputMediaInfo` specifies parameters for the composite video, such as track layout and the output video.
// `outputInfo` specifies information about the output video.
AliyunIMixRecorder.setMixMediaInfo(AliyunMixMediaInfoParam inputMediaInfo, MediaInfo outputInfo); // For parameter details, see the AliyunMixMediaInfoParam and MediaInfo API references.

// Set the output path for the recorded video.
AliyunIMixRecorder.setOutputPath(String path);

// Set the Group of Pictures (GOP) size for the output video.
AliyunIMixRecorder.setGop(int gop); // Unit: number of frames

Set up callbacks

Set up callbacks to receive real-time updates on the progress and status of audio and video processing. For more information about the parameters used in the code, see Related classes.

// Set the recording callback.
AliyunIMixRecorder.setRecordCallBack(RecordCallback callBack);

// Set the callback for captured video frames.
AliyunIMixRecorder.setOnFrameCallback(OnFrameCallBack callback);

// Set the callback for captured audio data.
AliyunIMixRecorder.setOnAudioCallback(OnAudioCallBack callback);

Enable preview

To enable preview, you must provide a SurfaceView container. Call startPreview() in the onResume() method of your Activity/Fragment and stopPreview() in the onPause() method. For more information about the parameters used in the code, see Related classes.

// Set the preview views.
// `cameraView` is the view for the camera feed. `videoView` is the view for the local video.
AliyunIMixRecorder.setDisplayView(SurfaceView cameraView, Surface videoView);

// Start the preview.
AliyunIMixRecorder.startPreview();

// Stop the preview.
// Call stopPreview() in the onPause() method of your Activity/Fragment.
AliyunIMixRecorder.stopPreview();

Start recording

You often need to start, stop, and restart recording multiple times to create the final video. Stopping the recording generates a video clip, while canceling the recording discards the current clip. For more information about the parameters used in the code, see Related classes.

// Start recording.
AliyunIMixRecorder.startRecording();

// Start recording the first clip.
AliyunIMixRecorder.startRecording();

// Stop recording to generate a video clip.
AliyunIMixRecorder.stopRecording();

// Start recording another clip.
AliyunIMixRecorder.startRecording();
// Cancel recording and discard the current clip.
AliyunIMixRecorder.cancelRecording();

// Continue to record the next video clip.
AliyunIMixRecorder.startRecording();
AliyunIMixRecorder.stopRecording();

Finish recording

When you finish recording, you can either merge all clips into a single video file or generate a configuration file containing information about the clips. For more information about the parameters used in the code, see Related classes.

// Finish recording and merge all recorded clips into a single video file.
AliyunIMixRecorder.finishRecording();

// Finish recording and generate a configuration file for the clips without merging them.
AliyunIMixRecorder.finishRecordingForEdit();

Control recording and manage clips

This section describes how to configure optional parameters for camera control (such as camera type and flash mode) and video clip management (such as setting recording duration, deleting clips, and getting the clip count). For more information about the parameters used in the code, see Related classes.

// Get the number of available cameras.
AliyunIMixRecorder.getCameraCount();

// Set the camera type.
AliyunIMixRecorder.setCamera(cameraType);

// Mute the recording.
AliyunIMixRecorder.setMute(boolean isMute);

// Set the sensor rotation angle.
// Important: Read the API documentation for this parameter carefully. 
AliyunIMixRecorder.setRotation(int rotation);

// Set the recording rotation angle.
// Important: Read the API documentation for this parameter carefully. 
AliyunIMixRecorder.setRecordRotation(int rotation);

// Set camera preview parameters such as flash, focus mode, zoom, and exposure.
// You can also set these parameters individually using the methods below.
AliyunIMixRecorder.setCameraParam(CameraParam cameraParam);

// Switch between the front and rear cameras.
AliyunIMixRecorder.switchCamera();

// Set the flash mode.
AliyunIMixRecorder.setLight(FlashType flashType);

// Set the zoom level. 
AliyunIMixRecorder.setZoom(float rate);

// Set the exposure compensation ratio. 
AliyunIMixRecorder.setExposureCompensationRatio(float value);

// Set the focus mode. 
AliyunIMixRecorder.setFocusMode(int mode);

// Trigger manual focus at a specific point. 
AliyunIMixRecorder.setFocus(float xRatio, float yRatio);

// Get the clip manager instance.
AliyunIClipManager manager = AliyunIRecorder.getClipManager();

// Set the maximum total recording duration in milliseconds (not for a single clip).
manager.setMaxDuration(int maxDurationMs);

// Set the minimum total recording duration in milliseconds (not for a single clip).
manager.setMinDuration(int minDurationMs);

// Delete the last recorded clip.
manager.deletePart();

// Delete a clip at a specific index.
manager.deletePart(int index);

// Delete all recorded clips.
manager.deleteAllPart();

// Get the total duration of all clips.
manager.getDuration();

// Get the total number of clips.
manager.getPartCount();

// Get a list of file paths for all clips.
manager.getVideoPathList();

Configure effects

This section describes how to configure optional recording effects such as retouching and filters. For more information about the parameters used in the code, see Related classes.

Filters

You can create custom filters. For more information, see Filters and transitions.

// Apply a filter.
AliyunIMixRecorder.applyFilter(effectFilter);

// Remove the filter.
// To remove the filter, pass a new EffectFilter with a null path.
AliyunIMixRecorder.applyFilter(new EffectFilter(null));

// Apply an animated filter.
AliyunIMixRecorder.applyAnimationFilter(effectFilter);

// Remove an animated filter.
AliyunIMixRecorder.removeAnimationFilter(effctFilter);

// Set the recording speed.
AliyunIMixRecorder.setRate(float rate);

// Add a static sticker or watermark.
AliyunIMixRecorder.addImage(effctImage);

// Remove a static sticker or watermark.
AliyunIMixRecorder.removeImage(effctImage);

// Update the position of a static sticker or watermark.
AliyunIMixRecorder.setEffectView(float xRatio,float yRatio,float widthRatio,float heightRatio,EffectBase effectBase);

Animated stickers

// Add an animated sticker.
AliyunIMixRecorder.addPaster(effectPaster,float sx,float sy,float sw,float sh,float rotation,boolean flip);

// Remove an animated sticker.
AliyunIMixRecorder.removePaster(effectPaster);

// Update the position of an animated sticker.
AliyunIMixRecorder.setEffectView(float xRatio,float yRatio,float widthRatio,float heightRatio,effectBase);

Advanced retouching

Other features

The SDK supports photo taking and facial recognition during video recording. For more information about the parameters used in the code, see Related classes.

Photo taking

// Take a photo with effects applied.
AliyunIMixRecorder.takePhoto(boolean needBitmap);

// Take a photo using the system camera (no effects).
AliyunIMixRecorder.takePicture(boolean needBitmap);

// Set the photo size for system photos. This is not supported for photos with effects.
AliyunIMixRecorder.setPictureSize(Camera.Size size);

Facial recognition

// Enable or disable facial recognition.
AliyunIMixRecorder.needFaceTrackInternal(boolean need);

// Set the path to the facial recognition model file.
AliyunIMixRecorder.setFaceTrackInternalModelPath(String path);

// Set the face detection rotation angle.
// Important: Read the API documentation for this parameter carefully. 
AliyunIMixRecorder.setFaceDetectRotation(int rotation);

// Set the maximum number of faces to detect.
// The maximum value for the built-in face tracking is 3.
AliyunIMixRecorder.setFaceTrackInternalMaxFaceCount(int maxFaceCount);

// Add an animated sticker that tracks a face.
AliyunIMixRecorder.addPaster(EffectPaster effectPaster);

Duet recording sample code

import android.graphics.Bitmap
import android.os.Bundle
import android.util.Log
import android.view.SurfaceView
import android.widget.ImageView
import android.widget.Toast
import androidx.appcompat.app.AppCompatActivity
import com.aliyun.svideosdk.common.struct.common.VideoDisplayMode
import com.aliyun.svideosdk.common.struct.encoder.VideoCodecs
import com.aliyun.svideosdk.common.struct.recorder.MediaInfo
import com.aliyun.svideosdk.mixrecorder.AliyunIMixRecorder
import com.aliyun.svideosdk.mixrecorder.AliyunMixMediaInfoParam
import com.aliyun.svideosdk.mixrecorder.AliyunMixRecorderDisplayParam
import com.aliyun.svideosdk.mixrecorder.AliyunMixTrackLayoutParam
import com.aliyun.svideosdk.mixrecorder.impl.AliyunMixRecorderCreator
import com.aliyun.svideosdk.recorder.RecordCallback


/**
 * Video recording example
 */
class MixRecordActivity : AppCompatActivity() {

    enum class RecordStatus {
        Idle,
        Recording
    }

    private lateinit var mAliyunRecord : AliyunIMixRecorder
    private lateinit var mVideoPreviewView : SurfaceView
    private lateinit var mCameraPreviewView : SurfaceView
    private lateinit var mRecordBtn : ImageView
    private var mRecordStatus = RecordStatus.Idle

    companion object {
        const val TAG = "MixRecordActivity"
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_mixrecord)

        mVideoPreviewView = findViewById(R.id.videoPreviewView)
        mCameraPreviewView = findViewById(R.id.cameraPreviewView)
        mRecordBtn = findViewById(R.id.btnRecordControl)
        // Recording button
        mRecordBtn.setOnClickListener {
            if(mRecordStatus == RecordStatus.Recording) {
                mAliyunRecord.finishRecording()
                Toast.makeText(this@MixRecordActivity, "Stop recording", Toast.LENGTH_SHORT).show()
                updateRecordStatus(RecordStatus.Idle)
            } else {
                val curTime = System.currentTimeMillis()
                mAliyunRecord.setOutputPath("/storage/emulated/0/DCIM/Camera/svideo_mixrecord_video_$curTime.mp4")
                mAliyunRecord.startRecording()
                Toast.makeText(this@MixRecordActivity, "Start recording", Toast.LENGTH_SHORT).show()
                updateRecordStatus(RecordStatus.Recording)
            }

        }

        mAliyunRecord = AliyunMixRecorderCreator.createAlivcMixRecorderInstance(this)


        val videoDisplayParam = AliyunMixRecorderDisplayParam.Builder()
                .displayMode(VideoDisplayMode.FILL)
                .layoutParam(
                        AliyunMixTrackLayoutParam.Builder()
                                .centerX(0.25f)
                                .centerY(0.5f)
                                .widthRatio(0.5f)
                                .heightRatio(1.0f)
                                .build()
                )
                .build()

        val cameraDisplayParam = AliyunMixRecorderDisplayParam.Builder()
                .displayMode(VideoDisplayMode.FILL)
                .layoutParam(AliyunMixTrackLayoutParam.Builder()
                        .centerX(0.75f)
                        .centerY(0.5f)
                        .widthRatio(0.5f)
                        .heightRatio(1.0f)
                        .build())
                .build()

        val mixMediaParam = AliyunMixMediaInfoParam.Builder()
                .streamStartTimeMills(0L)
                .streamEndTimeMills(0L) // Set to 0L to automatically use the full duration of the video.
                .mixVideoFilePath("/storage/emulated/0/DCIM/Camera/VID_20210317_174802.mp4")
                .mixDisplayParam(videoDisplayParam)
                .recordDisplayParam(cameraDisplayParam)
                .build()

        val mediaInfo = MediaInfo()
        mediaInfo.fps = 30
        mediaInfo.crf = 6
        mediaInfo.videoWidth = 720
        mediaInfo.videoHeight = 1080
        mediaInfo.videoCodec = VideoCodecs.H264_SOFT_OPENH264

        mAliyunRecord.setMixMediaInfo(mixMediaParam, mediaInfo)
        mAliyunRecord.setDisplayView(mCameraPreviewView, mVideoPreviewView)

        mAliyunRecord.setRecordCallback(object : RecordCallback {
            override fun onComplete(validClip: Boolean, clipDuration: Long) {
                Log.i(TAG, "onComplete")
            }

            override fun onFinish(outputPath: String?) {
                Log.i(TAG, "onFinish path : $outputPath")
                mAliyunRecord.clipManager.deleteAllPart()
            }

            override fun onProgress(progress: Long) {
                Log.i(TAG, "onProgress  : $progress")
            }

            override fun onMaxDuration() {
                Log.i(TAG, "onMaxDuration")
            }

            override fun onError(errorCode: Int) {
                Log.i(TAG, "onError : $errorCode")
            }

            override fun onInitReady() {
                Log.i(TAG, "onInitReady")
            }

            override fun onDrawReady() {
                Log.i(TAG, "onDrawReady")
            }

            override fun onPictureBack(bitmap: Bitmap?) {
                Log.i(TAG, "onPictureBack")
            }

            override fun onPictureDataBack(p0: ByteArray?) {
                Log.i(TAG, "onPictureDataBack")
            }

        })

    }

    private fun updateRecordStatus(recordStatus: RecordStatus)
    {
        mRecordStatus = recordStatus
        when(recordStatus) {
            RecordStatus.Idle -> {
                mRecordBtn.setImageResource(R.mipmap.alivc_svideo_bg_record_start)
            }
            RecordStatus.Recording -> {
                mRecordBtn.setImageResource(R.mipmap.alivc_svideo_bg_record_storp)
            }
        }
    }

    override fun onResume() {
        super.onResume()
        mAliyunRecord.startPreview()
    }

    override fun onPause() {
        super.onPause()
        mAliyunRecord.stopPreview()
    }

    override fun onDestroy() {
        super.onDestroy()
        mAliyunRecord.release()
    }
}

XML configuration example

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <androidx.constraintlayout.widget.Guideline
        android:id="@+id/guideline3"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:orientation="vertical"
        app:layout_constraintGuide_percent="0.5" />


    <SurfaceView
        android:id="@+id/videoPreviewView"
        android:layout_width="0dp"
        android:layout_height="400dp"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toStartOf="@+id/guideline3"
        app:layout_constraintHorizontal_bias="0.0"
        app:layout_constraintStart_toStartOf="parent"
        tools:layout_editor_absoluteY="0dp" />

    <SurfaceView
        android:id="@+id/cameraPreviewView"
        android:layout_width="0dp"
        android:layout_height="400dp"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintStart_toStartOf="@+id/guideline3"
        app:layout_constraintEnd_toEndOf="parent" />

    <ImageView
        android:id="@+id/btnRecordControl"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_centerInParent="true"
        android:layout_marginBottom="50dp"
        android:src="@mipmap/alivc_svideo_bg_record_start"
        android:visibility="visible"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent">

    </ImageView>

</androidx.constraintlayout.widget.ConstraintLayout>