The short video SDK provides the video merging class AliyunIMixComposer. You can call this class to merge multiple videos into a single one offline. The merged video can be arranged in a specified layout, such as picture-in-picture, nine-square grid, left-right split-screen, or up-down split-screen. Multiple video tracks can be added for video merging. This topic describes how to use the short video SDK for Android to merge videos. This topic also provides sample code for video merging.

Supported editions

Edition Supported
Professional Yes
Standard Yes
Basic No

Related classes

Class Description
AliyunIMixComposer A class that defines the core features of video merging, such as setting output parameters, creating tracks, adding video streams, starting merging, and setting callbacks.
AliyunMixComposerCreator A factory class that is used to create an instance of the AliyunIMixComposer implementation class.
AliyunMixTrack A class that defines video tracks. The class is used to add video streams to a track and set parameters about the layout and volume.
AliyunMixStream A class that defines video track streams. The class is used to obtain the display mode, file path, and end time of video streams.
AliyunMixOutputParam A class that defines merging output parameters. The class is used to set parameters such as the width and height, output bitrate, and quality level of the merged video.
AliyunMixCallback A class that defines merging callbacks. The class is used to set callbacks on merging completion, merging progress, and merging failures.

Merging process

Configuration Step Description Sample code
Basic 1 Create a merging instance. Create a merging instance
2 Create multiple tracks and video streams, and then add the video streams to the tracks separately. Create tracks
3 Set parameters such as the output path, and the width and height of the merged video. Set output parameters
4 Set callbacks and start merging. Start merging videos
5 Destroy the instance and release the resources. Release resources
Advanced 6 Set the cancel, pause, and continue features as needed. Control the video merging

Create a merging instance

Create a merging instance.

For information about the parameters that are used in the code, see Related classes.

// Create an instance.
AliyunIMixComposer mixComposer = AliyunMixComposerCreator.createMixComposerInstance();

Create tracks

Create multiple tracks and video streams, and then add the video streams to the tracks separately. For information about the parameters that are used in the code, see Related classes.

// Create multiple tracks.

// Create Track 1.
// The layout of Track 1.
AliyunMixTrackLayoutParam track1Layout = new AliyunMixTrackLayoutParam.Builder()
        .centerX(0.25f)
        .centerY(0.5f)
        .widthRatio(0.5f)
        .heightRatio(1.f)
        .build();
// Create the Track 1 instance.        
AliyunMixTrack track1 = mixComposer.createTrack(track1Layout);        
// Create the first video stream to be added to Track 1.
AliyunMixStream stream11 = new AliyunMixStream
        .Builder()
        .displayMode(VideoDisplayMode.FILL)
        .filePath("/storage/emulated/0/lesson_01.mp4")
        .streamEndTimeMills(20000)
        .build();
// Add the first video stream to Track 1. Note: The lately added video stream overwrites the previously added video stream. Only the last added video stream exists.
track1.addStream(stream11);


// Create Track 2.
// Parameters of Track 2.
AliyunMixTrackLayoutParam track2Layout = new AliyunMixTrackLayoutParam.Builder()
        .centerX(0.75f)
        .centerY(0.5f)
        .widthRatio(0.5f)
        .heightRatio(1.f)
        .build();
// Create the Track 2 instance.
AliyunMixTrack track2 = mixComposer.createTrack(track2Layout);        
// Create the first video stream to be added to Track 2.
AliyunMixStream stream21 = new AliyunMixStream
        .Builder()
        .displayMode(VideoDisplayMode.FILL)
        .filePath("/storage/emulated/0/lesson_02.mp4")
        .streamStartTimeMills(10000)
        .streamEndTimeMills(30000)
        .build();
// Add the first video stream to Track 2. Note: The lately added video stream overwrites the previously added video stream. Only the last added video stream exists.
track2.addStream(stream21);

Set output parameters

Set parameters such as the output path, and the width and height of the merged video. For information about the parameters that are used in the code, see Related classes.

// Set output parameters.
AliyunMixOutputParam outputParam = new AliyunMixOutputParam.Builder()
        .outputPath("/sdcard/output.mp4") // The path to the merged video.
        .outputAudioReferenceTrack(track2)// Specifies that the audio of Track 2 is used as the audio of the merged video. The merged video supports only one audio stream.
        .outputDurationReferenceTrack(track2) // Specifies that the duration of Track 2 is used as the duration of the merged video. If the duration of Track 1 is shorter than this value, the video of Track 1 stops at the last frame.
        .crf(6)
        .videoQuality(VideoQuality.HD)
        .outputWidth(720) // The width of the video.
        .outputHeight(1280) // The height of the video.
        .fps(30) //fps
        .gopSize(30) //gop
        .build();
mixComposer.setOutputParam(outputParam);

Start merging videos

Set callbacks and start merging the video. For information about the parameters that are used in the code, see Related classes.

// Start merging.
AliyunMixCallback callback = new AliyunMixCallback() {
            @Override
            public void onProgress(long progress) {// The merging progress.
                Log.e("MixRecord", "onProgress " + progress);
            }

            @Override
            public void onComplete() {
                Log.e("MixRecord", "onComplete"); // The merging is complete.
                runOnUiThread(new Runnable() {
                    @Override
                    public void run() {
                        // Do not call this API operation in a thread that is used by a callback.   
                        // Release the instance after the merging is complete.
                        mixComposer.release();
                    }
                });
            }

            @Override
            public void onError(int errorCode) { // The merging fails.
                Log.e("MixRecord", "onError " + errorCode);
            }
};

Release resources

After the video is merged, release the resources. Do not destroy the instance when the merging is in progress. For information about the parameters that are used in the code, see Related classes.

mixComposer.release();

Control the video merging

Set the cancel, pause, and continue features as needed. For information about the parameters that are used in the code, see Related classes.

// Pause the merging.
mixComposer.pause();

// Continue the merging.
mixComposer.resume();

// Cancel the merging.
mixComposer.cancel();

Sample code for video merging

/**
 * Video merging example
 */
class MixActivity : AppCompatActivity() {
    private val REQUEST_TRACK1_STREAM = 1001
    private val REQUEST_TRACK2_STREAM = 1002

    Private var example: AliyunIMixComposer? = null

    private lateinit var mVideoTrack1 : AliyunMixTrack
    private lateinit var mVideoTrack2 : AliyunMixTrack

    private var mVideoTrack1Duration = 0L
    private var mVideoTrack2Duration = 0L
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_mix)

        findViewById<Button>(R.id.btnReset).setOnClickListener {
            findViewById<Button>(R.id.btnMix).isEnabled = false

            mMixComposer?.release()

            init()
        }

        findViewById<Button>(R.id.btnAddTrack1Stream).setOnClickListener {
            PermissionX.init(this)
                .permissions(
                    Manifest.permission.READ_EXTERNAL_STORAGE,
                    Manifest.permission.WRITE_EXTERNAL_STORAGE
                )
                .request { allGranted, _, _ ->
                    if (allGranted) {
                        PictureSelector.create(this)
                            .openGallery(PictureMimeType.ofVideo())
                            .forResult(REQUEST_TRACK1_STREAM)
                    }
                }
        }

        findViewById<Button>(R.id.btnAddTrack2Stream).setOnClickListener {
            PermissionX.init(this)
                .permissions(
                    Manifest.permission.READ_EXTERNAL_STORAGE,
                    Manifest.permission.WRITE_EXTERNAL_STORAGE
                )
                .request { allGranted, _, _ ->
                    if (allGranted) {
                        PictureSelector.create(this)
                            .openGallery(PictureMimeType.ofVideo())
                            .forResult(REQUEST_TRACK2_STREAM)
                    }
                }
        }

        findViewById<Button>(R.id.btnMix).setOnClickListener {
            // Start producing the video.
            val callback: AliyunMixCallback = object : AliyunMixCallback {
                override fun onProgress(progress: Long) { // The production progress.
                    Log.e("MixActivity", "onProgress $progress")
                }

                override fun onComplete() {
                    Log.e("MixActivity", "onComplete")
                    ToastUtil.showToast(it.context, "The video is produced.")
                }

                override fun onError(errorCode: Int) {
                    Log.e("MixActivity", "onError $errorCode")
                    ToastUtil.showToast(it.context, "The video fails to be produced:$errorCode")
                }
            }

            // Set output parameters.
            val outputParamBuilder = AliyunMixOutputParam.Builder()
            outputParamBuilder
                .outputPath("/storage/emulated/0/DCIM/Camera/svideo_mix_demo.mp4")
                .crf(6)
                .videoQuality(VideoQuality.HD)
                .outputWidth(720)
                .outputHeight(1280)
                .fps(30)
                .gopSize(30)
            if(mVideoTrack1Duration > mVideoTrack2Duration) {
                outputParamBuilder.outputAudioReferenceTrack(mVideoTrack1)
                outputParamBuilder.outputDurationReferenceTrack(mVideoTrack1)
            } else {
                outputParamBuilder.outputAudioReferenceTrack(mVideoTrack2)
                outputParamBuilder.outputDurationReferenceTrack(mVideoTrack2)
            }

            mMixComposer!!.setOutputParam(outputParamBuilder.build())
            mMixComposer!!.start(callback)
        }

        init()
    }

    private fun init() {
        mMixComposer = AliyunMixComposerCreator.createMixComposerInstance()
        // Create Track 1.
        val track1Layout = AliyunMixTrackLayoutParam.Builder()
            .centerX(0.25f)
            .centerY(0.25f)
            .widthRatio(0.5f)
            .heightRatio(0.5f)
            .build()

        mVideoTrack1 = mMixComposer!!.createTrack(track1Layout)

        // Create Track 2.
        val track2Layout = AliyunMixTrackLayoutParam.Builder()
            .centerX(0.75f)
            .centerY(0.75f)
            .widthRatio(0.5f)
            .heightRatio(0.5f)
            .build()

        mVideoTrack2 = mMixComposer!!.createTrack(track2Layout)
    }
    override fun onResume() {
        super.onResume()
        mMixComposer?.resume()
    }

    override fun onPause() {
        super.onPause()
        mMixComposer?.pause()
    }

    override fun onDestroy() {
        super.onDestroy()
        mMixComposer?.release()
    }

    override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
        super.onActivityResult(requestCode, resultCode, data)
        if (resultCode == Activity.RESULT_OK) {
            when (requestCode) {
                REQUEST_TRACK1_STREAM -> {
                    // onResult Callback
                    val result = PictureSelector.obtainMultipleResult(data)

                    mVideoTrack1Duration = 0
                    for (streamBean in result) {

                        // Create the first video stream to be added to Track 1.
                        val stream1 = AliyunMixStream.Builder()
                            .displayMode(VideoDisplayMode.FILL)
                            .filePath(streamBean.realPath)
                            .streamStartTimeMills(mVideoTrack1Duration)
                            .streamEndTimeMills(streamBean.duration)
                            .build()
                        mVideoTrack1Duration += streamBean.duration
                        if(mVideoTrack1.addStream(stream1) == 0) {
                            ToastUtil.showToast(this, "The video stream is added to Track 1.")
                        }

                    }
                }
                REQUEST_TRACK2_STREAM -> {
                    // onResult Callback
                    val result = PictureSelector.obtainMultipleResult(data)

                    mVideoTrack2Duration = 0L
                    for (streamBean in result) {

                        // Create the first video stream to be added to Track 2.
                        val stream1 = AliyunMixStream.Builder()
                            .displayMode(VideoDisplayMode.FILL)
                            .filePath(streamBean.realPath)
                            .streamStartTimeMills(mVideoTrack2Duration)
                            .streamEndTimeMills(streamBean.duration)
                            .build()
                        mVideoTrack2Duration += streamBean.duration
                        if(mVideoTrack2.addStream(stream1) == 0) {
                            ToastUtil.showToast(this, "The video stream is added to Track 2.")
                        }
                    }
                }
            }
        }
        if(mVideoTrack1Duration > 0 && mVideoTrack2Duration > 0) {
            findViewById<Button>(R.id.btnMix).isEnabled = true
        }
    }
}

Sample configuration file in XML

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">


    <Button
        android:id="@+id/btnReset"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginTop="76dp"
        android:text="Reset"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

    <Button
        android:id="@+id/btnAddTrack1Stream"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginStart="56dp"
        android:layout_marginTop="64dp"
        android:text="Add Video 1"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toBottomOf="@id/btnReset"
        app:layout_constraintEnd_toEndOf="@id/btnReset"
         />

    <Button
        android:id="@+id/btnAddTrack2Stream"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginTop="64dp"
        android:layout_marginEnd="56dp"
        android:text="Add Video 2"
        app:layout_constraintStart_toStartOf="@id/btnReset"
        app:layout_constraintTop_toBottomOf="@id/btnReset"
        app:layout_constraintEnd_toEndOf="parent"
        />

    <Button
        android:id="@+id/btnMix"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginBottom="176dp"
        android:text="Start producing the video"
        android:enabled="false"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent" />
</androidx.constraintlayout.widget.ConstraintLayout>