All Products
Search
Document Center

MixRecording

Last Updated: Jul 30, 2019

The short video SDK provides the duet feature for recording a duet that consists of two videos. The core interface is AliyunIMixRecorder.

Concepts

This section describes some basic concepts that help you better understand the duet feature.

1. Duet

The duet feature allows you to produce a video from two videos. One is a sample video and the other is taken by the camera. The two videos are arranged in the specified layout, for example, left-right split-screen, up-down split-screen, or picture-in-picture. Each frame of the produced video contains images from the two videos, and the audio of the sample video is used as the audio of the produced video. The following figure shows some sample layouts. In fact, the short video SDK allows you to completely customize the layout, which will be described later in this topic.

track

2. Track

The two videos mentioned earlier are abstracted to two tracks in the short video SDK: track A and track B. Track A stores the video collected by the camera and track B stores the sample video. This abstraction helps you better understand the concept of track layout.

3. Track layout

The track layout is an attribute of a track. It specifies the position of the track in the produced video. In a normalized coordinate system, the track layout describes the center point of the track and its size, that is, the width and height of the track.

As shown in the following figure, track A occupies the left half of the screen and track B occupies the right half of the screen. Therefore, the width of both tracks is 0.5 and the height is 1.0. The center points of track A and track B are (0.25,0.5) and (0.75,0.5), respectively.

The track layout involves the AliyunMixRecorderDisplayParam and AliyunMixTrackLayoutParam interfaces. AliyunMixTrackLayoutParam specifies the center point and size of a track. In addition to the information specified by AliyunMixTrackLayoutParam, AliyunMixRecorderDisplayParam provides the displayMode and layoutLevel attributes to specify more information. The displayMode attribute specifies the scaling mode (padding or cropping) used when the aspect ratio of the video displayed in track differs from the aspect ratio of the track. The layoutLevel attribute specifies the layout level. If two tracks overlap, the track with a higher layout level is displayed above the one with a lower layer level. For more information, see the SDK documentation.

layout

Edition difference

Edition Description
Professional edition The duet feature is supported.
Standard edition The duet feature is supported.
Basic edition The duet feature is not supported.

Duet recording process

The camera and microphone permissions are required for duet recording. Otherwise, the recording fails. The duet recording process is as follows:

1. Set parameters.
2. Set callbacks.
3. Control preview.
4. Control and manage recording.
5. Configure effects.
6. Start recording.
7. Complete recording.

Set parameters

Initialization

  • Create a duet recording instance.
  1. AliyunIMixRecorder recorder = AliyunMixRecorderCreator.createAlivcMixRecorderInstance(Context context);// We recommend that you use Application Context as the context.
  • Destroy a duet recording instance.
  1. AliyunIMixRecorder#release();

Set input and output parameters

  • Set input and output parameters for duet recording, including information of the sample video, track display information, frame rate for preview, and frame rate, resolution, and encoder of the recorded video.
  1. AliyunIMixRecorder#setMixMediaInfo(AliyunMixMediaInfoParam inputMediaInfo, MediaInfo outputInfo);// For more information, see AliyunMixMediaInfoParam and MedianInfo in the SDK documentation.
  • Set the path of the recorded video.
  1. AliyunIMixRecorder#setOutputPath(String path);
  • Set the quality of the recorded video.
  1. AliyunIMixRecorder#setVideoQuality(VideoQuality quality);
  • Set the bitrate of the recorded video.
  1. AliyunIMixRecorder#setVideoBitrate(int bitrate);// Unit: Kbit/s.
  • Set the GOP size of the recorded video.
  1. AliyunIMixRecorder#setGop(int gop);// Unit: number of frames.

Set callbacks

The callback settings are the same as those for recording a common video.

Control preview

  • Start the preview.
  1. AliyunIMixRecorder#startPreview();
  • Stop the preview.
  1. AliyunIMixRecorder#stopPreview();

Control and manage recording

Control recording

  • Set the view of the duet recording preview window (SurfaceView).
  1. AliyunIMixRecorder#setDisplayView(SurfaceView previewView, SurfaceView playView);// **Note: Do not use the GLSurfaceView class or its subclass. **

The camera settings are the same as those for recording a common video.

Manage recording

  • Obtain the clip manager.
  1. AliyunIMixRecorder#getClipManager();// **Note: To delete the last clip, do not use the method of AliyunIClipManager. Instead, use AliyunIMixRecorder#deleteLastPart(). **
  • Set the maximum recording duration, which is the maximum duration of all clips instead of the maximum duration of a single clip.
  1. AliyunIClipManager#setMaxDuration(int maxDurationMs);
  • Set the minimum recording duration, which is the minimum duration of all clips instead of the minimum duration of a single clip.
  1. AliyunIClipManager#setMinDuration(int minDurationMs);
  • Obtain the total duration of clips.
  1. AliyunIClipManager#getDuration();
  • Obtain the total number of clips.
  1. AliyunIClipManager#getPartCount();
  • Delete the last clip. (Read the comment carefully.)
  1. AliyunIMixRecorder#deleteLastPart();// Do not use the AliyunIClipManager#deletePart() method. If you use the AliyunIClipManager#deletePart() method, the SDK does not seek to the last position where recording stops in the sample video.
  • Delete the specified clip.
  1. AliyunIClipManager#deletePart(int index);// This method only deletes the specified clip file and does not trigger seeking in the sample video.
  • Delete all clips.
  1. AliyunIClipManager#deleteAllPart();// This method only deletes the specified clip file and does not trigger seeking in the sample video.
  • Obtain the clip path list.
  1. AliyunIClipManager#getVideoPathList();

Configure effects

The effect settings are the same as those for recording a common video.

Start duet recording

  • Start recording a video clip.
  1. AliyunIMixRecorder#startRecording();
  • Stop recording a video clip.
  1. AliyunIMixRecorder#stopRecording();

Complete duet recording

  • Complete recording and merge recorded video clips into one video.
  1. AliyunIMixRecorder#finishRecording();

Sample code

  1. // The following code demonstrates how to record a duet. Before running the following code, make sure that a video named sample.mp4 exists in the root directory of the SD card on the mobile phone. That is, the /sdcard/sample.mp4 file exists. The code only demonstrates the core process and cannot be directly copied and pasted to your project. For more information about the detailed process, see the code of the demo.
  2. AliyunIMixRecorder mMixRecorder = AliyunIMixRecorder createAlivcMixRecorderInstance(getApplicationContext());
  3. if(WYSIWYG is required){
  4. mRecorder.setRecordRotation(0);// This setting enables WYSIWYG, so that a video recorded in landscape mode is not automatically rotated in the produced video.
  5. mRecorder.setFaceDetectRotation(getPictureRotation());// For more information about getPictureRotation, see the code of the demo.
  6. }else {
  7. mRecorder.setRotation(getPictureRotation());
  8. }
  9. mMixRecorder.setDisplayView(mCameraPrvSurfaceView, mPlayPrvSurfaceView);
  10. AliunIClipManager mClipManager = mRecorder.getClipManager();
  11. mClipManager.setMinDuration(mMinDuration);
  12. mClipManager.setMaxDuration(mMaxDuration);
  13. mRecordTimelineView.setMaxDuration(mClipManager.getMaxDuration());
  14. mRecordTimelineView.setMinDuration(mClipManager.getMinDuration());
  15. MediaInfo mOutputInfo = new MediaInfo();
  16. mOutputInfo.setVideoWidth(outputWidth);
  17. mOutputInfo.setVideoHeight(outputHeight);
  18. AliyunMixRecorderDisplayParam recorderDisplayParam = new AliyunMixRecorderDisplayParam.Builder()
  19. .displayMode(VideoDisplayMode.FILL)
  20. .layoutParam(
  21. new AliyunMixTrackLayoutParam.Builder()
  22. .centerX(0.25f)
  23. .centerY(0.5f)
  24. .widthRatio(0.5f)
  25. .heightRatio(1.0f)
  26. .build()
  27. )
  28. .build();
  29. AliyunMixRecorderDisplayParam sampleDisplayParam = new AliyunMixRecorderDisplayParam
  30. .Builder()
  31. .displayMode(VideoDisplayMode.FILL)
  32. .layoutParam(new AliyunMixTrackLayoutParam.Builder()
  33. .centerX(0.75f)
  34. .centerY(0.5f)
  35. .widthRatio(0.5f)
  36. .heightRatio(1.0f)
  37. .build())
  38. .build();
  39. AliyunMixMediaInfoParam mInputInfo = new AliyunMixMediaInfoParam
  40. .Builder()
  41. .streamStartTimeMills(0L)
  42. .streamEndTimeMills(0L)// The duration of the sample video is automatically applied when the two parameters are set to 0L.
  43. .mixVideoFilePath("/sdcard/sample.mp4")
  44. .mixDisplayParam(sampleDisplayParam)
  45. .recordDisplayParam(recorderDisplayParam)
  46. .build();
  47. mRecorder.setMixMediaInfo(mInputInfo, mOutputInfo);
  48. mRecorder.setRecordCallback(new RecordCallback() {
  49. @Override
  50. public void onComplete(boolean validClip, long clipDuration) {}
  51. @Override
  52. public void onFinish(String outputPath) {
  53. // The callback to be called when the production is completed.
  54. }
  55. @Override
  56. public void onProgress(final long duration) {}
  57. @Override
  58. public void onMaxDuration() {}
  59. @Override
  60. public void onError(int errorCode) {}
  61. @Override
  62. public void onInitReady() { }
  63. @Override
  64. public void onDrawReady() {}
  65. @Override
  66. public void onPictureBack(Bitmap bitmap) { }
  67. @Override
  68. public void onPictureDataBack(byte[] data) {}
  69. });
  70. mRecorder.startPreview();
  71. mRecorder.startRecording();
  72. mRecorder.stopRecording();
  73. mRecorder.finishRecording();

Produce a video

Feature

The AliyunIMixComposer interface is used to produce a video from multiple videos offline to achieve effects such as picture-in-picture, nine-square grid, left-right split-screen, and up-down split-screen. You can use this interface to produce a video from two or more videos, not only two videos in a duet.

Production process

1. Initialize a production instance.
2. Create tracks.
3. Add video streams to the tracks.
4. Set output parameters.
5. Start the production.
6. Destroy the production instance.

Initialize a production instance

  1. AliyunMixComposerCreator.createMixComposerInstance();

Create tracks

  1. AliyunIMixComposer#createTrack(AliyunMixTrackLayoutParam layoutParam);

Add video streams to the tracks

  • Create a video stream.
  1. new AliyunMixStream.Builder().build();
  • Add the video stream to a track.
  1. AliyunMixTrack#addStream(AliyunMixStream stream);

Set output parameters

  1. AliyunIMixComposer#setOutputParam(AliyunMixOutputParam param);

Start the production

  1. AliyunIMixComposer#start(AliyunMixCallback callback);

Pause the production

  1. AliyunIMixComposer#pause();

Resume the production

  1. AliyunIMixComposer#resume();

Cancel the production

  1. AliyunIMixComposer#cancel();

Destroy the production instance

  1. AliyunIMixComposer#release();

Sample code

  1. // Creates an instance.
  2. AliyunIMixComposer mixComposer = AliyunMixComposerCreator.createMixComposerInstance();
  3. // Creates track 1.
  4. AliyunMixTrackLayoutParam track1Layout = new AliyunMixTrackLayoutParam.Builder()
  5. .centerX(0.25f)
  6. .centerY(0.5f)
  7. .widthRatio(0.5f)
  8. .heightRatio(1.f)
  9. .build();
  10. // Creates the first video stream to be added to track 1.
  11. AliyunMixStream stream11 = new AliyunMixStream
  12. .Builder()
  13. .displayMode(VideoDisplayMode.FILL)
  14. .filePath("/storage/emulated/0/11.mp4")
  15. .streamEndTimeMills(20000)
  16. .build();
  17. // Adds this video stream to track 1.
  18. track1.addStream(stream11);
  19. // Creates the second video stream to be added to track 1.
  20. AliyunMixStream stream12 = new AliyunMixStream
  21. .Builder()
  22. .displayMode(VideoDisplayMode.FILL)
  23. .filePath("/storage/emulated/0/12.mp4")
  24. .streamEndTimeMills(20000)
  25. .build();
  26. // Adds this video stream to track 1.
  27. track1.addStream(stream12);
  28. // Creates track 2.
  29. AliyunMixTrackLayoutParam track2Layout = new AliyunMixTrackLayoutParam.Builder()
  30. .centerX(0.75f)
  31. .centerY(0.5f)
  32. .widthRatio(0.5f)
  33. .heightRatio(1.f)
  34. .build();
  35. // Creates the first video stream to be added to track 2.
  36. AliyunMixStream stream21 = new AliyunMixStream
  37. .Builder()
  38. .displayMode(VideoDisplayMode.FILL)
  39. .filePath("/storage/emulated/0/21.mp4")
  40. .streamEndTimeMills(20000)
  41. .build();
  42. // Adds this video stream to track 2.
  43. track2.addStream(stream21);
  44. // Creates the second video stream to be added to track 2.
  45. AliyunMixStream stream22 = new AliyunMixStream
  46. .Builder()
  47. .displayMode(VideoDisplayMode.FILL)
  48. .filePath("/storage/emulated/0/22.mp4")
  49. .streamEndTimeMills(20000)
  50. .build();
  51. // Adds this video stream to track 2.
  52. track2.addStream(stream22);
  53. // Sets output parameters.
  54. AliyunMixOutputParam outputParam = new AliyunMixOutputParam.Builder()
  55. .outputPath("/sdcard/output.mp4")
  56. .outputAudioReferenceTrack(track2)// Indicates that the audio of track 2 is used as the audio of the produced audio. Currently, the produced video supports only one audio stream, so the audio of second video stream of track 2 will not be added to the produced video.
  57. .outputDurationReferenceTrack(track2)// Indicates that the duration of track 2 is used as the duration of the produced video. If the duration of track 1 is shorter than this value, the video of track 1 stops at the last frame.
  58. .crf(6)
  59. .videoQuality(VideoQuality.HD)
  60. .outputWidth(720)
  61. .outputHeight(1280)
  62. .fps(30)
  63. .gopSize(30)
  64. .build();
  65. mixComposer.setOutputParam(outputParam);
  66. // Starts the production.
  67. AliyunMixCallback callback = new AliyunMixCallback() {
  68. @Override
  69. public void onProgress(long progress) {// The production progress callback.
  70. Log.e("MixRecord", "onProgress " + progress);
  71. }
  72. @Override
  73. public void onComplete() {
  74. Log.e("MixRecord", "onComplete");
  75. runOnUiThread(new Runnable() {
  76. @Override
  77. public void run() {
  78. // Do not call this method directly in the thread used by the callback.
  79. mixComposer.release();
  80. }
  81. });
  82. }
  83. @Override
  84. public void onError(int errorCode) {
  85. Log.e("MixRecord", "onError " + errorCode);
  86. }
  87. };
  88. mixComposer.start(callback);