All Products
Search
Document Center

:Video playback

Last Updated:Jul 27, 2023

LinkVisual SDK for Android allows you to implement the video playback feature. This topic describes how to implement the feature.

For convenience purposes, all mentions of LinkVisual SDK in this topic refer to LinkVisual SDK for Android.

Prerequisites

  • A product and device are created in the IoT Platform console. For more information, see Device Connection.

  • LinkVisual SDK is obtained. For more information, see Obtain the SDK.

  • LinkVisual SDK is initialized. For more information, see Initialize the SDK.

Background

The video playback feature supports the Real-Time Messaging Protocol (RTMP). The following video and audio coding formats are supported:

  • Video coding formats: H.264 and H.265.

  • Audio coding formats: G711a, G711u, and AAC-LC.

Procedure

Step 1: Register a video playback event listener and an error listener.

  • The video playback event listener OnVodStreamListener notifies the IP camera of events such as starting stream ingest, stopping stream ingest, pausing stream ingest, resuming stream ingest, and seeking to a specified position.

  • The error listener OnStreamErrorListener notifies the IP camera of errors in stream ingest.

Step 2: Process the request to obtain the list of video files on the IP camera.

  1. The app initiates a request to obtain the list of video files on the IP camera.

  2. The IP camera receives the request to obtain the list of its video files.

    The request to obtain the list of video files on the IP camera is made by synchronously calling the IP camera service. For more information about device services, see Device services.

  3. The IP camera responds to the request and returns the list of video files that meet the requirements to the app.

Important

The video file names must be encoded in Base64.

@Override
        public void onNotify(String connectId, String topic, AMessage aMessage){
            Log.d(TAG, "onNotify() called with: connectId = [" + connectId + "], topic = [" + topic + "], aMessage = ["
                    + new String((byte[]) aMessage.data) + "]");
            /**
             * Add a listener for the SDK.
             */
            IPCDev.getInstance().notifySyncTopicReceived(connectId, topic, aMessage);

            // Process the request for a synchronous service invocation.
            if (CONNECT_ID.equals(connectId) && !TextUtils.isEmpty(topic) &&
                    topic.contains("rrpc")) {
                Log.d(TAG, "IConnectNotifyListener   onNotify() called with: connectId = [" + connectId + "], topic = ["
                        + topic + "], aMessage = ["
                        + new String((byte[]) aMessage.data) + "]");

                int code = 200;
                String data = "{}";

                JSONObject json = JSON.parseObject(new String((byte[]) aMessage.data));
                if (json != null) {
                    String method = json.getString("method");
                    JSONObject params = json.getJSONObject("params");
                    switch (method) {
                        // Request to obtain the list of video files on the IP camera.
                        case "thing.service.QueryRecordList":
                            int beginTime = params.getIntValue("BeginTime");
                            int endTime = params.getIntValue("EndTime");
                            int querySize = params.getIntValue("QuerySize");
                            int type = params.getIntValue("Type");
                            appendLog("Receive the request to obtain the list of video files on the IP camera: beginTime=" + beginTime +
                                    "\tendTime=" + endTime + "\tquerySize=" + querySize + "\ttype=" + type);

                            JSONArray resultArray = new JSONArray();
                            JSONObject item1 = new JSONObject();
                            item1.put("FileName", Base64.encode("file1".getBytes(), Base64.DEFAULT));
                            item1.put("BeginTime", System.currentTimeMillis() / 1000 - 200);
                            item1.put("EndTime", System.currentTimeMillis() / 1000 - 100);
                            item1.put("Size", 1024000);
                            item1.put("Type", 0);
                            resultArray.add(item1);

                            JSONObject item2 = new JSONObject();
                            item2.put("FileName", Base64.encode("file2".getBytes(), Base64.DEFAULT));
                            item2.put("BeginTime", System.currentTimeMillis() / 1000 - 100);
                            item2.put("EndTime", System.currentTimeMillis() / 1000);
                            item2.put("Size", 1024000);
                            item2.put("Type", 0);
                            resultArray.add(item2);

                            JSONObject result = new JSONObject();
                            result.put("RecordList", resultArray);

                            code = 200;
                            data = result.toJSONString();
                            break;
                        default:
                            break;
                    }
                }

                MqttPublishRequest request = new MqttPublishRequest();
                request.isRPC = false;
                request.topic = topic.replace("request", "response");
                String resId = topic.substring(topic.indexOf("rrpc/request/") + 13);
                request.msgId = resId;
                request.payloadObj = "{\"id\":\"" + resId + "\", \"code\":" + code + ",\"data\":" + data + "}";
                LinkKit.getInstance().publish(request, new IConnectSendListener() {
                    @Override
                    public void onResponse(ARequest aRequest, AResponse aResponse) {
                        appendLog("The result is reported successfully");
                    }

                    @Override
                    public void onFailure(ARequest aRequest, AError aError) {
                        appendLog("The result fails to be reported:" + aError.toString());
                    }
                });
            }
        }

Step 3: Process the command of starting stream ingest.

  1. The app requests a video file in the list that was obtained in Step 2.

  2. The server sends the command of starting stream ingest, and provides the OnVodStreamListener.onStartPushVodStreaming(int streamId, String fileName) or OnVodStreamListener.onStartPushVodStreaming(int streamId, int beginTimeUtc, int endTimeUtc) callback to instruct the IP camera to ingest audio and video streams. The video file is played. LinkVisual SDK provides the following video playback methods:

    • Play the video of a specified file name.

          @Override
          public void onStartPushVodStreaming(int streamId, String fileName){
              appendLog("Start ingest of specified streams" + streamId + " File name:" + new String(Base64.decode(fileName, Base64.NO_WRAP)));
      
              try {
                  // Set video-related parameters.
                  VideoStreamParams videoStreamParams = new VideoStreamParams();
                  // The length of the video. Unit: seconds.
                  videoStreamParams.setDurationInS(H264_DURATION_IN_S);
                  videoStreamParams.setVideoFormat(VideoStreamParams.VIDEO_FORMAT_H264);
      
                  // Set audio-related parameters.
                  AudioStreamParams audioStreamParams = new AudioStreamParams();
                  audioStreamParams.setAudioChannel(AudioStreamParams.AUDIO_CHANNEL_MONO);
                  audioStreamParams.setAudioFormat(AudioStreamParams.AUDIO_FORMAT_G711A);
                  audioStreamParams.setAudioEncoding(AudioStreamParams.AUDIO_ENCODING_16BIT);
                  audioStreamParams.setAudioSampleRate(AudioStreamParams.AUDIO_SAMPLE_RATE_8000);
      
                  // Set parameters related to stream ingest.
                  IPCDev.getInstance().getIpcStreamManager().setStreamParams(streamId, videoStreamParams, audioStreamParams);
      
                  // TODO: Read the fileName file and call the operation to ingest audio and video streams.
                  // After stream ingest for the file is complete, call the IPCDev.getInstance().getIpcStreamManager().notifyVodComplete(streamId) operation to indicate that stream ingest is complete.
      
              } catch (NoSuchStreamException e) {
                  e.printStackTrace();
              }
          }
    • Play the video that was recorded in a specified time range.

          @Override
          public void onStartPushVodStreaming(int streamId, int beginTimeUtc, int endTimeUtc){
              appendLog("Start ingest of specified streams " + streamId + " beginTimeUtc: "+beginTimeUtc + " endTimeUtc:"+endTimeUtc);
      
              // TODO: Add the following stream ingest rules:
              // 1. beginTimeUtc , and endTimeUtc must be on the same day.
              // 2. After the onStartPushVodStreaming callback is received, the stream ingest starts from the nearest I-frame after beginTimeUtc. The timestamp is the UTC time of the frame.
              // 3. When there is no video recorded in the range from beginTimeUtc to endTimeUtc or all the streams of this time range has been ingested, the IPCDev.getInstance().getIpcStreamManager().notifyVodComplete(streamId) operation is called to indicate that the stream ingest is complete.
              // 4. When video data exists in the range from beginTimeUtc to endTimeUtc, even from different files, stream ingest continues.
          }

Step 4: Process the command of pausing or resuming stream ingest.

Sending audio and video data is paused or resumed by responding to the OnVodStreamListener command.

/**
     * Receive a request to pause stream ingest.
     *
     * @param streamId: The ID of the stream.
     */
    void onPausePushVodStreaming(int streamId);

    /**
     * Receive a request to resume stream ingest.
     *
     * @param streamId: The ID of the stream.
     */
    void onResumePushVodStreaming(int streamId);

Step 5: Process the seek command.

To respond to the seek command, the onSeekTo method is called back, and then stream ingest continues from the nearest I-frame after the timeStampInS point in time. For example, when you seek to the 80th second on the progress bar of the player app, stream ingest continues from the nearest I-frame after the 80th second.

/**
     * Receive a request to seek to a specified timestamp.
     *
     * @param streamId: The ID of the stream.
     * @param timeStampInS: The time offset, which is the duration between the start time of the video and the point in time that you seek to. Unit: seconds.
     */
    void onSeekTo(int streamId, long timeStampInS);

Step 6: Process the command of stopping stream ingest.

When the server sends a request to stop stream ingest:

  1. The OnVodStreamListener.onStopPushLiveStreaming() method is called back to notify the IP camera to stop ingesting streams.

  2. You must stop calling the operation of sending audio and video data, close the video file, and call the IPCStreamManager.getInstance().stopStreaming(int streamId) method.

/**
     * Receive the request to stop stream ingest.
     *
     * @param streamId: The ID of the stream.
     */
    @Override
    public void onStopPushStreaming(int streamId){
        // TODO: Stop sending audio and video data.
        try {
            // Call the stopStreaming operation.
            IPCDev.getInstance().getIpcStreamManager().stopStreaming(streamId);
         catch (NoSuchStreamException e) {
            e.printStackTrace();
        }
    }

Step 7: Handle streaming errors.

During stream ingest, the OnStreamErrorListener.onError(int streamId, StreamError error) method is called to receive and handle streaming errors. For more information about error codes, see Error codes.

Error codes

Streaming error codes

Error code

Identifier

Description

Solution

1

StreamError.ERROR_STREAM_CREATE_FAILED

Failed to create the streaming instance.

This error occurs due to insufficient system resources. Apply for memory and try again.

2

StreamError.ERROR_STREAM_START_FAILED

Failed to establish an RTMP connection.

Check whether the network is normal and try again.

3

StreamError.ERROR_STREAM_STOP_FAILED

Failed to stop ingesting streams.

This error occurs due to the invalid stream ID. You can ignore this error.

4

StreamError.ERROR_STREAM_SEND_VIDEO_FAILED

Failed to send the video data.

Troubleshoot the error cause based on the RTMP error code. For more information, see RTMP error codes.

5

StreamError.ERROR_STREAM_SEND_AUDIO_FAILED

Failed to send the audio data.

Troubleshoot the error cause based on the RTMP error code. For more information, see RTMP error codes.

6

StreamError.ERROR_STREAM_INVALID_PARAMS

The stream-related parameters are invalid.

The parameters that are set by using the setStreamParams operation are invalid. Check and modify the parameters and then try again.

RTMP error codes

Note

RTMP error codes are displayed only in logs for troubleshooting.

Error code

Identifier

Description

Solution

-1

RTMP_ILLEGAL_INPUT

The input parameters are invalid.

Check and modify the input parameters and then try again.

-2

RTMP_MALLOC_FAILED

Failed to allocate the memory.

Check the memory usage of the video device and then try again.

-3

RTMP_CONNECT_FAILED

Failed to establish an RTMP connection.

Check whether the network is normal and try again.

-4

RTMP_IS_DISCONNECTED

Failed to establish an RTMP connection.

This error occurs when the server ends the connection. You can ignore this error.

-5

RTMP_UNSUPPORT_FORMAT

The audio or video coding format is unsupported.

The parameters that are set by using the setStreamParams operation are invalid. Check and modify the parameters and then try again.

-6

RTMP_SEND_FAILED

Failed to send RTMP messages.

This error occurs when you call the send operation after the server ends the connection. You can ignore this error.

-7

RTMP_READ_MESSAGE_FAILED

Failed to read RTMP messages.

This error occurs when the server ends the connection. You can ignore this error.

-8

RTMP_READ_TIMESTAMP_ERROR

The timestamp is invalid.

During live streaming, make sure that the timestamp does not roll back. Check whether the timestamp is valid and try again.