Object Storage Service:HEIC and AVIF decoding on Android

HEIC decoding on Android

HEIC is an image format that uses a more modern compression algorithm than the JPEG format. An HEIC image is approximately 40% the size of a JPEG image with equivalent quality.

The high-performance HEIC decoding library of Alibaba Cloud is developed based on the open source libraries libheif and libde265 and is optimized for ARM. This end-to-end decoding optimization is based on the strengths of other similar open source libraries and significantly increases the decoding speed. The Alibaba Cloud HEIC decoding library is much more efficient than the original open source libraries. For the sample code for the high-performance HEIC decoding library of Alibaba Cloud, visit GitHub.

Configure HEIC decoding on Android

To configure HEIC decoding on Android, perform the following steps:

1. Add the Alibaba Cloud Maven repository configuration to the repositories node of build.gradle.

   maven { url 'https://maven.aliyun.com/repository/public' }

2. Add the following dependency to the dependencies node of the build.gradle file:

   implementation 'com.aliyun:libheif:1.0.0'

Configure obfuscation

If you want to configure obfuscation, add the following statement to code to ensure that HEIC decoding works as expected.

-keep class com.aliyun.libheif.**

Decode HEIC images

• Scenario 1:Call an API operation to decode HEIC images

      /**
       * decode HEIC file to RGBA format
       * @param length  the length of effective file memory
       * @param filebuf file pointer in memory
       * @return rgba byte, convenient to create a {@link android.graphics.Bitmap}
       */
      public static native boolean toRgba(long length, byte[] fileBuf, Bitmap bitmap);
  
      /**
       * judge file is heic format or not
       * @param length  the length of effective file memory
       * @param filebuf file pointer in memory
       * @return bool, if true, the format is heif;
       *               if false, the fromat is not heif
       */
      public static native boolean isHeic(long length, byte[] fileBuf);
  
      /**
       * get info from heic picture
       * @param HeifInfo info
       * @param length  the length of effective file memory
       * @param filebuf file pointer in memory
       * @return bool, if true, outSize is valid;
       *               if false, outSize is not valid
       */
      public static native boolean getInfo(HeifInfo info, long length, byte[] fileBuf);                

  The following sample code provides an example on how to decode the test.heic image in the asset directory as a bitmap image:

  val image = findViewById<ImageView>(R.id.image);
  val inputStream  = assets.open("test.heic")
  val buffer = ByteArray(8192)
  var bytesRead: Int
  val output = ByteArrayOutputStream()
  while (inputStream.read(buffer).also { bytesRead = it } != -1) {
              output.write(buffer, 0, bytesRead)
          }
  
  val heifInfo = HeifInfo()
  val fileBuffer: ByteArray = output.toByteArray()
  
  HeifNative.getInfo(heifInfo, fileBuffer.size.toLong(), fileBuffer)
  // Obtain the image height and width (the first element) from the frameList parameter. The image is a single frame image. 
  val heifSize = heifInfo.frameList.first()
  // Create an image based on the image height and width. 
  val bitmap = Bitmap.createBitmap(
              heifSize.width,
              heifSize.height,
              Bitmap.Config.ARGB_8888)
  // Decode the image. 
  val heifBuffer = HeifNative.toRgba(fileBuffer.size.toLong(), fileBuffer, bitmap)
  image.setImageBitmap(bitmap)

• Scenario 2: Integrate the decoding library with Glide and decode HEIC images by using the library

  Glide is a popular open source image caching library. For more information, see Glide on GitHub.

  1. Integrate the decoding library with Glide by modifying the build.gradle file.

     implementation 'com.aliyun:libheif:1.0.0'
     implementation 'com.github.bumptech.glide:glide:4.13.2'
     implementation 'com.github.bumptech.glide:compiler:4.13.2'

  2. Specify the high-performance HEIC decoding library of Alibaba Cloud in the decode method. For more information, see Custom module of Glide.

     class HeifByteBufferBitmapDecoder(bitmapPool: BitmapPool): ResourceDecoder<ByteBuffer, Bitmap> {
     
         private val bitmapPool: BitmapPool
     
         init {
             this.bitmapPool = Preconditions.checkNotNull(bitmapPool)
         }
     
         override fun handles(source: ByteBuffer, options: Options): Boolean {
             val buffer = ByteBufferUtil.toBytes(source)
             return HeifNative.isHeic(buffer.size.toLong(), buffer);
         }
     
         override fun decode(
             source: ByteBuffer,
             width: Int,
             height: Int,
             options: Options
         ): Resource<Bitmap>?  {
             val buffer = ByteBufferUtil.toBytes(source)
             var heifInfo = HeifInfo()
             HeifNative.getInfo(heifInfo, buffer.size.toLong(), buffer)
             val heifSize = heifInfo.frameList[0]
             val bitmap = Bitmap.createBitmap(heifSize.width, heifSize.height, Bitmap.Config.ARGB_8888)
             HeifNative.toRgba(buffer.size.toLong(), buffer, bitmap)
             return BitmapResource.obtain(bitmap, bitmapPool)
         }
     }

  3. Register the decoder module.

     @GlideModule(glideName = "HeifGlide")
     open class HeifGlideModule : LibraryGlideModule() {
         override fun registerComponents(
             context: Context, glide: Glide, registry: Registry
         ) {
             val byteBufferBitmapDecoder = HeifByteBufferBitmapDecoder(glide.bitmapPool)
             registry.prepend(
                 ByteBuffer::class.java,
                 Bitmap::class.java, byteBufferBitmapDecoder
             )
         }
     }

  4. Load and display the HEIF image.

     fun loadHeif(context: Context, imageView: ImageView, file: File) {
          Glide.with(context).asBitmap().load(file).into(imageView)
      }

• Scenario 3: Integrate the decoding library with Fresco and decode HEIF images by using the library

  Fresco is an open source Android image caching library that is developed by Facebook. For more information, see Fresco on GitHub.

  1. Integrate the decoding library with Fresco by modifying the build.gradle file.

     implementation 'com.aliyun:libheif:1.0.0'
     implementation 'com.facebook.fresco:fresco:2.6.0'

  2. Specify the high-performance HEIC decoding library of Alibaba Cloud in the decode method.

     class FrescoHeifDecoder: ImageDecoder {
     
         private var mFile: File?  = null
         private var mUri: Uri?  = null
     
         constructor(file: File) {
             mFile = file
         }
     
         constructor(uri: Uri) {
             mUri = uri
         }
     
         override fun decode(
             encodedImage: EncodedImage,
             length: Int,
             qualityInfo: QualityInfo,
             options: ImageDecodeOptions
         ): CloseableImage?  {
             var imageRequest: ImageRequest?  = null
             if (mFile != null) {
                 imageRequest = ImageRequest.fromFile(mFile)
             }
             if (mUri != null) {
                 imageRequest = ImageRequest.fromUri(mUri)
             }
             try {
                 val cacheKey = DefaultCacheKeyFactory.getInstance()
                     .getEncodedCacheKey(imageRequest, null)
                 val fileCache = ImagePipelineFactory.getInstance()
                     .mainFileCache
                 val resource = fileCache.getResource(cacheKey)
                 val file: File = if (resource == null) {
                     mFile!!
                 } else {
                     (resource as FileBinaryResource).file
                 }
     
                 val bufferFromFile = ByteBufferUtil.fromFile(file)
                 val bytes = ByteBufferUtil.toBytes(bufferFromFile)
                 val heifInfo = HeifInfo()
                 HeifNative.getInfo(heifInfo, file.length(), bytes)
                 val heifSize = heifInfo.frameList[0]
                 val bitmap =
                 Bitmap.createBitmap(heifSize.width, heifSize.height, Bitmap.Config.ARGB_8888)
     
                 HeifNative.toRgba(file.length(), bytes, bitmap)
     
                 return CloseableStaticBitmap(
                     pinBitmap(bitmap),
                     qualityInfo,
                     encodedImage.rotationAngle,
                     encodedImage.exifOrientation
                 )
             } catch (e: Exception) {
             }
             return null
         }
     
         private fun pinBitmap(bitmap: Bitmap?): CloseableReference<Bitmap>?  {
             return CloseableReference.of(Preconditions.checkNotNull(bitmap), BitmapCounterProvider.get().releaser)
         }
     }

     When you connect to the high-performance HEIC decoding library of Alibaba Cloud, call the ByteBufferUtil reference class:

     object ByteBufferUtil {
     
         @Throws(IOException::class)
         fun fromFile(file: File): ByteBuffer {
             var raf: RandomAccessFile?  = null
             var channel: FileChannel?  = null
             return try {
                 val fileLength = file.length()
                 if (fileLength > Int.MAX_VALUE) {
                     throw IOException("File too large to map into memory")
                 }
                 if (fileLength == 0L) {
                     throw IOException("File unsuitable for memory mapping")
                 }
                 raf = RandomAccessFile(file, "r")
                 channel = raf.channel
                 channel.map(FileChannel.MapMode.READ_ONLY, 0, fileLength).load()
             } finally {
                 if (channel != null) {
                     try {
                         channel.close()
                     } catch (e: IOException) {
                     }
                 }
                 if (raf != null) {
                     try {
                         raf.close()
                     } catch (e: IOException) {
                     }
                 }
             }
         }
     
         fun toBytes(byteBuffer: ByteBuffer): ByteArray {
             val result: ByteArray
             val safeArray = getSafeArray(byteBuffer)
             if (safeArray != null && safeArray.offset == 0 && safeArray.limit == safeArray.data.size) {
                 result = byteBuffer.array()
             } else {
                 val toCopy = byteBuffer.asReadOnlyBuffer()
                 result = ByteArray(toCopy.limit())
                 rewind(toCopy)
                 toCopy[result]
             }
             return result
         }
     
         private fun rewind(buffer: ByteBuffer): ByteBuffer {
             return buffer.position(0) as ByteBuffer
         }
     
         private fun getSafeArray(byteBuffer: ByteBuffer): SafeArray?  {
             return if (!byteBuffer.isReadOnly && byteBuffer.hasArray()) {
                 SafeArray(byteBuffer.array(), byteBuffer.arrayOffset(), byteBuffer.limit())
             } else null
         }
     
         internal class SafeArray
             (val data: ByteArray, val offset: Int, val limit: Int)
     }

  3. Load the HEIF image.

     fun loadHeif(simpleDraweeView: SimpleDraweeView, file: File) {
             val request =
             ImageRequestBuilder.newBuilderWithSource(Uri.fromFile(file)).setImageDecodeOptions(
                     ImageDecodeOptions.newBuilder().setCustomImageDecoder(FrescoHeifDecoder(file))
                         .build()
                 ).build()
             val controller = Fresco.newDraweeControllerBuilder()
                 .setImageRequest(request).build()
     
             simpleDraweeView.controller = controller
         }

AVIF decoding on Android

AVIF is a new image format based on AV1 video encoding and provides higher compression ratios and image details than JPEG and WebP. An AVIF image is approximately 35% the size of a JPEG image with equivalent quality.

You can use the libavif library to decode AVIF images. libavif encapsulates relevant Android API operations. The relevant code is stored in the android_jni directory. For more information, visit GitHub.

Integrate a decoding library with AVIF

Add the following dependencies to build.gradle:

implementation'org.aomedia.avif.android:avif:0.11.1.3c786d2'

Use direct decoding

The following sample code provides an example on how to directly use the decode method to decode the AVIF file in ByteBuffer:

/**
   * Decodes the AVIF image into the bitmap.
   *
   * @param encoded The encoded AVIF image. encoded.position() must be 0.
   * @param length Length of the encoded buffer.
   * @param bitmap The decoded pixels will be copied into the bitmap.
   * @param threads Number of threads to be used for the AVIF decode. Zero means use number of CPU
   *     cores as the thread count. Negative values are invalid. When this value is > 0, it is
   *     simply mapped to the maxThreads parameter in libavif. For more details, see the
   *     documentation for maxThreads variable in avif.h.
   * @return true on success and false on failure. A few possible reasons for failure are: 1) Input
   *     was not valid AVIF. 2) Bitmap was not large enough to store the decoded image. 3) Negative
   *     value was passed for the threads parameter.
   */
  public static native boolean decode(ByteBuffer encoded, int length, Bitmap bitmap, int threads);

We recommend that you set the threads parameter to 1. The value 1 specifies that single-thread decoding is used. If your image encoding method uses the AV1 tiles feature, you can enable multi-threading to improve efficiency. For more information, visit GitHub.

Integrate the decoding library with Glide

The custom module of Glide supports AVIF decoding. For the source code, see the code repository on GitHub.

Starting from Glide 4.13.0, you can add the following dependency to use Glide to decode AVIF files.

implementation "com.github.bumptech.glide:avif-integration:4.13.2"       

Integrate the decoding library with Fresco

For more information about how to integrate a decoding library with Fresco, see Customizing Image Formats.