Decompression tools for Snappy-compressed files

Snappy decompressor for C++

1. Download the C++ library from Snappy.
2. Use the Snappy.Uncompress method to decompress Snappy compressed OSS files.

Snappy decompressor for Java

1. Select one of the following methods to download the Java library.
   Note If you use Snappy decompressor for Java 1.1.2.1, some Snappy compressed OSS files may fail to be decompressed. To prevent this issue, we recommend that you use Snappy decompressor for Java 1.1.2.6 or later. For more information about the issue, visit bad handling of the MAGIC HEADER.
   • Click the following link to manually download the Java library:

     xerial snappy-java

   • Add the following dependency in a Maven project to download the Java library:

     <dependency>
     <groupId>org.xerial.snappy</groupId>
     <artifactId>snappy-java</artifactId>
     <version>1.0.4.1</version>
     <type>jar</type>
     <scope>compile</scope>
     </dependency>

2. Select one of the following methods to decompress the OSS files.
   Note SnappyFramedInputStream is not supported.
   • Snappy.Uncompress

     The following code is an example:

     String fileName = "C:\\Downloads\\36_1474212963188600684_4451886.snappy";
     RandomAccessFile randomFile = new RandomAccessFile(fileName, "r");
     int fileLength = (int) randomFile.length();
     randomFile.seek(0);
     byte[] bytes = new byte[fileLength];
     int byteread = randomFile.read(bytes);
     System.out.println(fileLength);
     System.out.println(byteread);
     byte[] uncompressed = Snappy.uncompress(bytes);
     String result = new String(uncompressed, "UTF-8");
     System.out.println(result);

   • Snappy.SnappyInputStream

     The following code is an example:

     String fileName = "C:\\Downloads\\36_1474212963188600684_4451886.snappy";
     SnappyInputStream sis = new SnappyInputStream(new FileInputStream(fileName));
     byte[] buffer = new byte[4096];
     int len = 0;
     while ((len = sis.read(buffer)) != -1) {
         System.out.println(new String(buffer, 0, len));
     }

Snappy decompressor for Python

1. Download and install Snappy decompressor for Python from python-snappy.
2. Use the snappy.uncompress method to decompress the OSS files.
   • Python 2 sample code:

     import snappy
     compressed = open('/tmp/temp.snappy').read()
     snappy.uncompress(compressed)

   • Python 3 sample code:

     import snappy
     compressed = open('/tmp/temp.snappy','rb').read()
     print(snappy.uncompress(compressed).decode(encoding='utf-8',errors="ignore"))

   Note

   • An error may occur when you use the snappy.uncompress method to decompress the OSS files in Windows. For example, the error UnicodeDecodeError: 'gbk' codec can't decode byte 0xff in position 0: illegal multibyte sequence may occur because the files in the Windows file system start with  byte order mark (BOM). We recommend that you use a UNIX-like system or specify a valid file number in the encoding of the open function.
   • The following command cannot be used to decompress the OSS files. This command can be used only in Hadoop mode (hadoop_stream_decompress) and streaming mode (stream_decompress).

     python -m snappy -d compressed_file.snappy uncompressed_file                            

Linux-based Snappy decompressor

Log Service provides a Linux-based Snappy decompressor that allows you to decompress Snappy compressed files.

1. Download the Linux-based Snappy decompressor from snappy_tool.
2. Run the following command to decompress the OSS files.
   Replace 03_1453457006548078722_44148.snappy and 03_1453457006548078722_44148 with the actual values.

   ./snappy_tool 03_1453457006548078722_44148.snappy 03_1453457006548078722_44148

   Sample output:

   compressed.size: 2217186
   snappy::Uncompress return: 1
   uncompressed.size: 25223660