Simple Log Service:Decompress Snappy files

Snappy library for C++

1. Download and install Snappy for C++.

2. Call snappy.uncompress to decompress the files.

Snappy library for Java

1. Download the Snappy library for Java.

   Note

   Version 1.1.2.1 may fail to decompress some files due to bad handling of the MAGIC HEADER. Use version 1.1.2.6 or later.

   • Manually download the library:

     xerial snappy-java

   • Add the following dependency to a Maven project:

     <dependency>
     <groupId>org.xerial.snappy</groupId>
     <artifactId>snappy-java</artifactId>
     <version>1.0.4.1</version>
     <type>jar</type>
     <scope>compile</scope>
     </dependency>

2. Decompress files by using one of the following methods.

   Note

   SnappyFramedInputStream is not supported.

   • Snappy.uncompress

     Sample code:

     String fileName = "C:\\Downloads\\36_1474212963188600684_4451886.snappy";
     RandomAccessFile randomFile = new RandomAccessFile(fileName, "r");
     int fileLength = (int) randomFile.length();
     randomFile.seek(0);
     byte[] bytes = new byte[fileLength];
     int byteread = randomFile.read(bytes);
     System.out.println(fileLength);
     System.out.println(byteread);
     byte[] uncompressed = Snappy.uncompress(bytes);
     String result = new String(uncompressed, "UTF-8");
     System.out.println(result);

   • Snappy.SnappyInputStream

     Sample code:

     String fileName = "C:\\Downloads\\36_1474212963188600684_4451886.snappy";
     SnappyInputStream sis = new SnappyInputStream(new FileInputStream(fileName));
     byte[] buffer = new byte[4096];
     int len = 0;
     while ((len = sis.read(buffer)) != -1) {
         System.out.println(new String(buffer, 0, len));
     }

Snappy library for Python

1. Download and install python-snappy.

2. Call snappy.uncompress to decompress the files.

   • Sample code for Python 2:

     import snappy
     compressed = open('/tmp/temp.snappy').read()
     snappy.uncompress(compressed)

   • Sample code for Python 3:

     import snappy
     compressed = open('/tmp/temp.snappy','rb').read()
     print(snappy.uncompress(compressed).decode(encoding='utf-8',errors="ignore"))

   Note

   • On Windows, decompression may fail with UnicodeDecodeError: 'gbk' codec can't decode byte 0xff in position 0: illegal multibyte sequence due to byte order mark (BOM) encoding. Use a UNIX-like system or specify a valid encoding in the open function.

   • The following command does not work for Snappy-compressed OSS files. To decompress from the CLI, use hadoop_stream_decompress or stream_decompress mode instead.

     python -m snappy -d compressed_file.snappy uncompressed_file                            

Open source Snappy tool for PHP

Use the php-ext-snappy extension to decompress Snappy files.

1. Download the source code from php-ext-snappy.

   Alternatively, clone the repository:

   git clone --recursive --depth=1 https://github.com/kjdev/php-ext-snappy.git

2. Compile the source code.

   % cd php-ext-snappy
   % phpize
   % ./configure
   % make
   % make install

3. Add the extension to php.ini.

   extension=snappy.so

   After the configuration, you can compress and decompress Snappy files. Sample code:

   $file_path = "test.snappy" ;
   if (file_exists($file_path)) {
     $str = file_get_contents($file_path); // Read the content of the file into a string. 
     $uncompressed = snappy_uncompress($str);
     echo $uncompressed;
   }

References

More Snappy resources are available at google/snappy.