Zstandard (real-time compression algorithm)

Share interesting information or links related to portable apps here.
Post Reply
Message
Author
User avatar
Midas
Posts: 6710
Joined: Mon Dec 07, 2009 7:09 am
Location: Sol3

Zstandard (real-time compression algorithm)

#1 Post by Midas »

Zstandard (http://facebook.github.io/zstd/) is a novel compression algorithm developed and open-sourced by Facebook.

Zstandard has been mentioned before by billon (viewtopic.php?p=84833#p84833) and a few others at the Jabut topic (viewtopic.php?t=22053), but with the increased attention it is garnering I suppose it deserves a dedicated post.
Zstandard is a real-time compression algorithm, providing high compression ratios. It offers a very wide range of compression/speed trade-off, while being backed by a very fast decoder. It also offers a special mode for small data, called dictionary compression, and can create dictionaries from any sample set. Zstandard library is provided as open source software using a BSD license.
Zstandard (or Zstd) is a lossless data compression algorithm developed by Yann Collet at Facebook. The name also refers to the reference implementation in C. Version 1 of the implementation was released as free software on 31 August 2016.
.
Here's the comparison table with the official benchmark from Zstandard homepage:
.
Image
.
FYI, there's a 7zip build with support for Zstandard (among some other exotic formats) at https://mcmilk.de/projects/7-Zip-zstd/ and a plugin for the regular 7zip with comparable scope at http://www.tc4shell.com/en/7zip/modern7z/.

User avatar
webfork
Posts: 10818
Joined: Wed Apr 11, 2007 8:06 pm
Location: US, Texas
Contact:

Re: Zstandard (real-time compression algorithm)

#2 Post by webfork »

Midas wrote: Tue Jul 03, 2018 5:44 amcan create dictionaries from any sample set
I was wondering when someone would build that kind of functionality. I remember years ago there was discussion of a compression set that used a CD-ROM full of data as a kind of starting point for compression references and I was confused why it didn't catch on.

Anyway, I can imagine a lot of applications for servers that download and decompress similar software installation tools. I imagine there are thousands of common data patterns throughout the last 10 Ubuntu installs.

Post Reply