Always wrap GZipStream with BufferedStream

GZipStream and DeflateStream don’t buffer their input, so they will only compress the individual chunks that are passed to Write (or, worst-case, WriteByte). Unless you can guarantee that large blocks of data are being passed to GZipStream.Write, always wrap GZipStream in a BufferedStream when using CompressionMode.Compress.

In fact, if you call only WriteByte on a compressing GZipStream, it will create “compressed” output that’s almost double the size of the input.

Experimentation determined that the optimal buffer size is 8K; above this, no further compression is gained. So, to create a GZipStream, use a method similar to the following:

public static Stream CreateCompressingGZipStream(Stream stream, bool leaveOpen)
{
return new BufferedStream(
new GZipStream(stream, CompressionMode.Compress, leaveOpen),
8192);
}

This chart shows how the compressed output keeps getting smaller as the buffer size increases (and how writing just one or two bytes at a time almost doubles the 1MB input):

GZipStream output size versus buffer size

// determine best buffer size

foreach (int bufferSize in new[] { 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 1536,
2048, 4096, 6144, 8192, 16384, 32768, 65536, 131072, 262144 })
{
using (MemoryStream output = new MemoryStream())
{
using (GZipStream compressor = new GZipStream(output, CompressionMode.Compress,
leaveOpen : true))
using (BufferedStream buffer = new BufferedStream(compressor, bufferSize))
{
// write simple data that will compress easily
for (int i = 0; i < 1000000; i++)
buffer.WriteByte(0);
}

Console.WriteLine("Buffer Size: {0:n0}, Compressed size: {1:n0}",
bufferSize, output.Length);
}
}

Posted by Bradley Grainger on June 08, 2012